CN111699682A - Method and apparatus for encoding and decoding using selective information sharing between channels - Google Patents

Abstract

A video decoding method, a video decoding apparatus, a video encoding method, and a video encoding apparatus are disclosed. The encoding decision information of the representative channel of the target block is shared as the encoding decision information of the target channel of the target block, and decoding of the target block is performed using the encoding decision information of the target channel. Since the coding decision information of the representative channel is shared with other channels, it is possible to prevent the same coding decision information from being repeatedly signaled. By this prevention, the efficiency of encoding and decoding for the target block, and the like can be improved.

Description

Method for encoding and decoding using selective information sharing between channels and equipment

technical field

The following embodiments relate generally to a video decoding method and apparatus and a video encoding method and apparatus, and more particularly, to a video decoding method and apparatus and video encoding method and apparatus using sharing of selective information between channels .

Background technique

With the continuous development of the information and communication industry, broadcasting services supporting high definition (HD) resolution have become widespread throughout the world. Through this popularity, a large number of users have become accustomed to high-resolution and high-definition images and/or videos.

In order to meet users' demands for high definition, a large number of institutions have accelerated the development of next-generation imaging devices. In addition to high definition TV (HDTV) and full high definition (FHD) TV, user interest in UHD TV has also increased, wherein the resolution of UHD TV is more than four times that of full high definition (FHD) TV. As its interest increases, there is a constant need for image encoding/decoding techniques for images with higher resolution and higher definition.

The image encoding/decoding apparatus and method may use an inter-frame prediction technique, an intra-frame prediction technique, an entropy encoding technique, etc., in order to perform encoding/decoding on high-resolution and high-definition images. The inter prediction technique may be a technique for predicting values of pixels included in a target picture using a temporally preceding picture and/or a temporally succeeding picture. The intra prediction technique may be a technique for predicting values of pixels included in a target picture using information about the pixels in the target picture. An entropy coding technique may be a technique for assigning short codewords to frequently occurring symbols and long codewords to infrequently occurring symbols.

Recently, there has been an increase in demand for high-quality images, such as Ultra High Definition (UHD) images, capable of providing high resolution, a further widened color space, and excellent image quality in various application fields. As images move toward higher resolution and higher quality images, the amount of image data required to provide an image may increase beyond the existing amount of image data. In the case of transmitting image data through a communication medium such as a wired/wireless broadband line or various broadcasting media such as satellite, ground wave, Internet Protocol (IP) network, wireless network, cable or mobile communication network, or in the case of In the case where image data is stored in various types of storage media such as compact disc (CD), digital versatile disc (DVD), universal serial bus (USB) media, and high definition (HD)-DVD, with the image As the amount of data increases, transmission costs and storage costs increase.

With the use of high-resolution and high-quality images, in order to solve inevitable and more serious problems in image data and provide images with higher resolution and higher image quality, efficient image encoding/decoding techniques are required.

SUMMARY OF THE INVENTION

technical problem

Embodiments aim to provide an encoding apparatus and method and a decoding apparatus and method using sharing of selective information between channels.

Technical solutions

According to one aspect, there is provided a decoding method, comprising: sharing encoding decision information of a representative channel of a target block as encoding decision information of a target channel of the target block; and performing on the target block using the encoding decision information of the target channel decoding.

The decoding method may further include receiving a bitstream including information on the target block.

The information about the target block may include encoding decision information for the representative channel.

The information about the target block may not include coding decision information of the target channel.

The encoding decision information of the representative channel may be transform skip information indicating whether the transform will be skipped.

The encoding decision information for the representative channel may indicate which transform is to be used for the channel's transform blocks.

The encoding decision information of the representative channel may be intra-frame encoding decision information of the representative channel.

The representative channel and the target channel may be channels in the YCbCr color space.

The representative channel may be a luminance channel.

The target channel may be a chrominance channel.

The representative channel may be the color channel with the highest correlation with the luminance signal.

The representative channel may be determined by an index in the bitstream indicating the selected representative channel.

The sharing operation may be performed when the image attributes of the plurality of channels of the target block are similar to each other.

When the intra prediction mode of the chroma channel of the target block is the direct mode, the image properties of the plurality of channels may be determined to be similar to each other.

The sharing operation may be performed when cross-channel prediction is used.

Whether to use cross-channel prediction may be derived based on information obtained from the bitstream.

The sharing operation may be performed when cross-channel prediction is used.

Whether to use cross-channel prediction may be determined based on the intra prediction mode of the target block.

When the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode, cross-channel prediction may be used.

Whether sharing is to be performed may be determined based on the size of the target block.

The encoding decision information of a representative channel of the plurality of channels of the target block may be used for all channels of the plurality of channels.

According to another aspect, there is provided an encoding method comprising: determining encoding decision information of a representative channel of a target block; and performing encoding on a target block using the encoding decision information of the representative channel, wherein the representative channel The encoding decision information of the channel is shared with the other channels of the target block.

The encoding method may further include generating a bitstream including information on the target block.

The information about the target block may include encoding decision information for the representative channel.

The information on the target block may not include coding decision information for the additional channel.

The representative channel and the additional channel may be channels in the YCbCr color space.

According to another aspect, there is provided a computer-readable storage medium storing a bitstream for image decoding, the bitstream including information about a target block, wherein the information about the target block includes a representation of the target block encoding decision information of a channel, wherein encoding decision information of a representative channel of the target block is used and shared as encoding decision information of a target channel of a target block, and wherein the encoding decision information of the target channel is used to perform Decoding of the target block.

beneficial effect

Provided are an encoding apparatus and method and a decoding apparatus and method using sharing of selectivity information between channels.

Description of drawings

1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present disclosure is applied;

2 is a block diagram showing a configuration of an embodiment of a decoding apparatus to which the present disclosure is applied;

3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded;

4 is a diagram illustrating a form of a prediction unit (PU) that a coding unit (CU) can include;

5 is a diagram illustrating the form of a transform unit (TU) that can be included in a CU;

Figure 6 shows the division of blocks according to an example;

7 is a diagram for explaining an embodiment of an intra prediction process;

8 is a diagram for explaining the positions of reference samples used in the intra prediction process;

9 is a diagram for explaining an embodiment of an inter prediction process;

Figure 10 illustrates a spatial candidate according to an embodiment;

11 illustrates the order of adding motion information of spatial candidates to a merge list, according to an embodiment;

Figure 12 illustrates transform and quantization processing according to an example;

Figure 13 shows a diagonal scan according to an example;

Figure 14 shows a horizontal scan according to an example;

Figure 15 shows vertical scanning according to an example;

16 is a configuration diagram of an encoding apparatus according to an embodiment;

17 is a configuration diagram of a decoding apparatus according to an embodiment;

18 is a flowchart of a method for decoding encoding decision information, according to an embodiment;

19 is a flowchart of a decoding method for determining whether a transform is to be skipped, according to an embodiment;

20 is a flowchart of a decoding method for determining whether a transform is to be skipped with reference to an intra-mode, according to an embodiment;

21 is a flowchart of a method for sharing transform selection information, according to an embodiment;

Figure 22 shows a single tree block partition structure according to an example;

Figure 23 shows a dual tree block partition structure according to an example;

24 illustrates a scheme for specifying a corresponding block based on a location in a corresponding region, according to an example;

25 illustrates a scheme for specifying corresponding blocks based on areas in corresponding regions, according to an example;

26 illustrates another scheme for specifying corresponding blocks based on areas in corresponding regions, according to an example;

27 illustrates a scheme for specifying a corresponding block based on the form of the block in the corresponding region, according to an example;

28 illustrates another scheme for specifying a corresponding block based on the form of the block in the corresponding region, according to an example;

29 illustrates a scheme for specifying a corresponding block based on an aspect ratio of a block in a corresponding region, according to an example;

30 illustrates another scheme for specifying a corresponding block based on an aspect ratio of a block in a corresponding region, according to an example;

31 illustrates a scheme for specifying corresponding blocks based on coding characteristics of blocks in corresponding regions, according to an example;

Figure 32 is a flowchart of an encoding method according to an embodiment; and

33 is a flowchart of a decoding method according to an embodiment.

Detailed ways

The present invention may be variously modified and may have various embodiments, and specific embodiments will be described in detail below with reference to the accompanying drawings. It should be understood, however, that these examples are not intended to limit the invention to the particular disclosed forms, and that they include all changes, equivalents, or modifications included within the spirit and scope of the invention.

The following exemplary embodiments will be described in detail with reference to the accompanying drawings that illustrate specific embodiments. These embodiments are described so that those of ordinary skill in the art to which this disclosure pertains can easily practice the embodiments. It should be noted that the various embodiments differ from each other, but need not be mutually exclusive. For example, the specific shapes, structures, and characteristics described herein can be implemented in other embodiments without departing from the spirit and scope of the various embodiments associated with one embodiment. Furthermore, it is to be understood that the location or arrangement of the various components in each disclosed embodiment can be changed without departing from the spirit and scope of the embodiments. Therefore, the appended detailed description is not intended to limit the scope of the present disclosure, and the scope of the exemplary embodiments is limited only by the appended claims and their equivalents, so long as they are appropriately described.

In the drawings, like reference numerals are used to designate the same or similar functions in various respects. The shapes, sizes, etc. of components in the drawings may be exaggerated for clarity of description.

Terms such as "first" and "second" may be used to describe various components, but the components are not limited by the terms. The terms are only used to distinguish one component from another. For example, a first component could be termed a second component without departing from the scope of this specification. Similarly, the second component may be referred to as the first component. The term "and/or" can include a combination of multiple associated descriptive items or any one of the multiple associated descriptive items.

It will be understood that when a component is referred to as being "connected" or "coupled" to another component, the two components can be directly connected or coupled to each other or intervening components may be present between the two components. It will be understood that when components are referred to as being "directly connected or coupled," there are no intervening components between the two components.

Furthermore, the components described in the embodiments are shown independently to represent different feature functions, but this does not mean that each component is formed by a separate piece of hardware or software. That is, for convenience of description, a plurality of components are individually arranged and included. For example, at least two of the plurality of components may be integrated into a single component. Conversely, a component can be divided into multiple components. Embodiments in which a plurality of components are integrated or embodiments in which some components are separated are included in the scope of the present specification as long as they do not depart from the essence of the present specification.

Furthermore, it should be noted that, in an exemplary embodiment, the expression describing that a component "includes" a particular component means that additional components may be included within the scope of the practical or technical spirit of the exemplary embodiment, but does not preclude the existence of Components other than a specific component.

The terms used in this specification are used to describe specific embodiments only, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context specifically indicates the contrary. In this specification, it should be understood that terms such as "comprising" or "having" are only intended to indicate the presence of features, numbers, steps, operations, components, parts or combinations thereof, and are not intended to preclude the presence or addition of one or more the possibility of other features, numbers, steps, operations, components, parts or combinations thereof.

The embodiments will be described in detail below with reference to the accompanying drawings so that those skilled in the art to which the embodiments pertain can easily practice the embodiments. In the following description of the embodiments, detailed descriptions of well-known functions or configurations that are considered to obscure the gist of the present specification will be omitted. Also, the same reference numerals are used throughout the drawings to designate the same components, and repeated descriptions of the same components will be omitted.

Hereinafter, "image" may refer to a single picture that constitutes a video, or may refer to the video itself. For example, "encoding and/or decoding an image" may mean "encoding and/or decoding a video", and may also mean "encoding and/or decoding any one of a plurality of images constituting a video".

Hereinafter, the terms "video" and "moving picture" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the target image may be an encoding target image that is a target to be encoded and/or a decoding target image that is a target to be decoded. Also, the target image may be an input image input to an encoding device or an input image input to a decoding device.

Hereinafter, the terms "image", "picture", "frame" and "screen" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the target block may be an encoding target block (ie, a target to be encoded) and/or a decoding target block (ie, a target to be decoded). Furthermore, the target block may be the current block, ie the target currently to be encoded and/or decoded. Here, the terms "target block" and "current block" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the terms "block" and "unit" may be used to have the same meaning and may be used interchangeably with each other. Alternatively, a "block" may refer to a specific unit.

In the following, the terms "region" and "fragment" are used interchangeably with each other.

Hereinafter, the specific signal may be a signal indicating a specific block. For example, the original signal may be a signal indicating the target block. The prediction signal may be a signal indicating a prediction block. The residual signal may be a signal indicative of a residual block.

In the following embodiments, certain information, data, flags, elements and attributes may have their respective values. A value of "0" corresponding to each of the information, data, flags, elements and attributes may indicate a logical false or a first predefined value. In other words, the value "0", false, logical false and the first predefined value can be used interchangeably with each other. A value of "1" corresponding to each of the information, data, flags, elements and attributes may indicate logical true or a second predefined value. In other words, the value "1", true, logical true and the second predefined value can be used interchangeably with each other.

When a variable such as i or j is used to denote a row, column, or index, the value i may be an integer of 0 or greater, or an integer of 1 or greater. In other words, in an embodiment, each of the row, column, and index may be counted from 0, or may be counted from 1.

Hereinafter, terms to be used in the embodiments will be described.

Encoder: An encoder represents a device for performing encoding.

Decoder: A decoder represents a device for performing decoding.

Unit: A "unit" may represent a unit of image encoding and decoding. The terms "unit" and "block" may be used to have the same meaning and may be used interchangeably with each other.

– A "cell" can be an array of MxN samples. M and N may each be a positive integer. The term "cell" may generally refer to a two-dimensional (2D) array of samples.

– During the encoding and decoding of images, a "unit" can be an area generated by partitioning an image. In other words, a "cell" can be a designated area in an image. A single image can be partitioned into multiple cells. Alternatively, a picture may be partitioned into sub-portions, and a unit may represent each partitioned sub-portion when encoding or decoding is performed on the partitioned sub-portions.

– During the encoding and decoding of images, pre-defined processing can be performed on each cell according to the cell type.

- Unit types can be classified into macrounits, coding units (CUs), prediction units (PUs), residual units, transform units (TUs), etc., according to function. Alternatively, depending on the function, a unit may represent a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, or the like.

- The term "unit" may denote information including a luma component block, a chroma component block corresponding to the luma component block, and syntax elements for each block such that the unit is designated as distinct from the block.

- The size and shape of the cells can be implemented differently. Furthermore, the cells may have any of a variety of sizes and shapes. Specifically, the shape of the cells may include not only squares, but also geometrical shapes (such as rectangles, trapezoids, triangles, and pentagons) that can be represented in two dimensions (2D).

Also, the unit information may include one or more of the type of the unit, the size of the unit, the depth of the unit, the encoding order of the unit, the decoding order of the unit, and the like. For example, the type of unit may indicate one of CU, PU, residual unit, and TU.

- A unit may be partitioned into subunits, each subunit having a smaller size than the size of the associated unit.

– Depth: Depth may represent the extent to which the cell is partitioned. Furthermore, the cell depth may indicate the level at which the corresponding cell exists when the cell is represented in a tree structure.

- The cell partition information may include a depth indicating the depth of the cell. The depth may indicate the number of times the cell is partitioned and/or the degree to which the cell is partitioned.

– In a tree structure, it can be considered that the depth of the root node is the smallest and the depth of the leaf nodes is the largest.

- A single unit can be hierarchically partitioned into multiple sub-units, while the single unit has depth information based on a tree structure. In other words, a unit and subunits generated by partitioning the unit may correspond to a node and a child node of the node, respectively. Each partitioned subunit may have a unit depth. Since the depth indicates the number of times the unit is partitioned and/or the degree to which the unit is partitioned, the partition information for a subunit may include information about the size of the subunit.

– In a tree structure, the top node may correspond to the initial node before partitioning. The top node may be referred to as the "root node". Also, the root node may have a minimum depth value. Here, the depth of the top node may be level "0".

- Nodes with a depth of level "1" may represent cells that were generated when the initial cell was partitioned once. Nodes with a depth of level "2" may represent cells that were generated when the original cell was partitioned twice.

- A leaf node with a depth of level "n" may represent a cell generated when the original cell is partitioned n times.

– Leaf nodes can be bottom nodes that cannot be further partitioned. The depth of a leaf node can be a maximum level. For example, the predefined value for the maximum level may be 3.

-QT depth may represent the depth for the quad partition. BT depth may represent the depth for two partitions. TT depth may represent the depth for three partitions.

– Sample: A sample can be the basic unit that makes up a block. A sample can be represented by a value from 0 to 2 ^Bd -1 according to the bit depth (Bd).

– Samples can be pixels or pixel values.

- In the following, the terms "pixel" and "sample" may be used to have the same meaning and may be used interchangeably with each other.

Coding Tree Unit (CTU): A CTU may consist of a single luma component (Y) coding tree block and two chroma component (Cb, Cr) coding tree blocks associated with the luma component coding tree block. Also, the CTU may represent information including the above-described blocks and syntax elements for each block.

– Each coding tree unit (CTU) may be partitioned using one or more partitioning methods, such as quad-tree (QT), binary-tree (BT), and ternary-tree (TT), in order to configure subunits, such as coding units , prediction unit and transform unit. Furthermore, each coding tree unit may be partitioned with a multi-type tree using one or more partitioning methods.

- "CTU" may be used as a term to designate a block of pixels that is a processing unit in image decoding and encoding processes, such as in the case of partitioning an input image.

Coding Tree Block (CTB): "CTB" may be used as a term to designate any one of a Y coding tree block, a Cb coding tree block, and a Cr coding tree block.

Neighboring Blocks: Neighboring blocks (or neighboring blocks) may represent blocks that are adjacent to the target block. Neighboring blocks may represent reconstructed neighboring blocks.

Hereinafter, the terms "adjacent block" and "adjacent block" may be used to have the same meaning and may be used interchangeably with each other.

Spatial Neighboring Block: A spatially neighboring block may be a block that is spatially adjacent to the target block. Neighboring blocks may include spatially neighboring blocks.

- The target block and spatially adjacent blocks may be included in the target picture.

- A spatially adjacent block may represent a block whose boundary is in contact with the target block or a block located within a predetermined distance from the target block.

- A spatially adjacent block may represent a block adjacent to a vertex of the target block. Here, the block adjacent to the vertex of the target block may represent a block that is vertically adjacent to a neighboring block that is horizontally adjacent to the target block or a block that is horizontally adjacent to a neighboring block that is vertically adjacent to the target block.

Temporal Neighboring Block: A temporally neighboring block may be a block that is temporally adjacent to the target block. The adjacent blocks may include temporally adjacent blocks.

- Temporally adjacent blocks may include co-located blocks (col blocks).

The -col block may be a block in a previously reconstructed co-located picture (col picture). The position of the col block in the col picture may correspond to the position of the target block in the target picture. Alternatively, the position of the col block in the col picture may be equal to the position of the target block in the target picture. The col picture may be a picture included in the reference picture list.

- A temporally adjacent block may be a block that is temporally adjacent to a spatially adjacent block of the target block.

Prediction unit: A prediction unit may be a basic unit for prediction such as inter prediction, intra prediction, inter compensation, intra compensation and motion compensation.

- A single prediction unit may be divided into multiple partitions or sub-prediction units of smaller size. The plurality of partitions may also be basic units when performing prediction or compensation. A partition generated by dividing a prediction unit may also be a prediction unit.

Prediction unit partition: A prediction unit partition may be the shape into which the prediction unit is divided.

Reconstructed Neighbors: Reconstructed neighbors may be units that have been decoded and reconstructed around the target unit.

- The reconstructed neighbor cells may be cells that are spatially adjacent to the target cell or temporally adjacent to the target cell.

- A reconstructed spatially adjacent unit may be a unit included in the target picture that has been reconstructed by encoding and/or decoding.

- A reconstructed temporally adjacent unit may be a unit included in the reference picture that has been reconstructed by encoding and/or decoding. The position of the reconstructed temporally adjacent unit in the reference image may be the same as the position of the target unit in the target picture, or may correspond to the position of the target unit in the target picture.

Parameter set: The parameter set can be header information in the structure of the bitstream. For example, parameter sets may include video parameter sets, sequence parameter sets, picture parameter sets, adaptation parameter sets, and the like.

Also, the parameter set may include slice header information and parallel block header information.

Rate-distortion optimization: An encoding device may use rate-distortion optimization in order to provide high encoding efficiency by utilizing a combination of: coding unit (CU) size, prediction mode, prediction unit (PU) size, motion information, and transform unit (TU) )size of.

- The rate-distortion optimization scheme can calculate the rate-distortion cost of each combination to select the optimal combination from these combinations. The rate-distortion penalty can be calculated using Equation 1 below. In general, the combination that minimizes the rate-distortion penalty can be selected as the optimal combination under the rate-distortion optimization scheme.

[Equation 1]

D+λ*R

-D for distortion. D may be the average of the squares of the differences between the original transform coefficients and the reconstructed transform coefficients in the transform unit (ie, the mean squared error).

-R may represent the rate, which may use relevant context information to represent the bit rate.

–λ represents the Lagrange multiplier. R may include not only encoding parameter information such as prediction mode, motion information, and encoding block flags, but also bits generated as a result of encoding transform coefficients.

- The encoding device may perform processes such as inter prediction and/or intra prediction, transform, quantization, entropy coding, inverse quantization (inverse quantization) and inverse transform in order to calculate exact D and R. These processes can greatly increase the complexity of the encoding device.

- Bitstream: A bitstream may represent a stream of bits comprising encoded image information.

- Parameter set: The parameter set may be header information in the structure of the bitstream.

The parameter set may include at least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set. Furthermore, the parameter set may include information on slice headers and information on parallel block headers.

Parsing: Parsing may be a decision on the value of a syntax element made by performing entropy decoding on the bitstream. Alternatively, the term "parse" may refer to this entropy decoding itself.

Symbol: A symbol may be at least one of a syntax element, an encoding parameter, and a transform coefficient of the encoding target unit and/or the decoding target unit. Furthermore, the symbols can be the target of entropy encoding or the result of entropy decoding.

Reference picture: A reference picture may be an image that is referenced by a unit in order to perform inter prediction or motion compensation. Alternatively, the reference picture may be a picture including a reference unit that is referenced by the target unit in order to perform inter prediction or motion compensation.

Hereinafter, the terms "reference picture" and "reference image" may be used to have the same meaning and may be used interchangeably with each other.

Reference picture list: The reference picture list may be a list including one or more reference pictures used for inter prediction or motion compensation.

- Types of reference picture lists may include merged list (LC), list 0 (L0), list 1 (L1), list 2 (L3), list 3 (L3), etc.

- For inter prediction, one or more reference picture lists may be used.

Inter prediction indicator: The inter prediction indicator may indicate the inter prediction direction for the target unit. Inter prediction may be one of unidirectional prediction and bidirectional prediction. Alternatively, the inter prediction indicator may represent the number of reference pictures used to generate the prediction unit of the target unit. Alternatively, the inter prediction indicator may represent the number of prediction blocks used for inter prediction or motion compensation of the target unit.

Reference picture index: The reference picture index may be an index indicating a specific reference picture in the reference picture list.

Motion Vector (MV): A motion vector can be a 2D vector used for inter prediction or motion compensation. The motion vector can represent the offset between the target image and the reference image.

- For example, the MV can be represented in a form such as (mv _x , mv _y ). mv _x may indicate the horizontal component and mv _y may indicate the vertical component.

- Search range: The search range may be the 2D area where the search for the MV is performed during inter prediction. For example, the size of the search range may be MxN. M and N may each be a positive integer.

Motion vector candidate: A motion vector candidate may be a block that is a prediction candidate when the motion vector is predicted or a motion vector of a block that is a prediction candidate.

- Motion vector candidates may be included in the motion vector candidate list.

Motion Vector Candidate List: A motion vector candidate list may be a list configured using one or more motion vector candidates.

Motion Vector Candidate Index: The motion vector candidate index may be an indicator for indicating a motion vector candidate in the motion vector candidate list. Alternatively, the motion vector candidate index may be an index of a motion vector predictor.

Motion information: The motion information may be information including at least one of a reference picture list, a reference picture, a motion vector candidate, a motion vector candidate index, a merge candidate, and a merge index, and a motion vector, a reference picture index, and an inter prediction indicator.

Merge candidate list: The merge candidate list may be a list using the merge candidate configuration.

Merging candidate: The merging candidate may be a spatial merging candidate, a temporal merging candidate, a combined merging candidate, a combined bi-predictive merging candidate, a zero merging candidate, and the like. Merge candidates may include motion information, such as prediction type information, reference picture indices for each list, and motion vectors.

Merge Index: The merge index may be an indicator for indicating a merge candidate in the merge candidate list.

- The merge index may indicate a reconstruction unit for deriving a merge candidate between a reconstruction unit that is spatially adjacent to the target unit and a reconstruction unit that is temporally adjacent to the target unit.

- The merge index may indicate at least one of multiple pieces of motion information of a merge candidate.

Transform unit: A transform unit may be a basic unit for residual signal encoding and/or residual signal decoding, such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. A single transform unit may be partitioned into multiple transform units with smaller sizes.

Scaling: Scaling can refer to the process of multiplying a factor by a transform coefficient level.

- Transform coefficients may be generated as a result of scaling transform coefficient levels. Scaling may also be referred to as "inverse quantization".

Quantization parameter (QP): A quantization parameter may be a value used to generate transform coefficient levels for transform coefficients in quantization. Optionally, the quantization parameter may also be a value used to generate transform coefficients by scaling transform coefficient levels in inverse quantization. Alternatively, the quantization parameter may be a value mapped to a quantization step size.

Delta quantization parameter: The delta quantization parameter is the difference between the quantization parameter of the target unit and the predicted quantization parameter.

Scanning: Scanning can represent a method of arranging the order of coefficients in a cell, block, or matrix. For example, a method for arranging a 2D array in the form of a one-dimensional (1D) array may be referred to as "scanning". Alternatively, the method for arranging a 1D array in the form of a 2D array may also be referred to as "scanning" or "inverse scanning".

Transform coefficients: Transform coefficients may be coefficient values generated when an encoding device performs transform. Alternatively, the transform coefficients may be coefficient values generated when the decoding apparatus performs at least one of entropy decoding and inverse quantization.

- Quantized levels or quantized transform coefficient levels generated by applying quantization to transform coefficients or residual signals may also be included in the meaning of the term "transform coefficients".

Level of quantization: The level of quantization may be a value generated when an encoding apparatus performs quantization on a transform coefficient or residual signal. Alternatively, the level of quantization may be a value that is a target of inverse quantization when the decoding apparatus performs inverse quantization.

- Quantized transform coefficient levels that are the result of transform and quantization may also be included in the meaning of quantized levels.

Non-zero transform coefficients: Non-zero transform coefficients may be transform coefficients having values other than zero, or may be transform coefficient levels having values other than zero. Alternatively, the non-zero transform coefficients may be transform coefficients whose magnitudes are not zero, or may be transform coefficient levels whose magnitudes are not zero.

Quantization matrix: A quantization matrix can be a matrix used in the quantization process or inverse quantization process in order to improve the subjective image quality or objective image quality of an image. The quantization matrix may also be referred to as a "scale list".

Quantization matrix coefficients: The quantization matrix coefficients can be each element in the quantization matrix. The quantization matrix coefficients may also be referred to as "matrix coefficients".

Default matrix: The default matrix can be a quantization matrix pre-defined by the encoding device and the decoding device.

Non-default matrix: The non-default matrix may be a quantization matrix that is not predefined by the encoding device and the decoding device. The non-default matrix may be signaled by the encoding device to the decoding device.

Most Probable Mode (MPM): MPM may represent an intra prediction mode that is used for intra prediction for a target block with a high probability.

The encoding apparatus and the decoding apparatus may determine one or more MPMs based on encoding parameters related to the target block and attributes of entities related to the target block.

The encoding apparatus and the decoding apparatus may determine one or more MPMs based on the intra prediction mode of the reference block. A reference block may include multiple reference blocks. The plurality of reference blocks may include a spatially adjacent block adjacent to the left of the target block and a spatially adjacent block adjacent to the upper side of the target block. In other words, one or more different MPMs may be determined depending on which intra prediction modes have been used for the reference block.

One or more MPMs may be determined in the same manner in both the encoding device and the decoding device. That is, the encoding apparatus and the decoding apparatus may share the same MPM list including one or more MPMs.

MPM list: An MPM list can be a list that includes one or more MPMs. The number of one or more MPMs in the MPM list may be predefined.

MPM indicator: The MPM indicator may indicate an MPM among one or more MPMs in the MPM list to be used for intra prediction for the target block. For example, the MPM indicator may be an index for the MPM list.

Since the MPM list is determined in the same way in both the encoding device and the decoding device, it may not be necessary to send the MPM list itself from the encoding device to the decoding device.

The MPM indicator may be signaled from the encoding device to the decoding device. Since the MPM indicator is signaled, the decoding apparatus may determine the MPM to be used for intra prediction for the target block among the MPMs in the MPM list.

MPM usage indicator: The MPM usage indicator may indicate whether the MPM usage mode is to be used for prediction for the target block. The MPM usage mode may be a mode in which the MPM to be used for intra prediction for the target block is determined using the MPM list.

The MPM usage indicator may be signaled from the encoding device to the decoding device.

Signaling: "Signaling" may mean that information is sent from an encoding device to a decoding device. Alternatively, "signaling" may mean that information is included in a bitstream or storage medium. The information signaled by the encoding device can be used by the decoding device.

FIG. 1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present disclosure is applied.

The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. A video may include one or more images (pictures). The encoding apparatus 100 may sequentially encode one or more pictures of the video.

1, the encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization (inverse quantization) unit 160, Inverse transform unit 170 , adder 175 , filter unit 180 and reference picture buffer 190 .

The encoding apparatus 100 may perform encoding on the target image using the intra mode and/or the inter mode.

Also, the encoding apparatus 100 may generate a bitstream including information on encoding by encoding the target image, and may output the generated bitstream. The generated bitstream can be stored in a computer-readable storage medium, and can be streamed over a wireless/wired transmission medium.

When the intra mode is used as the prediction mode, the switch 115 may switch to the intra mode. When the inter mode is used as the prediction mode, the switcher 115 may switch to the inter mode.

The encoding apparatus 100 may generate a prediction block of the target block. Also, after the prediction block has been generated, the encoding apparatus 100 may encode the residual between the target block and the prediction block.

When the prediction mode is the intra mode, the intra prediction unit 120 may use the pixels of the previously encoded/decoded neighboring blocks around the target block as reference samples. Intra-prediction unit 120 may perform spatial prediction on the target block using the reference samples, and may generate prediction samples for the target block via spatial prediction.

The inter prediction unit 110 may include a motion prediction unit and a motion compensation unit.

When the prediction mode is the inter mode, the motion prediction unit may search the reference image for a region that best matches the target block in the motion prediction process, and may derive a motion vector for the target block and the found region based on the found region.

Reference images may be stored in the reference picture buffer 190 . More specifically, the reference picture may be stored in the reference picture buffer 190 when encoding and/or decoding of the reference picture has been processed.

The motion compensation unit may generate a prediction block for the target block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional (2D) vector used for inter prediction. Furthermore, the motion vector may represent the offset between the target image and the reference image.

When the motion vector has a value other than an integer, the motion prediction unit and the motion compensation unit may generate a prediction block by applying an interpolation filter to a partial region of the reference image. In order to perform inter prediction or motion compensation, it may be determined which of skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and current picture reference mode corresponds to a CU-based PU for PUs included in the CU A method of predicting and compensating for motion, and inter-prediction or motion compensation may be performed according to this mode.

Subtractor 125 may generate a residual block, where the residual block is the difference between the target block and the prediction block. The residual block may also be referred to as a "residual signal".

The residual signal may be the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing the difference between the original signal and the predicted signal or a signal generated by transforming and quantizing the difference. The residual block may be a residual signal for block units.

The transform unit 130 may generate transform coefficients by transforming the residual block, and may output the generated transform coefficients. Here, the transform coefficients may be coefficient values generated by transforming the residual block.

Transform unit 130 may use one of a number of predefined transform methods when performing the transform.

A number of predefined transform methods may include discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loeve transform (KLT), and the like.

A transform method for transforming the residual block may be determined according to at least one of encoding parameters for the target block and/or neighboring blocks. For example, the transform method may be determined based on at least one of the inter prediction mode for the PU, the intra prediction mode for the PU, the size of the TU, and the shape of the TU. Alternatively, transform information indicating the transform method may be signaled from the encoding apparatus 100 to the decoding apparatus 200 .

When transform skip mode is used, transform unit 130 may omit transforming the residual block.

By quantizing the transform coefficients, a quantized transform coefficient level or a quantized level can be generated. Hereinafter, in an embodiment, each of the quantized transform coefficient level and the quantized level may also be referred to as a "transform coefficient".

The quantization unit 140 may generate a quantized transform coefficient level or a quantized level by quantizing the transform coefficient according to the quantization parameter. The quantization unit 140 may output the generated quantized transform coefficient level or quantized level. In this case, the quantization unit 140 may quantize the transform coefficients using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing probability distribution-based entropy encoding based on values calculated by the quantization unit 140 and/or encoding parameter values calculated during encoding. The entropy encoding unit 150 may output the generated bitstream.

The entropy encoding unit 150 may perform entropy encoding on information about pixels of the image and information required to decode the image. For example, information required to decode an image may include syntax elements and the like.

When entropy coding is applied, fewer bits can be allocated to more frequently occurring symbols, and more bits can be allocated to infrequently occurring symbols. Since the symbols are represented by this allocation, the size of the bit string for the target symbol to be encoded can be reduced. Therefore, the compression performance of video coding can be improved by entropy coding.

Also, for entropy encoding, the entropy encoding unit 150 may use an encoding method such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), or Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding/code (VLC) table. For example, entropy encoding unit 150 may derive a binarization method for the target symbol. Additionally, entropy encoding unit 150 may derive a probability model for the target symbol/bin. The entropy encoding unit 150 may perform arithmetic encoding using the derived binarization method, probability model, and context model.

The entropy encoding unit 150 may transform coefficients in a 2D block form into a 1D vector form through a transform coefficient scanning method in order to encode the quantized transform coefficient levels.

Encoding parameters may be information required for encoding and/or decoding. The encoding parameters may include information encoded by the encoding apparatus 100 and transmitted from the encoding apparatus 100 to the decoding apparatus, and may also include information that may be derived during encoding or decoding. For example, the information sent to the decoding device may include syntax elements.

Encoding parameters may include not only information such as syntax elements (or flags or indices) encoded by the encoding device and signaled by the encoding device to the decoding device, but also information derived in the encoding or decoding process. Furthermore, the encoding parameters may include information required to encode or decode the image. For example, the coding parameters may include at least one of the following values, a combination or statistics of: size of unit/block, depth of unit/block, partition information of unit/block, partition structure of unit/block, indicating unit/block Information whether the block is partitioned in a quad-tree structure, information indicating whether the unit/block is partitioned in a binary tree structure, the partition direction (horizontal direction or vertical direction) of the binary tree structure, the partition form of the binary tree structure (symmetric partition or asymmetric partition) ), information indicating whether the unit/block is partitioned in a ternary tree structure, the partition direction of the ternary tree structure (horizontal direction or vertical direction), the partition form of the ternary tree structure (symmetrical or asymmetrical partition, etc.), indicating the unit/block Information on whether to be partitioned in a composite tree structure, combination and direction (horizontal direction or vertical direction, etc.) of the partitions of the composite tree structure, prediction scheme (intra prediction or inter prediction), intra prediction mode/direction, reference samples filtering method, predictive block filtering method, predictive block boundary filtering method, filter taps for filtering, filter coefficients for filtering, inter prediction mode, motion information, motion vector, reference picture index, inter prediction direction, Inter prediction indicator, reference picture list, reference picture, motion vector predictor, motion vector prediction candidate, motion vector candidate list, information indicating whether merge mode is used, merge candidate, merge candidate list, indicating whether skip mode is used Information used, type of interpolation filter, taps of interpolation filter, filter coefficients of interpolation filter, size of motion vector, accuracy of motion vector representation, transform type, transform size, information indicating whether primary transform is used or not , information indicating whether an additional (secondary) transform is used, first transform selection information (or first transform index), second transform selection information (or second transform index), information indicating the presence or absence of a residual signal, coding Block style, coding block flag, quantization parameter, quantization matrix, information about the in-loop filter, information indicating whether the in-loop filter is applied, coefficients of the in-loop filter, taps of the in-loop filter, in-loop filter shape/form of DF, information indicating whether DF is applied, DF coefficients, DF taps, DF strength, DF shape/form, indicating adaptive samples Information on whether the offset is applied, value of adaptive sample offset, type of adaptive sample offset, type of adaptive sample offset, information indicating whether adaptive loop filter is applied, adaptive Coefficients of loop filter, taps of adaptive loop filter, shape/form of adaptive loop filter, binarization/de-binarization method, context model, context model determination method, context model update method, Information indicating whether the normal mode is executed, information indicating whether the bypass mode is executed, context bins, bypass bins, transform coefficients, transform coefficient levels, transform coefficient level scanning method, image display/output order, Stripe identification information, stripe type, stripe partition information, parallel block identification information, Parallel block type, parallel block partition information, picture type, bit depth, information on luma signals and information on chroma signals. The prediction scheme may represent one of an intra prediction mode and an inter prediction mode.

The first transform selection information may indicate the first transform applied to the target block.

The secondary transform selection information may indicate the secondary transform applied to the target block.

The residual signal may represent the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the predicted signal. The residual block may be a residual signal for the block.

Here, signaling the flag or index may mean that the encoding apparatus 100 includes an entropy-coded flag or an entropy-coded index generated by performing entropy encoding on the flag or index in the bitstream, and may mean that the decoding apparatus 200 performs entropy coding by the decoding apparatus 200 The entropy-coded flag or entropy-coded index extracted from the stream performs entropy decoding to obtain the flag or index.

Since the encoding apparatus 100 performs encoding via inter prediction, the encoded target image can be used as a reference image for another image to be subsequently processed. Accordingly, the encoding apparatus 100 may reconstruct or decode the encoded target image, and store the reconstructed or decoded image in the reference picture buffer 190 as a reference image. For decoding, inverse quantization and inverse transformation of the encoded target image may be performed.

The quantized levels may be inverse quantized by the inverse quantization unit 160 and inversely transformed by the inverse transform unit 170 . The coefficients that have been inverse quantized and/or inverse transformed may be added to the prediction block by an adder 175 . The inverse-quantized and/or inverse-transformed coefficients and the prediction block are added, and then the reconstructed block can be generated. Here, the inversely quantized and/or inversely transformed coefficients may represent coefficients on which one or more of inverse quantization and inverse transformations are performed, and may also represent a reconstructed residual block.

The reconstructed block may be filtered by filter unit 180 . Filter unit 180 may apply one or more of a deblocking filter, a sample adaptive offset (SAO) filter, an adaptive loop filter (ALF), and a non-local filter (NLF) for reconstructing blocks or reconstructing pictures. Filter unit 180 may also be referred to as a "loop filter".

A deblocking filter removes block distortions that occur at boundaries between blocks. In order to determine whether to apply the deblocking filter, a decision may be made on the number of columns or rows of pixels that are included in the block and include the determination of whether to apply the deblocking filter to the target block.

When a deblocking filter is applied to a target block, the applied filter may vary according to the required strength of deblocking filtering. In other words, among different filters, a filter decided in consideration of the strength of deblocking filtering may be applied to the target block. When the deblocking filter is applied to the target block, a filter corresponding to any one of the strong filter and the weak filter may be applied to the target block according to the required strength of the deblocking filter.

Also, when vertical filtering and horizontal filtering are performed on the target block, horizontal filtering and vertical filtering may be performed in parallel.

SAO can add the appropriate offset to the pixel value to compensate for encoding errors. SAO may perform a correction on the image to which deblocking is applied on a pixel-by-pixel basis, where the correction uses the offset of the difference between the original image and the image to which deblocking is applied. In order to perform offset correction for an image, a method for dividing pixels included in the image into a certain number of areas, determining an area to which offset is to be applied among the divided areas, and applying the offset to the determined area may be used. area method, and a method for applying offsets that takes into account edge information for each pixel may also be used.

The ALF may perform filtering based on values obtained by comparing the reconstructed image with the original image. After the pixels included in the image have been divided into a predetermined number of groups, a filter to be applied to each group may be determined, and filtering may be performed differently for each group. For luma signals, information on whether to apply an adaptive loop filter may be signaled for each CU. The shape and filter coefficients of the ALF to be applied to each block may be different for each block. Alternatively, ALF with a fixed form can be applied to the block regardless of the characteristics of the block.

The non-local filter may perform filtering based on a reconstructed block similar to the target block. A region similar to the target block may be selected from the reconstructed picture, and the filtering of the target block may be performed using statistical properties of the selected similar region. Information on whether to apply a non-local filter may be signaled for a coding unit (CU). Furthermore, the shape and filter coefficients of the non-local filters to be applied to a block may differ from block to block.

The reconstructed blocks or reconstructed images filtered by the filter unit 180 may be stored in the reference picture buffer 190 . The reconstructed block filtered by the filter unit 180 may be part of the reference picture. In other words, the reference picture may be a reconstructed picture composed of reconstructed blocks filtered by the filter unit 180 . The stored reference pictures can then be used for inter prediction.

FIG. 2 is a block diagram showing the configuration of an embodiment of a decoding apparatus to which the present disclosure is applied.

The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization (inverse quantization) unit 220, an inverse transform unit 230, an intra prediction unit 240, an inter prediction unit 250, a switch 245, an adder 255, a filter unit 260 and reference picture buffer 270.

The decoding apparatus 200 may receive the bit stream output from the encoding apparatus 100 . The decoding apparatus 200 may receive a bitstream stored in a computer-readable storage medium, and may receive a bitstream streamed through a wired/wireless transmission medium.

The decoding apparatus 200 may perform decoding on the bitstream in the intra mode and/or the inter mode. Also, the decoding apparatus 200 may generate a reconstructed image or a decoded image through decoding, and may output the reconstructed image or the decoded image.

For example, the operation of switching to the intra mode or the inter mode based on the prediction mode used for decoding may be performed by the switch 245 . When the prediction mode for decoding is the intra mode, the switch 245 may be operated to switch to the intra mode. When the prediction mode for decoding is the inter mode, the switch 245 may be operated to switch to the inter mode.

The decoding apparatus 200 may obtain a reconstructed residual block by decoding an input bitstream, and may generate a prediction block. When the reconstructed residual block and the prediction block are acquired, the decoding apparatus 200 may generate a reconstructed block that is a target of decoding by adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream based on a probability distribution of the bitstream. The generated symbols may include quantized transform coefficient level format symbols. Here, the entropy decoding method may be similar to the entropy encoding method described above. That is, the entropy decoding method may be an inverse process of the entropy encoding method described above.

The entropy decoding unit 210 may change coefficients in the form of a one-dimensional (1D) vector into a 2D block shape through a transform coefficient scanning method in order to decode the quantized transform coefficient levels.

For example, the coefficients of the block can be changed to a 2D block shape by scanning the block coefficients using an upper right diagonal scan. Alternatively, which of the upper right diagonal scan, vertical scan and horizontal scan will be used may be determined according to the size of the corresponding block and/or the intra prediction mode.

The quantized coefficients may be inverse quantized by the inverse quantization unit 220 . The inverse quantization unit 220 may generate inverse quantized coefficients by performing inverse quantization on the quantized coefficients. In addition, the inversely quantized coefficients may be inversely transformed by the inverse transform unit 230 . The inverse transform unit 230 may generate a reconstructed residual block by performing an inverse transform on the inversely quantized coefficients. As a result of performing inverse quantization and inverse transform on the quantized coefficients, a reconstructed residual block may be generated. Here, when generating the reconstructed residual block, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When using the intra mode, the intra prediction unit 240 may generate a prediction block by performing spatial prediction using pixel values of previously decoded neighboring blocks around the target block.

Inter prediction unit 250 may include a motion compensation unit. Alternatively, the inter prediction unit 250 may be designated as a "motion compensation unit".

When the inter mode is used, the motion compensation unit 250 may generate a prediction block by performing motion compensation using motion vectors and reference images stored in the reference picture buffer 270 .

The motion compensation unit may apply an interpolation filter to a partial region of the reference image when the motion vector has a value other than an integer, and may generate a prediction block using the reference image to which the interpolation filter is applied. To perform motion compensation, the motion compensation unit may determine, on a CU basis, which of skip mode, merge mode, advanced motion vector prediction (AMVP) mode, and current picture reference mode corresponds to motion compensation for PUs included in the CU method, and motion compensation may be performed according to the determined mode.

The reconstructed residual block and the prediction block may be added to each other by the adder 255 . The adder 255 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The reconstructed block may be filtered by filter unit 260 . The filter unit 260 may apply at least one of a deblocking filter, a SAO filter, an ALF, and an NLF to the reconstructed block or the reconstructed image. The reconstructed image may be a picture including reconstructed blocks.

The filtered reconstructed image may be output by the encoding device 100 and used by the encoding device.

The reconstructed image filtered by the filter unit 260 may be stored in the reference picture buffer 270 as a reference picture. The reconstructed block filtered by the filter unit 260 may be part of the reference picture. In other words, the reference picture may be an image composed of reconstructed blocks filtered by the filter unit 260 . The stored reference pictures can then be used for inter prediction.

FIG. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded.

FIG. 3 may schematically illustrate an example in which a single unit is partitioned into a plurality of subunits.

To efficiently partition an image, coding units (CUs) may be used in encoding and decoding. The term "unit" may be used to collectively designate 1) a block comprising image samples and 2) a syntax element. For example, "partition of a unit" may mean "partition of a block corresponding to a unit".

A CU may be used as a basic unit for image encoding/decoding. The CU may be used as a unit to which one mode selected from the intra mode and the inter mode is applied in image encoding/decoding. In other words, in image encoding/decoding, it may be determined which of the intra mode and the inter mode is to be applied to each CU.

Also, a CU may be a basic unit for predicting, transforming, quantizing, inverse transforming, inverse quantizing, and encoding/decoding transform coefficients.

3, a picture 300 may be sequentially partitioned into units corresponding to a largest coding unit (LCU), and a partition structure may be determined for each LCU. Here, LCU may be used to have the same meaning as Coding Tree Unit (CTU).

Partitioning a cell may mean partitioning a block corresponding to the cell. The block partition information may include depth information about the depth of the unit. The depth information may indicate the number of times the cell is partitioned and/or the extent to which the cell is partitioned. A single unit can be hierarchically partitioned into subunits, while the single unit has depth information based on a tree structure. Each partitioned subunit may have depth information. The depth information may be information indicating the size of the CU. Depth information may be stored for each CU.

Each CU may have depth information. When a CU is partitioned, the depth of the CU generated from the partition may be increased by 1 from the depth of the partitioned CU.

The partition structure may represent the distribution of coding units (CUs) in the LCU 310 that are used to efficiently encode pictures. This distribution may be determined depending on whether a single CU is to be partitioned into multiple CUs. The number of CUs generated by partitioning may be a positive integer of 2 or more, including 2, 3, 4, 8, 16, and so on. According to the number of CUs generated by partitioning, the horizontal size and vertical size of each CU generated by partitioning may be smaller than those of the CU before being partitioned.

Each partitioned CU may be recursively partitioned into four CUs in the same manner. Via recursive partitioning, at least one of the horizontal size and the vertical size of each partitioned CU may be reduced compared to at least one of the horizontal size and the vertical size of the CU before being partitioned.

Partitioning of a CU may be performed recursively up to a predefined depth or a predefined size. For example, the depth of a CU may have a value ranging from 0 to 3. The size of the CU may range from a size of 64×64 to a size of 8×8 depending on the depth of the CU.

For example, the depth of the LCU may be 0, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be the CU having the largest coding unit size, and the SCU may be the CU having the smallest coding unit size.

Partitioning may begin at LCU 310, and the depth of the CU may increase by one whenever the horizontal size and/or vertical size of the CU is reduced by partitioning.

For example, an unpartitioned CU may have a size of 2Nx2N for each depth. Also, in the case where a CU is partitioned, a CU of size 2N×2N may be partitioned into four CUs each of size N×N. The value of N can be halved each time the depth increases by 1.

Referring to FIG. 3 , an LCU with a depth of 0 may have 64×64 pixels or a block of 64×64. 0 can be the minimum depth. A depth-3 SCU may have 8x8 pixels or 8x8 blocks. 3 can be the maximum depth. Here, a CU having a block of 64×64, which is an LCU, may be represented by depth 0. A CU with a block of 32x32 may be represented with a depth of 1. A CU with 16x16 blocks may be represented with a depth of 2. A CU having a block of 8×8 that is an SCU may be represented by a depth of 3.

The information on whether the corresponding CU is partitioned may be represented by the partition information of the CU. The partition information may be 1-bit information. All CUs except SCUs may include partition information. For example, the value of partition information of a CU that is not partitioned may be 0. The value of the partition information of the partitioned CU may be 1.

For example, when a single CU is partitioned into four CUs, the horizontal and vertical sizes of each of the four CUs generated by partitioning may be half of those of the CU before being partitioned. When a CU having a size of 32×32 is partitioned into four CUs, the size of each of the four partitioned CUs may be 16×16. When a single CU is partitioned into four CUs, the CUs may be considered to have been partitioned in a quad-tree structure.

For example, when a single CU is partitioned into two CUs, the horizontal or vertical size of each of the two CUs generated by partitioning may be half the horizontal or vertical size of the CU before being partitioned. When a CU having a size of 32×32 is vertically partitioned into two CUs, the size of each of the two partitioned CUs may be 16×32. When a CU having a size of 32×32 is horizontally partitioned into two CUs, the size of each of the partitioned two CUs may be 32×16. When a single CU is partitioned into two CUs, the CUs may be considered to have been partitioned in a binary tree structure.

Both quad-tree partitioning and binary-tree partitioning are applied to the LCU 310 of FIG. 3 .

In the encoding apparatus 100, a coding tree unit (CTU) having a size of 64×64 may be partitioned into a plurality of smaller CUs through a recursive quad-tree structure. A single CU may be partitioned into four CUs with the same size. Each CU may be recursively partitioned and may have a quadtree structure.

Through recursive partitioning of the CU, the optimal partitioning method that incurs the least rate-distortion cost can be selected.

FIG. 4 is a diagram illustrating a form of a prediction unit (PU) that a coding unit (CU) can include.

Among the CUs partitioned from the LCU, the CUs that are no longer partitioned may be divided into one or more prediction units (PUs). This division is also called "partitioning".

A PU may be a basic unit for prediction. The PU may be encoded and decoded in any one of skip mode, inter mode, and intra mode. The PU can be partitioned into various shapes according to each mode. For example, the target block described above with reference to FIG. 1 and the target block described above with reference to FIG. 2 may both be PUs.

A CU may not be divided into PUs. When the CU is not divided into PUs, the size of the CU and the size of the PU may be equal to each other.

In skip mode, there may be no partitions in the CU. In skip mode, the 2Nx2N mode 410, in which the size of the PU and the size of the CU are the same as each other, may be supported without partitioning.

In inter mode, there can be 8 types of partition shapes in a CU. For example, in inter mode, 2N×2N mode 410, 2N×N mode 415, N×2N mode 420, N×N mode 425, 2N×nU mode 430, 2N×nD mode 435, nL×2N mode can be supported 440 and nR×2N mode 445.

In intra mode, 2Nx2N mode 410 and NxN mode 425 may be supported.

In 2Nx2N mode 410, a PU of size 2Nx2N may be encoded. A PU of size 2N×2N may represent a PU of the same size as that of a CU. For example, a PU of size 2Nx2N may have size 64x64, 32x32, 16x16, or 8x8.

In NxN mode 425, a PU of size NxN may be encoded.

For example, in intra prediction, when the size of the PU is 8×8, four partitioned PUs may be encoded. The size of each partitioned PU may be 4×4.

When the PU is encoded in intra mode, the PU may be encoded using any one of multiple intra prediction modes. For example, HEVC techniques may provide 35 intra-prediction modes under which a PU may be encoded.

Which of the 2Nx2N mode 410 and the NxN mode 425 will be used to encode the PU may be determined based on a rate-distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2N×2N. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100 . Through encoding operations, the optimal intra prediction mode for a PU of size 2Nx2N can be derived. The optimal intra prediction mode may be an intra prediction mode in which the smallest rate-distortion cost occurs when encoding a PU of size 2N×2N among a plurality of intra prediction modes that can be used by the encoding apparatus 100 .

Also, the encoding apparatus 100 may sequentially perform encoding operations on respective PUs obtained by performing N×N partitioning. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100 . Through encoding operations, the optimal intra prediction mode for a PU of size NxN can be derived. The optimal intra prediction mode may be an intra prediction mode in which the smallest rate-distortion cost occurs when encoding a PU of size N×N among a plurality of intra prediction modes that can be used by the encoding apparatus 100 .

The encoding apparatus 100 may determine, based on a comparison between a rate-distortion cost of a PU of size 2N×2N and a rate-distortion cost of a PU of size N×N, among a PU of size 2N×2N and a PU of size N×N which will be encoded.

A single CU may be partitioned into one or more PUs, and a PU may be partitioned into multiple PUs.

For example, when a single PU is partitioned into four PUs, the horizontal and vertical sizes of each of the four PUs generated by partitioning may be half of those of the PU before being partitioned. When a PU of size 32x32 is partitioned into four PUs, the size of each of the four partitioned PUs may be 16x16. When a single PU is partitioned into four PUs, the PUs can be considered to have been partitioned in a quad-tree structure.

For example, when a single PU is partitioned into two PUs, the horizontal or vertical size of each of the two PUs generated by partitioning may be half the horizontal or vertical size of the PU before being partitioned. When a PU of size 32x32 is vertically partitioned into two PUs, the size of each of the two partitioned PUs may be 16x32. When a PU of size 32x32 is horizontally partitioned into two PUs, the size of each of the two partitioned PUs may be 32x16. When a single PU is partitioned into two PUs, the PUs can be considered to have been partitioned in a binary tree structure.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that can be included in a CU.

A transform unit (TU) may be a basic unit in a CU that is used for processes such as transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding.

A TU may have a square shape or a rectangular shape. The shape of the TU may be determined based on the size and/or shape of the CU.

In a CU partitioned from an LCU, a CU that is no longer partitioned as a CU may be partitioned into one or more TUs. Here, the partition structure of the TU may be a quadtree structure. For example, as shown in FIG. 5, a single CU 510 may be partitioned one or more times according to a quadtree structure. Through such partitioning, a single CU 510 may be composed of TUs of various sizes.

It can be considered that a CU is recursively divided when a single CU is divided two or more times. Through partitioning, a single CU may be composed of transform units (TUs) with various sizes.

Optionally, a single CU may be divided into one or more TUs based on the number of vertical and/or horizontal lines dividing the CU.

CUs can be divided into symmetric TUs or asymmetric TUs. In order to divide into asymmetric TUs, information about the size and/or shape of each TU may be signaled from the encoding apparatus 100 to the decoding apparatus 200 . Optionally, the size and/or shape of each TU may be derived from information about the size and/or shape of the CU.

A CU may not be divided into TUs. When the CU is not divided into TUs, the size of the CU and the size of the TU may be equal to each other.

A single CU may be partitioned into one or more TUs, and a TU may be partitioned into multiple TUs.

For example, when a single TU is partitioned into four TUs, the horizontal and vertical sizes of each of the four TUs generated by partitioning may be half of those of the TU before being partitioned. When a TU of size 32x32 is partitioned into four TUs, the size of each of the four partitioned TUs may be 16x16. When a single TU is partitioned into four TUs, the TUs can be considered to have been partitioned in a quad-tree structure.

For example, when a single TU is partitioned into two TUs, the horizontal or vertical size of each of the two TUs generated by partitioning may be half the horizontal or vertical size of the TU before being partitioned. When a TU of size 32x32 is vertically partitioned into two TUs, the size of each of the two partitioned TUs may be 16x32. When a TU of size 32x32 is horizontally partitioned into two TUs, the size of each of the two partitioned TUs may be 32x16. When a single TU is partitioned into two TUs, the TUs can be considered to have been partitioned in a binary tree structure.

The CUs may be divided in a different manner than that shown in FIG. 5 .

For example, a single CU may be divided into three CUs. The horizontal size or vertical size of the three CUs generated by the division may be 1/4, 1/2, and 1/4 of the horizontal size or vertical size of the original CU before being divided, respectively.

For example, when a CU having a size of 32×32 is vertically divided into three CUs, the sizes of the three CUs generated by the division may be 8×32, 16×32, and 8×32, respectively. In this way, when a single CU is divided into three CUs, the CU can be considered to be divided in the form of a ternary tree.

One of the exemplary partitioning forms (ie, quadtree partitioning, binary tree partitioning, and ternary tree partitioning) may be applied to the partitioning of a CU, and multiple partitioning schemes may be combined and used together for the partitioning of a CU. Here, the case where a plurality of division schemes are combined and used together may be referred to as "compound tree format division".

FIG. 6 shows the division of blocks according to an example.

In the video encoding and/or decoding process, as shown in FIG. 6, target blocks may be divided.

For the division of the target block, an indicator indicating division information may be signaled from the encoding apparatus 100 to the decoding apparatus 200 . The division information may be information indicating how the target block is divided.

The division information may be a division flag (hereinafter referred to as "split_flag"), a quad-binary flag (hereinafter referred to as "QB_flag"), a quadtree flag (hereinafter referred to as "quadtree_flag"), a binary tree flag (hereinafter referred to as "binarytree_flag") ) and one or more of a binary type flag (hereinafter referred to as "Btype_flag").

'split_flag' may be a flag indicating whether the block is divided. For example, a split_flag value of 1 may indicate that the corresponding block is split. A split_flag value of 0 may indicate that the corresponding block is not split.

"QB_flag" may be a flag indicating which of the quad-tree form and the binary-tree form corresponds to the shape in which the block is divided. For example, a QB_flag value of 0 may indicate that the block is partitioned in a quadtree. A QB_flag value of 1 may indicate that the block is partitioned in a binary tree. Optionally, a QB_flag value of 0 may indicate that the block is divided in a binary tree. A QB_flag value of 1 may indicate that the block is partitioned in a quadtree.

'quadtree_flag' may be a flag indicating whether a block is divided in a quadtree form. For example, a quadtree_flag value of 1 may indicate that the block is divided in a quadtree. A quadtree_flag value of 0 may indicate that the block is not partitioned in a quadtree.

"binarytree_flag" may be a flag indicating whether the block is divided in a binary tree form. For example, a binarytree_flag value of 1 may indicate that the block is divided in a binary tree. A binarytree_flag value of 0 may indicate that the block is not divided in a binary tree.

'Btype_flag' may be a flag indicating which of vertical division and horizontal division corresponds to the division direction when the block is divided in a binary tree form. For example, a Btype_flag value of 0 may indicate that the block is divided in the horizontal direction. A Btype_flag value of 1 may indicate that the block is divided in the vertical direction. Alternatively, a Btype_flag value of 0 may indicate that the block is divided in the vertical direction. A Btype_flag value of 1 may indicate that the block is divided in the horizontal direction.

For example, partition information for the block in FIG. 6 may be derived by signaling at least one of quadtree_flag, binarytree_flag, and Btype_flag, as shown in Table 1 below.

Table 1

For example, the split information for the block in FIG. 6 may be derived by signaling at least one of split_flag, QB_flag, and Btype_flag, as shown in Table 2 below.

Table 2

The partitioning method may be limited to quadtree or binary tree depending on the size and/or shape of the block. When this restriction is applied, split_flag may be a flag indicating whether the block is divided in the form of a quad-tree or a flag indicating whether the block is divided in the form of a binary tree. The size and shape of the block may be derived from the depth information of the block, and the depth information may be signaled from the encoding apparatus 100 to the decoding apparatus 200 .

Partitioning in the form of a quadtree is only possible when the size of the block falls within a certain range. For example, the specific range may be defined by at least one of a maximum block size and a minimum block size that can be divided only in the form of a quadtree.

Information indicating the maximum block size and the minimum block size that can be divided only in the form of a quadtree may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Furthermore, this information may be signaled for at least one of units such as videos, sequences, pictures, and slices (or segments).

Alternatively, the maximum block size and/or the minimum block size may be fixed sizes predefined by the encoding apparatus 100 and the decoding apparatus 200 . For example, when the size of the block is larger than 64x64 and smaller than 256x256, only division in the form of a quadtree is possible. In this case, split_flag may be a flag indicating whether to perform division in a quadtree form.

Partitioning in binary tree form is only possible when the size of the block falls within a certain range. For example, the specific range may be defined by at least one of a maximum block size and a minimum block size that can be divided only in the form of a binary tree.

Information indicating the maximum block size and/or the minimum block size that can be divided only in a binary tree form may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Furthermore, this information may be signaled for at least one of units such as sequences, pictures, and slices (or slices).

Alternatively, the maximum block size and/or the minimum block size may be fixed sizes predefined by the encoding apparatus 100 and the decoding apparatus 200 . For example, when the size of the block is larger than 8×8 and smaller than 16×16, only division in the form of a binary tree is possible. In this case, split_flag may be a flag indicating whether to perform division in the form of a binary tree.

The division of blocks may be restricted by previous divisions. For example, when a block is divided in the form of a binary tree and a plurality of divided blocks are generated, each divided block may be additionally divided only in the form of a binary tree.

When the horizontal size or vertical size of the partition block is a size that cannot be further divided, the above-mentioned indicator may not be signaled.

FIG. 7 is a diagram for explaining an embodiment of intra prediction processing.

Arrows extending radially from the center of the graph in FIG. 7 indicate the prediction directions of the intra prediction modes. Also, numbers appearing near the arrows indicate examples of mode values assigned to the intra prediction mode or assigned to the prediction direction of the intra prediction mode.

Intra-coding and/or decoding may be performed using reference samples of blocks adjacent to the target block. Neighboring blocks may be neighboring reconstructed blocks. For example, intra-frame encoding and/or decoding may be performed using values of reference samples included in each adjacent reconstructed block or encoding parameters of adjacent reconstructed blocks.

The encoding apparatus 100 and/or the decoding apparatus 200 may generate the prediction block by performing intra prediction on the target block based on the information on the samples in the target image. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block for the target block by performing intra prediction based on information about samples in the target image. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may perform directional prediction and/or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may be a block generated as a result of performing intra prediction. A prediction block may correspond to at least one of a CU, PU, and TU.

A unit of a prediction block may have a size corresponding to at least one of a CU, PU, and TU. The prediction block may have a square shape of size 2Nx2N or NxN. Dimensions NxN may include dimensions 4x4, 8x8, 16x16, 32x32, 64x64, and the like.

Alternatively, the prediction block may be a square block of size 2x2, 4x4, 8x8, 16x16, 32x32, 64x64, etc. or a size of 2x8, 4x8, 2x Rectangular blocks of 16, 4x16, 8x16, etc.

Intra prediction may be performed considering the intra prediction mode for the target block. The number of intra prediction modes that the target block may have may be a predefined fixed value, and may be a value determined differently according to properties of the prediction block. For example, the properties of the prediction block may include the size of the prediction block, the type of the prediction block, and the like.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction block. Alternatively, the number of intra prediction modes may be, for example, 3, 5, 9, 17, 34, 35 or 36.

The intra prediction mode may be a non-directional mode or a directional mode. For example, as shown in FIG. 7, the intra prediction modes may include two non-directional modes and 33 directional modes.

The two non-directional modes may include DC mode and planar mode.

The directional pattern may be a pattern with a specific direction or a specific angle.

All intra prediction modes can be represented by at least one of a mode number, a mode value, and a mode angle. The number of intra prediction modes may be M. The value of M can be 1 or greater. In other words, the number of intra prediction modes may be M, where M includes the number of non-directional modes and the number of directional modes.

The number of intra prediction modes may be fixed to M regardless of the size and/or color components of the block. For example, the number of intra prediction modes may be fixed to any one of 35 and 67 regardless of the size of the block.

Alternatively, the number of intra prediction modes may vary according to the size of the block and/or the type of color component.

For example, the larger the size of the block, the larger the number of intra prediction modes. Optionally, the larger the size of the block, the smaller the number of intra prediction modes. When the size of the block is 4×4 or 8×8, the number of intra prediction modes may be 67. When the size of the block is 16×16, the number of intra prediction modes may be 35. When the size of the block is 32×32, the number of intra prediction modes may be 19. When the size of the block is 64×64, the number of intra prediction modes may be seven.

For example, the number of intra prediction modes may differ depending on whether the color component is a luminance signal or a chrominance signal. Alternatively, the number of intra prediction modes corresponding to luma component blocks may be greater than the number of intra prediction modes corresponding to chroma component blocks.

For example, in a vertical mode with a mode value of 26, prediction may be performed in the vertical direction based on pixel values of reference samples. For example, in a horizontal mode with a mode value of 10, prediction may be performed in the horizontal direction based on pixel values of reference samples.

Even in a direction mode other than the above-described modes, the encoding apparatus 100 and the decoding apparatus 200 may perform intra prediction on the target unit using reference samples according to angles corresponding to the direction mode.

The intra prediction mode located on the right side with respect to the vertical mode may be referred to as "vertical-right mode". The intra prediction mode located below the horizontal mode may be referred to as a "horizontal-down mode". For example, in FIG. 7 , the intra prediction mode whose mode value is one of 27, 28, 29, 30, 31, 32, 33, and 34 may be the vertical-right mode 613. The intra prediction mode whose mode value is one of 2, 3, 4, 5, 6, 7, 8, and 9 may be the horizontal-down mode 616 .

Non-directional modes may include DC mode and planar mode. For example, the value of DC mode may be 1. The value of flat mode can be 0.

Orientation patterns may include angle patterns. Among the plurality of intra prediction modes, the remaining modes other than the DC mode and the planar mode may be directional modes.

When the intra prediction mode is the DC mode, the prediction block may be generated based on an average value of pixel values of a plurality of reference pixels. For example, the value of the pixel of the prediction block may be determined based on an average value of pixel values of a plurality of reference pixels.

The number of intra prediction modes and the mode value of each intra prediction mode described above are only exemplary. The number of intra-prediction modes described above and the mode value of each intra-prediction mode may be differently defined according to embodiments, implementations, and/or requirements.

In order to perform intra prediction on the target block, a step of checking whether samples included in the reconstructed adjacent blocks can be used as reference samples for the target block may be performed. When a sample that cannot be used as a reference sample of the target block exists among the samples in the neighboring block, via interpolation using at least one sample value among the samples included in the reconstructed neighboring block and/or Or copy the resulting value to replace the sample value of a sample that cannot be used as a reference sample. When a sample value of an existing sample is replaced by a value generated via copying and/or interpolation, that sample may be used as a reference sample for the target block.

In intra prediction, a filter may be applied to at least one of reference samples and prediction samples based on at least one of an intra prediction mode and a size of a target block.

The type of filter to be applied to at least one of the reference samples and the prediction samples may be different according to at least one of the intra prediction mode of the target block, the size of the target block, and the shape of the target block. The types of filters can be classified according to one or more of the number of filter taps, the value of the filter coefficients, and the filter strength.

When the intra prediction mode is the plane mode, the upper reference sample of the target block, the left reference sample of the target block, the A weighted sum of the upper right reference samples and the lower left reference samples of the target block is used to generate the sample value of the prediction target block.

When the intra prediction mode is the DC mode, the average value of the reference samples above the target block and the reference samples to the left of the target block may be used when generating the prediction block of the target block. Also, filtering using the values of the reference samples may be performed on a specific row or a specific column in the target block. The particular row may be one or more upper rows adjacent to the reference sample. The particular column may be one or more left columns adjacent to the reference sample.

When the intra prediction mode is the directional mode, the prediction block may be generated using upper reference samples, left reference samples, upper right reference samples and/or lower left reference samples of the target block.

In order to generate the predicted samples described above, real-number based interpolation may be performed.

The intra prediction mode of the target block may be predicted from the intra prediction modes of neighboring blocks adjacent to the target block, and information for prediction may be entropy encoded/entropy decoded.

For example, when the intra prediction modes of the target block and the adjacent blocks are the same as each other, a predefined flag may be used to signal that the intra prediction modes of the target block and the adjacent blocks are the same.

For example, an indicator for indicating the same intra prediction mode as the intra prediction mode of the target block among the intra prediction modes of a plurality of adjacent blocks may be signaled.

When the intra prediction modes of the target block and the neighboring blocks are different from each other, the information on the intra prediction mode of the target block may be encoded and/or decoded using entropy encoding and/or entropy decoding.

FIG. 8 is a diagram for explaining the positions of reference samples used in the intra prediction process.

FIG. 8 shows the positions of reference samples used for intra prediction of the target block. Referring to FIG. 8 , the reconstructed reference samples used for intra prediction of the target block may include a lower left reference sample 831, a left reference sample 833, an upper left reference sample 835, an upper reference sample 837, and an upper right reference sample 839 .

For example, the left reference sample 833 may represent the reconstructed reference pixel adjacent to the left side of the target block. Upper reference samples 837 may represent reconstructed reference pixels adjacent to the top of the target block. The upper left reference sample 835 may represent the reconstructed reference pixel located at the upper left corner of the target block. The lower left reference sample 831 may represent a reference sample located below the left sample line among samples located on the same line as the left sample line composed of the left reference sample 833 . The upper right reference sample 839 may represent a reference sample located to the right of the upper sample line among samples located on the same line as the upper sample line composed of the upper reference sample 837 .

When the size of the target block is N×N, the numbers of the lower left reference samples 831 , the left reference samples 833 , the upper reference samples 837 , and the upper right reference samples 839 may all be N.

The predicted block may be generated by performing intra prediction on the target block. The process of generating the prediction block may include determining values for pixels in the prediction block. The size of the target block and the prediction block may be the same.

The reference samples used for intra prediction of the target block may vary according to the intra prediction mode of the target block. The direction of the intra prediction mode may represent the dependency between the reference samples and the pixels of the prediction block. For example, a value specifying a reference sample may be used as a value specifying one or more pixels in the prediction block. In this case, the designated reference samples and the one or more designated pixels in the prediction block may be samples and pixels located on a straight line in the direction of the intra prediction mode. In other words, the value of the specified reference sample can be copied as the value of the pixel located in the direction opposite to that of the intra prediction mode. Alternatively, the value of a pixel in the prediction block may be the value of a reference sample located in the direction of the intra prediction mode with respect to the position of the pixel.

In an example, when the intra prediction mode of the target block is a vertical mode with a mode value of 26, the upper reference sample 837 may be used for intra prediction. When the intra prediction mode is the vertical mode, the value of a pixel in the prediction block may be the value of a reference sample located vertically above the position of the pixel. Therefore, the upper reference sample 837 adjacent to the top of the target block can be used for intra prediction. Furthermore, the values of the pixels in a row of the prediction block may be the same as the values of the pixels of the upper reference sample 837 .

In an example, when the intra prediction mode of the target block is a horizontal mode with a mode value of 10, the left reference sample 833 may be used for intra prediction. When the intra prediction mode is the horizontal mode, the value of a pixel in the prediction block may be the value of a reference sample located horizontally to the left of the position of the pixel. Therefore, the left reference sample 833 adjacent to the left side of the target block can be used for intra prediction. Furthermore, the values of the pixels in one column of the prediction block may be the same as the values of the pixels of the left reference sample 833 .

In an example, when the mode value of the intra prediction mode of the current block is 18, at least some of the left reference samples 833, the upper left reference samples 835, and the upper reference samples 837 may be used for Intra prediction. When the mode value of the intra prediction mode is 18, the value of the pixel in the prediction block may be the value of the reference sample diagonally located at the upper left corner of the pixel.

Also, in the case where an intra prediction mode with a mode value of 27, 28, 29, 30, 31, 32, 33 or 34 is used, at least a part of the upper right reference samples 839 may be used for intra prediction.

Also, in the case where an intra prediction mode with a mode value of 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least a part of the lower left reference samples 831 may be used for intra prediction.

Also, in the case of an intra prediction mode in which the mode value is a value ranging from 11 to 25, the upper left reference sample 835 may be used for intra prediction.

The number of reference samples used to determine the pixel value of one pixel in the prediction block may be 1 or 2 or more.

As described above, the pixel value of the pixel in the prediction block may be determined according to the position of the pixel and the position of the reference sample indicated by the direction of the intra prediction mode. When the position of the pixel and the position of the reference sample indicated by the direction of the intra prediction mode are integer positions, the value of one reference sample indicated by the integer position may be used to determine the pixel value of the pixel in the prediction block .

When the position of the pixel and the position of the reference sample indicated by the direction of the intra prediction mode are not integer positions, an interpolated reference sample based on the two closest reference samples to the position of the reference sample may be generated. The values of the interpolated reference samples may be used to determine pixel values for pixels in the prediction block. In other words, when the position of the pixel in the prediction block and the position of the reference sample indicated by the direction of the intra prediction mode indicate the position between the two reference samples, a value based on the two samples may be generated interpolation.

The prediction block generated via prediction may be different from the original target block. In other words, there may be a prediction error, which is the difference between the target block and the prediction block, and there may also be a prediction error between the pixels of the target block and the pixels of the prediction block.

In the following, the terms "difference", "error" and "residual" may be used to have the same meaning and may be used interchangeably with each other.

For example, in the case of directional intra prediction, the longer the distance between the pixels of the prediction block and the reference samples, the larger the prediction error that may occur. Such prediction errors can lead to discontinuities between the generated prediction block and neighboring blocks.

To reduce prediction errors, filtering operations for prediction blocks may be used. The filtering operation may be configured to adaptively apply filters to regions in the prediction block that are considered to have larger prediction errors. For example, a region considered to have a larger prediction error may be the boundary of a prediction block. Also, a region considered to have a large prediction error in a prediction block may differ according to intra prediction modes, and characteristics of filters may also differ according to intra prediction modes.

FIG. 9 is a diagram for explaining an embodiment of an inter prediction process.

The rectangles shown in FIG. 9 may represent images (or pictures). Furthermore, in FIG. 9, arrows may indicate prediction directions. That is, each picture may be encoded and/or decoded according to the prediction direction.

Pictures may be classified into intra pictures (I pictures), uni-predictive or predictively encoded pictures (P pictures), and bi-predictive or bi-predictive encoded pictures (B pictures) according to encoding types. Each picture may be encoded and/or decoded according to its encoding type.

When a target image to be encoded is an I picture, the target image can be encoded using data contained in the image itself without performing inter prediction with reference to other images. For example, I pictures may be encoded via intra prediction only.

When the target image is a P picture, the target image may be encoded via inter prediction using reference pictures existing in one direction. Here, the one direction may be a forward direction or a backward direction.

When the target image is a B-picture, the image may be encoded via inter-frame prediction using reference pictures existing in both directions, or may be encoded via frames using reference pictures existing in one of the forward and backward directions Inter prediction encodes images. Here, the two directions may be a forward direction and a backward direction.

P pictures and B pictures encoded and/or decoded using reference pictures may be regarded as pictures using inter prediction.

Hereinafter, the inter prediction in the inter mode according to the embodiment will be described in detail.

Inter prediction may be performed using motion information.

In the inter mode, the encoding apparatus 100 may perform inter prediction and/or motion compensation on the target block. The decoding apparatus 200 may perform inter prediction and/or motion compensation corresponding to the inter prediction and/or motion compensation performed by the encoding apparatus 100 on the target block.

The motion information of the target block may be derived separately by the encoding apparatus 100 and the decoding apparatus 200 during inter prediction. The motion information may be derived using motion information of reconstructed neighboring blocks, motion information of the col block, and/or motion information of blocks adjacent to the col block.

For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and/or motion compensation by using motion information of spatial candidates and/or temporal candidates as motion information of the target block. A target block may represent a PU and/or a PU partition.

Spatial candidates may be reconstructed blocks that are spatially adjacent to the target block.

A temporal candidate may be a reconstructed block corresponding to the target block in a previously reconstructed co-located picture (col picture).

In inter prediction, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency and decoding efficiency by utilizing motion information of spatial candidates and/or temporal candidates. The motion information of the spatial candidates may be referred to as "spatial motion information". The motion information of the temporal candidates may be referred to as "temporal motion information".

Hereinafter, the motion information of the spatial candidate may be motion information of a PU including the spatial candidate. The motion information of the temporal candidate may be motion information of a PU including the temporal candidate. The motion information of the candidate block may be motion information of a PU including the candidate block.

Inter prediction may be performed using reference pictures.

The reference picture may be at least one of a picture preceding the target picture and a picture following the target picture. The reference picture may be a picture used for prediction of the target block.

In inter prediction, a region in a reference picture may be specified using a reference picture index (or refIdx) for indicating a reference picture, a motion vector to be described later, and the like. Here, the area specified in the reference picture may indicate a reference block.

The inter prediction can select a reference picture, and can also select a reference block corresponding to the target block from the reference picture. Also, inter prediction may use the selected reference block to generate a prediction block for the target block.

Motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200 .

Spatial candidates may be 1) blocks that exist in the target picture 2) have been previously reconstructed via encoding and/or decoding and 3) are adjacent to or at the corners of the target block. Here, the "block at the corner of the target block" may be a block that is vertically adjacent to a neighboring block that is horizontally adjacent to the target block, or a block that is horizontally adjacent to a neighboring block that is vertically adjacent to the target block. Also, "a block located at the corner of the target block" may have the same meaning as "a block adjacent to the corner of the target block". The meaning of "a block located at the corner of the target block" may be included in the meaning of "a block adjacent to the target block".

For example, a spatial candidate can be a reconstructed block located to the left of the target block, a reconstructed block located above the target block, a reconstructed block located in the lower left corner of the target block, a reconstructed block located in the upper right corner of the target block, or a target block located in the upper left corner of the target block.

Each of the encoding apparatus 100 and the decoding apparatus 200 may identify a block existing at a position spatially corresponding to the target block in the col picture. The position of the target block in the target picture and the position of the identified block in the col picture may correspond to each other.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a col block that exists at a predefined relevant position for the identified block as a temporal candidate. The predefined relevant locations may be locations that exist inside and/or outside the identified block.

For example, a col block may include a first col block and a second col block. When the coordinates of the identified block are (xP, yP) and the size of the identified block is denoted by (nPSW, nPSH), the first col block may be the block located at coordinates (xP+nPSW, yP+nPSH). The second col block may be the block located at coordinates (xP+(nPSW>>1), yP+(nPSH>>1)). The second col block is optionally used when the first col block is not available.

The motion vector of the target block may be determined based on the motion vector of the col block. Each of the encoding apparatus 100 and the decoding apparatus 200 may scale the motion vector of the col block. The scaled motion vector of the col block may be used as the motion vector of the target block. Also, the motion vector of the running information of the temporal candidates stored in the list may be a scaled motion vector.

The ratio of the motion vector of the target block to the motion vector of the col block may be the same as the ratio of the first distance to the second distance. The first distance may be a distance between the reference picture and the target picture of the target block. The second distance may be a distance between the reference picture and the col picture of the col block.

The scheme for deriving motion information may vary according to the inter prediction mode of the target block. For example, as the inter prediction mode applied to the inter prediction, there may be an Advanced Motion Vector Predictor (AMVP) mode, a merge mode, a skip mode, a current picture reference mode, and the like. Merge mode may also be referred to as "motion merge mode". Each mode will be described in detail below.

1) AMVP mode

When the AMVP mode is used, the encoding apparatus 100 may search for similar blocks in a neighboring area of the target block. The encoding apparatus 100 may acquire a prediction block by performing prediction on a target block using motion information of the found similar blocks. The encoding apparatus 100 may encode the residual block, which is the difference between the target block and the prediction block.

1-1) Create a list of predicted motion vector candidates

When the AMVP mode is used as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may create a list of predicted motion vector candidates using motion vectors of spatial candidates, motion vectors of temporal candidates, and zero vectors. The motion vector predictor candidate list may include one or more motion vector predictor candidates. At least one of a motion vector of a spatial candidate, a motion vector of a temporal candidate, and a null vector may be determined and used as a predicted motion vector candidate.

Hereinafter, the terms "prediction motion vector (candidate)" and "motion vector (candidate)" may be used to have the same meaning, and may be used interchangeably with each other.

Hereinafter, the terms "motion vector predictor candidate" and "AMVP candidate" may be used to have the same meaning, and may be used interchangeably with each other.

Hereinafter, the terms "motion vector predictor candidate list" and "AMVP candidate list" may be used to have the same meaning, and may be used interchangeably with each other.

Spatial candidates may include reconstructed spatial neighboring blocks. In other words, the motion vectors of the reconstructed neighboring blocks may be referred to as "spatially predicted motion vector candidates".

Temporal candidates may include the col block and blocks adjacent to the col block. In other words, the motion vector of the col block or the motion vector of the block adjacent to the col block may be referred to as a "temporal motion vector predictor candidate".

The zero vector can be the (0,0) motion vector.

A motion vector predictor candidate may be a motion vector predictor for predicting a motion vector. Also, in the encoding apparatus 100, each predicted motion vector candidate may be an initial search position for a motion vector.

1-2) Searching for motion vectors using the list of predicted motion vector candidates

The encoding apparatus 100 may determine a motion vector to be used for encoding the target block within the search range using the list of predicted motion vector candidates. Also, the encoding apparatus 100 may determine a motion vector predictor candidate to be used as a motion vector predictor of the target block among the motion vector predictor candidates existing in the motion vector predictor candidate list.

The motion vector to be used for encoding the target block may be a motion vector that can be encoded with minimum cost.

Also, the encoding apparatus 100 may determine whether to encode the target block using the AMVP mode.

1-3) Transmission of inter-frame prediction information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The decoding apparatus 200 may perform inter prediction on the target block using the inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether the AMVP mode is used, 2) a predicted motion vector index, 3) a motion vector difference (MVD), 4) a reference direction, and 5) a reference picture index.

Hereinafter, the terms "predictor motion vector index" and "AMVP index" may be used to have the same meaning, and may be used interchangeably with each other.

Furthermore, the inter prediction information may include residual signals.

When the mode information indicates that the AMVP mode is used, the decoding apparatus 200 may acquire the predicted motion vector index, the MVD, the reference direction, and the reference picture index from the bitstream through entropy decoding.

The motion vector predictor index may indicate a motion vector predictor candidate to be used for prediction of the target block among the motion vector predictor candidates included in the motion vector predictor candidate list.

1-4) Inter prediction in AMVP mode using inter prediction information

The decoding apparatus 200 may derive motion vector predictor candidates using the motion vector predictor candidate list, and may determine motion information of the target block based on the derived motion vector predictor candidates.

The decoding apparatus 200 may determine a motion vector candidate for the target block from among the predicted motion vector candidates included in the predicted motion vector candidate list using the predicted motion vector index. The decoding apparatus 200 may select the motion vector predictor candidate indicated by the motion vector predictor index from among the motion vector predictor candidates included in the motion vector predictor candidate list as the motion vector predictor of the target block.

The motion vector that will actually be used for inter prediction of the target block may not match the predicted motion vector. In order to indicate the difference between the motion vector that will actually be used for inter-predicting the target block and the predicted motion vector, MVD may be used. The encoding apparatus 100 may derive a predicted motion vector similar to the motion vector actually to be used for inter prediction of the target block, so as to use the MVD as small as possible.

The MVD may be the difference between the motion vector of the target block and the predicted motion vector. The encoding apparatus 100 may calculate the MVD, and may perform entropy encoding on the MVD.

The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive the motion vector of the target block by summing the decoded MVD and the predicted motion vector. In other words, the motion vector of the target block derived by the decoding apparatus 200 may be the sum of the entropy decoded MVD and the motion vector candidates.

The reference direction may indicate a list of reference pictures to be used for prediction of the target block. For example, the reference direction may indicate one of the reference picture list L0 and the reference picture list L1.

The reference direction only indicates a list of reference pictures to be used for prediction of the target block, and may not mean that the direction of the reference pictures is limited to the forward direction or the backward direction. In other words, each of the reference picture list L0 and the reference picture list L1 may include pictures in the forward direction and/or the backward direction.

The reference direction being unidirectional may mean using a single reference picture list. The reference direction being bidirectional may mean using two reference picture lists. In other words, the reference direction may indicate one of the following cases: a case where only the reference picture list L0 is used, a case where only the reference picture list L1 is used, and a case where two reference picture lists are used.

The reference picture index may indicate a reference picture to be used for prediction of the target block among the reference pictures in the reference picture list. The reference picture index may be entropy encoded by the encoding apparatus 100 . The entropy-encoded reference picture index may be signaled by the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

When two reference picture lists are used to predict the target block, a single reference picture index and a single motion vector may be used for each of the reference picture lists. Also, when two reference picture lists are used to predict the target block, two prediction blocks may be designated for the target block. For example, the average or weighted sum of the two predicted blocks for the target block may be used to generate the (final) predicted block of the target block.

The motion vector of the target block may be derived through the predicted motion vector index, MVD, reference direction, and reference picture index.

The decoding apparatus 200 may generate a prediction block for the target block based on the derived motion vector and the reference picture index. For example, the prediction block may be the reference block indicated by the derived motion vector in the reference picture indicated by the reference picture index.

Since the predicted motion vector index and the MVD are encoded while the motion vector of the target block itself is not encoded, the number of bits transmitted from the encoding apparatus 100 to the decoding apparatus 200 can be reduced, and encoding efficiency can be improved.

The motion information of the reconstructed neighboring blocks may be used for the target block. In a specific inter prediction mode, the encoding apparatus 100 may not separately encode the actual motion information of the target block. The motion information of the target block is not encoded, but additional information may be encoded, wherein the additional information enables the motion information of the target block to be derived using the motion information of the reconstructed neighboring blocks. Since the additional information is encoded, the number of bits transmitted to the decoding apparatus 200 can be reduced, and encoding efficiency can be improved.

For example, as an inter prediction mode in which the motion information of the target block is not directly encoded, there may be a skip mode and/or a merge mode. Here, each of the encoding apparatus 100 and the decoding apparatus 200 may use an indicator and/or an index indicating a unit whose motion information is to be used as the motion information of the target unit among reconstructed neighboring units.

2) Merge mode

As a scheme for deriving motion information of a target block, there is merging. The term "merging" may mean merging the motions of multiple blocks. "Merge" may mean that the motion information of one block is also applied to other blocks. In other words, the merge mode may be a mode in which the motion information of the target block is derived from the motion information of the neighboring blocks.

When the merge mode is used, the encoding apparatus 100 may predict the motion information of the target block using the motion information of the spatial candidate and/or the motion information of the temporal candidate. The spatial candidates may include reconstructed spatially neighboring blocks that are spatially adjacent to the target block. The spatial neighboring blocks may include left neighboring blocks and upper neighboring blocks. Temporal candidates may include col blocks. The terms "spatial candidate" and "spatial merging candidate" may be used to have the same meaning and may be used interchangeably with each other. The terms "temporal candidate" and "temporal merge candidate" may be used to have the same meaning and may be used interchangeably with each other.

The encoding apparatus 100 may obtain a prediction block through prediction. The encoding apparatus 100 may encode the residual block, which is the difference between the target block and the prediction block.

2-1) Create a merge candidate list

When the merge mode is used, each of the encoding apparatus 100 and the decoding apparatus 200 may create a merge candidate list using motion information of spatial candidates and/or motion information of temporal candidates. The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction can be unidirectional or bidirectional.

The merge candidate list may include merge candidates. Merge candidates may be motion information. In other words, the merging candidate list may be a list storing a plurality of pieces of motion information.

The merging candidates may be pieces of motion information of temporal candidates and/or spatial candidates. Also, the merge candidate list may include new merge candidates generated by combining merge candidates already existing in the merge candidate list. In other words, the merge candidate list may include new motion information generated by combining pieces of motion information previously existing in the merge candidate list.

A merge candidate may be a specific mode in which inter prediction information is derived. The merge candidate may be information indicating a specific mode of deriving inter prediction information. The inter prediction information of the target block may be derived according to the specific mode indicated by the merge candidate. Furthermore, the specific mode may include a process of deriving a series of inter prediction information. This specific mode may be an inter prediction information derivation mode or a motion information derivation mode.

The inter prediction information of the target block may be derived according to the mode indicated by the merge candidate selected from among the merge candidates in the merge candidate list by the merge index.

For example, the motion information derivation mode in the merge candidate list may be at least one of the following modes: 1) a motion information derivation mode for sub-block units; 2) an affine motion information derivation mode.

Also, the merge candidate list may include motion information of the zero vector. A zero vector may also be referred to as a "zero merge candidate".

In other words, the pieces of motion information in the merge candidate list may be at least one of the following information: 1) motion information of spatial candidates, 2) motion information of temporal candidates, 3) motion information of Motion information generated by combining a plurality of pieces of motion information, and 4) a zero vector.

The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction may also be referred to as an "inter prediction indicator". The reference direction can be unidirectional or bidirectional. A unidirectional reference direction may indicate L0 prediction or L1 prediction.

A merge candidate list may be created before prediction in merge mode is performed.

The number of merge candidates in the merge candidate list may be predefined. Each of the encoding apparatus 100 and the decoding apparatus 200 may add merging candidates to the merging candidate list according to a predefined scheme and a predefined priority so that the merging candidate list has a predefined number of merging candidates. The merging candidate list of the encoding apparatus 100 and the merging candidate list of the decoding apparatus 200 may be made identical to each other using a predefined scheme and a predefined priority.

Merging may be applied on a CU or PU basis. When combining is performed on a CU or PU basis, the encoding apparatus 100 may transmit a bitstream including predefined information to the decoding apparatus 200 . For example, the predefined information may include 1) information indicating whether or not to perform merging for each block partition, and 2) information on blocks to be merged among blocks that are spatial candidates and/or temporal candidates for the target block block information.

2-2) Search motion vector using merge candidate list

The encoding apparatus 100 may determine merge candidates to be used for encoding the target block. For example, the encoding apparatus 100 may perform prediction on the target block using the merge candidates in the merge candidate list, and may generate a residual block for the merge candidates. The encoding apparatus 100 may encode the target block using a merge candidate that generates the least cost in encoding of prediction and residual blocks.

Also, the encoding apparatus 100 may determine whether to encode the target block using the merge mode.

2-3) Transmission of inter-frame prediction information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The encoding apparatus 100 may generate entropy-encoded inter-prediction information by performing entropy encoding on the inter-prediction information, and may transmit a bitstream including the entropy-encoded inter-prediction information to the decoding apparatus 200 . The entropy-encoded inter prediction information may be signaled by the encoding apparatus 100 to the decoding apparatus 200 through a bitstream.

The decoding apparatus 200 may perform inter prediction on the target block using the inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether a merge mode is used and 2) a merge index.

Furthermore, the inter prediction information may include residual signals.

The decoding apparatus 200 may acquire the merge index from the bitstream only when the mode information indicates that the merge mode is used.

The mode information may be a merge flag. The unit of mode information may be a block. The information about the block may include mode information, and the mode information may indicate whether the merge mode is applied to the block.

The merge index may indicate a merge candidate to be used for prediction of the target block among the merge candidates included in the merge candidate list. Alternatively, the merge index may indicate a block to be merged with the target block among neighboring blocks that are spatially or temporally adjacent to the target block.

The encoding apparatus 100 may select a merge candidate having the highest encoding performance among the merge candidates included in the merge candidate list, and set the value of the merge index to indicate the selected merge candidate.

2-4) Inter prediction of merge mode using inter prediction information

The decoding apparatus 200 may perform prediction on the target block using the merging candidate indicated by the merging index among the merging candidates included in the merging candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the merge index, the reference picture index, and the reference direction.

3) Skip mode

The skip mode may be a mode in which motion information of a spatial candidate or motion information of a temporal candidate is applied to the target block without change. Also, the skip mode may be a mode in which the residual signal is not used. In other words, when the skip mode is used, the reconstructed block may be the predicted block.

The difference between merge mode and skip mode is whether a residual signal is sent or used. That is, skip mode may be similar to merge mode, except that no residual signal is sent or used.

When the skip mode is used, the encoding apparatus 100 may transmit, to the decoding apparatus 200 through a bitstream, information about a block whose motion information is to be used as the motion information of the target block among the blocks that are spatial candidates or temporal candidates . The encoding apparatus 100 may generate entropy-encoded information by performing entropy encoding on the information, and may signal the entropy-encoded information to the decoding apparatus 200 through a bitstream.

Also, when the skip mode is used, the encoding apparatus 100 may not transmit other syntax information such as MVD to the decoding apparatus 200 . For example, when the skip mode is used, the encoding apparatus 100 may not signal syntax elements related to at least one of MVC, encoding block flags, and transform coefficient levels to the decoding apparatus 200 .

3-1) Create a merge candidate list

The skip mode can also use a merge candidate list. In other words, the merge candidate list can be used in both merge mode and skip mode. In this regard, the merge candidate list may also be referred to as a "skip candidate list" or a "merge/skip candidate list".

Optionally, skip mode may use an additional candidate list that is different from that of merge mode. In this case, in the following description, the merge candidate list and the merge candidate may be replaced with a skip candidate list and a skip candidate, respectively.

A merge candidate list may be created before prediction in skip mode is performed.

3-2) Search motion vector using merge candidate list

The encoding apparatus 100 may determine merge candidates to be used for encoding the target block. For example, the encoding apparatus 100 may perform prediction on the target block using merge candidates in the merge candidate list. The encoding apparatus 100 may encode the target block using a merge candidate that generates the least cost in prediction.

Also, the encoding apparatus 100 may determine whether to encode the target block using the skip mode.

3-3) Transmission of inter-frame prediction information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The decoding apparatus 200 may perform inter prediction on the target block using the inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether a skip mode is used and 2) a skip index.

The skip index may be the same as the merge index described above.

When using skip mode, the target block can be encoded without using a residual signal. The inter prediction information may not include residual signals. Alternatively, the bitstream may not include a residual signal.

The decoding apparatus 200 may acquire the skip index from the bitstream only when the mode information indicates that the skip mode is used. As described above, the merge index and the skip index may be the same as each other. The decoding apparatus 200 may acquire the skip index from the bitstream only when the mode information indicates that the merge mode or the skip mode is used.

The skip index may indicate a merging candidate to be used for prediction of the target block among the merging candidates included in the merging candidate list.

3-4) Inter prediction in skip mode using inter prediction information

The decoding apparatus 200 may perform prediction on the target block using the merge candidate indicated by the skip index among the merge candidates included in the merge candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the skip index, the reference picture index, and the reference direction.

4) Current screen reference mode

The current picture reference mode may represent a prediction mode that uses a previously reconstructed region in the target picture to which the target block belongs.

Motion vectors that specify previously reconstructed regions can be used. The reference picture index of the target block may be used to determine whether the target block has been encoded in the current picture reference mode.

A flag or index indicating whether the target block is a block encoded in the current picture reference mode may be signaled by the encoding apparatus 100 to the decoding apparatus 200 . Alternatively, whether the target block is a block encoded in the current picture reference mode may be inferred from the reference picture index of the target block.

When the target block is encoded in the current picture reference mode, the current picture may exist at a fixed position or an arbitrary position in the reference picture list for the target block.

For example, the fixed position may be the position where the value of the reference picture index is 0 or the last position.

When a target picture exists at an arbitrary position in the reference picture list, an additional reference picture index indicating such an arbitrary position may be signaled by the encoding apparatus 100 to the decoding apparatus 200 .

In the AMVP mode, merge mode, and skip mode described above, motion information to be used for prediction of a target block among pieces of motion information in the list may be specified using an index of the list.

In order to improve encoding efficiency, the encoding apparatus 100 may signal only an index of an element that generates a minimum cost in inter prediction on the target block among the elements in the list. The encoding apparatus 100 may encode the index, and may signal the encoded index.

Therefore, it must be possible to derive the above-described lists (ie, the motion vector predictor candidate list and the merge candidate list) based on the same data using the same scheme by the encoding apparatus 100 and the decoding apparatus 200 . Here, the same data may include reconstructed pictures and reconstructed blocks. Also, in order to specify an element using an index, the order of the elements in the list must be fixed.

FIG. 10 shows spatial candidates according to an embodiment.

In Fig. 10, the positions of the spatial candidates are shown.

The large block at the center of the graph may represent the target block. Five tiles can represent spatial candidates.

The coordinates of the target block can be (xP, yP), and the size of the target block can be represented by (nPSW, nPSH).

The spatial candidate A ₀ may be a block adjacent to the lower left corner of the target block. A ₀ may be a block occupying the pixel located at coordinates (xP-1, yP+nPSH+1).

The spatial candidate A1 may be _a block adjacent to the left of the target block. A ₁ may be the lowermost block among the blocks adjacent to the left of the target block. Alternatively, A1 may be the block adjacent to the top _{of A0} . A1 may be _a block occupying the pixel located at coordinates (xP-1, yP+nPSH).

The spatial candidate B ₀ may be a block adjacent to the upper right corner of the target block. B ₀ may be a block occupying the pixel located at coordinates (xP+nPSW+1, yP-1).

The spatial candidate B1 may be _a block adjacent to the top of the target block. B ₁ may be the rightmost block among the blocks adjacent to the top of the target block. Alternatively, B1 may be the block adjacent to the left _{of B0} . B1 may be _a block occupying the pixel located at coordinates (xP+nPSW, yP-1).

The spatial candidate B2 may be a block adjacent to the upper left corner of the target block. B2 may be a block occupying the pixel located at coordinates (xP-1, yP-1).

Determination of availability of spatial and temporal candidates

In order to include the motion information of the spatial candidate or the motion information of the temporal candidate in the list, it must be determined whether the motion information of the spatial candidate or the motion information of the temporal candidate is available.

Hereinafter, candidate blocks may include spatial candidates and temporal candidates.

For example, the determination may be performed by sequentially applying the following steps 1) to 4).

Step 1) When the PU including the candidate block is located outside the boundary of the picture, the availability of the candidate block may be set to "false". The expression "availability is set to false" may have the same meaning as "set to unavailable".

Step 2) When the PU including the candidate block is located outside the boundary of the slice, the availability of the candidate block may be set to "false". When the target block and the candidate block are located in different slices, the availability of the candidate block may be set to "false".

Step 3) When the PU including the candidate block is located outside the boundary of the parallel block, the availability of the candidate block may be set to "false". When the target block and the candidate block are located in different parallel blocks, the availability of the candidate block may be set to "false".

Step 4) When the prediction mode of the PU including the candidate block is the intra prediction mode, the availability of the candidate block may be set to "false". When the PU including the candidate block does not use inter prediction, the availability of the candidate block may be set to "false".

FIG. 11 illustrates an order of adding motion information of spatial candidates to a merge list according to an embodiment.

As shown in FIG. 11 , when pieces of motion information of spatial candidates are added to the merge list, the order of A ₁ , B ₁ , B ₀ , A ₀ , and B ₂ may be used. That is, pieces of motion information of available space candidates may be added to the merge list in the order of A ₁ , B ₁ , B ₀ , A ₀ , and B ₂ .

Methods for deriving merged lists in merge mode and skip mode

As described above, the maximum number of merge candidates in the merge list can be set. "N" can be used to indicate the maximum number of settings. The set number may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 . The strip header of the strip may include N. In other words, the maximum number of merge candidates in the merge list for the target block of the slice can be set by the slice header. For example, the value of N can be substantially 5.

A plurality of pieces of motion information (ie, merge candidates) may be added to the merge list in the order of steps 1) to 4) below.

Step 1) Among the space candidates, the available space candidates may be added to the merged list. The pieces of motion information of the available space candidates may be added to the merge list in the order shown in FIG. 10 . Here, when the motion information of the available space candidates overlaps with other motion information already existing in the merge list, the motion information of the available space candidates may not be added to the merge list. The operation of checking whether the corresponding motion information overlaps with other motion information existing in the list may be simply referred to as "overlap checking".

The maximum number of pieces of motion information to be added may be N.

Step 2) When the number of pieces of motion information in the merge list is less than N and the temporal candidates are available, the motion information of the temporal candidates can be added to the merge list. Here, when the motion information of the available temporal candidates overlaps with other motion information already existing in the merge list, the motion information of the available temporal candidates may not be added to the merge list.

Step 3) When the number of pieces of motion information in the merge list is less than N and the type of the target slice is "B", the combined motion information generated by combining bidirectional prediction (bi-prediction) may be added to the merge list.

The target slice may be a slice including target blocks.

The combined motion information may be a combination of L0 motion information and L1 motion information. The L0 motion information may be motion information referring only to the reference picture list L0. The L1 motion information may be motion information referring only to the reference picture list L1.

In the merged list, there may be one or more pieces of L0 motion information. Also, in the merged list, there may be one or more pieces of L1 motion information.

The combined motion information may include one or more pieces of combined motion information. When generating combined motion information, L0 motion information and L1 to be used in the step of generating combined motion information among the one or more pieces of L0 motion information and the one or more pieces of L1 motion information may be predefined sports information. One or more pieces of combined motion information may be generated in a predefined order via combined bi-directional prediction using a pair of different motion information in the merged list. One piece of motion information in the pair of different motion information may be L0 motion information, and the other piece of motion information in the pair of different motion information may be L1 motion information.

For example, the combined motion information added with the highest priority may be a combination of L0 motion information with merge index 0 and L1 motion information with merge index 1. When motion information with merge index 0 is not L0 motion information or when motion information with merge index 1 is not L1 motion information, combined motion information may neither be generated nor added. Next, the combined motion information added with the next priority may be a combination of L0 motion information with merge index 1 and L1 motion information with merge index 0. The detailed combinations that follow may conform to other combinations in the field of video encoding/decoding.

Here, when the combined motion information overlaps with other motion information already existing in the merge list, the combined motion information may not be added to the merge list.

Step 4) When the number of pieces of motion information in the merge list is less than N, the motion information of the zero vector can be added to the merge list.

The zero vector motion information may be motion information in which the motion vector is a zero vector.

The number of pieces of zero vector motion information may be one or more. The reference picture indices of one or more pieces of zero vector motion information may be different from each other. For example, the value of the reference picture index of the first zero vector motion information may be 0. The value of the reference picture index of the second zero vector motion information may be 1.

The number of pieces of zero vector motion information may be the same as the number of reference pictures in the reference picture list.

The reference direction of the zero vector motion information can be bidirectional. The two motion vectors can be zero vectors. The number of pieces of zero vector motion information may be the smaller one of the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1. Alternatively, when the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1 are different from each other, a reference direction that is a unidirectional can be used for a reference that can only be applied to a single reference picture list screen index.

The encoding apparatus 100 and/or the decoding apparatus 200 may then add the zero vector motion information to the merge list while changing the reference picture index.

When the zero vector motion information overlaps with other motion information already present in the merge list, the zero vector motion information may not be added to the merge list.

The order of the above steps 1) to 4) is only exemplary and may be changed. Furthermore, some of the above steps may be omitted according to predefined conditions.

Method for deriving motion vector predictor candidate list in AMVP mode

The maximum number of motion vector predictor candidates in the motion vector predictor candidate list may be predefined. N can be used to indicate a predefined maximum number. For example, the predefined maximum number can be 2.

A plurality of pieces of motion information (ie, motion vector predictor candidates) may be added to the motion vector predictor candidate list in the order of step 1) to step 3) below.

Step 1) The available spatial candidates among the spatial candidates may be added to the predicted motion vector candidate list. The spatial candidates may include a first spatial candidate and a second spatial candidate.

The first spatial candidate may be one of A ₀ , A ₁ , scaled A ₀ , and scaled A ₁ . The second spatial candidate may be one of B ₀ , B ₁ , B ₂ , scaled B ₀ , scaled B ₁ , and scaled B ₂ .

The pieces of motion information of the available spatial candidates may be added to the motion vector predictor candidate list in the order of the first spatial candidate and the second spatial candidate. In this case, when the motion information of the available space candidates overlaps with other motion information already existing in the motion vector predictor candidate list, the motion information of the available space candidates may not be added to the motion vector predictor candidate list. In other words, when the value of N is 2, if the motion information of the second spatial candidate is the same as that of the first spatial candidate, the motion information of the second spatial candidate may not be added to the motion vector predictor candidate list.

The maximum number of pieces of motion information to be added may be N.

Step 2) When the number of pieces of motion information in the motion vector predictor candidate list is less than N and the temporal candidates are available, the motion information of the temporal candidates may be added to the motion vector predictor candidate list. In this case, when the motion information of the available temporal candidates overlaps with other motion information already existing in the motion vector predictor candidate list, the motion information of the available temporal candidates may not be added to the motion vector predictor candidate list.

Step 3) When the number of pieces of motion information in the motion vector predictor candidate list is less than N, zero vector motion information may be added to the motion vector predictor candidate list.

The zero vector motion information may include one or more pieces of zero vector motion information. The reference picture indices of the one or more pieces of zero vector motion information may be different from each other.

The encoding apparatus 100 and/or the decoding apparatus 200 may sequentially add pieces of zero vector motion information to the predicted motion vector candidate list while changing the reference picture index.

When the zero vector motion information overlaps with other motion information already present in the motion vector predictor candidate list, the zero vector motion information may not be added to the motion vector predictor candidate list.

The description of the zero vector motion information made above in connection with the merge list can also be applied to the zero vector motion information. Duplicate descriptions thereof will be omitted.

The order of step 1) to step 3) described above is only exemplary and may be changed. Furthermore, some of the steps may be omitted according to predefined conditions.

Figure 12 shows transform and quantization processing according to an example.

As shown in FIG. 12, the quantized levels may be generated by performing transform and/or quantization processing on the residual signal.

The residual signal may be generated as the difference between the original block and the predicted block. Here, the prediction block may be a block generated via intra prediction or inter prediction.

The residual signal can be transformed into a signal in the frequency domain by a transformation process as part of the quantization process.

Transform kernels for transforms may include various DCT kernels, such as discrete cosine transform (DCT) type 2 (DCT-II) and discrete sine transform (DST) kernels.

These transform kernels can perform a separable transform or a two-dimensional (2D) non-separable transform on the residual signal. A separable transform may be a transform that indicates that a one-dimensional (1D) transform is performed on the residual signal in each of the horizontal and vertical directions.

The DCT types and DST types adaptively used for the 1D transform may include DCT-V, DCT-VIII, DST-I, and DST-VII in addition to DCT-II, as shown in Table 3 below.

table 3

As shown in Table 3, a transform set can be used when deriving the DCT type or DST type to be used for the transform. Each transform set may include multiple transform candidates. Each transform candidate may be of DCT type or DST type.

Table 4 below shows an example of a transform set applied to the horizontal direction according to the intra prediction mode.

Table 4

In Table 4, the number of each transform set to be applied to the horizontal direction of the residual signal is indicated according to the intra prediction mode of the target block.

Table 5 below shows an example of a transform set applied to the vertical direction of the residual signal according to the intra prediction mode.

table 5

As exemplified in Tables 4 and 5, transform sets to be applied to the horizontal and vertical directions may be predefined according to the intra prediction mode of the target block. The encoding apparatus 100 may perform transform and inverse transform on the residual signal using the transform included in the transform set corresponding to the intra prediction mode of the target block. Also, the decoding apparatus 200 may perform inverse transform on the residual signal using the transform included in the transform set corresponding to the intra prediction mode of the target block.

In transform and inverse transform, as exemplified in Table 3, Table 4, and Table 5, the set of transforms to be applied to the residual signal may be determined, and may not be signaled. The transformation indication information may be signaled from the encoding apparatus 100 to the decoding apparatus 200 . The transform indication information may be information indicating which of a plurality of transform candidates included in the transform set to be applied to the residual signal is used.

As described above, methods using various transforms may be applied to residual signals generated via intra prediction or inter prediction.

The transformation may include at least one of a first transformation and a second transformation. The transform coefficients may be generated by performing a first transform on the residual signal, and the second transform coefficients may be generated by performing a second transform on the transform coefficients.

The first transformation may be referred to as the "first transformation." Furthermore, the first transform may also be referred to as an "adaptive multi-transform (AMT) scheme". As mentioned above, AMT may represent the application of different transforms to each 1D direction (ie, vertical and/or horizontal) or selected directions.

Alternatively, AMT may be referred to as Multiple Transform Selection (MTS) or Extended Multiple Transform (EMT).

The secondary transform may be a transform for increasing the energy concentration of the transform coefficients generated by the first transform. Similar to the first transform, the second transform can be a separable transform or a non-separable transform. Such a non-separable transform may be a non-separable quadratic transform (NSST).

The first transformation may be performed using at least one of a number of predefined transformation methods. For example, the predefined multiple transform methods may include discrete cosine transform (DCT), discrete sine transform (DST), Karhunen-Loeve transform (KLT), and the like.

Furthermore, the first transformation can be of various types according to the kernel function that defines the DCT or DST.

For example, according to the transform kernel presented in Table 6 below, the first transform may include transforms such as DCT-2, DCT-5, DST-7, DCT-7, DST-8, DST-1, and DCT-8. In Table 6 below, various transform types and transform kernels for multiple transform selection (MTS) are illustrated.

MTS may refer to the selection of a combination of one or more DCT and/or DST cores to transform the residual signal in horizontal and/or vertical directions.

Table 6

In Table 6, i and j may be integer values equal to or greater than 0 and less than or equal to N-1.

The second transform may be performed on the transform coefficients generated by performing the first transform.

The first transform and/or the second transform may be applied to signal components corresponding to one or more of a luma (luma) component and a chroma (chroma) component. Whether to apply the primary transform and/or the secondary transform may be determined according to at least one of encoding parameters for the target block and/or neighboring blocks. For example, whether to apply the primary transform and/or the secondary transform may be determined according to the size and/or shape of the target block.

The transform method to be applied to the first transform and/or the second transform may be determined according to at least one of encoding parameters for the target block and/or neighboring blocks. The determined transform method may also indicate that the first transform and/or the second transform is not used.

Alternatively, transform information indicating the transform method may be signaled from the encoding apparatus 100 to the decoding apparatus 200 . For example, the transform information may include indices of transforms to be used for the first transform and/or the second transform.

The quantized transform coefficients (ie, the levels of quantization) may be generated by performing quantization on the result generated by performing the primary transform and/or the secondary transform or by performing quantization on the residual signal.

Figure 13 shows a diagonal scan according to an example.

FIG. 14 shows horizontal scanning according to an example.

FIG. 15 shows vertical scanning according to an example.

The quantized transform coefficients may be scanned via (top right) at least one of diagonal scanning, vertical scanning, and horizontal scanning according to at least one of intra prediction mode, block size, and block shape. The block may be a transform unit (TU).

Each scan can start at a specific start point and can end at a specific end point.

For example, the quantized transform coefficients may be changed into 1D vector form by scanning the coefficients of the block using the diagonal scanning of FIG. 13 . Alternatively, the horizontal scan of FIG. 14 or the vertical scan of FIG. 15 may be used instead of the diagonal scan according to the size of the block and/or the intra prediction mode.

Vertical scanning may be an operation of scanning 2D block type coefficients in the column direction. Horizontal scanning may be an operation of scanning 2D block type coefficients in the row direction.

In other words, which of the diagonal scan, the vertical scan and the horizontal scan will be used may be determined according to the size of the block and/or the inter prediction mode.

As shown in FIGS. 13 , 14 and 15 , the quantized transform coefficients may be scanned in a diagonal direction, a horizontal direction, or a vertical direction.

The quantized transform coefficients may be represented by a block shape. Each block may include multiple sub-blocks. Each sub-block may be defined according to a minimum block size or minimum block shape.

In scanning, a scanning order according to the type or direction of scanning may be applied to the sub-blocks first. Also, a scanning order according to the direction of scanning may be applied to the quantized transform coefficients in each sub-block.

For example, as shown in FIGS. 13 , 14 and 15 , when the size of the target block is 8×8, quantized transform coefficients may be generated by primary transform, secondary transform, and quantization of the residual signal of the target block . Thus, one of three types of scan orders can be applied to the four 4x4 sub-blocks, and the quantized transform coefficients can also be scanned for each 4x4 sub-block according to the scan order.

The scanned quantized transform coefficients may be entropy encoded, and the bitstream may include the entropy encoded quantized transform coefficients.

The decoding apparatus 200 may generate quantized transform coefficients via entropy decoding of the bitstream. The quantized transform coefficients may be arranged in 2D blocks via inverse scanning. Here, as a method of inverse scanning, at least one of upper-right diagonal scanning, vertical scanning, and horizontal scanning may be performed.

Inverse quantization may be performed on the quantized transform coefficients. The result generated by performing inverse quantization may be subjected to inverse quadratic transformation according to whether or not inverse quadratic transformation is performed. Also, the first inverse transformation may be performed on the result generated by performing the second inverse transformation according to whether the first inverse transformation will be performed. The reconstructed residual signal may be generated by performing a first inverse transform on a result generated by performing a second inverse transform.

FIG. 16 is a configuration diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1600 may correspond to the encoding apparatus 100 described above.

The encoding apparatus 1600 may include a processing unit 1610, a memory 1630, a user interface (UI) input device 1650, a UI output device 1660, and a memory 1640 in communication with each other through a bus 1690. The encoding device 1600 may also include a communication unit 1620 connected to the network 1699 .

The processing unit 1610 may be a central processing unit (CPU) or a semiconductor device for executing processing instructions stored in the memory 1630 or the memory 1640 . The processing unit 1610 may be at least one hardware processor.

The processing unit 1610 may generate and process signals, data, or information input to the encoding apparatus 1600, output from the encoding apparatus 1600, or used in the encoding apparatus 1600, and may perform checking, comparison on the signals, data, or information , OK, etc. In other words, in an embodiment, the generation and processing of data or information and the checking, comparison and determination of data or information may be performed by the processing unit 1610 .

The processing unit 1610 may include an inter prediction unit 110, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175 , filter unit 180 and reference picture buffer 190 .

Inter prediction unit 110, intra prediction unit 120, switch 115, subtractor 125, transform unit 130, quantization unit 140, entropy encoding unit 150, inverse quantization unit 160, inverse transform unit 170, adder 175, filter unit At least some of 180 and reference picture buffer 190 may be program modules and may communicate with an external device or system. The program modules may be included in the encoding device 1600 in the form of an operating system, application program modules, or other program modules.

The program modules may be physically stored in various types of well-known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the encoding apparatus 1200 .

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to embodiments or for implementing abstract data types according to embodiments.

The program modules may be implemented using instructions or code executed by at least one processor of the encoding device 1600 .

Processing unit 1610 may operate inter prediction unit 110, intra prediction unit 120, switch 115, subtractor 125, transform unit 130, quantization unit 140, entropy encoding unit 150, inverse quantization unit 160, inverse transform unit 170, adders 175 , the filter unit 180 and the instructions or code in the reference picture buffer 190 .

The storage unit may represent memory 1630 and/or storage 1640 . Each of memory 1630 and storage 1640 may be any of various types of volatile or nonvolatile storage media. For example, the memory 1630 may include at least one of a read only memory (ROM) 1631 and a random access memory (RAM) 1632 .

The storage unit may store data or information used for the operation of the encoding apparatus 1600 . In an embodiment, the data or information of the encoding apparatus 1600 may be stored in a storage unit.

For example, the storage unit may store pictures, blocks, lists, motion information, inter prediction information, bitstreams, and the like.

The encoding apparatus 1600 may be implemented in a computer system including a computer-readable storage medium.

The storage medium may store at least one module required for the operation of the encoding apparatus 1600 . The memory 1630 may store at least one module and may be configured such that the at least one module is executed by the processing unit 1610 .

Functions related to communication of data or information of the encoding apparatus 1600 may be performed through the communication unit 1620 .

For example, the communication unit 1620 may transmit the bitstream to the decoding apparatus 1600 which will be described later.

FIG. 17 is a configuration diagram of a decoding apparatus according to an embodiment.

The decoding apparatus 1700 may correspond to the decoding apparatus 200 described above.

The decoding apparatus 1700 may include a processing unit 1710, a memory 1730, a user interface (UI) input device 1750, a UI output device 1760, and a memory 1740 in communication with each other through a bus 1790. The decoding device 1700 may also include a communication unit 1720 connected to the network 1799 .

The processing unit 1710 may be a central processing unit (CPU) or a semiconductor device for executing processing instructions stored in the memory 1730 or the memory 1740 . The processing unit 1710 may be at least one hardware processor.

The processing unit 1710 may generate and process signals, data or information input to the decoding apparatus 1700, output from the decoding apparatus 1700, or used in the decoding apparatus 1700, and may perform checking, comparison on the signals, data or information , OK, etc. In other words, in an embodiment, the generation and processing of data or information and the checking, comparison and determination of data or information may be performed by the processing unit 1710 .

The processing unit 1710 may include an entropy decoding unit 210 , an inverse quantization unit 220 , an inverse transform unit 230 , an intra prediction unit 240 , an inter prediction unit 250 , a switch 245 , an adder 255 , a filter unit 260 , and a reference picture buffer 270 .

In the entropy decoding unit 210 , the inverse quantization unit 220 , the inverse transform unit 230 , the intra prediction unit 240 , the inter prediction unit 250 , the adder 255 , the switch 245 , the filter unit 260 and the reference picture buffer 270 of the decoding apparatus 200 At least some of the modules may be program modules and may communicate with external devices or systems. The program modules may be included in the decoding device 1700 in the form of an operating system, application program modules, or other program modules.

Program modules may be physically stored in various types of well-known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the decoding apparatus 1700 .

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to embodiments or for implementing abstract data types according to embodiments.

The program modules may be implemented using instructions or code executed by at least one processor of the decoding apparatus 1700 .

Processing unit 1710 may operate entropy decoding unit 210 , inverse quantization unit 220 , inverse transform unit 230 , intra prediction unit 240 , inter prediction unit 250 , switch 245 , adder 255 , filter unit 260 , and reference picture buffer 270 instruction or code in .

The storage unit may represent memory 1730 and/or storage 1740 . Each of memory 1730 and storage 1740 may be any of various types of volatile or non-volatile storage media. For example, the memory 1730 may include at least one of the ROM 1731 and the RAM 1732 .

The storage unit may store data or information for the operation of the decoding apparatus 1700 . In an embodiment, the data or information of the decoding apparatus 1700 may be stored in a storage unit.

For example, the storage unit may store pictures, blocks, lists, motion information, inter prediction information, bitstreams, and the like.

The decoding apparatus 1700 may be implemented in a computer system including a computer-readable storage medium.

The storage medium may store at least one module required for the operation of the decoding apparatus 1700 . The memory 1730 may store at least one module and may be configured such that the at least one module is executed by the processing unit 1710 .

Functions related to communication of data or information of the decoding apparatus 1700 may be performed through the communication unit 1720 .

For example, the communication unit 1720 may receive the bitstream from the encoding apparatus 1600 .

Image processing method using sharing of information between channels

Methods and apparatuses according to embodiments may apply transform coding (transcoding) techniques using prediction and various transforms to high-resolution images, such as 4K or 8K resolution images, by sharing various types of The encoding decision information is predefined to encode and/or decode the image, and the compressed bitstream or compressed data for the encoded image may be decoded by sharing the transmitted encoding decision information between channels.

Multiple channels may represent multiple components representing a block. For example, the plurality of channels may include color channels, depth channels, alpha channels, and the like.

Hereinafter, the terms "channel" and "color" may have the same meaning and may be used interchangeably with each other. Furthermore, the term "color" may indicate one of the channels. The term "channel" may be used interchangeably with one or more of the terms "color," "depth," and "alpha."

The technique in the present embodiment can be used, and thus the problems of degradation in compression ratio and image quality that occur when conventional techniques are applied to encoding and decoding of images can be solved. In particular, when the conventional technique is applied to an image in which changes in pixel values are spatially concentrated, the problems of degradation of compression ratio and image quality may be serious.

In an example, as various types of encoding decision information shared between channels to perform encoding and decoding according to an embodiment, there are the following pieces of information. In the names of the following information, "flag" may be omitted.

1) Transform skip flag (transform_skip_flag) information may indicate whether to selectively skip transform. Alternatively, the transform_skip_flag information may indicate one of using transform and skipping transform.

2) Intra smoothing filtering information may indicate whether smoothing filtering is applied to reference pixels used in intra prediction.

3) Position Dependent Intra Prediction Combination (PDPC) flag (PDPC_flag) information may indicate whether a smoothing (i.e., filtering) is applied by using adjacent pixels and not applied when a specific intra prediction (eg, planar prediction) is performed Smoothing (ie, filtering) neighboring pixels to perform intra prediction.

4) Residual Differential Pulse Code Modulation (RDPCM) flag (rdpcm_flag) information may indicate whether RDPCM that additionally performs Differential Pulse Code Modulation (DPCM) on the residual signal obtained through one prediction and obtains the residual signal again will be performed.

5) The multiple transform selection (MTS) flag (mts_flag) information may indicate whether an extended multiple transform (EMT) based encoding method is to be used.

EMT may be an encoding method that selects and uses a specified transform for a transform block serving as a target block from among a plurality of transforms provided.

EMT may also stand for "Enhanced Multi-Transform" and may also indicate "Multi-Transform Selection (MTS)".

6) EMT flag information may indicate whether EMT is to be used.

7) The MTS index (mts_idx) information may indicate which transforms will be used in the horizontal and vertical directions when using MTS.

A part of the mts_idx information (eg, a designated bit in mts_idx) may be information indicating the transform used in the horizontal direction of the residual signal.

Another part of the mts_idx information or part of the rest of the mts_idx information (eg, one other specified bit in mts_idx) may be information indicating the transform used in the vertical direction of the residual signal.

For example, the determination of the transformation according to the mts_idx information may be configured as shown in Table 7 and Table 8 below.

[Table 7]

[Table 8]

In Table 7, the transform in the horizontal direction and the transform in the vertical direction used according to the intra prediction mode and the value of the mts_idx information are exemplified.

According to Table 7, when the mts_idx information is acquired in order to perform intra prediction or inter prediction of the target block, the transformation in the horizontal direction and the transformation in the vertical direction to be used for the transformation of the target block may be determined according to the value of the mts_idx information .

For example, when the intra prediction mode of the target block is 6 and the value of mts_idx information is 2, DCT-7 may be used as transform in the horizontal direction, and DCT-8 may be used as transform in the vertical direction.

Table 8 shows a modified example of Table 7. In Table 8, MTS_CU_flag may indicate that flag information mts_flag is determined and transmitted based on the CU, wherein the flag information mts_flag indicates whether a multiple transform selection (MTS) method is used. Also, MTS_Hor_flag and MTS_Ver_flag may indicate the transform used in the horizontal direction and the transform used in the vertical direction, respectively. Table 8 may illustrate transforms used in the horizontal and vertical directions by the values of MTS_Hor_flag and MTS_Ver_flag.

Optionally, the determination of the transformation according to the mts_idx information of Table 7 may be configured as shown in Table 9 below.

[Table 9]

In Table 9, in intra prediction and inter prediction, the horizontal transform type and the vertical transform type used according to the value of mts_idx information are exemplified.

Each value of the horizontal transform type can indicate a specific transform. For example, a value of "1" for the horizontal transform type may represent DST-7. A value of "2" for the horizontal transform type may represent DCT-8.

8) Non-separable secondary transform (NSST) flag (nsst_flag) information may indicate whether an NSST encoding method for additionally performing non-separable secondary transform on all or some of the transform coefficients obtained through the primary transform is to be used.

9) NSST index (nsst_idx) information may indicate the type of secondary transform to be applied to all or some transform coefficients when the NSST encoding method is used.

The nsst_idx information may indicate the transform to be used for the non-separable secondary transform.

10) CU skip flag information may indicate whether the step of sending encoded data for the CU is to be skipped.

11) CU local illuminance compensation (LIC) flag (CU_lic_flag) information may indicate whether to compensate for differences between luminance values of blocks.

12) Overlapping block motion compensation (OBMC) flag (obmc_flag) information may indicate whether multiple overlapping motion compensation blocks are used to generate the final motion compensation block.

13) The codeAlfCtuEnable flag (codeAlfCtuEnable_flag) information may indicate whether an adaptive loop filter (ALF) can be applied to the pixel value of the current CTU.

When such encoding decision information is shared among channels, an image with excellent image quality can be obtained while improving the image compression rate.

When describing encoding decision information to be shared between channels according to the present embodiment, in order to facilitate the overall description and understanding of the embodiment (such as the description of operations, figures, and equations), transform_skip_flag information may be used as the information to be shared between channels An example of encoding decision information.

However, the transform_skip_flag information is only a single example, and the encoding decision information to be shared between channels to which the present embodiment is applied does not necessarily represent only the transform_skip_flag information.

For example, it should be understood that one or more pieces of the above-mentioned pieces of encoding decision information required for decoding (such as 1) rdpcm_flag information, 2) pieces of transform-related selection information such as mts_flag information, mts_idx information, nsst_flag information, and nsst_idx information , 3) obmc_flag information and 4) PDPC_flag information) are included in coding decision information to be shared between channels.

Furthermore, the YCbCr color space may be used as an example when describing channels that will share encoding decision information required for decoding according to an embodiment. However, the YCbCr color space is only a single detailed example, and embodiments can be applied to various color spaces, such as YUV color space, XYZ color space, and RGB color space.

The color index cIDX may be a channel index indicating one of the channels in the color space.

For the YCbCr color space and the YUV color space, cIDX may have values such as "0/1/2" for the channels displayed sequentially in the corresponding color space. The value "a/b/c" may indicate that the value of cIDX indicating the first channel is 'a', the value of cIDX indicating the second channel is 'b', and the value of cIDX indicating the third channel is 'c'.

Alternatively, for the YCbCr color space and the YUV color space, cIDX may have values such as "0/2/1" for the channels displayed sequentially in the corresponding color space.

For the RGB color space and the XYZ color space, cIDX may have values such as "1/0/2" or "2/0/1" for the channels displayed sequentially in the corresponding color space.

As image compression techniques that have been developed or are being developed for the purpose of realizing high-efficiency image encoding/decoding, there may be various techniques such as 1) predicting values of pixels included in a target picture from pictures before or after the target picture 2) An intra prediction technique that predicts the value of a pixel included in the current target picture using the information of the pixel in the target picture, 3) Compresses the energy of the residual signal remaining as a prediction error 4) Entropy coding techniques that assign short codes to more frequently occurring values and long codes to less frequently occurring values, and arithmetic coding techniques. By utilizing these image compression techniques, image data can be efficiently compressed, transmitted and stored.

There are various compression techniques that can be applied to the encoding of images. Furthermore, depending on the properties of the image to be encoded, certain compression techniques may be more advantageous than others. Accordingly, the encoding apparatus 1600 may perform the most favorable compression on the target block by adaptively determining whether to use any one of various types of compression techniques for the target block.

Therefore, in order to select the most advantageous compression technique for the target block from among various alternative compression techniques, the encoding apparatus 1600 typically may perform rate-distortion optimization (RDO). From a rate-distortion perspective, it may not be known in advance which of the various image encoding decisions that may be selected for encoding of an image is optimal. Accordingly, the encoding apparatus 1600 may calculate rate-distortion values for the combinations of all available image encoding decisions by performing encoding (or simplified encoding) for each combination of all available image encoding decisions, and may determine and use the rate-distortion values with the calculated rate-distortion values The image encoding decision with the smallest rate-distortion value in is used as the final image encoding decision for the target block.

In addition, the encoding apparatus 1600 may record in the bitstream encoding decisions derived by performing such RDO or derived using an additional decision method selected by the encoding apparatus 1600 . The decoding apparatus 1700 may read (ie, parse) the encoding decision recorded in the bitstream, and may accurately perform decoding on the target block by performing inverse processing corresponding to encoding according to the encoding decision.

Here, the information indicating the encoding decision may be referred to as "encoding decision information" or "encoding information" required for decoding.

Hereinafter, the terms "coding decision information" and "coding information" may have the same meaning and may be used interchangeably with each other.

Often, the multiple channels used for an image (eg, YUV, YCbCr, RGB, and XYZ) may not always have the same or similar properties. Therefore, making encoding decisions independently of each of the multiple channels generally achieves better performance from a compression ratio perspective.

For example, as one of the above-described encoding decisions, there is transform_skip_flag as an encoding decision indicating whether to perform transform on the target block. That is, it may be determined whether transform is to be skipped for each of the blocks, and transform_skip_flag information indicating such a decision may be recorded in the bitstream for each of the plurality of channels as encoding decision information.

Generally, in encoding for image compression, it has been considered to always perform a transform for a target block. However, when the spatial change of the value of the pixel in the target block that is the target of compression is very large, or especially when the change of the pixel value is very locally limited, the degree to which the image energy is concentrated on the low frequency even if the transformation is applied It may not be very large, and instead, there may be a larger number of transform coefficients for high frequency regions with relatively large values.

Therefore, when the low-frequency signal components are largely maintained and the high-frequency signal components are eliminated by transform and quantization processing, or when transform and quantization processing for reducing the amount of data by applying strong quantization is applied, severe degradation of image quality may occur. In particular, this degradation of image quality may be further enhanced when the spatial changes in the values of pixels are very large or the changes in pixel values are concentrated on very locally restricted areas.

To address the above problems, instead of uniformly applying the transform to the target block, a method for directly encoding the value of a pixel in the spatial domain without transforming can be used. According to this method, it can be determined whether to perform transform on each transform block. By performing transforms or skipping transforms based on such decisions, encoding of transform blocks may be performed. In the bitstream, transform_skip_flag information, which is encoding decision information indicating whether or not to perform transform is to be skipped, may be recorded.

For example, when the value of the transform_skip_flag information is 1, the transform may be skipped. When the value of the transform_skip_flag information is 0, transformation can be performed. The encoding apparatus 1600 may transmit the information on whether to skip the transform for the target block to the decoding apparatus 1700 through the transform_skip_flag information, and may solve the above-mentioned problem by means of such transmission.

Also, pieces of transform_skip_flag information may be respectively set for a luma channel (ie, a Y channel) and a chroma channel (ie, a Cb channel and a Cr channel), and then the pieces of transform_skip_flag information may be transmitted. The decoding apparatus 1700 may perform decoding on the target block by skipping or performing transformation on the channels of the target block according to the value of transform_skip_flag information for each channel read (ie, parsed) from the bitstream.

However, when multiple pieces of transform_skip_flag information for channels such as Y, Cb, and Cr are transmitted for all transform blocks, there may be additional problems that overhead may increase due to signaling multiple pieces of transform_skip_flag information and the compression rate of the image may be degraded.

In order to alleviate problems such as degradation of compression ratio, flag information indicating whether transform is to be skipped may be limitedly transmitted only when the size of the transform block is less than or equal to a specific transform block size. However, even with this scheme, pieces of flag information indicating whether the transform will be skipped must be sent for each transform block with a size larger than the specific block size for all channels, which may still degrade the compression rate of the image. Furthermore, this degraded compression ratio inevitably reduces the quality of the compressed image.

In order to solve the degradation of compression ratio caused by transmitting multiple pieces of encoding decision information selected by the encoding apparatus 1600 for all channels, encoding and/or decoding methods using sharing of information between channels are disclosed in the embodiments.

First, the image properties of the predefinable channels are determined as conditions similar to each other. When these conditions are satisfied, the encoding decision information for the image or block determined by the encoding apparatus 1600 for a representative channel among the plurality of channels may be sent to the decoding apparatus 1700 .

The encoding decision information communicated for the representative channel may be shared and used for all or selected channels of the plurality of channels except the representative channel. By this means of sharing and use, the compression ratio of the image can be improved. Therefore, the encoding and/or decoding method according to the present embodiment can provide excellent encoding efficiency even without transmitting pieces of encoding decision information for multiple channels.

Here, the coding decision information to be shared may include the above-mentioned transform_skip_flag information, intra smoothing filter information, rdpcm_flag information, mts_flag information, mts_idx information, PDPC_flag information, MTS_CU_flag information, MTS_Hor_flag information, MTS_Ver_flag information, nsst_flag information, nsst_idx information, CU skip One or more of flag information, CU_lic_flag information, obmc_flag information, codeAlfCtuEnable_flag information, and PDPC_flag information.

The image properties of the individual channels are determined to be conditions that are similar to each other

In order to determine that the image properties of the channels are similar to each other, it may be checked whether cross-channel prediction (inter-channel prediction) has been used for the target block.

That is, in order to predict the decoding target channel of the target block, it may be checked whether the prediction value for decoding the target channel is obtained using reconstruction information by applying a specific model to another channel (eg, the luma channel). forecasting method. For example, the reconstruction information may be pixel values of reconstructed pixels or values of transform coefficients. The specific model may be a linear model.

The decoding target channel may be a channel that is currently a target to be decoded among the plurality of channels. The encoding target channel may be a channel that is a target to be currently encoded among the plurality of channels. In the following, the encoding target channel and/or the decoding target channel may also be simply referred to as "target channel".

For example, it may be checked whether the intra prediction for the target block uses an intra prediction mode in which a prediction value for the target channel is derived by using reconstruction information of another channel.

In order to use the reconstruction information of another channel to derive the predicted value for the target channel, a cross-component linear model (CCLM) using a single linear model, a multimodal linear model (MMLM) using multiple linear models, and a multimodal linear model using multiple filters Filter linear models can be used. In CCLM, the term "component" may be replaced by "channel".

The INTRA_CCLM mode may be an intra prediction mode using CCLM. INTRA_MMLM mode may be an intra prediction mode using MMLM. INTRA_MFLM mode may be an intra prediction mode using MFLM.

Alternatively, the determination that the image attributes of the channels are similar to each other can be determined by checking whether the intra prediction mode of the target block (eg, intra_chroma_pred_mode information indicating the intra prediction mode for the chroma channel of the target block) is INTRA_CCLM mode, INTRA_MMLM mode and one of INTRA_MFLM mode.

Alternatively, the determination that the image properties of the channels are similar to each other can be achieved by checking whether the encoding mode of the target channel of the target block uses the encoding mode of another channel (eg, the luma channel) unchanged. For example, the determination that the image properties of the channels are similar to each other may be realized by checking whether the intra prediction mode (eg, intra_chroma_pred_mode information) of the target block is direct mode (DM). Direct mode may also be referred to as "derived mode". DM may be a mode indicating that the intra prediction mode of the luma channel is used as the intra prediction mode of the chroma channel without change due to the characteristic that the correlation between the luma channel and the chroma channel may be high.

Features of DM as one of the intra prediction modes and detailed operations thereof may be defined in more detail with reference to Table 10 and Table 11 below.

Table 10 shows (when the value of sps_cclm_enabled_flag information is 0) a method for setting the IntraPredModeC value for intra prediction of chroma signals.

Table 11 shows (when the value of sps_cclm_enabled_flag information is 1) a method for setting the IntraPredModeC value for intra prediction of chroma signals.

[Table 10]

[Table 11]

In general, in intra prediction, it may be determined whether to use intra cross-component linear model (CCLM) mode, intra multi-model LM (MMLM) mode or intra multi-filter LM (MFLM) mode, where in the The pixel values of the reconstructed pixels in the mode for a single channel (eg, the luma channel, or more generally, the representative channel) are used to compute predictions for additional channels (eg, the chroma channel, or more generally, the target channel) value.

The indication of the case of using INTRA_CCLM mode, INTRA_MMLM mode, or INTRA_MFLM mode can be classified into two types, and is defined in detail according to the value of sps_cclm_enabled_flag information, as shown in Table 10 and Table 11.

sps_cclm_enabled_flag information may be information indicating whether INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode are to be enabled. Alternatively, sps_cclm_enabled_flag information may be information indicating whether INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode have been enabled.

When the value of sps_cclm_enabled_flag is 0, the INTRA_CCLM mode, INTRA_MMLM mode and INTRA_MFLM mode may not be used, and when the value of sps_cclm_enabled_flag is 1, the INTRA_CCLM mode may be used. Optionally, when the value of sps_cclm_enabled_flag is 1, at least one of INTRA_MMLM mode and INTRA_MFLM mode may be used.

The intra prediction mode (eg, intra_chroma_pred_mode) of the target channel may be determined by checking whether the intra prediction mode of the chroma channel (ie, the value of intra_chroma_pred_mode) is a specific value (eg, 4 in Table 10 and 5 in Table 11) information) is a DM. In the description of this operation, when the value of sps_cclm_enabled_flag is 0, reference may be made to Table 10, and when the value of sps_cclm_enabled_flag is 1, reference may be made to Table 11.

When the value of sps_cclm_enabled_flag is 0 and the value of the intra prediction mode (eg, intra_chroma_pred_mode) of the target channel is 4, it may be considered that DM is applied. Optionally, when the value of sps_cclm_enabled_flag is 1 and the value of the intra prediction mode (eg, intra_chroma_pred_mode) of the target channel is 5, DM may be considered to be applied. For intra prediction of the target channel (eg, chroma channel) of the target block indicated by DM, the value of IntraPredModeY indicating the intra prediction mode of the representative channel (eg, luma channel) may be changed without change Used as the value for IntraPredModeC.

Here, the intra prediction mode intra_chroma_pred_mode of the chroma signal may be index information indicating which type of intra prediction is to be used for the chroma signal.

With this index information, the final value indicating the intra prediction mode actually used for the intra prediction of the chrominance signal may be the value of IntraPredModeC. In other words, IntraPredModeC may indicate an intra prediction mode actually used for intra prediction of chroma signals.

When the value of sps_cclm_enabled_flag is 0 and DM is applied (ie, the value of intra_chroma_pred_mode is 4), if the value of IntraPredModeY is 0, 50, 18 or 1, the value of IntraPredModeC may also be 0, 50, 18 or 1.

Here, a value of 0 may represent planar mode (ie, planar prediction or planar direction), a value of 1 may represent DC mode, a value of 18 may represent horizontal mode, a value of 50 may represent vertical mode, and a value of 66 may represent diagonal mode.

When the value of IntraPredModeY is another value X different from any one of the four values 0, 50, 18 and 1, the value of IntraPredModeC may also be X equal to the value of IntraPredModeY at this time.

In addition, as shown in the first four rows in Table 10, when the value of cclm_enabled_flag is 0, if the value of IntraPredModeY is 0, 50, 18, or 1, the value of IntraPredModeC may be determined according to the value of IntraPredModeY.

For example, when the value of IntraPredModeY is 0, 50, 18, or 1, as described in the first row of Table 10, the value of IntraPredModeC can be 66, 0, 0, or 0. When the value of IntraPredModeY is another value other than 0, 50, 18, and 1, the value of IntraPredModeC may be 0.

Furthermore, when the value of sps_cclm_enabled_flag is 1 and DM is applied (ie, the value of intra_chroma_pred_mode is 5), if the value of IntraPredModeY is 0, 50, 18 or 1, the value of IntraPredModeC can also be 0, 50, 18 or 1 .

Here, a value of 0 may represent planar mode (ie, planar prediction or planar direction), a value of 1 may represent DC mode, a value of 18 may represent horizontal mode, a value of 50 may represent vertical mode, and a value of 66 may represent diagonal mode.

When the value of IntraPredModeY is another value X different from any one of the four values 0, 50, 18 and 1, the value of IntraPredModeC may also be X equal to the value of IntraPredModeY at this time.

In addition, as shown in the first five rows in Table 11, when the value of cclm_enabled_flag is 1, if the value of IntraPredModeY is 0, 50, 18, or 1, the value of IntraPredModeC may be determined according to the value of IntraPredModeY.

For example, when the value of IntraPredModeY is 0, 50, 18, or 1, the value of IntraPredModeC can be 66, 0, 0, or 0, as described in the first row in Table 11. When the value of IntraPredModeY is another value other than 0, 50, 18, and 1, the value of IntraPredModeC may be 0.

In another embodiment, the determination that the image properties of the channels are similar to each other can be determined by checking whether a mode indicating that only a specific mode limited by the encoding mode of another channel (eg, the luma channel) will be used is used as the mode for the target block The encoding mode of the target channel is implemented. For example, the determination that the image properties of the channels are similar to each other can be made by checking whether the intra prediction mode of the target channel is direct mode (DM).

Cross-channel prediction using correlation between channels

Cross-channel prediction may be a technique that uses pixel values of pixels in another channel instead of intra- or inter-prediction when predicting pixel values of pixels in a target channel.

When the target block is encoded, the fact that the performance of cross-channel prediction is better than that of other types of prediction may indicate that there is considerable similarity between the pixel values of the pixels in the channels of the target block.

Therefore, in this case, when the value of the encoding decision information of the representative channel is determined, the determined value of the encoding decision information of the representative channel is likewise used for the encoding decision information of the other channel or will be determined by the representative channel It may be advantageous for the value of the encoding decision information to indicate a specific value for the encoding decision information of another channel.

For example, when the value of the transform_skip_flag information of the representative channel is 0 (indicating that no transform is skipped), the probability that the determined value of the transform_skip_flag information will be '0' may be high even in other channels.

Therefore, for a picture or block for which cross-channel prediction is valid, there may be cases where multiple pieces of transform_skip_flag information need not be individually specified for multiple channels. The reason for this is that the similarity between channels is high, and thus the probability that pieces of transform_skip_flag information of each channel will be the same as each other may be high.

Despite such image attributes, when multiple pieces of transform_skip_flag information are respectively transmitted for channels of an image, the compression rate and image quality may be degraded.

This principle can also be applied to additional coding decision information, namely mts_flag information, mts_idx information, nsst_flag information, nsst_idx information, intra smoothing information, PDPC_flag information and rdpcm_flag information, and the coding decision information for representative channels The probability that the value will be the same as the value of the encoding decision information for the other channel may be higher.

Therefore, based on the condition that the image attributes of the channels are determined to be similar to each other, it may be determined whether or not the cross-channel prediction using the correlation between the channels has been determined as the encoding mode of the target block.

For example, the determination of whether cross-channel prediction has been determined as the encoding mode of the target block may be aimed at determining whether the image properties of the channels are similar to each other according to whether a color component linear prediction mode (CCLM) indicating cross-channel prediction is applied to the target block . In order to determine whether CCLM is applied to the target block, it may be checked whether the prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode.

For example, the determination of whether cross-channel prediction has been determined to be the encoding mode of the target block may be aimed at determining whether the intra-mode of a representative channel (eg, the luma channel) is used unchanged for another channel (eg, the luma channel) , chrominance channels Cb and Cr). Alternatively, the determination of whether the cross-channel prediction has been determined to be the coding mode of the target block may be aimed at determining whether a particular intra-mode indicated by the intra-mode of the representative channel is used for another channel. Alternatively, the determination of whether the cross-channel prediction has been determined to be the coding mode of the target block may be aimed at determining whether a particular intra-mode derived from the intra-mode of the representative channel is used for the other channel.

For example, the determination of whether cross-channel prediction has been determined to be the encoding mode of the target block may be aimed at determining whether the encoding apparatus 1600 and the decoding apparatus 1700 use a particular encoding mode (eg, an inter-channel sharing mode) consistently with each other.

For example, the determination of whether cross-channel prediction has been determined to be the encoding mode of the target block may be aimed at determining whether the intra-prediction mode of the chroma channel is DM.

Such a DM may be an indication to use the intra-prediction mode of the chroma channel as the intra-prediction mode of the luma channel without change due to the characteristic that the correlation between the luma channel and the chroma channel may be high model. Therefore, when the intra prediction mode of the chroma channel of the target block is DM, it can be determined that the condition that the image attributes of the channels are determined to be similar to each other is satisfied.

In addition to the conditions described in the above example, it may also be determined based on the size of the block whether cross-channel prediction has been determined as the encoding mode of the target block.

For example, the larger the size of a block, the higher the probability that pixels with heterogeneous properties will exist in the corresponding block. Therefore, the similarity between channels of blocks with larger size may be less than the similarity between channels of blocks with smaller size. Furthermore, when the size of the blocks is too small, the similarity between the channels of the blocks may be unstable.

For example, sharing of information between channels may be performed only for blocks whose size is less than or equal to a certain size. This particular size can be 64x64, 32x32 or 16x16. When the block size is less than or equal to the specific size, sharing of information between channels is performed, and thus the condition that the image attributes of the channels are determined to be similar to each other can be more reliably satisfied.

Alternatively, the sharing of information between channels may be performed only for blocks whose size is greater than a certain size. This particular size may be 4x4. When the size of the block is larger than the specific size, sharing of information between channels is performed, and thus the condition that the image attributes of the channels are determined to be similar to each other can be more reliably satisfied.

Optionally, inter-channel information can be performed only if the size of the block is larger than a first specific size (eg, 4×4) and less than or equal to a second specific size (eg, 32×32 or 64×64) of sharing. Sharing of information between channels can be performed only when the size of the block falls within a certain range, and thus the condition that the image attributes of the channels are determined to be similar to each other can be more reliably satisfied.

In an embodiment, a method and apparatus for encoding a target block through sharing of information between channels will be described below, and the following functions may be provided.

- Coding decision information for one channel of the target block can be parsed from the compressed bitstream, and the one can be used encoding decision information for each channel to perform decoding for all or some selected channels of the target block.

- The bitstream may be configured such that encoding decision information is sent only for a representative channel of the target block or for some selected channels.

- It may be determined whether the transform will be skipped for one channel, and the determination of whether the transform will be skipped may be applied to the other channel.

- For one channel of a transform block, the transform_skip_flag information may be parsed from the compressed bitstream. Whether the transform is to be skipped may be determined for one channel or multiple channels of the transform block by utilizing the parsed transform_skip_flag information.

- For one channel of a transform block, transform_skip_flag information may be signaled. The transform_skip_flag information for one channel can even be used for another channel.

- Encoding decision information can be efficiently signaled through the sharing of information between channels. Through this efficient signaling, coding efficiency and subjective image quality can be improved.

- In particular, when the spatial changes of pixel values in a block are very large or very sharp, the degree of image energy concentration on low frequencies may not be large even if the transform is applied to the target block. Furthermore, when low-frequency signal components are largely maintained and high-frequency signal components are eliminated by applying transform and quantization processing to such blocks, or when strong quantization is applied to such blocks, severe degradation of image quality may occur. In an embodiment, whether the transform for a block is to be skipped may be indicated economically based on the determination of the encoding apparatus 1600 without incurring a large overhead. With this saving indication, the compression ratio of the image can be improved, and the degradation of the image quality can be minimized.

- When a cross-channel prediction technique using correlation between channels is used, multiple pieces of encoding decision information may not be used separately for multiple channels. In an embodiment, the encoding decision information may be sent for one channel, and the encoding decision information sent for the one channel may be shared and used with all the remaining channels or some channels selected from the remaining channels. By this sharing, the problems of degradation of compression rate and image quality can be solved.

Determining representative channels from color space

As color spaces for image encoding and decoding, there are YCbCr and YUV spaces for encoding and decoding of general images, and further, RGB, XYZ, and YCoCg spaces exist. When one of the various color spaces is the target color space for encoding and decoding of the image, one of the channels of the target color space may be determined as a representative channel of the target color space.

In an embodiment, the color channel of the channels having the highest correlation with the luminance signal may be determined as the representative channel. For example, in the RGB color space, the G channel may have the highest correlation with the luminance signal, and thus the G channel may be selected as the representative channel. In the XYZ color space, the Y channel may have the highest correlation with the luminance signal, and thus the Y channel may be selected as the representative channel. In the YCoCg color space, the Y channel may have the highest correlation with the luminance signal, and thus the Y channel may be selected as the representative channel.

Channels in the color space may be represented by index values such as "0/1/2". SelectedCIDX can be the index of the selected color. Optionally, SelectedCIDX may be an index value indicating the selected representative channel. The representative channel may be determined by an index SelectedCIDX indicating the selected representative channel among the pieces of information about the target block in the bitstream.

For example, in the YCbCr color space, the value of SelectedCIDX may be 0 indicating the Y channel.

For example, in the YCbCr color space, the Cb channel can be determined as the representative channel. When the Cb channel is determined to be the representative channel, the value of SelectedCIDX may be 1 indicating the Cb channel.

For example, in the YUV color space, the U channel may be determined as the representative channel. When the U channel is determined to be the representative channel, the value of SelectedCIDX may be 1 indicating the U channel.

For the sharing of coding decision information between channels, a specific channel in the color space can be selected as a representative channel. In encoding and decoding of images, encoding decision information for a representative channel may be shared among one or more of the remaining channels.

For example, the encoding apparatus 1600 may signal only the encoding decision information of the representative channel to the decoding apparatus 1700 through the bitstream. Alternatively, the decoding apparatus 1700 may use the bitstream to derive coding decision information for the representative channel. The encoding decision information for at least some of the remaining channels may not be separately signaled. The decoding apparatus 1700 may use the encoding decision information for the representative channel to derive encoding decision information for at least some of the remaining channels. In other words, the encoding decision information for the representative channel may be shared with at least some of the remaining channels.

For example, when the Y channel with the highest correlation with the luma signal is selected as the representative channel in the YCbCr color space, there may be differences between the luma channel (ie, Y) and the chroma channel (ie, Cb and/or Cr) There is a correlation. Therefore, when prediction is performed for image compression, encoding decision information for the luma channel as a representative channel can be implicitly shared as pieces of encoding decision information for one or more chroma blocks, instead of Apply independent predictions to each of the three channels in the color space. The one or more chroma blocks may include one or more of Cb blocks and Cr blocks.

For example, when the Cb channel is selected as the representative channel in the YCbCr color space, there may be a correlation between the Cb signal and the Cr signal that make up the chrominance channel. Therefore, when prediction is performed for image compression, the encoding decision information for the Cb channel, which is the representative channel, can be implicitly shared as the encoding decision information for the Cr block, instead of applying independent prediction to the two chrominance channel.

Encoding decision information shared between channels

The encoding decision information that can be shared between channels may be information such as syntax elements, which are encoded by the encoding device 1600 and signaled to the decoding device 1700 as information included in the bitstream. For example, encoding decision information may include flags, indexes, and the like. Additionally, encoding decision information may include information derived during encoding and/or decoding processes. Furthermore, encoding decision information may represent information required to encode and/or decode an image.

For example, the encoding decision information may include the size of the unit/block, the depth of the unit/block, the partition information of the unit/block, the partition structure of the unit/block, the partition flag information indicating whether the unit/block is partitioned in the form of a quadtree , partition flag information indicating whether the unit/block is partitioned in the form of a binary tree, the partition direction (horizontal or vertical direction) in the form of a binary tree, the partition form in the form of a binary tree (symmetrical or asymmetrical), whether the unit/block is partitioned in the form of a binary tree. Partition flag information to be partitioned in the form of a ternary tree, partition direction (horizontal direction or vertical direction) in the form of a ternary tree, partition form in the form of a ternary tree (symmetrical partition or asymmetrical partition), indicating whether the unit/block is in the form of a compound tree Partitioned information, combination and direction (horizontal direction or vertical direction) of partitions in composite tree, prediction scheme (intra prediction or inter prediction), intra prediction mode/direction, reference sample filtering method, prediction block filtering method, prediction block boundary filtering method, filtered filter taps, filtered filter coefficients, inter prediction mode, motion information, motion vector, reference picture index, inter prediction direction, inter prediction indicator, reference picture list, reference image, motion vector predictor, motion vector prediction candidate, motion vector candidate list, information indicating whether merge mode is used, merge candidate, merge candidate list, information indicating whether skip mode is used, type of interpolation filter, Filter taps of interpolation filter, filter coefficients of interpolation filter, size of motion vector, precision of representation of motion vector, transform type, transform size, information indicating whether primary transform is used, indicating additional (secondary) transform Information whether to be used, primary transform selection information (or primary transform index), secondary transform selection information (or secondary transform index), information indicating the presence or absence of residual signal, coding block style, coding block flag, quantization parameters, quantization matrix, information about the in-loop filter, information about whether the in-loop filter is applied, coefficients of the in-loop filter, filter taps of the in-loop filter, shape/form of the in-loop filter, indication Information whether the DF is applied, the coefficients of the DF, the taps of the DF, the strength of the DF, the shape/form of the DF, indicating whether the adaptive sample offset is applied or not information, information indicating whether adaptive sample offset is applied, value of adaptive sample offset, type of adaptive sample offset, type of adaptive sample offset, indicating adaptive loop filter Information whether it is applied, coefficients of the adaptive loop filter, taps of the adaptive loop filter, shape/form of the adaptive loop filter, binarization/debinarization method, context model, context model Determination method, context model update method, information indicating whether normal mode is executed, information indicating whether bypass mode is executed, context bin, bypass bin, transform coefficient, transform coefficient level, transform coefficient level scan method, image Display/output sequence, stripe identification information, stripe type, stripe partition information, Parallel block identification information, parallel block type information, parallel block partition information, picture type, bit depth, information on luminance signals and information on chroma signals, transform_skip_flag information, primary transform selection information, secondary transform selection information, reference samples At least one or a combination of point filter information, PDPC_flag information, rdpcm_flag information, EMT flag information, mts_flag information, mts_idx information, nsst_flag information, and nsst_idx information.

Among the pieces of encoding decision information that can be shared between channels, the primary transform selection information may be to use a combination of one or more DCT transform kernels and/or DST transform kernels related to the horizontal direction and/or the vertical direction. Transform information required to perform the transform process on the residual signal. For example, the primary transformation selection information may be information required to use the MTS in the primary transformation. The primary transform selection information may include mts_flag information and mts_idx information.

The primary transform selection information applied to the target block may be explicitly signaled, or alternatively, may be implicitly derived by the encoding apparatus 1600 and decoding apparatus 1700 using the encoding decision information for the target block and the encoding decision information for neighboring blocks out.

After the primary transform is completed in the encoding apparatus 1600, secondary transform may be performed in order to improve the energy concentration of the transform coefficients.

The secondary transform selection information applied to the target block may be explicitly signaled, or alternatively, may be implicitly signaled by the encoding apparatus 1600 and the decoding apparatus 1700 using the encoding decision information of the target block and the encoding decision information of neighboring blocks. Deduced. The decoding apparatus 1700 may perform the second inverse transformation according to whether the second inverse transformation is to be performed, and may perform the primary inverse transformation on the result of performing the second inverse transformation according to whether the first inverse transformation is to be performed.

The encoding apparatus 1600 may generate rdpcm_flag information for the target block, and may record the rdpcm_flag information in the bitstream. The decoding apparatus 1700 may acquire rdpcm_flag information through a bitstream, and may perform RDPCM according to the information indicated by the rdpcm_flag information, or may not perform RDPCM.

18 is a flowchart of a method for decoding encoding decision information, according to an embodiment.

According to an embodiment, when a specific channel in the color space is selected as the representative channel in order to share information among the plurality of channels, the encoding decision information of the representative channel of the target block can be obtained from the plurality of channels other than the representative channel One or more of the remaining channels are shared.

For example, when performing intra prediction for the target block in the YCbCr color space, the Y channel may be set as the representative channel, and thereafter, the intra coding decision information of the representative channel may be shared and used to perform in addition to the representative channel Instead of sending multiple pieces of intra-coding decision information for each of the three channels in the color space independently, the decoding of channels other than the sex channel (ie, the Cb channel and/or the Cr channel). Optionally, after the Cb channel is assumed to be the representative channel in the YCbCr color space, the intra-coding decision information of the representative channel can be implicitly used for decoding the Cr channel, which is in addition to the representative channel. channel outside the sexual channel.

For example, intra-coding decision information of a representative channel that can be shared by the remaining channels may include intra-prediction mode, intra-prediction direction, prediction block boundary filtering method, filter taps for prediction block boundary filtering, prediction block boundary filtering Filtered filter coefficients, transform_skip_flag information, primary transform selection information, secondary transform selection information, mts_flag information, mts_idx information, PDPC_flag information, rdpcm_flag information, EMT flag information, nsst_flag information, nsst_idx information, intra frame smoothing filter information, CU skip One or more of pass flag information, CU_lic flag information, obmc_flag information, codeAlfCtuEnable_flag information, and PDPC_flag information.

The case where cross-channel prediction is selected from among various techniques for predicting chrominance signals (including angle prediction, DC prediction, planar prediction, etc.), or the case where cross-channel prediction is more favorable, may represent properties of the luma signal (ie, the Y signal) Very similar properties to chrominance signals (ie, Cb and/or Cr signals).

In this case, when the channel of the Y signal block is the representative channel, the coding decision information determined in the process of encoding and/or decoding the representative channel can be applied equally to the chroma block. With this application, the number of bits required to transmit coding decision information can be reduced. Thus, encoding and decoding can be performed such that a single piece of encoding decision information is used for multiple channels via cross-channel prediction.

For example, after encoding decision information for the luma channel has been determined, the determined encoding decision information may be shared among the remaining channels, and encoding may be performed based on the shared encoding decision information. Alternatively, pieces of encoding decision information may be applied independently to the three channels rather than shared between channels, and the three channels may be encoded and/or decoded independently. The encoding apparatus 1600 may determine a method that is advantageous from the viewpoint of rate-distortion as an encoding method in a method for sharing encoding decision information between channels and a method for encoding channels independently. Based on the determination, the encoding apparatus 1600 may explicitly write information on whether encoding decision information is shared between channels into the bitstream, and this information may be sent to the decoding apparatus 1700 through the bitstream.

For example, when certain encoding conditions are met in the encoding and/or decoding process, information on whether encoding decision information is shared between channels may not be explicitly signaled, and encoding decision information for a representative channel may be shared by the remaining channels.

For example, the specific coding condition may be a condition indicating whether cross-component prediction (CCP), DM or cross-component linear model (CCLM) is used.

For example, when the remaining chrominance signals (ie, the Cb signal and/or the Cr signal) are predicted using at least one of the original signal, the reconstructed signal, the residual signal, and the predicted signal of the luma signal that has been encoded or decoded, Sharing of coding decision information between channels may be applied.

For example, when a luma signal is predicted using at least one of an original signal, a reconstructed signal, a residual signal, and a prediction signal of a chroma signal that has been encoded or decoded, sharing of encoding decision information between channels may be applied.

For example, when a Cr signal is predicted using at least one of an original signal, a reconstructed signal, a residual signal, and a prediction signal of the Cb signal that has completed encoding or decoding, sharing of encoding decision information between channels may be applied.

For example, when the Cb signal is predicted using at least one of the original, reconstructed, residual and predicted signals of the Cr signal that has been encoded or decoded, sharing of coding decision information between channels may be applied.

When the luma channel is set as the representative channel in the YCbCr color space, the encoding decision information may be signaled only for the luma signal, and may not be signaled separately for the rest of the chroma channels, rather than to all channels other than the luma channel. The remaining channels signal encoding decision information. By such selective transmission, the compressibility ratio can be improved.

Also, the encoding decision information can be transmitted only for the Cb signal, and the transmitted encoding decision information can be shared for the Cr signal, so the compression rate can be improved. Alternatively, the encoding decision information may be transmitted only for the Cr signal, and the transmitted encoding decision information may be shared for the Cb signal, and thus, the compression rate may be improved.

At step 1810, the communication unit 1720 may receive the bitstream. The bitstream may include coding decision information.

At step 1820, the processing unit 1710 may determine whether the sharing of encoding decision information is to be used for the target channel of the target block.

When the encoding decision information is not shared, step 1830 may be performed.

When the encoding decision information is to be shared, step 1840 may be performed.

In step 1830, the processing unit 1710 may obtain coding decision information of the target channel from the bitstream. The processing unit 1710 may parse and read the encoding decision information of the target channel from the bitstream.

In step 1840, the processing unit 1710 may set the encoding decision information such that the encoding decision information of the representative channel is used as the encoding decision information of the target channel.

Step 1820, Step 1830 and Step 1840 can be represented by the following code 1:

[Code 1]

cIdx may indicate the target channel of the target block. For example, when the number of channels in the target image is 3, cIdx may be one of predefined specific values, and may be one of {0, 1, 2}.

In an embodiment, the cIdx of the representative channel may be assumed to be 0.

"cidx!=0" may indicate that the target channel is not a representative channel (eg, a luma channel). In other words, a cIdx value of "0" may indicate that the target channel is the representative channel.

In other words, at step 1820, when the target channel is not a representative channel and cross-channel prediction is used for the target block, the processing unit 1710 may determine to use sharing of coding decision information for the target channel. When the target channel is a representative channel or when cross-channel prediction is not used for the target block, the processing unit 1710 may determine that sharing of coding decision information is not used for the target channel.

Whether cross-channel prediction is used for the target block may be 1) derived based on information obtained from the bitstream and 2) implicitly derived according to whether certain conditions are met.

As described above, whether cross-channel prediction is used may be determined based on the intra prediction mode of the target block. Whether cross-channel prediction is used may be determined based on whether the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode. For example, when the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode, the processing unit 1710 may determine that cross-channel prediction is used.

As described above, whether or not cross-channel prediction is used may be determined based on whether the intra prediction mode of the chroma channel of the target block has a specific value. For example, when the intra prediction mode of the chroma channel of the target block has a specific value, the processing unit 1710 may determine that cross-channel prediction is used.

The intra prediction mode of the chroma channel of the target block may be indicated by intra_chroma_pred_mode information.

As described above, whether cross-channel prediction is used may be determined based on whether the intra prediction mode of the target channel is DM. For example, when the intra prediction mode of the target channel is DM, the processing unit 1710 may determine that cross-channel prediction is used.

At step 1830, the processing unit 1710 may obtain coding decision information for the block of the channel indicated by cIdx from the bitstream.

The processing unit 1710 may parse and read the coding decision information of the block of the channel indicated by cIdx from the bitstream.

At step 1840, the processing unit 1710 may share the encoding decision information for the representative channel as the encoding decision information for the block of the channel indicated by cIdx. In other words, the processing unit 1710 may set the encoding decision information of the representative channel as the encoding decision information of the block of the channel indicated by cIdx.

Depending on the embodiment, an operation corresponding to the condition or execution may be additionally performed before step 1820 or between steps 1820 and 1840 .

According to an embodiment, for Cb signals, coding decision information may be sent, and for Cr signals, coding decision information for Cb signals may be shared without sending coding decision information. In this case, the above code 1 can be modified to the following code 2:

[Code 2]

19 is a flowchart of a decoding method for determining whether a transform is to be skipped, according to an embodiment.

The encoding apparatus 1600 may determine whether to skip transform (eg, primary transform and/or secondary transform) according to the size of the target block.

The target block may be a transform block.

log2TrafoSize may represent the size of the target block.

For example, when the size of the target block is less than or equal to a threshold value indicating a boundary value of the block size, the encoding apparatus 1600 may skip transformation of the target block.

Log2MaxTransformSkipSize may represent a threshold value indicating the boundary value of the block size.

When the transformation of the target block is skipped, the encoding apparatus 1600 may set the value of transform_skip_flag information to 1 without performing transformation. transform_skip_flag information may be transmitted to the decoding apparatus 1700 through a bitstream.

Also, when performing transformation of the target block, the encoding apparatus 1600 may perform transformation, and may set the value of transform_skip_flag information to 0. transform_skip_flag information may be transmitted to the decoding apparatus 1700 through a bitstream.

Here, a plurality of pieces of transform_skip_flag information may be individually transmitted for channels constituting a color space of an image.

The decoding apparatus 1700 may acquire the value of transform_skip_flag information from the bitstream. In other words, the decoding apparatus 1700 may parse and read transform_skip_flag information from the bitstream.

Here, the decoding apparatus 1700 may acquire the value of the transform_skip_flag information from the bitstream only when the size of the block is less than or equal to the threshold value indicating the boundary value of the block size.

Also, the decoding apparatus 1700 may acquire pieces of transform_skip_flag information for multiple channels of an image.

The acquisition of transform_skip_flag information can be represented by the following code 3:

[Code 3]

If(log2TrafoSize<=Log2MaxTransformSkipSize)

transform_skip_flag[x0][y0][cIdx]

x0 and y0 may represent spatial coordinates indicating the location of the target block.

cIdx may indicate the target channel of the target block information.

When there are three image channels, cIdx may have one of the predefined values {0, 1, 2}. The value of the representative channel can be 0.

Code 3 can be modified to code 4 below:

[Code 4]

If((log2TbWidth<=Log2MaxTransformSkipSize_W)&&(log2TbHeight<=Log2MaxTransformSkipSize_H))

transform_skip_flag[x0][y0][cIdx]

log2TbWidth may be a value based on Equation 11 below. "width" may be the width of the target block (ie, the horizontal length of the target block).

[Equation 11]

log2TbWidth= _log2width

log2TbHeight may have a value based on Equation 12 below. "height" may be the height of the target block (ie, the vertical length of the target block).

[Equation 12]

log2TbHeight=log ₂ height

The predefined thresholds Log2MaxTransformSkipSize_W and Log2MaxTransformSkipSize_H may be equal to each other or may be different from each other. For example, the value of Log2MaxTransformSkipSize_W may be 2, and the value of Log2MaxTransformSkipSize_H may be 2.

Code 3 can be modified to code 5 below:

[Code 5]

If((log2TbWidth<=2)&&(log2TbHeight<=2))

transform_skip_flag[x0][y0][cIdx]

As described above, instead of individually signaling multiple pieces of transform_skip_flag information for multiple channels, transform_skip_flag information may be signaled only for luma (Y) signals, and transform_skip_flag information may not be individually signaled for the remaining chroma channels. Alternatively, the transform_skip_flag information may be signaled only for the Cb signal, and the transform_skip_flag information may not be separately signaled for the Cr signal, and the transform_skip_flag information signaled for the Cb signal may be shared for the Cr signal.

Next, an embodiment of sharing transform_skip_flag information will be described.

In an embodiment, the decoding apparatus 1700 may obtain transform_skip_flag information from the bitstream. This acquisition can be represented by the following code 6:

[Code 6]

x0 and y0 may be spatial coordinates indicating the location of the target block.

cIdx may indicate the target channel of the target block.

In code 6 and other codes including the condition "if(log2TrafoSize<=Log2MaxTransformSkipSize)", the condition "if((log2TbWidth<=Log2MaxTransformSkipSize_W)&&(log2TbHeight<=Log2MaxTransformSkipSize_H))" or the condition "if((log2TbWidth<=2 )&&(log2TbHeight<=2))" instead of the condition "if(log2TrafoSize<=Log2MaxTransformSkipSize)".

When the number of channels of an image is 3, cIdx may have one of the predefined values {0, 1, 2}. The value of the representative channel can be 0. Alternatively, the value of the representative channel may be 1 or 2.

At step 1910, the communication unit 1720 may receive the bitstream.

At step 1920, the processing unit 1710 may determine whether it is possible to skip the transform for the target block.

If it is determined that the transformation may be skipped, step 1930 may be performed.

If it is determined that it is not possible to skip the transformation, step 1960 may be performed.

For example, when the size of the target block is less than or equal to a certain size, the processing unit 1710 may determine that it is not possible to skip the transform.

For example, when the size of the target block is larger than a certain size, the processing unit 1710 may determine that it is not possible to skip the transform.

Here, the specific size may be a boundary value of a block size that allows skipping of transforms.

For example, the processing unit 1710 may determine that it is not possible to skip the transformation when the conditions in Code 7 below are satisfied (ie, when the results of the conditions in Code 7 below are true), and when the conditions in Code 7 below are not satisfied ( That is, when the result of the condition in Code 7 is false), processing unit 1710 may determine that the transform may be skipped.

[Code 7]

if(log2TrafoSize<=Log2MaxTransformSkipSize)

At step 1930, the processing unit 1710 may determine whether sharing of transform_skip_flag information with the target channel of the target block is to be used.

If it is determined that transform_skip_flag information will not be shared, step 1940 may be performed.

If it is determined that transform_skip_flag information is to be shared, step 1950 may be performed.

For example, the processing unit 1710 may determine that transform_skip_flag information will be shared when the conditions in Code 8 below are satisfied (ie, when the result of the conditions in Code 8 is true), and when the conditions in Code 8 below are not satisfied (ie, when the results of the conditions in Code 8 below are true) , when the result of the condition in code 8 is false), processing unit 1710 may determine that transform_skip_flag information will not be shared.

[Code 8]

if((cIdx != 0) && (cross-channel prediction is used))

In other words, at step 1930, when the target channel is not a representative channel and cross-channel prediction is used for the target block, the processing unit 1710 may determine that transform_skip_flag information is to be shared with the target channel. Conversely, when the target channel is a representative channel or when cross-channel prediction is not used for the target block, the processing unit 1710 may determine that transform_skip_flag information will not be shared.

In step 1940, the processing unit 1710 may obtain the transform_skip_flag information of the target channel from the bitstream. The processing unit 1710 may parse and read the transform_skip_flag information of the target channel from the bitstream.

transform_skip_flag information may be stored in transform_skip_flag[x0][y0][cIdx].

In step 1950, the processing unit 1710 may set the transform_skip_flag information such that the transform_skip_flag information of the representative channel is used as the transform_skip_flag information of the target channel.

The processing unit 1710 may use the transform_skip_flag information of the representative channel as the transform_skip_flag information of the target channel without parsing and reading the transform_skip_flag information of the target channel from the bitstream. In other words, the processing unit 1710 may store the value of transform_skip_flag[x0][y0][0] in transform_skip_flag[x0][y0][cIdx].

That is, in the case where the process for parsing and reading the transform_skip_flag information of the target channel from the bitstream is not required, the value previously stored in transform_skip_flag[x0][y0][0] can likewise be stored in transform_skip_flag[x0 ][y0][cIdx] is used.

For example, the transform_skip_flag information signaled for the luma (Y) channel as the representative channel may even be used for the chroma channels (Cb and/or Cr).

According to an embodiment, for the Cb signal, transform_skip_flag information may be transmitted, and for the Cr signal, the transform_skip_flag information for the Cb signal may be shared without transmitting the transform_skip_flag information. In this case, the above code 6 can be modified to the following code 9:

[Code 9]

When it is not possible to skip the transform for the target block, step 1960 may be performed.

At step 1960, information indicating that the transform will not be skipped for the target block may be set. The value of transform_skip_flag[x0][y0][cIdx] may be set to 0 because it is not allowed to skip the transform for the target block.

20 is a flowchart of a decoding method for determining whether a transform is to be skipped according to an intra mode, according to an embodiment.

There may be considerable correlation between the luminance channel (ie, Y) and the chrominance channel (ie, Cb and/or Cr) of an image. For example, the luma channel may contain a lot of information about the texture of the image, and the Cb channel and the Cr channel, which are chroma channels, may additionally provide color information to be added to the texture.

Therefore, when performing prediction required for compression and reconstruction of an image, the prediction values for the Cb block and the Cr block can be calculated without performing independent prediction for each of the three channels of the color space, which is based on the previously obtained through decoding The signal of the luma channel performs prediction for the Cb and Cr blocks.

The technique used to calculate these predicted values may be referred to as "Cross-Channel Prediction (CCP)" or "CCLM" above.

The decoding apparatus 1700 may determine whether cross-channel prediction has been used by checking whether the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode.

Such cross-channel prediction may be effective since a substantial portion of the texture information of the chroma signal is also included in the luma signal. Similarly, a prediction value for a Cr block targeted for prediction may be calculated from the signal of the Cb channel using cross-channel prediction.

The case where cross-channel prediction is selected for prediction of chrominance signals, or the case where cross-channel prediction is favorable, from various techniques including angle prediction, DC prediction, and planar prediction may indicate that the signal characteristics of the channel corresponding to SelectedCIDX are very similar to that of another Signal characteristics of the channel.

Therefore, when it is advantageous to skip (or perform) the transform for the block for the channel corresponding to SelectedCIDX, it may be equally advantageous to also skip (or perform) the transform for the blocks of the remaining channels.

Therefore, when cross-channel prediction is used, the parsing of the bitstream may not be performed separately for the three channels in order to obtain transform_skip_flag information. When the transform_skip_flag information of the representative channel is parsed out, the transform_skip_flag information of the remaining channels may not be parsed separately. The transform_skip_flag information of the representative channel may be shared and used as the transform_skip_flag information of the remaining channels, and information indicating such sharing may be recorded in the bitstream. For example, for such sharing, the channel corresponding to SelectedCIDX may be used to determine whether the transform is to be skipped.

Alternatively, rate-distortion values may be calculated for cases where transforms are skipped equally for three channels, and rate-distortion values may be calculated for cases where transforms are performed equally for three channels. The rate-distortion value calculated when the transform is skipped and the rate-distortion value calculated when the transform is performed may be compared with each other, and one of the scheme for skipping the transform and the scheme for performing the transform may be used based on the result of the comparison. between the more advantageous schemes to perform the encoding of the channel.

In an embodiment, instead of signaling multiple pieces of transform_skip_flag information for multiple lanes, transform_skip_flag information may be signaled only for lanes corresponding to SelectedCIDX, and transform_skip_flag information may not be separately signaled for the remaining lanes.

Next, an embodiment in which such transform_skip_flag information is shared will be described.

In an embodiment, the decoding apparatus 1700 may obtain transform_skip_flag information from the bitstream. Such acquisition can be represented by the following code 10:

[Code 10]

x0 and y0 may be spatial coordinates indicating the location of the target block.

cIdx may indicate the target channel of the target block.

When the number of channels in the image is 3, the value of cIdx in code 10 may be one of the values {0, 1, 2}. For example, the value of cIdx may be one of the predefined values {0,1,2}.

At step 2010, the communication unit 1720 may receive the bitstream.

At step 2020, the processing unit 1710 may determine whether it is possible to skip the transform for the target block.

When it is possible to skip the transformation, step 2030 may be performed.

When it is not possible to skip the transformation, step 2060 may be performed.

For example, when the size of the target block is less than or equal to a certain size, the processing unit 1710 may determine that it is not possible to skip the transform.

For example, when the size of the target block is larger than a certain size, the processing unit 1710 may determine that it is not possible to skip the transform.

Here, the specific size may be a boundary value of a block size that allows skipping of transforms.

For example, the processing unit 1710 may determine that it is not possible to skip the transformation when the conditions in Code 7 below are satisfied (ie, when the results of the conditions in Code 7 below are true), and when the conditions in Code 7 below are not satisfied ( That is, when the result of the condition in Code 7 is false), processing unit 1710 may determine that the transform may be skipped.

[Code 11]

if(log2TrafoSize<=Log2MaxTransformSkipSize)

At step 2030, the processing unit 1710 may determine whether to use sharing of transform_skip_flag information with the target channel of the target block based on the selected representative channel.

If it is determined that the transform_skip_flag information will not be shared, step 2040 may be performed.

If it is determined that transform_skip_flag information is to be shared, step 2050 may be performed.

For example, the processing unit 1710 may determine that transform_skip_flag information will be shared when the conditions in Code 8 below are satisfied (ie, when the result of the conditions in Code 8 is true), and when the conditions in Code 8 below are not satisfied (ie, when the results of the conditions in Code 8 below are true) , when the result of the condition in code 8 is false), processing unit 1710 may determine that transform_skip_flag information will not be shared.

[Code 12]

if((cIdx !=SelectedCIDX) && "Cross-channel prediction is used")

In other words, in step 2030, when the target channel is not the selected representative channel indicated by SelectedCIDX and cross-channel prediction is used for the target block, the processing unit 1710 may determine to share transform_skip_flag information with the target channel. Furthermore, when the target channel is the selected representative channel indicated by SelectedCIDX or when cross-channel prediction is not used for the target block, the processing unit 1710 may determine that transform_skip_flag information is not shared.

In step 2040, the processing unit 1710 may obtain the transform_skip_flag information of the target channel from the bitstream. The processing unit 1710 may parse and read the transform_skip_flag information of the target channel from the bitstream.

transform_skip_flag information may be stored in transform_skip_flag[x0][y0][cIdx].

In step 2050, the processing unit 1710 may set the transform_skip_flag information such that the transform_skip_flag information of the selected representative channel indicated by SelectedCIDX is used as the transform_skip_flag information of the target channel.

The processing unit 1710 may use the transform_skip_flag information of the selected representative channel indicated by SelectedCIDX as the transform_skip_flag information of the target channel without parsing and reading the transform_skip_flag information of the target channel from the bitstream. In other words, the processing unit 1710 may store the value of transform_skip_flag[x0][y0][SelectedCIDX] in transform_skip_flag[x0][y0][cIdx].

That is, the value previously stored in transform_skip_flag[x0][y0][SelectedCIDX] can likewise be used in transform_skip_flag[x0][y0][cIdx] without needing to be used for parsing and reading from the bitstream The process of taking the transform_skip_flag information of the target channel.

In step 2060, information indicating that the target channel for the target block will not skip transforms may be set. The value of transform_skip_flag[x0][y0][cIdx] may be set to 0 because it is not allowed to skip the transform for the target block.

In other words, for the target channel of the target block indicated by cIdx, a predefined value of 0 may be set in the transform_skip_flag information to indicate that the transform will not be skipped for the target channel, without parsing and reading from the bitstream indicating whether the skip will be skipped Transform_skip_flag information for transform.

As described above, in the above-described embodiments, the encoding decision information of the selected representative channel may be shared with all channels except the selected representative channel.

The above-described embodiments may be partially modified. In other words, the encoding decision information of the selected representative channel can be shared as the encoding decision information of another designated channel. For example, the encoding decision information may include transform_skip_flag information.

For example, when the value of SelectedCIDX is 1, a cIDX value of 1 indicates a Cb signal, and a cIDX value of 2 indicates a Cr signal, coding decision information for the Cb signal may be shared as coding decision information for the Cr signal.

In other words, the encoding decision information for the Cb signal may be transmitted from the encoding apparatus 1600 to the decoding apparatus 1700, and the encoding decision information for the Cr signal may be set using the encoding decision information for the Cb signal without separately transmitting the encoding decision information for the Cr signal Encoding decision information.

When the coding decision information for the Cb signal is shared as the coding decision information for the Cr signal, the above code 10 can be modified to the following code 13:

[Code 13]

At step 2030, the processing unit 1710 may determine that transform_skip_flag information will be shared when the conditions in Code 8 below are satisfied (ie, when the results of the conditions in Code 8 below are true), and when the conditions in Code 8 below are not satisfied (ie, when the result of the condition in Code 8 is false), processing unit 1710 may determine that transform_skip_flag information will not be shared.

[Code 14]

if((cIdx != 2)or(!"Cross-channel prediction is used"))

In other words, at step 2030, 1) when the target channel is not a channel that shares the coding decision information of the selected representative channel indicated by SelectedCIDX, or 2) when cross-channel prediction is not used for the target block, the processing unit 1710 may determine not to share transform_skip_flag information with the target channel. Furthermore, 1) when the target channel is a channel that shares encoding decision information for the selected representative channel indicated by SelectedCIDX, and 2) when cross-channel prediction is used for the target block, the processing unit 1710 may determine to share transform_skip_flag information.

The steps in Code 13 can be implemented as other steps that maintain the same meaning. For example, code 13 can be modified to code 15 as follows:

[Code 15]

Sharing of transformation selection information

21 is a flowchart of a method for sharing transform selection information, according to an embodiment.

In the above-described embodiments, the transform_skip_flag information has been described as encoding decision information to be shared. The transform_skip_flag information in the above embodiment may be replaced with another type of encoding decision information. In the following, transform selection information is described as encoding decision information to be shared.

The transform selection information may be information indicating which transform is to be used for the transform block of the target channel. The transformation selection information may include the above-mentioned primary transformation selection information and/or secondary transformation selection information.

There may be considerable correlation between the luminance channel (ie, Y) and the chrominance channel (ie, Cb and/or Cr) of an image. For example, the luma channel may contain a lot of information about the texture of the image, and the Cb channel and the Cr channel, which are chroma channels, may additionally provide color information to be added to the texture.

Therefore, when performing prediction required for compression and reconstruction of an image, the prediction values for the Cb block and the Cr block can be calculated without performing independent prediction for each of the three channels of the color space, which is based on the previously obtained through decoding The signal of the luma channel performs prediction for the Cb and Cr blocks. Such cross-channel prediction may be effective since a considerable amount of texture information of the chroma signal can be included in the luma signal.

The case where cross-channel prediction is chosen from among various techniques for predicting chroma signals (including angular prediction, DC prediction, planar prediction, etc.), or the case where cross-channel prediction is more favorable, may indicate that the signal properties of the luma channel are very similar to those of chroma Signal properties of the channel (ie, Cb and/or Cr).

Thus, while it may be advantageous to use a particular one of multiple transforms for a luma block, it may be advantageous to use the same transform for a chroma block (ie, a Cb block and/or a Cr block).

In other words, in general, the same transform can be used for luma blocks and chroma blocks. Alternatively, when a specific transform is used for luma blocks, another specific transform corresponding to the specific transform used for luma blocks may be used for chroma blocks.

Thus, when using cross-channel prediction, if a transform is determined, the determined transform can be equally used for the luma and chroma channels (ie, the three channels) without being separately signaled for the luma channel and chroma channel transformation.

Alternatively, if one transform is determined for the luma channel, a transform corresponding to the transform determined for the luma channel may be used for the chroma channel.

When the transform to be used for the channel is determined in this way, the luma channel and the chroma channel may be encoded based on the determination. For such encoding, only one of the available transforms can be selected for the luma channel, and the one transform for the luma channel is selected, so the transform for the chroma channel can be automatically determined.

Alternatively, encoding using the same transform can be performed on the three channels. With this encoding, rate-distortion values can be calculated for multiple transforms. Thereafter, through a comparison between the rate-distortion values of the applicable transforms, the transform with the most favorable rate-distortion value can be selected, and encoding can be performed according to the selected transform.

Optionally, encoding using transform sets may be performed on three channels. A transform set may include a specific transform for the luma channel and a transform for the chroma channel that corresponds to the specific transform.

With this encoding, rate-distortion values can be calculated for multiple transform sets. Thereafter, through a comparison between rate-distortion values of applicable transform sets, a transform set with the most favorable rate-distortion value can be selected, and encoding can be performed according to the selected transform set.

In an embodiment, instead of individually signaling multiple pieces of transform selection information for multiple channels, transform selection information may only be signaled for the luma signal (the luma channel), and may not be separately signaled for the remaining channels that are chroma channels Transform selection information.

In an embodiment, the decoding apparatus 1700 may obtain transform selection information from the bitstream.

At step 2110, the communication unit 1720 may receive the bitstream.

At step 2120, the processing unit 1710 may determine whether sharing of transform selection information with the target channel of the target block is to be used.

When the transform selection information is not to be shared, step 2130 may be performed.

When the transform selection information is to be shared, step 2140 may be performed.

In step 2130, the processing unit 1710 may obtain transform selection information of the target channel from the bitstream. The processing unit 1710 may parse and read the transform selection information of the target channel from the bitstream.

In step 2140, the processing unit 1710 may set the transform selection information such that the transform selection information of the representative channel is used as the transform selection information of the target channel.

Step 2120, Step 2130 and Step 2140 can be represented by the following code 16:

[Code 16]

x0 and y0 may be spatial coordinates indicating the location of the target block.

cIdx may indicate the target channel of the target block.

When the number of channels in the image is 3, the value of cIdx in code 10 may be one of the values {0, 1, 2}. For example, the value of cIdx may be one of the predefined values {0,1,2}.

The cIdx of the representative channel can be assumed to be 0.

"cIdx!=0" may indicate that the target channel is not a representative channel (eg, a luma channel). A cIdx value of 0 may indicate that the target channel is the representative channel.

In other words, at step 2120, when the target channel is not a representative channel and cross-channel prediction is used for the target block, the processing unit 1710 may determine that the sharing of transform selection information with the target channel is to be used. When the target channel is a representative channel or when cross-channel prediction is not used for the target block, the processing unit 1710 may determine that the sharing of transform selection information with the target channel will not be used.

In step 2130, the processing unit 1710 may obtain transform selection information of the target channel from the bitstream. The processing unit 1710 may parse and read the transform selection information of the target channel from the bitstream.

Transform selection information may be stored in transform selection information[x0][y0][cIdx].

In step 2140, the processing unit 1710 may set the transform selection information such that the transform selection information of the representative channel is used as the transform selection information of the target channel.

The processing unit 1710 may use the transform selection information of the representative channel as the transform selection information of the target channel without parsing and reading the transform selection information of the target channel from the bitstream. That is, the processing unit 1710 may store the value of transform selection information[x0][y0][0] in transform selection information[x0][y0][cIdx].

That is, the values previously stored in the transform selection information[x0][y0][0] can be equally For transform selection information[x0][y0][cIdx].

Determine if cross-channel prediction will be used

In the embodiments described above with reference to FIGS. 18 to 21 , it has been exemplified whether cross-channel prediction is to be used by checking whether the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode. In other words, when the intra prediction mode of the target block is one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode, cross-channel prediction may be used. When the intra prediction mode of the target block is not one of INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode, cross-channel prediction may not be used.

The above-described determination of whether cross-channel prediction will be used is merely an example, and whether cross-channel prediction will be used may be determined by one of the following codes 17 to 23. For example, cross-channel prediction may be used when the value of the condition of each of the following codes is true, and cross-channel prediction may not be used when the value of the condition of each of the following codes is false. "intra_chroma_pred_mode" may be the intra prediction mode of the chroma channel.

[Code 17]

if(intra_chroma_pred_mode==CCLM mode)

[Code 18]

if(intra_chroma_pred_mode==DM mode)

[Code 19]

if(intra_chroma_pred_mode==INTRA_CCLM mode)

[Code 20]

if(intra_chroma_pred_mode==INTRA_MMLM mode)

[Code 21]

if(intra_chroma_pred_mode==INTRA_MFLM mode)

[Code 22]

if((intra_chroma_pred_mode==INTRA_CCLM mode)||

(intra_chroma_pred_mode==INTRA_MMLM mode)||

(intra_chroma_pred_mode==INTRA_MFLM mode))

[Code 23]

if((intra_chroma_pred_mode==DM mode)||

(intra_chroma_pred_mode==INTRA_CCLM mode)||

(intra_chroma_pred_mode==INTRA_MMLM mode)||

(intra_chroma_pred_mode==INTRA_MFLM mode))

Each of CCLM mode, DM mode, INTRA_CCLM mode, INTRA_MMLM mode, and INTRA_MFLM mode described in Code 17 to Code 23 may indicate one value of intra_chroma_pred_mode presented in the first column in Table 10 and Table 11 above. Regarding the CCLM mode, the DM mode, the INTRA_CCLM mode, the INTRA_MMLM mode, and the INTRA_MFLM mode, reference may be made to the foregoing descriptions made with respect to Table 10 and Table 11 above.

The block size may additionally be considered in the schemes in Code 17 to Code 23 above when aiming to determine whether cross-channel prediction is to be used.

Determining whether cross-channel prediction is to be used may also be performed by one of Code 24 to Code 30 below. For example, cross-channel prediction may be used when the value of the condition in each of the following codes is true, and may not be used when the value of the condition in each of the following code is false.

[Code 24]

if((intra_chroma_pred_mode==CCLM mode) && block size condition)

[Code 25]

if((intra_chroma_pred_mode==DM mode) && block size condition)

[Code 26]

if((intra_chroma_pred_mode==INTRA_CCLM mode) && block size condition)

[Code 27]

if((intra_chroma_pred_mode==INTRA_MMLM mode) && block size condition)

[Code 28]

if((intra_chroma_pred_mode==INTRA_MFLM mode) && block size condition)

[Code 29]

if((intra_chroma_pred_mode==INTRA_CCLM mode)||

(intra_chroma_pred_mode==INTRA_MMLM mode)||

(intra_chroma_pred_mode==INTRA_MFLM mode) && block size condition)

[Code 30]

if((intra_chroma_pred_mode==DM(direct mode)mode)||

(intra_chroma_pred_mode==INTRA_CCLM mode)||

(intra_chroma_pred_mode==INTRA_MMLM mode)||

(intra_chroma_pred_mode==INTRA_MFLM mode) && block size condition)

The block size condition presented in Code 24 to Code 30 may be replaced by one of the following codes Code 31, Code 32, Code 33 and Code 34.

[Code 31]

((log2TbWidth<=Log2MaxSizeWidth)&&(log2TbHeight<=Log2MaxSizeHeight))

[Code 32]

((log2TbWidth>=Log2MinSizeWidth)&&(log2TbHeight>=Log2MinSizeHeight))

[Code 33]

((log2TbWidth>Log2MinSizeWidth)&&(log2TbHeight<=Log2MaxSizeHeight))

[Code 34]

((log2TbWidth>Log2MinSizeWidth)&&(log2TbHeight>Log2MaxSizeHeight))

log2TbWidth and log2TbHeight have been described above with reference to Equation 11 and Equation 12.

Log2MaxSizeWidth, Log2MaxSizeHeight, Log2MinSizeWidth and Log2MinSizeHeight may be predefined values. Log2MaxSizeWidth may be the width of the block with the largest size. Log2MaxSizeHeight can be the height of the block with the largest size. Log2MinSizeWidth can be the width of the block with the smallest size. Log2MinSizeHeight can be the height of the block with the smallest size.

For example, the value of Log2MaxSizeWidth may be 16, the value of Log2MaxSizeHeight may be 16, the value of Log2MinSizeWidth may be 4, and the value of Log2MinSizeHeight may be 4.

Alternatively, the value of Log2MaxSizeWidth may be 32, and the value of Log2MaxSizeHeight may be 32.

When using DM, the intra prediction mode of the chrominance signal may not be signaled separately. When DM is used, the intra-prediction mode signaled for the luma signal can also be used unchanged in the chroma mode.

Encoding and decoding using sharing of selective information between channels under a block partitioning structure

In general, when an image is encoded, an appropriate encoding scheme may be used individually for multiple spatial regions, taking into account the spatial characteristics in the image. For this encoding, a picture may be partitioned into CUs, and CUs generated from the partitions may be encoded separately.

To perform this encoding, the same block partition structure can be used for the luma and chroma channels.

However, the characteristics of the luminance signal and the characteristics of the chrominance signal may be different from each other. Considering the difference between characteristics, different block partition structures can be used for luma channel and chroma channel respectively, in order to achieve more efficient coding.

In the following, the case where the block partition structure of an image is the same for the luma signal and the chroma signal (or multiple channels) is referred to as "single tree block partition structure" or "single tree".

Hereinafter, the case where the block partition structure of an image is different for the luma signal and the chroma signal (or multiple channels) is referred to as "dual tree block partition structure" or "dual tree".

In an embodiment, a block of additional channels corresponding to the target block of the target channel may be specified between the luma channel and the chroma channel (or channels). A block of another channel corresponding to the target block of the target channel is called a "col-block".

In an embodiment, when the luma channel and the chroma channel (or channels) have the same block partition structure (ie, when a single tree is applied), the block of the additional channel may be specified corresponding to the target block of the target channel .

In an embodiment, when the luma channel and the chroma channel (or channels) have different block partition structures (ie, when a dual tree is applied), the block of the additional channel may be specified corresponding to the target block of the target channel .

When the luma channel and the chroma channel (or channels) have different block partition structures (ie, when dual trees are applied), the coding decision information of the corresponding block can be shared with the target block of the target channel. By sharing the encoding decision information, the target block is encoded, and thus encoding efficiency can be improved.

The encoding decision information for the corresponding block corresponding to the target block of the target channel may be parsed from the compressed bitstream.

The encoding decision information of the corresponding block corresponding to the target block of the target channel may be parsed from the compressed bitstream, and the target block of the target channel may be decoded using the encoding decision information of the corresponding block.

For example, transform_skip_flag information of the corresponding block corresponding to the target block of the target channel may be parsed, and the transform_skip_flag information of the corresponding block may be used to determine whether transform is to be skipped for the target block.

Shared information may be coding decision information shared between luma blocks and chroma blocks.

When the block partition structure of the first channel and the block partition structure of the second channel are the same as each other, if the spatial position of the first block of the first channel corresponds to the spatial position of the second block of the second channel, the first block and the second block The two blocks may correspond to each other (ie, may be co-located). In other words, corresponding blocks of different channels may be blocks in different channels with corresponding (co-located) spatial locations. The shared information of the second block corresponding to the first block may be used for encoding and/or decoding of the first block.

When the block partition structure of the chroma channel is the same as the block partition structure of the luma channel, the luma block corresponding to a specific chroma block of the chroma channel may be at a spatial position corresponding to the spatial position of the specific chroma block Luminance block. In this case, shared information of luma blocks corresponding to chroma blocks may be used for encoding and/or decoding of chroma blocks.

Figure 22 shows a single tree block partition structure.

Figure 23 shows a dual tree block partition structure.

Under the 4:2:0 color subsampling structure, the luma signal region spatially corresponding to the chroma block may occupy an area four times larger than that of the chroma block. In other words, the horizontal length (width) and vertical length (height) of the luminance signal region may be twice as large as the horizontal length and vertical length of the chrominance block.

As shown in FIG. 22 , in the corresponding image regions of the chroma channel and the luma channel, the block partition structure of the chroma channel and the block partition structure of the luma channel may be the same as each other. In other words, a single tree can be used for both chroma and luma channels.

In FIG. 22 and subsequent figures, image areas corresponding to each other are indicated by "corresponding areas".

As shown in FIG. 23 , in the corresponding image regions of the chroma channel and the luma channel, the block partition structure of the chroma channel and the block partition structure of the luma channel may be different from each other. In other words, dual trees can be used for both chroma and luma channels.

For example, in FIG. 23, the area of the luma channel spatially corresponding to one chroma block may be partitioned into eight blocks.

When the block partition structures in the corresponding image regions of the luma channel and the chroma channel are identical to each other, the block of the luma channel that spatially corresponds to a particular chroma block can be unambiguously specified.

Conversely, when the block partition structure of the luma channel is different from the block partition structure of the chroma channel, for a specified chroma block determined by partitioning according to the block partition structure of the chroma channel, the luma block corresponding to the specified chroma block May not be explicitly specified in the luma channel. As shown in FIG. 23, this ambiguity is due to the fact that the block partition structure of the chroma channel and the block partition structure of the luma channel are different from each other.

In an embodiment, a method for specifying a luma block corresponding to a chroma block will be described for a case where the block partition structure of the luma channel is different from that of the chroma channel.

For example, multiple luma blocks corresponding to chroma blocks may be specified. With this designation, one piece(s) of shared information can be acquired from one or more luma blocks corresponding to the chroma blocks as target blocks, and the chrominance can be performed using the acquired piece(s) of shared information Encoding and/or decoding of degree blocks.

Hereinafter, a method for specifying one or more blocks of the second channel corresponding to the blocks of the first channel will be described for a case where the block partition structure of the first channel is different from that of the second channel. In the following description, although the first channel will be described as a chrominance channel and the second channel will be described as a luma channel, the chrominance channel and the luma channel are only exemplary, and as described above, the first channel and the second channel will be described as a luma channel. The two channels can be different types of channels.

24 illustrates a scheme for specifying corresponding blocks based on positions in corresponding regions, according to an example.

A corresponding area indicating a luma block corresponding to a chroma block may be designated as a rectangular area. The position of the uppermost leftmost pixel in the rectangular area may be (xCb, yCb). The position of the lowermost and rightmost pixel in the rectangular area may be (xCb+cbWidth-1, yCb+cbHeight-1).

Position (xCb, yCb) may indicate the position of the luma pixel corresponding to the position of the uppermost leftmost pixel in the chroma block (ie, the chroma coded block).

cbWidth and cbHeight may be values indicating the width and height of the target block based on luma pixels, respectively.

In other words, a corresponding area indicating a luma block corresponding to a chroma block may be defined as a rectangular area in which the position of the uppermost leftmost pixel based on the position of the luma pixel is (xCb, yCb), And the rectangular area has a horizontal width cbWidth and a vertical height cbHeight.

The corresponding regions indicating the luminance blocks corresponding to the above-described chrominance blocks may be applied to the embodiments described above with reference to FIGS. 24 to 31 .

In FIG. 24, a luma block corresponding to a chroma block may be a luma block existing at a predefined position in a corresponding region of a luma channel corresponding to a chroma block in space.

In other words, a luma block that exists at a predefined position in a corresponding region of a luma channel that spatially corresponds to a chroma block may be designated as one or more luma blocks corresponding to a chroma block. Alternatively, luma blocks occupying predefined positions in corresponding regions of the luma channel spatially corresponding to the chroma blocks may be designated as one or more luma blocks corresponding to the chroma blocks.

For example, the predefined positions may be a center (CR) position, an upper left (TL) position, an upper right (TR) position, a lower left (BL) position, and a lower right (BR) position in the region of the luma channel spatially corresponding to the chroma block. )Location.

The CR position may indicate (xCb+cbWidth/2, yCb+cbHeight/2). The TL position may indicate (xCb, yCb). The TR position may indicate (xCb+cbWidth-1, yCb). The BL position may indicate (xCb, yCb+cbHeight-1). The BR position may indicate (xCb+cbWidth-1, yCb+cbHeight-1).

In an embodiment, blocks of additional channels corresponding to the target block of the target channel may be specified in a plurality of channels, such as the luma channel and the chrominance channel. Blocks of another channel corresponding to the target block of the target channel are called "corresponding blocks".

In an embodiment, in the region of the luma channel spatially corresponding to the chroma block, the luma block including the luma pixel present at the position (xCb+cbWidth/2, yCb+cbHeight/2) indicating the center (CR) may be is the corresponding block. Thus, when a target block is partitioned in a dual-tree, a particular block of another channel (eg, luma channel) of the plurality of channels corresponding to the target block of the target channel (eg, chroma channel) can be explicitly specified. In an embodiment, in the encoding and/or decoding of the chroma channels, a luma block including a luma pixel present at position (xCb+cbWidth/2, yCb+cbHeight/2) may be designated as the corresponding block. Information about the specified corresponding block may be used to encode and/or decode the target block.

For example, the predefined positions may be some of the CR positions, the TL positions, the TR positions, the BL positions, and the BR positions in the region of the luma channel that spatially corresponds to the chroma block.

For example, a luma block corresponding to a chroma block may be a block including at least one of pixels located at the following positions in the region of the luma channel spatially corresponding to the chroma block: center position, upper left position, upper right position , the lower left position and the lower right position.

For example, the luma blocks corresponding to the chroma blocks may be some of the blocks including at least one of the pixels located in the area of the luma channel spatially corresponding to the chroma blocks: center position, upper left position, upper right position, lower left position, and lower right position.

For example, a luma block corresponding to a chroma block may include a block containing at least one of the pixels located at the following positions in the region of the luma channel spatially corresponding to the chroma block: center position, upper left position, upper right position , the lower left position and the lower right position.

25 illustrates a scheme for specifying corresponding blocks based on areas in corresponding regions, according to an example.

26 illustrates another scheme for specifying corresponding blocks based on areas in corresponding regions, according to an example.

The luma block corresponding to the chroma block may indicate the luma block having the largest area in the region of the luma channel spatially corresponding to the chroma block.

Alternatively, the luma block corresponding to the chroma block may be a predefined number of luma blocks with the largest area in the region of the luma channel spatially corresponding to the chroma block.

Such a designation scheme can be attributed to the fact that there is a high probability that the feature of the luma block with the largest area or the predefined number of luma blocks with the largest area in the region of the luma channel corresponding to the chroma block will be the same as Chroma blocks have similar characteristics.

As shown in FIG. 25, the predetermined number may be two. In FIG. 25, two blocks having the largest area (ie, block 1 and block 2) may be selected from the eight luma blocks in the region of the luma channel corresponding to the chroma block.

As shown in FIG. 26, the predetermined number may be three. In FIG. 25, three blocks having the largest area (ie, block 1, block 2, and block 3) may be selected from the eight luma blocks in the region of the luma channel corresponding to the chroma block.

With this designation scheme, even if the block partition structures of the luma channel and the chroma channel are different from each other, the coding efficiency can be improved by utilizing the shared information.

FIG. 27 illustrates a scheme for specifying a corresponding block based on the form of the block in the corresponding area, according to an example.

FIG. 28 illustrates another scheme for specifying corresponding blocks based on the form of blocks in corresponding regions, according to an example.

A luma block corresponding to a chroma block may indicate a luma block having the same form as a chroma block in a region of a luma channel that spatially corresponds to the chroma block.

For example, the form of the block may include the size of the block.

According to dual-tree block partition structures, such as those shown in FIGS. 27 and 28 , the block partition structure of CUs for chroma channels may be different from the block partition structure of CUs in regions of luma channels corresponding to CUs for chroma channels . Even in this case, the region of the luma channel corresponding to the region of the target block in the CU of the chroma channel may exist as a single block.

In this case, a single luma block in the region of the luma channel corresponding to the chroma block can be accurately matched to the chroma channel. Shared information for matched luma blocks may be shared as coding decision information for chroma blocks.

29 illustrates a scheme for specifying a corresponding block based on the aspect ratio of the block in the corresponding region, according to an example.

30 illustrates another scheme for specifying a corresponding block based on an aspect ratio of a block in a corresponding region, according to an example.

The luma block corresponding to the chroma block may be a luma block having the same aspect ratio as the chroma block in the region of the luma channel spatially corresponding to the chroma block.

Alternatively, the luma block corresponding to the chroma block may be a luma block having an aspect ratio similar to that of the chroma block in the region of the luma channel spatially corresponding to the chroma block.

For example, in the area of the luma channel of FIG. 29 , luma block 2 may be selected, and in the area of the luma channel of FIG. 30 , luma block 1 may be selected.

Here, the aspect ratio of the block may be the ratio of the horizontal length to the vertical length of the corresponding block. In other words, the aspect ratio of the block may be a value obtained by dividing the horizontal length of the block by the vertical length of the block.

For example, the following Equation 13 can be used to determine whether the aspect ratios of blocks are equal to each other:

[Equation 13]

(log ₂ Width _Chroma - log ₂ Height _Chroma )==(log ₂ Width _Luma - log ₂ Height _Luma )

Width _Chroma can be the width of the chroma block. Height _Chroma can be the height of the chroma block.

Width _Luma may be the width of the luma block corresponding to the chroma block. Height _Luma may be the height of the luma block corresponding to the chroma block.

The following Equation 14 can be used to determine whether the aspect ratios are similar to each other:

[Equation 14]

|(log ₂ WidthChroma-log ₂ HeightChroma)==(log ₂ WidthLuma-log ₂ HeightLuma)|<THD

"|x|" may indicate the absolute value of x.

THD can be the threshold. For example, the value of THD may be 2.

According to Equation 13 and Equation 14, one or more luma blocks having the same aspect ratio as the chroma blocks may be designated as corresponding blocks. Alternatively, one or more luma blocks having an aspect ratio similar to that of the chroma blocks may be designated as corresponding blocks.

31 illustrates a scheme for specifying corresponding blocks based on coding attributes of blocks in corresponding regions, according to an example.

Although the block partition structure of the chroma channel and the block partition structure of the luma channel are independent of each other, if there is a luma block in the region of the luma channel spatially corresponding to the chroma block, there is the same coding decision information as the chroma block. , the shared information of the chrominance block and the shared information of the luma block may be the same as each other.

To take advantage of these properties, a luma block that has the same value as a chroma block for predefined coding decision information can be designated as a luma block corresponding to a chroma block. Alternatively, a luma block having a value similar to that of the chroma block for the predefined encoding decision information may be designated as the luma block corresponding to the chroma block.

For example, the predefined encoding decision information may be information on whether intra prediction is used, intra prediction mode, motion prediction information, motion vector, information on whether merge mode is used, derivation mode, transform selection information, and the like.

For example, intra prediction mode may be used as predefined encoding decision information. In FIG. 31, Luma Block 1, Luma Block 2, and Luma Block 3 are shown in the region of the luma channel. Luma Block 1, Luma Block 2, and Luma Block 3 may be luma blocks having the same (or similar) intra prediction mode as that of the chroma blocks. Luma block 1, luma block 2, and luma block 3 may be designated as luma blocks corresponding to chroma blocks.

According to the above specific scheme, a luma block corresponding to a chroma block may include a plurality of luma blocks. When pieces of shared information of a plurality of luma blocks are identical to each other, a problem may not occur when the pieces of shared information are used for chroma blocks. Conversely, when pieces of shared information of multiple luma blocks are different from each other, shared information to be used for encoding and/or decoding of chroma blocks may be ambiguous.

Values corresponding to most of the values of the pieces of shared information of the plurality of corresponding blocks may be used as the values of the shared information of the chroma blocks. This decision-making method can be called "majority-based shared information decision-making method". In this way, the information to be shared can be efficiently shared without additional signaling.

For example, when transform_skip_flag information is shared, a value occupying most of the values of a plurality of pieces of transform_skip_flag information of a plurality of luma blocks corresponding to a chroma block may be shared as a value of the transform_skip_flag information of the chroma block. With this sharing, encoding and/or decoding of chroma blocks may be performed.

For example, shared information is used only when the number of luma blocks corresponding to chroma blocks is only 1. Shared information can be used only when there is only one luma block satisfying the above-mentioned specific condition in the region of the luma channel spatially corresponding to the chroma block. Alternatively, coding decision information may be shared between the channels when the region of another channel that spatially corresponds to the target block of the target channel is specified by only a single block.

Referring to FIG. 27 , when a region of a luma channel spatially corresponding to a target block of a chroma channel is partitioned into only one block, the one block may be a corresponding block of the luma channel. This shared information for the corresponding block may be used to encode and/or decode chroma blocks.

When the region of the luma channel spatially corresponding to the target block of the chroma channel is not partitioned into one block, the coding decision information may not be shared without separate signaling.

32 is a flowchart of an encoding method according to an embodiment.

The encoding method and the bitstream generation method according to the present embodiment may be performed by the encoding apparatus 1600 . This embodiment may be part of a target encoding method or a video encoding method.

At step 3210, the processing unit 1610 may determine encoding decision information for the representative channel of the target block.

In step 3220, the processing unit 1610 may generate information about the target block by performing encoding on the target block using the encoding decision information of the representative channel of the target block.

At step 3230, the processing unit 1610 may generate a bitstream including information about the target block.

The information about the target block may include coding decision information of the representative channel. Also, the bitstream and the information about the target block may not include coding decision information of the target channel.

The embodiment described with reference to FIG. 32 may be combined with the further embodiments described above. Duplicate description will be omitted here

33 is a flowchart of a decoding method according to an embodiment.

The decoding method according to the present embodiment may be performed by the decoding apparatus 1600 .

At step 3310, the communication unit 1710 may receive a bitstream including information about the target block.

The information about the target block may include coding decision information of the representative channel. Also, the bitstream and the information about the target block may not include coding decision information of the target channel.

In step 3320, the processing unit 1720 may share the encoding decision information of the representative channel of the target block as the encoding decision information of the target channel of the target block.

The encoding decision information of the representative channel may be shared as the encoding decision information of the target channel.

In step 3320, the processing unit 1610 may perform decoding on the target block using the encoding decision information of the target channel.

The embodiment described with reference to FIG. 33 may be combined with the further embodiments described above. Duplicate descriptions will be omitted here.

The above-described embodiments may be performed using the same method in the encoding apparatus 1600 and the decoding apparatus 1700 .

The order of steps, operations, and processes to be applied in the embodiments may be different from each other in the encoding apparatus 1600 and the decoding apparatus 1700 . Optionally, the order of steps, operations and processes to be applied in the embodiment may be the same as each other in the encoding apparatus 1600 and the decoding apparatus 1700 .

Embodiments may be performed separately for luminance and chrominance signals. Alternatively, the embodiments may be performed equally on luminance and chrominance signals.

The form of each block to which the embodiment will be applied may be a square form or a non-square form.

Whether to apply an embodiment may be determined based on a size of at least one of a CU, a PU, a TU, and a target block. Here, a size may be defined as a minimum size and/or a maximum size that enables an embodiment to be applied to a target, and may be defined as a fixed size that enables an embodiment to be applied to a target.

Furthermore, the first embodiment can be applied to a first size, and the second embodiment can be applied to a second size. That is, the embodiments can be applied compoundly according to the size of the target. Furthermore, the embodiments may only be applied to the case where the size of the target is equal to or larger than the minimum size and smaller than or equal to the maximum size. That is, the embodiments may only be applied to the case where the size of the target falls within a specific range.

For example, the embodiments may only be applied to the case where the size of the target block is equal to or larger than 8×8. For example, the embodiments may only be applied to the case where the size of the target block is 4x4. For example, the embodiments may only be applied when the size of the target block is less than or equal to 16×16. For example, the embodiments may only be applied to the case where the size of the target block is equal to or greater than 16×16 and less than or equal to 64×64.

Whether or not to apply an embodiment may be determined according to a time layer. In order to identify the time layer to which an embodiment is to be applied, a separate identifier may be signaled. Embodiments may be selectively applied to the time horizon specified by the identifier. Here, such an identifier may indicate the lowest layer and/or the highest layer to which the embodiment is to be applied, and may also indicate a specific layer to which the embodiment is to be applied. Furthermore, the time horizon to which the embodiments will be applied may be predefined.

For example, embodiments may be applied only to the case where the temporal layer of the target image is the lowest layer. For example, embodiments may only be applied when the temporal layer identifier of the target image is equal to or greater than one. For example, the embodiments may be applied only to the case where the temporal layer of the target image is the highest layer.

The stripe type to which the embodiment will be applied can be defined. Embodiments may be selectively applied depending on the stripe type.

In the above-described embodiments, although the method has been described based on the flowchart as a series of steps or units, the present disclosure is not limited to the order of the steps, and some steps may be in a different order than the described order of steps to be performed or concurrently with other steps. Furthermore, those skilled in the art will understand that the steps shown in the flowcharts are not exclusive and may include other steps, or that one or more of the flowcharts may be deleted without departing from the scope of the present disclosure steps.

The above-described embodiments according to the present disclosure can be implemented as programs that can be executed by various computer devices, and can be recorded on a computer-readable storage medium. Computer-readable storage media may include program instructions, data files, and data structures, alone or in combination. The program instructions recorded on the storage medium may be specially designed or configured for the present disclosure, or may be known or available to those of ordinary skill in the computer software arts.

Computer-readable storage media may include information used in embodiments of the present disclosure. For example, a computer-readable storage medium may include a bitstream, and the bitstream may contain the information described above in the embodiments of the present invention.

Computer-readable storage media may include non-transitory computer-readable media.

Examples of computer-readable storage media may include all types of hardware devices specially configured to record and execute program instructions, such as magnetic media (such as hard disks, floppy disks, and magnetic tapes), optical media (such as compact disk (CD)-ROMs) and Digital Versatile Disc (DVD)), magneto-optical media such as floppy disk, ROM, RAM and flash memory. Examples of program instructions include machine code, such as code created by a compiler, and high-level language code that can be executed by a computer using an interpreter. A hardware device may be configured to operate as one or more software modules to perform the operations of the present disclosure, and vice versa.

As described above, although the present disclosure has been described based on specific details, such as detailed components and a limited number of embodiments and drawings, the specific details are provided only to facilitate understanding of the present disclosure and the present disclosure is not limited to these embodiments , those skilled in the art will practice various changes and modifications based on the above description.

Therefore, it should be understood that the spirit of the present embodiment is not limited to the above-described embodiment and that the appended claims and their equivalents and modifications thereto fall within the scope of the present disclosure.

Claims (20)

1. A decoding method, comprising:

sharing the coding decision information of the representative channel of the target block as the coding decision information of the target channel of the target block; and is

Decoding a target block using coding decision information of the target channel.

2. The decoding method of claim 1, further comprising: receiving a bitstream including information on a target block,

wherein the information on the target block includes coding decision information of the representative channel, and

wherein the information on the target block does not include coding decision information of the target channel.

3. The decoding method of claim 1, wherein the coding decision information of the representative channel is transform skip information indicating whether a transform is to be skipped.

4. The decoding method of claim 1, wherein the coding decision information of the representative channel is transform selection information indicating which transform is to be used for a transform block of the channel.

5. The decoding method of claim 1, wherein the coding decision information of the representative channel is intra-coding decision information of the representative channel.

6. The decoding method of claim 1, wherein the representative channel and the target channel are channels in a YCbCr color space.

7. The decoding method of claim 1, wherein:

the representative channel is a luminance channel, and

the target channel is a chrominance channel.

8. The decoding method of claim 1, wherein the representative channel is a color channel having a highest correlation with a luminance signal.

9. The decoding method of claim 1, wherein the representative channel is determined by an index in the bitstream indicating the selected representative channel.

10. The decoding method according to claim 1, wherein the sharing operation is performed when image properties of a plurality of channels of the target block are similar to each other.

11. The decoding method of claim 10, wherein when the intra prediction mode of the chroma channel of the target block is a direct mode, the image properties of the plurality of channels are determined to be similar to each other.

12. The decoding method of claim 1, wherein:

when cross-channel prediction is used, the sharing operation is performed, and

whether cross-channel prediction is used is derived based on information obtained from the bitstream.

13. The decoding method of claim 1, wherein:

when cross-channel prediction is used, the sharing operation is performed, and

determining whether to use cross-channel prediction based on an intra prediction mode of a target block.

14. The decoding method of claim 13, wherein when the INTRA prediction mode of the target block is one of an INTRA _ CCLM mode, an INTRA _ MMLM mode, and an INTRA _ MFLM mode, the cross-channel prediction is used.

15. The decoding method of claim 1, wherein whether the sharing operation is to be performed is determined based on a size of a target block.

16. The decoding method of claim 1, wherein the coding decision information of the representative channel of the plurality of channels of the target block is used for all of the plurality of channels.

17. An encoding method, comprising:

determining coding decision information of a representative channel of a target block; and is

Performing encoding on a target block using the encoding decision information of the representative channel,

Wherein the coding decision information of the representative channel is shared with a further channel of the target block.

18. The encoding method of claim 17, further comprising: generating a bitstream including information on the target block,

wherein the information on the target block includes coding decision information of the representative channel, and

wherein the information on the target block does not include coding decision information of the further channel.

19. The encoding method of claim 17, wherein the representative channel and the further channel are channels in a YCbCr color space.

20. A computer-readable storage medium storing a bitstream for image decoding, the bitstream comprising:

information on the target block is transmitted to the mobile station,

wherein the information on the target block includes encoding decision information of a representative channel of the target block,

wherein the coding decision information of the representative channel of the target block is used and shared as the coding decision information of the target channel of the target block, and

wherein decoding of the target block is performed using the coding decision information of the target channel.

Images