CN119404506A

CN119404506A - Method, apparatus and recording medium for image encoding/decoding

Info

Publication number: CN119404506A
Application number: CN202380046324.6A
Authority: CN
Inventors: 林雄; 权亨晋; 金东炫; 金钟昊; 林成昶; 崔振秀; 崔海哲; 韩熙智
Original assignee: Electronics and Telecommunications Research Institute ETRI; Industry Academic Cooperation Foundation of Hanbat National University
Current assignee: Electronics and Telecommunications Research Institute ETRI; Industry Academic Cooperation Foundation of Hanbat National University
Priority date: 2022-04-12
Filing date: 2023-04-12
Publication date: 2025-02-07
Also published as: KR20230146483A

Abstract

Disclosed herein is a device, method, and storage medium for performing prediction on a target block. An image decoding method may include selecting whether to use a mixed reference line, deriving a mixed reference line, generating one or more sample points on the mixed reference line, and performing prediction on the target block using the mixed reference line. The mixed reference line may be generated when two or more reference lines around the target block are mixed. Various methods may be used to derive the mixed reference line and the sample points on the mixed reference line.

Description

Method, apparatus and recording medium for image encoding/decoding

Technical Field

The present disclosure relates generally to methods, apparatuses, and storage media for image encoding/decoding. More particularly, the present disclosure relates to a method, apparatus, and storage medium for performing prediction on a target block.

The present disclosure claims the benefits of korean patent application No. 10-2022-0045428, filed on 12 at 4 months of 2022, application No. 10-2022-0130316, filed on 12 at 10 months of 2022, and application No. 10-2023-0048169, filed on 12 at 4 months of 2023, which are hereby incorporated by reference in their entirety.

Background

With the continued development of the information and communication industry, broadcast services supporting High Definition (HD) resolution have been popular throughout the world. Through this popularity, a large number of users have become accustomed to high resolution and high definition images and/or videos.

In order to meet the demands of users for high definition, a large number of institutions have accelerated the development of next-generation imaging devices. In addition to High Definition TV (HDTV) and Full High Definition (FHD) TV, user interest in UHD TV has increased, where the resolution of UHD TV is more than four times that of Full High Definition (FHD) TV. With the increasing interest thereof, image encoding/decoding techniques for images with higher resolution and higher definition are now required.

As an image compression technique, there are various techniques (such as an inter-prediction technique, an intra-prediction technique, a transform, a quantization technique, and an entropy encoding technique).

Inter prediction technology is a technology for predicting values of pixels included in a current picture using a picture before the current picture and/or a picture after the current picture. The intra prediction technique is a technique for predicting values of pixels included in a current picture using information about pixels in the current picture. The transform and quantization technique may be a technique for compressing the energy of the residual signal. Entropy coding techniques are techniques for assigning short codewords to frequently occurring values and assigning long codewords to less frequently occurring values.

By using these image compression techniques, data about an image can be efficiently compressed, transmitted, and stored.

Disclosure of Invention

Technical problem

Embodiments are directed to an apparatus, method, and storage medium for performing prediction of a target block.

Embodiments are directed to an apparatus, method, and storage medium for performing prediction using a hybrid reference line.

Technical proposal

According to one aspect, there is provided an image decoding method including selecting whether to use a hybrid reference line, deriving the hybrid reference line, generating one or more samples on the hybrid reference line, and performing prediction of a target block using the hybrid reference line.

Whether to use the hybrid reference line may be selected based on a direction of an intra prediction mode.

Whether to use the hybrid reference line may be selected based on the size of the target block.

The mixed reference line may be generated when two or more reference lines are mixed.

The two or more reference lines may be selected based on a reference cue. The sample points on the hybrid reference line may be generated based on the distance between the two or more reference lines.

The prediction may be performed based on a distance between the two or more reference lines.

According to another aspect, there is provided an image encoding method including selecting whether to use a hybrid reference line, deriving the hybrid reference line, generating one or more samples on the hybrid reference line, and performing prediction on a target block using the hybrid reference line.

The two or more reference lines may be selected based on a reference cue. The sample points on the hybrid reference line may be generated based on the distance between two or more reference lines.

According to another aspect, there is provided a computer readable storage medium for storing a bitstream for image decoding, the bitstream including encoding information about a target block, whether to be selected using a hybrid reference line, the hybrid reference line being derived, one or more samples on the hybrid reference line being generated, and prediction of the target block using the hybrid reference line being performed.

The two or more reference lines may be selected based on a reference cue.

The sample points on the hybrid reference line may be generated based on the distance between the two or more reference lines.

Advantageous effects

An apparatus, method, and storage medium for performing prediction on a target block are provided.

An apparatus, method, and storage medium for performing prediction using a hybrid reference line are provided.

Drawings

Fig. 1 is a block diagram showing a configuration of an embodiment of an encoding apparatus to which the present disclosure is applied;

Fig. 2 is a block diagram showing a configuration of an embodiment of a decoding apparatus to which the present disclosure is applied;

fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded;

Fig. 4 is a diagram showing a form of a Prediction Unit (PU) that an encoding unit (CU) can include;

fig. 5 is a diagram illustrating a form of a Transform Unit (TU) that can be included in a CU;

FIG. 6 illustrates partitioning of blocks according to an example;

fig. 7 is a diagram for explaining an embodiment of an intra prediction process;

Fig. 8 is a diagram illustrating reference samples used in an intra prediction process;

fig. 9 is a diagram for explaining an embodiment of an inter prediction process;

FIG. 10 illustrates spatial candidates according to an embodiment;

fig. 11 illustrates an order of adding motion information of spatial candidates to a merge list according to an embodiment;

FIG. 12 illustrates a transform and quantization process according to an example;

FIG. 13 illustrates a diagonal scan according to an example;

FIG. 14 illustrates a horizontal scan according to an example;

FIG. 15 illustrates a vertical scan according to an example;

Fig. 16 is a configuration diagram of an encoding apparatus according to an embodiment;

fig. 17 is a configuration diagram of a decoding apparatus according to an embodiment;

Fig. 18 is a flowchart illustrating a target block prediction method and a bitstream generation method according to an embodiment;

fig. 19 is a flowchart of a target block prediction method using a bitstream according to an embodiment;

FIG. 20 is a flow chart of a prediction method in encoding according to an embodiment;

FIG. 21 is a flow chart of a prediction method in decoding according to an embodiment;

Fig. 22 illustrates directions of intra prediction modes of a target block according to an example;

FIG. 23 illustrates N adjacent reference lines of a target block according to an example;

fig. 24 illustrates a case where a hybrid reference line is located on a first reference line according to an example;

FIG. 25 illustrates sample values at locations of samples on a horizontal line according to an example;

Fig. 26 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the first direction;

fig. 27 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the second direction;

fig. 28 illustrates mixing the positions of the samples on the reference line and the positions of the samples on the reference line when the direction of the intra prediction mode is the third direction;

fig. 29 shows mixing the positions of the samples on the reference line and the positions of the samples on the reference line when the direction of the intra prediction mode is the fourth direction;

FIG. 30 shows target blocks, reference lines, and fragments, according to an example, and

FIG. 31 illustrates a target block, a template, and a reference line according to an example.

Detailed Description

The present invention is susceptible to various modifications and alternative embodiments, and specific embodiments thereof are described in detail below with reference to the accompanying drawings. It should be understood, however, that the examples are not intended to limit the invention to the particular forms disclosed, but to include all changes, equivalents, or modifications falling within the spirit and scope of the invention.

The following exemplary embodiments will be described in detail with reference to the accompanying drawings showing specific embodiments. These embodiments are described so that those of ordinary skill in the art to which the present disclosure pertains will be readily able to practice them. It should be noted that the various embodiments are different from each other, but need not be mutually exclusive. For example, the specific shapes, structures and characteristics described herein may be implemented as other embodiments related to one embodiment without departing from the spirit and scope of the other embodiments. Further, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. Accordingly, the following detailed description is not intended to limit the scope of the disclosure, and the scope of the exemplary embodiments is defined only by the appended claims and their equivalents (as long as they are properly described).

In the drawings, like reference numerals are used to designate the same or similar functions in all respects. The shapes, sizes, etc. of components in the drawings may be exaggerated to make the description clear.

Terms such as "first" and "second" may be used to describe various components, but the components are not limited by the terms. The terms are used only to distinguish one component from another. For example, a first component may be referred to as a second component without departing from the scope of the present description. Similarly, the second component may be referred to as a first component. The term "and/or" may include a combination of a plurality of related descriptive items or any of a plurality of related descriptive items.

It will be understood that when an element is referred to as being "connected" or "coupled" to another element, the two elements can be directly connected or coupled to each other or intervening elements may be present between the two elements. On the other hand, it will be understood that when components are referred to as being "directly connected or joined," there are no intervening components between the two components.

The components described in the embodiments are shown separately to indicate different characteristic functions, but this does not mean that each component is formed of a separate piece of hardware or software. That is, for convenience of description, a plurality of components are individually arranged and included. For example, at least two of the plurality of components may be integrated into a single component. Instead, one component may be divided into a plurality of components. Embodiments in which multiple components are integrated or embodiments in which some components are separated are included in the scope of the present specification as long as they do not depart from the essence of the present specification.

The terminology used in the embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. In an embodiment, it should be understood that terms such as "comprises" or "comprising" are intended to merely indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof, and are not intended to exclude the possibility that one or more other features, numbers, steps, operations, components, parts, or combinations thereof will be present or added. That is, in the embodiment, the expression that describes a component "including" a specific component means that another component may be included within the scope of the practice of the present invention or the technical spirit of the present invention, but does not exclude the presence of components other than the specific component.

In an embodiment, the term "at least one" may mean one of one or more amounts (such as 1, 2, 3, and 4).

In phrases listing items, such as "a or B", "at least one of a and B", "at least one of a or B", "A, B and C" and "A, B or at least one of C" as used in embodiments, each of the phrases may indicate any one of the items listed in the respective phrase, or may indicate all possible combinations of the listed items.

In an embodiment, the term "plurality" may mean one of two or more numbers (such as 2, 3, and 4).

Some components of the embodiments are not necessary components for performing necessary functions, but may be optional components for improving performance only. Embodiments may be implemented using only the necessary components to implement the essence of the embodiments. For example, a structure including only necessary components (not including only optional components for improving performance) is also included within the scope of the embodiments.

The embodiments will be described in detail below with reference to the drawings so that those skilled in the art to which the embodiments pertain can easily implement the embodiments. In the following description of the embodiments, a detailed description of known functions or configurations that are considered to obscure the gist of the present specification will be omitted. In addition, the same reference numerals are used to designate the same components throughout the drawings, and repeated descriptions of the same components will be omitted.

Hereinafter, "image" may represent a single picture constituting a video, or may represent the video itself. For example, "encoding and/or decoding an image" may mean "encoding and/or decoding a video" and may also mean "encoding and/or decoding any one of a plurality of images constituting a video".

Hereinafter, the terms "video" and "moving picture" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the target image may be an encoding target image that is a target to be encoded and/or a decoding target image that is a target to be decoded. Further, the target image may be an input image input to the encoding apparatus or an input image input to the decoding apparatus. And, the target image may be a current image, i.e., a target to be currently encoded and/or decoded. For example, the terms "target image" and "current image" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the terms "image", "picture", "frame" and "screen" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the target block may be an encoding target block (i.e., a target to be encoded) and/or a decoding target block (i.e., a target to be decoded). Furthermore, the target block may be a current block, i.e., a target that is currently to be encoded and/or decoded. Here, the terms "target block" and "current block" may be used to have the same meaning and may be used interchangeably with each other. The current block may represent an encoding target block that is an encoding target during encoding and/or a decoding target block that is a decoding target during decoding. Further, the current block may be at least one of a coded block, a predicted block, a residual block, and a transformed block.

Hereinafter, the terms "block" and "unit" may be used to have the same meaning and may be used interchangeably with each other. Alternatively, "block" may represent a particular unit.

Hereinafter, the terms "region" and "fragment" are used interchangeably.

In the following embodiments, specific information, data, flags, indexes, elements, and attributes may have their respective values. A value of "0" corresponding to each of the information, data, flags, indexes, elements, and attributes may indicate false, logical false, or a first predefined value. In other words, the values "0", false, logical false and first predefined value may be used interchangeably with each other. The value "1" corresponding to each of the information, data, flags, indexes, elements, and attributes may indicate true, logical true, or a second predefined value. In other words, the values "1", true, logical true and second predefined values may be used interchangeably with each other.

When a variable such as i or j is used to indicate a row, column, or index, the value i may be an integer of 0 or an integer greater than 0, or may be an integer of 1 or an integer greater than 1. In other words, in an embodiment, each of the rows, columns, and indexes may be counted starting from 0, or may be counted starting from 1.

In an embodiment, the term "one or more" or the term "at least one" may mean the term "plurality". The terms "one or more" or the term "at least one" may be used interchangeably with "plurality.

Next, terms to be used in the embodiments will be described.

Encoder the encoder means for performing the encoding. That is, the encoder may represent an encoding device.

Decoder the decoder means for performing decoding. That is, the decoder may represent a decoding device.

Units may represent units of image encoding and decoding. The terms "unit" and "block" may be used with the same meaning and are used interchangeably with each other.

The cell may be an array of M x N samples. Each of M and N may be a positive integer. The cells may generally represent an array of samples in two dimensions.

During the encoding and decoding of the images, a "unit" may be a region generated by partitioning one image. In other words, a "cell" may be an area specified in one image. A single image may be partitioned into multiple units. Alternatively, one image may be partitioned into sub-portions, and a unit may represent each partitioned sub-portion when encoding or decoding is performed on the partitioned sub-portion.

During the encoding and decoding of the image, a predefined process may be performed on each unit according to the type of unit.

Unit types can be classified into macro units, coding Units (CUs), prediction Units (PUs), residual units, transform Units (TUs), etc. according to functions. Alternatively, the units may represent blocks, macro blocks, coding tree units, coding tree blocks, coding units, coding blocks, prediction units, prediction blocks, residual units, residual blocks, transform units, transform blocks, etc., according to the function. For example, the target unit that is the target of encoding and/or decoding may be at least one of a CU, a PU, a residual unit, and a TU.

The term "unit" may denote information including a luminance (luma) component block, a chrominance (chroma) component block corresponding to the luminance component block, and syntax elements for the respective blocks, such that the unit is designated as being distinguished from the block.

The size and shape of the cells may be implemented differently. Further, the cells may have any of a variety of sizes and shapes. In particular, the shape of the cell may include not only square, but also geometric shapes (such as rectangle, trapezoid, triangle, and pentagon) that may be represented in two dimensions (2D).

Further, the unit information may include one or more of a type of a unit, a size of a unit, a depth of a unit, an encoding order of a unit, a decoding order of a unit, and the like. For example, the type of unit may indicate one of a CU, a PU, a residual unit, and a TU.

One unit may be partitioned into sub-units, each sub-unit having a smaller size than the size of the associated unit.

Depth may represent the degree to which a cell is partitioned. Further, the depth of a cell may indicate the level at which the corresponding cell exists when the cell is represented by a tree structure.

The cell partition information may comprise a depth indicating the depth of the cell. The depth may indicate the number of times a unit is partitioned and/or the degree to which the unit is partitioned.

In a tree structure, the depth of the root node may be considered to be minimum and the depth of the leaf node to be maximum. The root node may be the highest (top) node. The leaf node may be the lowest node.

A single unit may be hierarchically partitioned into a plurality of sub-units, while the single unit has tree-structure based depth information. In other words, a unit and a sub-unit generated by partitioning the unit may correspond to a node and a sub-node of the node, respectively. Each partitioned sub-unit may have a unit depth. Since the depth indicates the number of times a unit is partitioned and/or the degree to which a unit is partitioned, the partition information of a sub-unit may include information about the size of the sub-unit.

In a tree structure, the top node may correspond to the initial node before partitioning. The top node may be referred to as the "root node". Further, the root node may have a minimum depth value. Here, the depth of the top node may be the level "0".

A node of depth level "1" may represent a unit generated when the initial unit is partitioned once. A node of depth level "2" may represent a cell that is generated when an initial cell is partitioned twice.

A leaf node of depth level "n" may represent a unit that is generated when the initial unit is partitioned n times.

A leaf node may be a bottom node that cannot be partitioned further. The depth of the leaf node may be the maximum level. For example, the predefined value for the maximum level may be 3.

QT depth may represent depth for a quadrant. BT depth may represent depth for a bi-partition. The TT depth may represent the depth for the tri-partition.

The samples, which may be the basic units constituting the block. The samples may be represented by values from 0 to 2 ^Bd -1 according to the bit depth (Bd).

The samples may be pixels or pixel values.

In the following, the terms "pixel" and "sample" may be used with the same meaning and are used interchangeably with each other.

The Coding Tree Unit (CTU) may be composed of a single luma component (Y) coding tree block and two chroma component (i.e., cb, cr) coding tree blocks associated with the luma component coding tree block. Further, the CTU may represent information including the above-described blocks and syntax elements for each block.

Each Coding Tree Unit (CTU) may be partitioned using one or more partitioning methods, such as Quadtree (QT), binary Tree (BT) and Trigeminal Tree (TT), in order to configure sub-units, such as coding units, prediction units and transform units. The quadtree may represent a quad-tree. Further, each coding tree unit may be partitioned using a multi-type tree (MTT) using one or more partitioning methods.

"CTU" may be used as a term designating a block of pixels as a processing unit in image decoding and encoding processes (as in the case of partitioning an input image).

Coding Tree Block (CTB): "CTB" may be used as a term designating any one of a Y coding tree block, a Cb coding tree block, and a Cr coding tree block.

Neighboring blocks (or neighboring blocks) may represent blocks adjacent to the target block. The neighboring blocks may represent reconstructed neighboring blocks.

Hereinafter, the terms "adjacent block" and "neighboring block" may be used to have the same meaning and may be used interchangeably with each other.

The neighboring blocks may represent reconstructed neighboring blocks.

Spatially adjacent blocks the spatially adjacent blocks may be blocks spatially adjacent to the target block. The neighboring blocks may include spatially neighboring blocks.

The target block and the spatial neighboring block may be included in the target picture.

Spatially adjacent blocks may represent blocks whose boundaries are in contact with the target block or blocks located within a predetermined distance from the target block.

The spatially neighboring blocks may represent blocks adjacent to the vertices of the target block. Here, a block adjacent to a vertex of the target block may represent a block vertically adjacent to an adjacent block horizontally adjacent to the target block or a block horizontally adjacent to an adjacent block vertically adjacent to the target block.

Time-adjacent block-the time-adjacent block may be a block that is adjacent in time to the target block. The neighboring blocks may include temporal neighboring blocks.

The temporal neighboring blocks may comprise co-located blocks (col blocks).

The col block may be a block in a co-located picture (col picture) reconstructed previously. The location of the col block in the col picture may correspond to the location of the target block in the target picture. Alternatively, the location of the col block in the col picture may be equal to the location of the target block in the target picture. The col picture may be a picture included in the reference picture list.

The temporal neighboring blocks may be blocks temporally adjacent to the spatial neighboring blocks of the target block.

Prediction mode the prediction mode may be information indicating a mode for intra prediction or a mode for inter prediction.

Prediction unit the prediction unit may be a basic unit for prediction such as inter prediction, intra prediction, inter compensation, intra compensation and motion compensation.

A single prediction unit may be divided into a plurality of partitions or sub-prediction units having smaller sizes. The plurality of partitions may also be basic units in performing prediction or compensation. The partition generated by dividing the prediction unit may also be the prediction unit.

Prediction unit partition-the prediction unit partition may be the shape into which the prediction unit is divided.

Reconstructed neighboring cells the reconstructed neighboring cells may be cells that have been decoded and reconstructed that are neighboring the target cell.

The reconstructed neighboring cells may be cells spatially or temporally adjacent to the target cell.

The reconstructed spatial neighboring units may be units comprised in the target picture that have been reconstructed by encoding and/or decoding.

The reconstructed temporal neighboring units may be units comprised in the reference image that have been reconstructed by encoding and/or decoding. The position of the reconstructed temporal neighboring unit in the reference image may be the same as the position of the target unit in the target picture or may correspond to the position of the target unit in the target picture. Furthermore, the reconstructed temporal neighboring unit may be a block neighboring a corresponding block in the reference image. Here, the position of the corresponding block in the reference image may correspond to the position of the target block in the target image. Here, the fact that the positions of the blocks correspond to each other may mean that the positions of the blocks are identical to each other, that one block is included in another block, or that one block occupies a specific position in another block.

Sprites a picture may be divided into one or more sprites. A sprite may be composed of one or more parallel block rows and one or more parallel block columns.

The sprite may be an area in the picture having a square shape or a rectangular (i.e. non-square rectangular) shape. Further, the sprite may include one or more CTUs.

A sub-picture may be a rectangular area of one or more slices in the picture.

A sprite may comprise one or more parallel blocks, one or more tiles (blocks) and/or one or more slices.

Parallel blocks-parallel blocks may be areas in a picture that have a square shape or a rectangular (i.e., non-square rectangular) shape.

The parallel block may comprise one or more CTUs.

The parallel blocks may be partitioned into one or more partitions.

Blocking-a blocking may represent one or more CTU rows in a parallel block.

The parallel blocks may be partitioned into one or more partitions. Each chunk may include one or more CTU rows.

Parallel blocks that are not partitioned into two parts may also represent partitions.

The slice may include one or more parallel blocks in the picture. Alternatively, the stripe may include one or more partitions in parallel blocks.

The sprite may comprise one or more strips that collectively cover a rectangular area of the picture. Thus, each sub-picture boundary is also always a slice boundary, and each vertical sub-picture boundary is also always a vertical parallel block boundary.

Parameter set-the parameter set may correspond to header information in the internal structure of the bitstream.

The parameter set may include at least one of a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), a Picture Parameter Set (PPS), an Adaptive Parameter Set (APS), a Decoding Parameter Set (DPS), and the like.

The information signaled by each parameter set can be applied to the picture referencing the corresponding parameter set. For example, information in a VPS may be applied to a picture referencing the VPS. The information in the SPS may be applied to pictures referencing the SPS. The information in the PPS may be applied to a picture referencing the PPS.

Each parameter set may refer to a higher parameter set. For example, PPS may refer to SPS. SPS may refer to VPS.

Furthermore, the parameter set may comprise parallel block groups, slice header information and parallel block header information. The parallel block group may be a group including a plurality of parallel blocks. Further, the meaning of "parallel block group" may be the same as that of "stripe".

Rate-distortion optimization an encoding device may use rate-distortion optimization to provide high encoding efficiency by utilizing a combination of the size of the Coding Unit (CU), the prediction mode, the size of the Prediction Unit (PU), the motion information, and the size of the Transform Unit (TU).

The rate-distortion optimization scheme may calculate the rate-distortion costs for each combination to select the optimal combination from among the combinations. The rate distortion cost may be calculated using the equation "d+λ x R". In general, the combination that minimizes the rate-distortion cost may be selected as the optimal combination under the rate-distortion optimization scheme.

D may represent distortion. D may be an average value of squares (i.e., mean square error) of differences between original transform coefficients and reconstructed transform coefficients in the transform unit.

R may represent the rate, which may represent the bit rate using the relevant context information.

- Λ represents the lagrangian multiplier. R may include not only coding parameter information (such as a prediction mode, motion information, and a coding block flag), but also bits generated due to coding of transform coefficients.

The encoding device may perform processes such as inter-prediction and/or intra-prediction, transformation, quantization, entropy coding, inverse quantization (inverse quantization) and/or inverse transformation in order to calculate accurate D and R. These processes can greatly increase the complexity of the encoding device.

Bit stream the bit stream may represent a stream of bits comprising encoded image information.

Parsing-parsing may be a decision on a value of a syntax element made by performing entropy decoding on a bitstream. Alternatively, the term "parsing" may refer to such entropy decoding itself.

The symbol may be at least one of a syntax element, an encoding parameter, and a transform coefficient of the encoding target unit and/or the decoding target unit. Furthermore, the symbol may be a target of entropy encoding or a result of entropy decoding.

Reference pictures a reference picture may be an image that is referenced by a unit in order to perform inter prediction or motion compensation. Alternatively, the reference picture may be an image including a reference unit that is referenced by the target unit in order to perform inter prediction or motion compensation.

Hereinafter, the terms "reference picture" and "reference image" may be used to have the same meaning and may be used interchangeably with each other.

Reference picture list the reference picture list may be a list comprising one or more reference pictures that are used for inter prediction or motion compensation.

Types of reference picture lists may include combined List (LC), list 0 (L0), list 1 (L1), list 2 (L2), list 3 (L3), etc.

For inter prediction, one or more reference picture lists may be used.

Inter prediction indicator-the inter prediction indicator may indicate an inter prediction direction for the target unit. Inter prediction may be one of unidirectional prediction and bidirectional prediction. Alternatively, the inter prediction indicator may represent the number of reference pictures used to generate the prediction unit of the target unit. Alternatively, the inter prediction indicator may represent the number of prediction blocks used for inter prediction or motion compensation of the target unit.

Prediction list utilization flag the prediction list utilization flag may indicate whether to use at least one reference picture in a particular reference picture list to generate a prediction unit.

The inter prediction indicator may be derived using the prediction list utilization flag. Instead, the prediction list utilization flag may be derived using an inter prediction indicator. For example, a case where the prediction list indicates "0" (as the first value) with a flag may indicate that, for the target unit, the reference pictures in the reference picture list are not used to generate the prediction block. The case where the prediction list utilization flag indicates "1" (as the second value) may indicate that for the target unit, the prediction unit is generated using the reference picture list.

Reference picture index the reference picture index may be an index indicating a particular reference picture in a reference picture list.

Picture Order Count (POC): the POC value of a picture may represent the order in which the corresponding picture is displayed.

Motion Vector (MV) the motion vector may be a 2D vector for inter prediction or motion compensation. The motion vector may represent an offset between the target image and the reference image.

MV may be represented, for example, in a form such as (MV _x,mv_y). mv _x may indicate a horizontal component and mv _y may indicate a vertical component.

Search scope the search scope may be a 2D region where a search for MVs is performed during inter prediction. For example, the size of the search range may be mxn. M and N may each be positive integers.

Motion vector candidates the motion vector candidates may be motion vectors of blocks that are prediction candidates or blocks that are prediction candidates when the motion vector is predicted.

The motion vector candidates may be included in a motion vector candidate list.

Motion vector candidate list the motion vector candidate list may be a list configured using one or more motion vector candidates.

Motion vector candidate index the motion vector candidate index may be an indicator for indicating a motion vector candidate in the motion vector candidate list. Alternatively, the motion vector candidate index may be an index of a motion vector predictor.

Motion information the motion information may be information including a reference picture list, a reference picture, a motion vector candidate index, at least one of a merge candidate and a merge index, and a motion vector, a reference picture index, and an inter prediction indicator.

Merge candidates-the merge candidate list may be a list configured using one or more merge candidates.

The merge candidates may be spatial merge candidates, temporal merge candidates, combined bi-predictive merge candidates, history-based candidates, average merge candidates based on the average of two candidates, zero merge candidates, etc. The merge candidate may include an inter prediction indicator, and may include motion information such as prediction type information, a reference picture index for each list, a motion vector, a prediction list utilization flag, and an inter prediction indicator.

Merge index-the merge index may be an indicator for indicating a merge candidate in the merge candidate list.

The merge index may indicate a reconstruction unit for deriving a merge candidate among a reconstruction unit spatially adjacent to the target unit and a reconstruction unit temporally adjacent to the target unit.

The merge index may indicate at least one of a plurality of pieces of motion information of the merge candidate.

Transform unit the transform unit may be a basic unit of residual signal encoding and/or residual signal decoding, such as transformation, inverse transformation, quantization, inverse quantization, transform coefficient encoding and transform coefficient decoding. A single transform unit may be partitioned into multiple sub-transform units having smaller sizes. Here, the transforms may include one or more of primary transforms and secondary transforms, and the inverse transforms may include one or more of primary inverse transforms and secondary inverse transforms.

Scaling may represent the process of multiplying a factor by a transform coefficient level.

As a result of scaling the transform coefficient levels, transform coefficients may be generated. Scaling may also be referred to as "dequantizing".

Quantization Parameter (QP) the quantization parameter may be a value used to generate a transform coefficient level for a transform coefficient in quantization. Alternatively, the quantization parameter may also be a value for generating the transform coefficient by scaling the transform coefficient level in inverse quantization. Alternatively, the quantization parameter may be a value mapped to a quantization step size.

Delta quantization parameter the Delta quantization parameter may represent the difference between the quantization parameter of the target unit and the predicted quantization parameter.

Scanning may represent a method of arranging the order of coefficients in a unit, block or matrix. For example, a method for arranging a 2D array in the form of a one-dimensional (1D) array may be referred to as "scanning". Alternatively, a method for arranging the 1D array in the form of a 2D array may also be referred to as "scanning" or "inverse scanning".

Transform coefficients the transform coefficients may be coefficient values generated when the encoding device performs the transform. Alternatively, the transform coefficient may be a coefficient value generated when the decoding apparatus performs at least one of entropy decoding and dequantization.

The quantized level or quantized transform coefficient level generated by applying quantization to the transform coefficients or residual signal may also be included in the meaning of the term "transform coefficient".

Quantization level the quantization level may be a value generated when the encoding device performs quantization on the transform coefficient or the residual signal. Alternatively, the level of quantization may be a value that is a target of inverse quantization when the decoding apparatus performs inverse quantization.

Quantized transform coefficient levels as a result of the transform and quantization may also be included in the meaning of quantized levels.

Non-zero transform coefficients the non-zero transform coefficients may be transform coefficients having values other than 0 or may be transform coefficient levels having values other than 0. Alternatively, the non-zero transform coefficient may be a transform coefficient whose magnitude is not 0, or may be a transform coefficient level whose magnitude is not 0.

Quantization matrix the quantization matrix may be a matrix used in a quantization process or an inverse quantization process in order to improve subjective image quality or objective image quality of an image. The quantization matrix may also be referred to as a "scaling list".

Quantization matrix coefficients the quantization matrix coefficients may be each element in the quantization matrix. The quantized matrix coefficients may also be referred to as "matrix coefficients".

Default matrix the default matrix may be a quantization matrix predefined by the encoding device and decoding device.

Non-default matrix-the non-default matrix may be a quantization matrix that is not predefined by the encoding device and decoding device. The non-default matrix may represent a quantization matrix signaled by a user from the encoding device to the decoding device.

Most Probable Mode (MPM) MPM may represent an intra-prediction mode that is used with high probability for intra-prediction for a target block.

The encoding device and the decoding device may determine one or more MPMs based on the encoding parameters associated with the target block and the attributes of the entity associated with the target block.

The encoding device and the decoding device may determine one or more MPMs based on the intra-prediction mode of the reference block. The reference block may include a plurality of reference blocks. The plurality of reference blocks may include a spatially neighboring block adjacent to the left side of the target block and a spatially neighboring block adjacent to the upper side of the target block. In other words, one or more different MPMs may be determined depending on which intra prediction modes have been used for the reference block.

One or more MPMs may be determined in the same way in both the encoding device and the decoding device. That is, the encoding device and the decoding device may share the same MPM list including one or more MPMs.

MPM list the MPM list may be a list comprising one or more MPMs. The number of one or more MPMs in the MPM list may be predefined.

MPM indicator the MPM indicator may indicate an MPM to be used for intra-prediction for a target block among one or more MPMs in the MPM list. For example, the MPM indicator may be an index for the MPM list.

Since the MPM list is determined in the same way in both the encoding device and the decoding device, there may be no need to send the MPM list itself from the encoding device to the decoding device.

The MPM indicator may be signaled from the encoding device to the decoding device. Since the MPM indicator is signaled, the decoding apparatus may determine an MPM to be used for intra prediction for the target block among MPMs in the MPM list.

MPM usage indicator may indicate whether MPM usage patterns are to be used for prediction for target blocks. The MPM usage pattern may be a pattern in which an MPM list is used to determine MPMs to be used for intra prediction for a target block.

The MPM use indicator may be signaled from the encoding device to the decoding device.

Signaling "may mean that information is sent from the encoding device to the decoding device. Alternatively, "signaling" may mean that the information is included in a bitstream or a recording medium by an encoding apparatus. The information signaled by the encoding device may be used by the decoding device.

The encoding device may generate encoded information by performing encoding of the information to be signaled. The encoded information may be transmitted from the encoding device to the decoding device. The decoding device may obtain the information by decoding the transmitted encoded information. Here, the encoding may be entropy encoding, and the decoding may be entropy decoding.

Selective signaling information may be selectively signaled. Selective signaling for information may mean that the encoding device selectively includes information in a bitstream or a recording medium (according to specific conditions). Selective signaling for information may mean that the decoding device selectively extracts information from the bitstream (according to certain conditions).

Omission of signalling for information may be omitted. The omission of signaling for information with respect to information may mean that the encoding device does not include information in the bitstream or the recording medium (according to a specific condition). Omission of signaling for information may mean that the decoding device does not extract information from the bitstream (according to certain conditions).

Statistical values variables, coding parameters, constants, etc. may have computable values. The statistical value may be a value generated by performing calculation (operation) on a value of a specified target. For example, the statistics may indicate one or more of an average, a weighted sum, a minimum, a maximum, a mode, a median, and an interpolation of values of a particular variable, a particular encoding parameter, a particular constant, and the like.

Fig. 1 is a block diagram showing a configuration of an embodiment of an encoding apparatus to which the present disclosure is applied.

The encoding apparatus 100 may be an encoder, a video encoding apparatus, or an image encoding apparatus. The video may include one or more images (pictures). The encoding apparatus 100 may sequentially encode one or more images of the video.

Referring to fig. 1, the encoding apparatus 100 includes an inter prediction unit 110, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization (inverse quantization) unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may perform encoding on the target image using intra mode and/or inter mode. In other words, the prediction mode of the target block may be one of an intra mode and an inter mode.

Hereinafter, the terms "intra mode", "intra prediction mode", "intra mode", and "intra prediction mode" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the terms "inter mode", "inter prediction mode", "inter mode", and "inter prediction mode" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the term "image" may indicate only a partial image, or may indicate a block. Further, the processing of the "image" may indicate sequential processing of a plurality of blocks.

Further, the encoding apparatus 100 may generate a bitstream including encoded information by encoding a target image, and may output and store the generated bitstream. The generated bit stream may be stored in a computer readable storage medium and may be streamed over a wired and/or wireless transmission medium.

When the intra mode is used as the prediction mode, the switcher 115 can switch to the intra mode. When the inter mode is used as the prediction mode, the switcher 115 may switch to the inter mode.

The encoding apparatus 100 may generate a prediction block of the target block. Further, after the prediction block has been generated, the encoding apparatus 100 may encode a residual block for the target block using a residual between the target block and the prediction block.

When the prediction mode is an intra mode, the intra prediction unit 120 may use pixels of a neighboring block adjacent to the target block, which is previously encoded/decoded, as a reference sample. The intra prediction unit 120 may perform spatial prediction on the target block using the reference samples, and may generate prediction samples for the target block via spatial prediction. The predicted samples may represent samples in a prediction block.

The inter prediction unit 110 may include a motion prediction unit and a motion compensation unit.

When the prediction mode is an inter mode, the motion prediction unit may search for a region in the reference image that best matches the target block in the motion prediction process, and may derive a motion vector for the target block and the found region based on the found region. Here, the motion prediction unit may use the search range as a target area for searching.

The reference picture may be stored in a reference picture buffer 190. More specifically, when encoding and/or decoding of a reference picture has been processed, the encoded and/or decoded reference picture may be stored in the reference picture buffer 190.

Since decoded pictures are stored, the reference picture buffer 190 may be a Decoded Picture Buffer (DPB).

The motion compensation unit may generate a prediction block for the target block by performing motion compensation using the motion vector. Here, the motion vector may be a two-dimensional (2D) vector for inter prediction. Further, the motion vector may indicate an offset between the target image and the reference image.

When the motion vector has a value other than an integer, the motion prediction unit and the motion compensation unit may generate the prediction block by applying an interpolation filter to a partial region of the reference image. In order to perform inter prediction or motion compensation, it may be determined which mode of a skip mode, a merge mode, an Advanced Motion Vector Prediction (AMVP) mode, and a current picture reference mode corresponds to a method for predicting and compensating for motion of a PU included in a CU based on the CU, and inter prediction or motion compensation may be performed according to the mode.

The subtractor 125 may generate a residual block, where the residual block is the difference between the target block and the prediction block. The residual block may also be referred to as a "residual signal".

The residual signal may be the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming or quantizing a difference between the original signal and the predicted signal or a signal generated by transforming and quantizing the difference. The residual block may be a residual signal for a block unit.

The transform unit 130 may generate transform coefficients by transforming the residual block, and may output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by transforming the residual block.

The transformation unit 130 may use one of a plurality of predefined transformation methods in performing the transformation.

The plurality of predefined transform methods may include Discrete Cosine Transform (DCT), discrete Sine Transform (DST), karhunen-Loeve transform (KLT), and the like.

A transform method for transforming the residual block may be determined according to at least one of the encoding parameters for the target block and/or the neighboring block. For example, the transform method may be determined based on at least one of an inter prediction mode for the PU, an intra prediction mode for the PU, a size of the TU, and a shape of the TU. Alternatively, the transform information indicating the transform method may be signaled from the encoding apparatus 100 to the decoding apparatus 200.

When the transform skip mode is used, the transform unit 130 may omit an operation of transforming the residual block.

By quantizing the transform coefficients, quantized transform coefficient levels or quantized levels may be generated. Hereinafter, in the embodiments, each of the quantized transform coefficient level and the quantized level may also be referred to as a "transform coefficient".

The quantization unit 140 may generate quantized transform coefficient levels (i.e., quantized levels or quantized coefficients) by quantizing the transform coefficients according to quantization parameters. The quantization unit 140 may output the generated quantized transform coefficient level. In this case, the quantization unit 140 may quantize the transform coefficient using a quantization matrix.

The entropy encoding unit 150 may generate a bitstream by performing entropy encoding based on probability distribution based on the values calculated by the quantization unit 140 and/or the encoding parameter values calculated in the encoding process. The entropy encoding unit 150 may output the generated bitstream.

The entropy encoding unit 150 may perform entropy encoding on information about pixels of an image and information required to decode the image. For example, information required for decoding an image may include syntax elements and the like.

When entropy coding is applied, fewer bits may be allocated to more frequently occurring symbols and more bits may be allocated to less frequently occurring symbols. Since the symbol is represented by this allocation, the size of the bit string for the target symbol to be encoded can be reduced. Accordingly, the compression performance of video coding can be improved by entropy coding.

Further, for entropy encoding, the entropy encoding unit 150 may use an encoding method such as exponential golomb, context Adaptive Variable Length Coding (CAVLC), or Context Adaptive Binary Arithmetic Coding (CABAC). For example, the entropy encoding unit 150 may perform entropy encoding using a variable length coding/coding (VLC) table. For example, entropy encoding unit 150 may derive a binarization method for the target symbol. Furthermore, entropy encoding unit 150 may derive a probability model for the target symbol/binary bit. The entropy encoding unit 150 may perform arithmetic encoding using the derived binarization method, probability model, and context model.

The entropy encoding unit 150 may transform coefficients in the form of 2D blocks into the form of 1D vectors through a transform coefficient scanning method in order to encode quantized transform coefficient levels.

The encoding parameters may be information required for encoding and/or decoding. The encoding parameters may include information encoded by the encoding device 100 and transmitted from the encoding device 100 to the decoding device, and may also include information that may be derived during encoding or decoding. For example, the information transmitted to the decoding device may include a syntax element.

The encoding parameters may include not only information (or flags or indexes) such as syntax elements encoded by the encoding device and signaled by the encoding device to the decoding device, but also information derived in the encoding or decoding process. Furthermore, the encoding parameters may include information required to encode or decode the image. For example, the encoding parameters may include at least one value, a combination or statistics of a size of a unit/block, a shape/form of a unit/block, a depth of a unit/block, partition information of a unit/block, a partition structure of a unit/block, information indicating whether a unit/block is partitioned in a quadtree structure, information indicating whether a unit/block is partitioned in a binary tree structure, a partition direction (horizontal direction or vertical direction) of a binary tree structure, a partition form (symmetric partition or asymmetric partition) of a binary tree structure, information indicating whether a unit/block is partitioned in a trigeminal tree structure, a partition direction (horizontal direction or vertical direction) of a trigeminal tree structure, a partition direction (horizontal direction or vertical direction), the partition form of the trigeminal tree structure (symmetric partition or asymmetric partition, etc.), information indicating whether units/blocks are partitioned in the multi-type tree structure, the combination of the partitions of the multi-type tree structure and the direction (horizontal direction or vertical direction, etc.), the partition form of the multi-type tree structure (symmetric partition or asymmetric partition, etc.), the partition form of the multi-type tree structure partition tree (binary tree or trigeminal tree) in the form of multi-type tree, prediction type (intra-or inter-prediction), intra-prediction mode/direction, intra-luminance prediction mode/direction, intra-chrominance prediction mode/direction, intra-partition information, inter-partition information, coding block partition flag, prediction block partition flag, transform block partition flag, intra-luminance prediction mode/direction, inter-partition information, coding block partition flag, intra-frame chrominance prediction mode/direction, intra-frame partition information, inter-frame partition information, coding block partition flag, intra-frame chrominance prediction mode/direction, reference sample filtering method, reference sample filter tap, reference sample filter coefficient, prediction block filtering method, prediction block filter tap, prediction block filter coefficient, prediction block boundary filtering method, prediction block boundary filter tap, prediction block boundary filter coefficient, inter prediction mode, motion information, motion vector difference, reference picture index, inter prediction direction, inter prediction indicator, prediction list utilization flag, reference picture list, reference picture, POC, motion vector predictor, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, information indicating whether merge mode is used, information, Merge index, merge candidate list, information indicating whether skip mode is used, type of interpolation filter, tap of interpolation filter, filter coefficient of interpolation filter, size of motion vector, accuracy of motion vector representation, type of transform, transform size, information indicating whether first transform is used, information indicating whether additional (second) transform is used, first transform selection information (or first transform index), second transform selection information (or second transform index), information indicating presence or absence of residual signal, coding block pattern, coding block flag, quantization parameter, residual signal, and method of processing a video signal, Quantization matrix, information about loop filter, information indicating whether loop filter is applied, coefficients of loop filter, taps of loop filter, shape/form of loop filter, information indicating whether deblocking filter is applied, coefficients of deblocking filter, taps of deblocking filter, deblocking filter strength, shape/form of deblocking filter, information indicating whether adaptive sample offset is applied, value of adaptive sample offset, class of adaptive sample offset, type of adaptive sample offset, information indicating whether adaptive loop filter is applied, coefficients of adaptive loop filter, taps of adaptive loop filter, Shape/form of adaptive loop filter, binarization/inverse binarization method, context model decision method, context model update method, information indicating whether normal mode is performed, information indicating whether bypass (bypass) mode is performed, significant coefficient flag, last significant coefficient flag, coding flag of coefficient group, position of last significant coefficient, information indicating whether value of coefficient is greater than 1, information indicating whether value of coefficient is greater than 2, information indicating whether value of coefficient is greater than 3, residual coefficient value information, sign information, reconstructed luminance sample point, reconstructed chrominance sample point, context binary bit, bypass binary bit, residual luminance sample point, residual chroma sample point, transform coefficient, luminance transform coefficient, chroma transform coefficient, quantized level, luminance quantized level, chroma quantized level, transform coefficient level scanning method, size of motion vector search area on decoding apparatus side, and method for decoding apparatus shape/form of motion vector search area on decoding device side, number of times of motion vector search on decoding device side, size of CTU, minimum block size, maximum block size maximum block depth, minimum block depth, image display/output order, stripe identification information, stripe type, stripe partition information, parallel block group identification information, and image display/output order, Parallel block group type, parallel block group partition information, parallel block identification information, parallel block type, parallel block partition information, picture type, bit depth, input sample bit depth, reconstructed sample bit depth, residual sample bit depth, transform coefficient bit depth, quantized rank bit depth, information on luminance signal, information on chrominance signal, color space of a target block, and color space of a residual block. In addition, the above-described encoding parameter related information may also be included in the encoding parameters. Information for calculating and/or deriving the above-mentioned encoding parameters may also be included in the encoding parameters. Information calculated or derived using the above-described encoding parameters may also be included in the encoding parameters.

In an embodiment, the encoding parameters may refer to encoding parameters of the target. The target may refer to a target of a specific process, such as a target block and a target picture. The encoding parameters of the target block may refer to encoding parameters of neighboring blocks of the target block, encoding parameters of a reference block for the target block, or encoding parameters of a reference picture for the target block. The encoding parameter of the target picture may refer to an encoding parameter of a reference picture for the target picture, or an encoding parameter of a block included in the target picture.

The first transform selection information may indicate a first transform applied to the target block.

The second transform selection information may indicate a second transform applied to the target block.

The residual signal may represent a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing a difference between the original signal and the predicted signal. The residual block may be a residual signal for the block.

Here, signaling information may mean that the encoding apparatus 100 includes entropy-encoded information generated by performing entropy encoding on a flag or an index in a bitstream, and may mean that the decoding apparatus 200 acquires information by performing entropy decoding on the entropy-encoded information extracted from the bitstream. Here, the information may include a flag, an index, and the like.

A signal may mean information to be signaled. Hereinafter, information for the image and the block may be referred to as a "signal". Further, in the following, the terms "information" and "signal" may be used to have the same meaning and may be used interchangeably with each other. For example, the specific signal may be a signal representing a specific block. The original signal may be a signal representing the target block. The prediction signal may be a signal representing a prediction block. The residual signal may be a signal representing a residual block.

The bitstream may include information based on a specific syntax. The encoding apparatus 100 may generate a bitstream including information according to a specific syntax. The decoding apparatus 200 may acquire information from the bitstream according to a specific syntax.

Since the encoding apparatus 100 performs encoding via inter prediction, the encoded target image may be used as a reference image for another image to be subsequently processed. Accordingly, the encoding apparatus 100 may reconstruct or decode the encoded target image and store the reconstructed or decoded image as a reference image in the reference picture buffer 190. For decoding, inverse quantization and inverse transformation of the encoded target image may be performed.

The quantized level may be inverse quantized by the inverse quantization unit 160 and may be inverse transformed by the inverse transformation unit 170. The inverse quantization unit 160 may generate an inverse quantized coefficient by performing inverse transform for the quantized level. The inverse transform unit 170 may generate inverse quantized and inverse transformed coefficients by performing inverse transform on the inverse quantized coefficients.

The inverse quantized and inverse transformed coefficients may be added to the prediction block by adder 175. The inverse quantized and inverse transformed coefficients and the prediction block are added, and then a reconstructed block may be generated. Here, the inversely quantized and/or inverse transformed coefficients may represent coefficients on which one or more of inverse quantization and inverse transformation are performed, and may also represent a reconstructed residual block. Here, the reconstructed block may represent a restored block or a decoded block.

The reconstructed block may be filtered by a filter unit 180. The filter unit 180 may apply one or more of a deblocking filter, a Sample Adaptive Offset (SAO) filter, an Adaptive Loop Filter (ALF), and a non-local filter (NLF) to a reconstructed sample, a reconstructed block, or a reconstructed picture. The filter unit 180 may also be referred to as a "loop filter".

The deblocking filter may remove block distortion that occurs at boundaries between blocks in the reconstructed picture. To determine whether to apply the deblocking filter, a decision may be made to be included in the block and include the number of columns or rows of pixels based on which the determination is made as to whether to apply the deblocking filter to the target block.

When a deblocking filter is applied to a target block, the applied filter may be different depending on the strength of the deblocking filter required. In other words, among different filters, a filter determined in consideration of the intensity of deblocking filtering may be applied to the target block. When the deblocking filter is applied to the target block, one or more of a long tap filter, a strong filter, a weak filter, and a gaussian filter may be applied to the target block according to the required intensity of the deblocking filter.

Further, when vertical filtering and horizontal filtering are performed on the target block, horizontal filtering and vertical filtering may be performed in parallel.

SAO may add the appropriate offset to the pixel value to compensate for the coding error. The SAO may perform a correction on the image to which deblocking is applied based on pixels, wherein the correction uses an offset of a difference between the original image and the image to which deblocking is applied. In order to perform offset correction for an image, a method for dividing pixels included in the image into a certain number of regions, determining a region to which an offset is to be applied among the divided regions, and applying the offset to the determined region may be used, and a method for applying the offset in consideration of edge information of each pixel may also be used.

The ALF may perform filtering based on a value obtained by comparing the reconstructed image with the original image. After pixels included in an image have been divided into a predetermined number of groups, filters to be applied to each group may be determined, and filtering may be differently performed for the respective groups. Information regarding whether to apply the adaptive loop filter may be signaled for each CU. Such information may be signaled for the luminance signal. The shape and filter coefficients of the ALF to be applied to each block may be different for each block. Alternatively, ALF having a fixed form may be applied to a block regardless of the characteristics of the block.

The non-local filter may perform filtering based on a reconstructed block similar to the target block. Regions similar to the target block may be selected from the reconstructed picture and filtering of the target block may be performed using statistical properties of the selected similar regions. Information about whether to apply the non-local filter may be signaled for the Coding Unit (CU). Furthermore, the shape and filter coefficients of the non-local filter to be applied to a block may be different depending on the block.

The reconstructed block or the reconstructed image filtered by the filter unit 180 may be stored as a reference picture in a reference picture buffer 190. The reconstructed block filtered by the filter unit 180 may be part of a reference picture. In other words, the reference picture may be a reconstructed picture composed of the reconstructed blocks filtered by the filter unit 180. The stored reference pictures may then be used for inter prediction or motion compensation.

Fig. 2 is a block diagram showing a configuration of an embodiment of a decoding apparatus to which the present disclosure is applied.

The decoding apparatus 200 may be a decoder, a video decoding apparatus, or an image decoding apparatus.

Referring to fig. 2, the decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization (inverse quantization) unit 220, an inverse transformation unit 230, an intra prediction unit 240, an inter prediction unit 250, a switch 245, an adder 255, a filter unit 260, and a reference picture buffer 270.

The decoding apparatus 200 may receive the bit stream output from the encoding apparatus 100. The decoding apparatus 200 may receive a bit stream stored in a computer-readable storage medium and may receive a bit stream transmitted through a wired/wireless transmission medium stream.

The decoding apparatus 200 may perform decoding on the bit stream in an intra mode and/or an inter mode. Further, the decoding apparatus 200 may generate a reconstructed image or a decoded image via decoding, and may output the reconstructed image or the decoded image.

For example, an operation of switching to the intra mode or the inter mode based on the prediction mode for decoding may be performed by the switch 245. When the prediction mode for decoding is an intra mode, the switch 245 may be operated to switch to the intra mode. When the prediction mode for decoding is an inter mode, the switch 245 may be operated to switch to the inter mode.

The decoding apparatus 200 may acquire a reconstructed residual block by decoding an input bitstream, and may generate a prediction block. When the reconstructed residual block and the prediction block are acquired, the decoding apparatus 200 may generate a reconstructed block as a target to be decoded by adding the reconstructed residual block and the prediction block.

The entropy decoding unit 210 may generate symbols by performing entropy decoding on the bitstream based on probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized transform coefficient levels (i.e., quantized levels or quantized coefficients). Here, the entropy decoding method may be similar to the entropy encoding method described above. That is, the entropy decoding method may be an inverse of the entropy encoding method described above.

The entropy decoding unit 210 may change coefficients having a one-dimensional (1D) vector form into a 2D block shape through a transform coefficient scanning method in order to decode quantized transform coefficient levels.

For example, the coefficients of a block may be changed to a 2D block shape by scanning the block coefficients using an upper right diagonal scan. Alternatively, which of the upper right diagonal scan, the vertical scan, and the horizontal scan is to be used may be determined according to the size of the corresponding block and/or the intra prediction mode.

The quantized coefficients may be dequantized by dequantization unit 220. The dequantization unit 220 may generate dequantized coefficients by performing dequantization on the quantized coefficients. Further, the inversely quantized coefficients may be inversely transformed by the inverse transformation unit 230. The inverse transform unit 230 may generate a reconstructed residual block by performing inverse transform on the inversely quantized coefficients. As a result of performing inverse quantization and inverse transformation on the quantized coefficients, a reconstructed residual block may be generated. Here, when generating the reconstructed residual block, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra prediction unit 240 may generate a prediction block by performing spatial prediction for a target block, wherein the spatial prediction uses pixel values of previously decoded neighboring blocks adjacent to the target block.

The inter prediction unit 250 may include a motion compensation unit. Alternatively, the inter prediction unit 250 may be designated as a "motion compensation unit".

When the inter mode is used, the motion compensation unit may generate a prediction block by performing motion compensation for the target block, wherein the motion compensation uses a motion vector and a reference image stored in the reference picture buffer 270.

The motion compensation unit may apply an interpolation filter to a partial region of the reference image when the motion vector has a value other than an integer, and may generate the prediction block using the reference image to which the interpolation filter is applied. In order to perform motion compensation, the motion compensation unit may determine which mode of a skip mode, a merge mode, an Advanced Motion Vector Prediction (AMVP) mode, and a current picture reference mode corresponds to a motion compensation method for a PU included in the CU, based on the CU, and may perform motion compensation according to the determined mode.

The reconstructed residual block and the prediction block may be added to each other by an adder 255. The adder 255 may generate a reconstructed block by adding the reconstructed residual block and the prediction block.

The reconstructed block may be filtered by a filter unit 260. The filter unit 260 may apply at least one of a deblocking filter, an SAO filter, an ALF, and an NLF to the reconstructed block or the reconstructed image. The reconstructed image may be a picture comprising reconstructed blocks.

The filter unit may output the reconstructed image.

The reconstructed image and/or the reconstructed block filtered by the filter unit 260 may be stored as a reference picture in a reference picture buffer 270. The reconstructed block filtered by the filter unit 260 may be part of a reference picture. In other words, the reference picture may be an image composed of the reconstructed block filtered by the filter unit 260. The stored reference pictures may then be used for inter prediction or motion compensation.

Fig. 3 is a diagram schematically showing a partition structure of an image when the image is encoded and decoded.

Fig. 3 may schematically illustrate an example in which a single unit is partitioned into a plurality of sub-units.

In order to partition an image efficiently, a Coding Unit (CU) may be used in encoding and decoding. The term "unit" may be used to collectively designate 1) a block including image samples and 2) a syntax element. For example, "partition of a unit" may represent "partition of a block corresponding to the unit".

A CU may be used as a basic unit for image encoding/decoding. A CU may be used as a unit to which one mode selected from an intra mode and an inter mode is applied in image encoding/decoding. In other words, in image encoding/decoding, it may be determined which one of the intra mode and the inter mode is to be applied to each CU.

Furthermore, a CU may be a basic unit that predicts, transforms, quantizes, inverse transforms, inverse quantizes, and encodes/decodes transform coefficients.

Referring to fig. 3, an image 300 may be sequentially partitioned into units corresponding to a Largest Coding Unit (LCU), and a partition structure may be determined for each LCU. Here, LCUs may be used to have the same meaning as Code Tree Units (CTUs).

Partitioning a unit may mean partitioning a block corresponding to the unit. The block partition information may include depth information regarding the depth of the unit. The depth information may indicate the number of times a unit is partitioned and/or the degree to which the unit is partitioned. A single unit may be hierarchically partitioned into multiple sub-units while having tree-structure-based depth information.

Each partitioned subunit may have depth information. The depth information may be information indicating the size of the CU. The depth information may be stored for each CU.

Each CU may have depth information. When a CU is partitioned, the depth of the CU generated from the partition may be increased by 1 from the depth of the partitioned CU.

The partition structure may represent a distribution of Coding Units (CUs) in LCU 310 for efficiently encoding an image. Such a distribution may be determined according to whether a single CU is to be partitioned into multiple CUs. The number of CUs generated by partitioning may be a positive integer of 2 or more, including 2,3, 4, 8, 16, and the like.

Depending on the number of CUs generated by performing partitioning, the horizontal and vertical sizes of each CU generated by performing partitioning may be smaller than those of the CU before being partitioned. For example, the horizontal and vertical sizes of each CU generated by partitioning may be half the horizontal and vertical sizes of the CU before partitioning.

Each partitioned CU may be recursively partitioned into four CUs in the same manner. The at least one of the horizontal and vertical dimensions of each partitioned CU may be reduced via recursive partitioning as compared to the at least one of the horizontal and vertical dimensions of the CU prior to partitioning.

Partitioning of the CU may be performed recursively until a predefined depth or a predefined size.

For example, the depth of a CU may have a value ranging from 0to 3. The size of a CU may range from a 64 x 64 size to an 8 x 8 size depending on the depth of the CU.

For example, the depth of LCU 310 may be 0 and the depth of the Smallest Coding Unit (SCU) may be a predefined maximum depth. Here, as described above, the LCU may be a CU having a maximum coding unit size, and the SCU may be a CU having a minimum coding unit size.

Partitioning may begin at LCU 310 and the depth of a CU may increase by 1 each time the horizontal and/or vertical size of the CU is reduced by partitioning.

For example, for each depth, a CU that is not partitioned may have a size of 2n×2n. Further, in the case where a CU is partitioned, a CU having a size of 2n×2n may be partitioned into four CUs each having a size of n×n. The value of N may be halved each time the depth increases by 1.

Referring to fig. 3, an LCU of depth 0 may have 64×64 pixels or a 64×64 block. 0 may be the minimum depth. An SCU of depth 3 may have 8 x 8 pixels or 8 x 8 blocks. 3 may be the maximum depth. Here, a CU having a block of 64×64 as an LCU may be represented by a depth of 0. A CU with 32 x 32 blocks may be represented by a depth of 1. A CU with 16 x 16 blocks may be represented by depth 2. A CU with 8 x 8 blocks as SCU may be represented by depth 3.

The information on whether the corresponding CU is partitioned may be represented by partition information of the CU. The partition information may be 1-bit information. All CUs except SCU may include partition information. For example, the value of partition information of a CU that is not partitioned may be a first value. The value of partition information of the partitioned CU may be a second value. When the partition information indicates whether the CU is partitioned, the first value may be "0" and the second value may be "1".

For example, when a single CU is partitioned into four CUs, the horizontal size and the vertical size of each of the four CUs generated by performing the partitioning may be half of the horizontal size and the vertical size of the CU before being partitioned. When a CU of size 32×32 is partitioned into four CUs, the size of each of the partitioned four CUs may be 16×16. When a single CU is partitioned into four CUs, the CUs may be considered to have been partitioned in a quadtree structure. In other words, quadtree partitioning may be considered as already applied to CUs.

For example, when a single CU is partitioned into two CUs, the horizontal size or the vertical size of each of the two CUs generated by performing the partitioning may be half of the horizontal size or the vertical size of the CU before being partitioned. When a CU of size 32×32 is vertically partitioned into two CUs, the size of each of the partitioned two CUs may be 16×32. When a CU of size 32×32 is horizontally partitioned into two CUs, the size of each of the partitioned two CUs may be 32×16. When a single CU is partitioned into two CUs, the CUs may be considered to have been partitioned in a binary tree structure. In other words, the binary tree partition may be considered to have been applied to the CU.

For example, when a single CU is partitioned (or divided) into three CUs, the original CU before being partitioned is partitioned such that its horizontal size or vertical size is divided at a ratio of 1:2:1, thus enabling three sub-CUs to be generated. For example, when a CU of size 16×32 is horizontally partitioned into three sub-CUs, the three sub-CUs generated by the partition may have sizes of 16×8, 16×16, and 16×8, respectively, in a top-to-bottom direction. For example, when a CU of size 32×32 is vertically partitioned into three sub-CUs, the three sub-CUs generated by the partition may have sizes of 8×32, 16×32, and 8×32, respectively, in the left-to-right direction. When a single CU is partitioned into three CUs, the CUs may be considered to be partitioned in a trigeminal form. In other words, the trigeminal tree partition can be considered to have been applied to the CU.

Both the quadtree partition and the binary tree partition are applied to LCU 310 of fig. 3.

In the encoding apparatus 100, a Coding Tree Unit (CTU) having a size of 64×64 may be partitioned into a plurality of smaller CUs by a recursive quadtree structure. A single CU may be partitioned into four CUs having the same size. Each CU may be recursively partitioned and may have a quadtree structure.

By recursive partitioning of the CU, an optimal partitioning method that incurs the minimum rate-distortion cost may be selected.

Coding Tree Unit (CTU) 320 in fig. 3 is an example of a CTU in which quadtree partitions, binary tree partitions, and trigeminal tree partitions are all applied.

As described above, in order to partition the CTU, at least one of the quadtree partition, the binary tree partition, and the trigeminal tree partition may be applied to the CTU. Partitioning may be applied based on a particular priority.

For example, quadtree partitions may be preferentially applied to CTUs. A CU that cannot be further partitioned in a quadtree form may correspond to a leaf node of the quadtree. The CUs corresponding to the leaf nodes of the quadtree may be root nodes of the binary tree and/or the trigeminal tree. That is, the CUs corresponding to leaf nodes of the quadtree may be partitioned in a binary tree form or a trigeminal tree form, or may not be further partitioned. In this case, each CU generated by applying the binary tree partition or the trigeminal tree partition to the CU corresponding to the leaf node of the quadtree is prevented from being partitioned again by the quadtree, thereby effectively performing the partitioning of the block and/or the operation of signaling the block partition information.

The partition information may be used to signal the partition of the CU corresponding to each node of the quadtree. The four partition information having a first value (e.g., "1") may indicate that the corresponding CU is partitioned in a quadtree form. The four partition information having a second value (e.g., "0") may indicate that the corresponding CU is not partitioned in quadtree form. The quarter-zone information may be a flag having a specific length (e.g., 1 bit).

There may be no priority between the binary tree partition and the trigeminal tree partition. That is, CUs corresponding to leaf nodes of the quadtree may be partitioned in a binary tree form or a trigeminal tree form. Furthermore, a CU generated by binary tree partitioning or trigeminal tree partitioning may or may not be further partitioned in binary tree form or trigeminal tree form.

Partitions that execute when there is no priority between a binary tree partition and a trigeminal tree partition may be referred to as "multi-type tree partitions". That is, a CU corresponding to a leaf node of a quadtree may be a root node of a multi-type tree. The partition of the CU corresponding to each node of the multi-type tree may be signaled using at least one of information indicating whether the CU is partitioned in the multi-type tree, partition direction information, and partition tree information. For the partition of the CU corresponding to each node of the multi-type tree, information indicating whether the partition of the multi-type tree is performed, partition direction information, and partition tree information may be sequentially signaled.

For example, information indicating whether a CU is partitioned in a multi-type tree and has a first value (e.g., "1") may indicate that the corresponding CU is partitioned in a multi-type tree form. The information indicating whether the CU is partitioned in the multi-type tree and has a second value (e.g., "0") may indicate that the corresponding CU is not partitioned in the multi-type tree form.

When a CU corresponding to each node of the multi-type tree is partitioned in the form of the multi-type tree, the corresponding CU may further include partition direction information.

The partition direction information may indicate a partition direction of the multi-type tree partition. Partition direction information having a first value (e.g., "1") may indicate that the corresponding CU is partitioned in a vertical direction. Partition direction information having a second value (e.g., "0") may indicate that the corresponding CU is partitioned in the horizontal direction.

When a CU corresponding to each node of the multi-type tree is partitioned in the form of the multi-type tree, the corresponding CU may further include partition tree information. The partition tree information may indicate a tree that is used for multi-type tree partitions.

For example, partition tree information having a first value (e.g., "1") may indicate that the corresponding CU is partitioned in a binary tree form. Partition tree information having a second value (e.g., "0") may indicate that the corresponding CU is partitioned in a trigeminal tree form.

Here, each of the above information indicating whether partitioning by the multi-type tree is performed, partition tree information, and partition direction information may be a flag having a specific length (e.g., 1 bit).

At least one of the above-described four-partition information, information indicating whether partitioning by multi-type tree is performed, partition direction information, and partition tree information may be entropy-encoded and/or entropy-decoded. To perform entropy encoding/decoding of such information, information of neighboring CUs adjacent to the target CU may be used.

For example, it may be considered that the partition forms (i.e., partition/non-partition, partition tree, and/or partition direction) of the left CU and/or the upper CU and the partition forms of the target CU may be similar to each other with a high probability. Thus, based on the information of the neighboring CUs, context information for entropy encoding and/or entropy decoding of the information of the target CU may be derived. Here, the information of the neighboring CU may include at least one of 1) four partition information of the neighboring CU, 2) information indicating whether the neighboring CU is partitioned in a multi-type tree, 3) partition direction information of the neighboring CU, and 4) partition tree information of the neighboring CU.

In another embodiment of the binary tree partition and the trigeminal tree partition, the binary tree partition may be preferentially executed. That is, the binary tree partition may be applied first, and then the CU corresponding to the leaf node of the binary tree may be set as the root node of the trigeminal tree. In this case, quad-tree partitioning or binary tree partitioning may not be performed on CUs corresponding to nodes of the trigeminal tree.

A CU that is not further partitioned by quadtree partitions, binary tree partitions, and/or trigeminal tree partitions may be a unit of coding, prediction, and/or transformation. That is, the CU may not be further partitioned for prediction and/or transformation. Accordingly, a partition structure for partitioning a CU into Prediction Units (PUs)/or Transform Units (TUs), partition information thereof, and the like may not exist in the bitstream.

However, when the size of the CU as a unit of partition is greater than the size of the maximum transform block, the CU may be recursively partitioned until the size of the CU becomes less than or equal to the size of the maximum transform block. For example, when the size of a CU is 64×64 and the size of the largest transform block is 32×32, the CU may be partitioned into four 32×32 blocks in order to perform the transform. For example, when the size of a CU is 32×64 and the size of the largest transform block is 32×32, the CU may be partitioned into two 32×32 blocks.

In this case, information indicating whether the CU is partitioned for transformation may not be separately signaled. Without signaling, it may be determined whether the CU is partitioned via a comparison between the horizontal size (and/or vertical size) of the CU and the horizontal size (and/or vertical size) of the largest transform block. For example, a CU may be vertically halved when the horizontal size of the CU is greater than the horizontal size of the largest transform block. Further, when the vertical size of the CU is greater than that of the largest transform block, the CU may be horizontally halved.

Information about the maximum size and/or minimum size of the CU and information about the maximum size and/or minimum size of the transform block may be signaled or determined at a level higher than the level of the CU. For example, the higher level may be a sequence level, a picture level, a parallel block group level, or a slice level. For example, the minimum size of the CU may be set to 4×4. For example, the maximum size of the transform block may be set to 64×64. For example, the maximum size of the transform block may be set to 4×4.

Information about the minimum size of a CU corresponding to a leaf node of the quadtree (i.e., the minimum size of the quadtree) and/or information about the maximum depth of a path from the root node of the multi-type tree to the leaf node (i.e., the maximum depth of the multi-type tree) may be signaled or determined at a level higher than the level of the CU. For example, the higher level may be a sequence level, a picture level, a slice level, a parallel block group level, or a parallel block level. Information about the minimum size of the quadtree and/or information about the maximum depth of the multi-type tree may be signaled or determined separately at each of the intra-stripe and inter-stripe levels.

Information about the difference between the size of the CTU and the maximum size of the transform block may be signaled or determined at a level higher than the level of the CU. For example, the higher level may be a sequence level, a picture level, a slice level, a parallel block group level, or a parallel block level. Information about the maximum size of the CU corresponding to each node of the binary tree (i.e., the maximum size of the binary tree) may be determined based on the size of the CTU and the information of the difference. The maximum size of the CU corresponding to each node of the trigeminal tree (i.e., the maximum size of the trigeminal tree) may have different values depending on the type of stripe. For example, the maximum size of the trigeminal tree of intra-stripe levels may be 32×32. For example, the maximum size of the trigeminal tree of inter-stripe levels may be 128×128. For example, the minimum size of the CU corresponding to each node of the binary tree (i.e., the minimum size of the binary tree) and/or the minimum size of the CU corresponding to each node of the trigeminal tree (i.e., the minimum size of the trigeminal tree) may be set to the minimum size of the CU.

In another example, the maximum size of the binary tree and/or the maximum size of the trigeminal tree may be signaled or determined at the stripe level. Further, the minimum size of the binary tree and/or the minimum size of the trigeminal tree may be signaled or determined at the stripe level.

Based on the various block sizes and depths described above, the four-partition information, information indicating whether partitioning by multi-type tree is performed, partition tree information, and/or partition direction information may or may not be present in the bitstream.

For example, when the size of the CU is not greater than the minimum size of the quadtree, the CU may not include the quadbregion information, and the quadbregion information of the CU may be inferred as the second value.

For example, when the size (horizontal and vertical) of a CU corresponding to each node of the multi-type tree is greater than the maximum size (horizontal and vertical) of the binary tree and/or the maximum size (horizontal and vertical) of the trigeminal tree, the CU may not be partitioned in the binary tree form and/or the trigeminal tree form. In this manner of determination, information indicating whether a partition in the multi-type tree is executed may not be signaled, but may be inferred as a second value.

Alternatively, a CU may not be partitioned in a binary tree form and/or a trigeminal tree form when the size (horizontal size and vertical size) of the CU corresponding to each node of the multi-type tree is equal to the minimum size (horizontal size and vertical size) of the binary tree, or when the size (horizontal size and vertical size) of the CU is equal to twice the minimum size (horizontal size and vertical size) of the trigeminal tree. In this manner of determination, information indicating whether a partition in the multi-type tree is executed may be not signaled, but may be inferred as a second value. The reason for this is that when a CU is partitioned in a binary tree form and/or a trigeminal tree form, a CU smaller than the minimum size of the binary tree and/or the minimum size of the trigeminal tree is generated.

Alternatively, the binary tree partition or the trigeminal tree partition may be limited based on the size of the virtual pipeline data unit (i.e., the size of the pipeline buffer). For example, a binary tree partition or a trigeminal tree partition may be limited when a CU is partitioned into sub-CUs that are unsuitable for the size of the pipeline buffer by the binary tree partition or the trigeminal tree partition. The size of the pipeline buffer may be equal to the maximum size of the transform block (e.g., 64 x 64).

For example, when the size of the pipeline buffer is 64×64, the following partitions may be limited.

Trigeminal tree partition for NxM CUs (where N and/or M is 128)

Horizontal binary tree partition for 128×n CUs (where N < = 64)

Vertical binary tree partition for n×128 CUs (where N < = 64)

Alternatively, when the depth of a CU corresponding to each node of the multi-type tree is equal to the maximum depth of the multi-type tree, the CU may not be partitioned in binary tree form and/or in trigeminal tree form. In this manner of determination, information indicating whether a partition in the multi-type tree is executed may be not signaled, but may be inferred as a second value.

Alternatively, the information indicating whether partitioning by the multi-type tree is performed may be signaled only when at least one of the vertical binary tree partition, the horizontal binary tree partition, the vertical trigeminal tree partition, and the horizontal trigeminal tree partition is possible for a CU corresponding to each node of the multi-type tree. Otherwise, the CUs may not be partitioned in binary tree form and/or trigeminal tree form. In this way, information indicating whether a partition in the multi-type tree is executed may not be signaled, but may be inferred as a second value.

Alternatively, for a CU corresponding to each node of the multi-type tree, partition direction information may be signaled only if both vertical and horizontal binary tree partitions are available or only if both vertical and horizontal trigeminal tree partitions are available. Otherwise, partition direction information may not be signaled, but may be inferred as a value indicating the direction in which a CU may be partitioned.

Alternatively, for CUs corresponding to each node of the multi-type tree, partition tree information may be signaled only if both vertical binary tree partitions and vertical trigeminal tree partitions are available or only if both horizontal binary tree partitions and horizontal trigeminal tree partitions are available. Otherwise, partition tree information may not be signaled, but may be inferred as a value indicating a tree applicable to the partition of the CU.

Fig. 4 is a diagram illustrating a form of a prediction unit that an encoding unit can include.

Among CUs partitioned from LCUs, a CU that is no longer partitioned may be partitioned into one or more Prediction Units (PUs). This partitioning is also referred to as "partitioning".

A PU may be a base unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. The PU may be partitioned into various shapes according to various modes. For example, the target block described above with reference to fig. 1 and the target block described above with reference to fig. 2 may both be PUs.

A CU may not be partitioned into PUs. When a CU is not divided into PUs, the size of the CU and the size of the PU may be equal to each other.

In skip mode, no partition may be present in the CU. In the skip mode, the 2n×2n mode 410 may be supported without partitioning, wherein in the 2n×2n mode 410, the size of the PU and the size of the CU are identical to each other.

In inter mode, there may be 8 types of partition shapes in a CU. For example, in the inter mode, a 2n×2n mode 410, a 2n×n mode 415, an n×2n mode 420, an n×n mode 425, a 2n×nu mode 430, a 2n×nd mode 435, an nl×2n mode 440, and an nr×2n mode 445 may be supported.

In intra mode, a 2nx2n mode 410 and an nxn mode 425 may be supported.

In the 2n×2n mode 410, PUs of size 2n×2n may be encoded. A PU of size 2N x 2N may represent a PU of the same size as a CU. For example, a PU of size 2N x 2N may have a size of 64 x 64, 32 x 32, 16 x 16, or 8 x 8.

In the nxn mode 425, a PU of size nxn may be encoded.

For example, in intra prediction, four partitioned PUs may be encoded when the PU size is 8 x 8. The size of each partitioned PU may be 4 x 4.

When encoding a PU in intra mode, the PU may be encoded using any of a plurality of intra prediction modes. For example, HEVC techniques may provide 35 intra-prediction modes, and a PU may be encoded in any of the 35 intra-prediction modes.

Which of the 2nx2n mode 410 and the nxn mode 425 is to be used to encode the PU may be determined based on the rate distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2nx2n. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. By the encoding operation, the best intra prediction mode for a PU of size 2N x 2N can be derived. The optimal intra prediction mode may be an intra prediction mode that exhibits a minimum rate distortion cost when encoding a PU having a size of 2n×2n among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

Further, the encoding apparatus 100 may sequentially perform encoding operations on respective PUs obtained by performing nxn partitioning. Here, the encoding operation may be an operation of encoding the PU in each of a plurality of intra prediction modes that can be used by the encoding apparatus 100. By the encoding operation, the best intra prediction mode for a PU of size nxn may be derived. The optimal intra prediction mode may be an intra prediction mode that exhibits a minimum rate distortion cost when encoding a PU of size nxn among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

The encoding apparatus 100 may determine which of a PU of size 2nx2n and a PU of size nxn is to be encoded based on a comparison between the rate-distortion cost of the PU of size 2nx2n and the rate-distortion cost of the PU of size nxn.

A single CU may be partitioned into one or more PUs, and a PU may be partitioned into multiple PUs.

For example, when a single PU is partitioned into four PUs, the horizontal and vertical dimensions of each of the four PUs generated by the partitioning may be half the horizontal and vertical dimensions of the PU prior to being partitioned. When a PU of size 32 x 32 is partitioned into four PUs, the size of each of the four partitioned PUs may be 16 x 16. When a single PU is partitioned into four PUs, the PUs may be considered to have been partitioned in a quadtree structure.

For example, when a single PU is partitioned into two PUs, the horizontal or vertical size of each of the two PUs generated by the partitioning may be half the horizontal or vertical size of the PU prior to being partitioned. When a PU of size 32 x 32 is vertically partitioned into two PUs, the size of each of the two partitioned PUs may be 16 x 32. When a PU of size 32 x 32 is horizontally partitioned into two PUs, the size of each of the two partitioned PUs may be 32 x 16. When a single PU is partitioned into two PUs, the PUs may be considered to have been partitioned in a binary tree structure.

Fig. 5 is a diagram showing a form of a transform unit that can be included in an encoding unit.

A Transform Unit (TU) may be a basic unit in a CU used for processes such as transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding.

The TUs may have a square shape or a rectangular shape. The shape of the TU may be determined based on the size and/or shape of the CU.

Among CUs partitioned from LCUs, a CU that is no longer partitioned into CUs may be partitioned into one or more TUs. Here, the partition structure of the TUs may be a quadtree structure. For example, as shown in FIG. 5, a single CU 510 may be partitioned one or more times according to a quadtree structure. With such partitioning, a single CU 510 may be composed of TUs having various sizes.

A CU may be considered to be recursively partitioned when a single CU is partitioned two or more times. By partitioning, a single CU may be composed of Transform Units (TUs) having various sizes.

Alternatively, a single CU may be partitioned into one or more TUs based on the number of vertical and/or horizontal lines that partition the CU.

A CU may be divided into symmetric TUs or asymmetric TUs. For division into asymmetric TUs, information about the size and/or shape of each TU may be signaled from the encoding apparatus 100 to the decoding apparatus 200. Alternatively, the size and/or shape of each TU may be derived from information about the size and/or shape of the CU.

A CU may not be partitioned into TUs. When a CU is not divided into TUs, the size of the CU and the size of the TUs may be equal to each other.

A single CU may be partitioned into one or more TUs, and a TU may be partitioned into multiple TUs.

For example, when a single TU is partitioned into four TUs, the horizontal and vertical sizes of each of the four TUs generated by the partitioning may be half the horizontal and vertical sizes of the TUs before being partitioned. When a TU of size 32×32 is partitioned into four TUs, the size of each of the four partitioned TUs may be 16×16. When a single TU is partitioned into four TUs, the TUs may be considered to have been partitioned in a quadtree structure.

For example, when a single TU is partitioned into two TUs, the horizontal size or vertical size of each of the two TUs generated by the partitioning may be half the horizontal size or vertical size of the TUs before being partitioned. When a TU of size 32×32 is vertically partitioned into two TUs, the size of each of the two partitioned TUs may be 16×32. When a TU of size 32×32 is horizontally partitioned into two TUs, the size of each of the two partitioned TUs may be 32×16. When a single TU is partitioned into two TUs, the TUs may be considered to have been partitioned in a binary tree structure.

The CUs may be partitioned differently than shown in fig. 5.

For example, a single CU may be divided into three CUs. The horizontal or vertical sizes of the three CUs generated by the division may be 1/4, 1/2, and 1/4 of the horizontal or vertical sizes of the original CU before the division, respectively.

For example, when a CU having a size of 32×32 is vertically divided into three CUs, the sizes of the three CUs generated by the division may be 8×32, 16×32, and 8×32, respectively. In this way, when a single CU is divided into three CUs, the CUs can be considered to be divided in a form of a trigeminal tree.

One of the exemplary partition forms (i.e., quadtree partition, binary tree partition, and trigeminal tree partition) may be applied to the partition of the CU, and multiple partition schemes may be combined and used together for the partition of the CU. Here, a case where a plurality of division schemes are combined and used together may be referred to as "compound tree-type division".

Fig. 6 illustrates partitioning of blocks according to an example.

In the video encoding and/or decoding process, as shown in fig. 6, target blocks may be partitioned. For example, the target block may be a CU.

For the division of the target block, an indicator indicating the division information may be signaled from the encoding apparatus 100 to the decoding apparatus 200. The partition information may be information indicating how the target block is partitioned.

The division information may be one or more of a division flag (hereinafter referred to as "split_flag"), a four-binary flag (hereinafter referred to as "qb_flag"), a quadtree flag (hereinafter referred to as "quadtree _flag"), a binary tree flag (hereinafter referred to as "binarytree _flag"), and a binary type flag (hereinafter referred to as "Btype _flag").

The "split_flag" may be a flag indicating whether a block is divided. For example, a split_flag value of 1 may indicate that the corresponding block is partitioned. A split flag value of 0 may indicate that the corresponding block is not partitioned.

The "qb_flag" may be a flag indicating which of the quadtree form and the binary tree form corresponds to a shape in which the block is divided. For example, a qb_flag value of 0 may indicate that the blocks are divided in a quadtree form. A qb_flag value of 1 may indicate that the blocks are partitioned in a binary tree form. Alternatively, a qb_flag value of 0 may indicate that the blocks are partitioned in a binary tree form. A qb_flag value of 1 may indicate that the blocks are divided in a quadtree form.

"Quadtree _flag" may be a flag indicating whether a block is divided in a quadtree form. For example, a quadtree _flag value of 1 may indicate that the block is partitioned in a quadtree form. A quadtree _flag value of 0 may indicate that the block is not divided in a quadtree form.

"Binarytree _flag" may be a flag indicating whether a block is divided in a binary tree form. For example, a binarytree _flag value of 1 may indicate that the block is partitioned in a binary tree form. A binarytree _flag value of 0 may indicate that the block is not partitioned in a binary tree form.

"Btype _flag" may be a flag indicating which of the vertical division and the horizontal division corresponds to the division direction when the block is divided in the form of a binary tree. For example, a Btype _flag value of 0 may indicate that the block is divided in the horizontal direction. A Btype _flag value of 1 may indicate that the block is divided in the vertical direction. Alternatively, a Btype _flag value of 0 may indicate that the block is divided in the vertical direction. A Btype _flag value of 1 may indicate that the block is divided in the horizontal direction.

For example, the partition information of the block in fig. 6 may be derived by signaling at least one of quadtree _ flag, binarytree _flag and Btype _flag, as shown in table 1 below.

TABLE 1

For example, the partition information of the block in fig. 6 may be derived by signaling at least one of split_flag, qb_flag, and Btype _flag, as shown in table 2 below.

TABLE 2

The partitioning method may be limited to a quadtree or a binary tree depending on the size and/or shape of the block. When this restriction is applied, the split_flag may be a flag indicating whether the block is divided in a quadtree form or a flag indicating whether the block is divided in a binary tree form. The size and shape of the block may be derived from the depth information of the block, and the depth information may be signaled from the encoding apparatus 100 to the decoding apparatus 200.

When the size of the block falls within a specific range, division in only a quadtree form is possible. For example, the specific range may be defined by at least one of a maximum block size and a minimum block size that can be divided only in a quadtree form.

Information indicating the maximum block size and the minimum block size that can be divided only in the quadtree form may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Furthermore, this information may be signaled for at least one of the units such as video, sequence, picture, parameter, parallel block group and slice (or slice).

Alternatively, the maximum block size and/or the minimum block size may be a fixed size predefined by the encoding apparatus 100 and the decoding apparatus 200. For example, when the block size is greater than 64×64 and less than 256×256, it is possible to divide only in a quadtree form. In this case, the split_flag may be a flag indicating whether to perform division in a quadtree form.

Partitioning in quadtree form only is possible when the size of the block is larger than the maximum size of the transformed block. Here, the sub-block generated by the partition may be at least one of a CU and a TU.

In this case, the split_flag may be a flag indicating whether or not the CU is partitioned in a quadtree form.

When the size of the block falls within a specific range, division in only a binary tree form or a trigeminal tree form is possible. For example, the specific range may be defined by at least one of a maximum block size and a minimum block size that can be divided only in a binary tree form or a trigeminal tree form.

Information indicating the maximum block size and/or the minimum block size that can be divided in a binary tree form only or in a trigeminal tree form can be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. Furthermore, this information may be signaled for at least one of the units such as sequence, picture and slice (or slice).

Alternatively, the maximum block size and/or the minimum block size may be a fixed size predefined by the encoding apparatus 100 and the decoding apparatus 200. For example, when the block size is greater than 8×8 and less than 16×16, it is possible to divide only in a binary tree form. In this case, the split_flag may be a flag indicating whether to perform division in a binary tree form or a trigeminal tree form.

The above description of partitioning in a quadtree form may be equally applied to binary tree forms and/or trigeminal tree forms.

The partitioning of a block may be limited by the previous partitioning. For example, when a block is partitioned in a particular binary tree form and multiple sub-blocks are generated from the partition, each sub-block may be further partitioned only in a particular tree form. Here, the specific tree form may be at least one of a binary tree form, a trigeminal tree form, and a quadtree form.

When the horizontal size or the vertical size of the tiles is a size that cannot be further divided, the above-described indicator may be signaled without a signal.

Fig. 7 is a diagram for explaining an embodiment of intra prediction processing.

Arrows extending radially from the center of the graph in fig. 7 indicate the prediction direction of the intra prediction mode. Further, numbers appearing near the arrows indicate examples of mode values assigned to intra prediction modes or prediction directions of intra prediction modes.

In fig. 7, the number 0 may represent a plane mode as a non-directional intra prediction mode. The number 1 may represent a DC mode as a non-directional intra prediction mode.

Intra-coding and/or decoding may be performed using reference samples of neighboring blocks of the target block. The neighboring blocks may be reconstructed neighboring blocks. The reference sample point may represent a neighboring sample point.

For example, intra-coding and/or decoding may be performed using values of reference samples included in the reconstructed neighboring blocks or coding parameters of the reconstructed neighboring blocks.

The encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block by performing intra prediction on a target block based on information about samples in a target image. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may generate a prediction block for a target block by performing intra prediction based on information about samples in a target image. When intra prediction is performed, the encoding apparatus 100 and/or the decoding apparatus 200 may perform directional prediction and/or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may be a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, PU, and TU.

The unit of the prediction block may have a size corresponding to at least one of the CU, PU, and TU. The prediction block may have a square shape with a size of 2n×2n or n×n. The dimensions N x N may include dimensions 4 x 4, 8 x 8, 16 x 16, 32 x 32, 64 x 64, etc.

Alternatively, the prediction block may be a square block of size 2×2,4×4, 8×8, 16×16, 32×32, 64×64, etc., or a rectangular block of size 2×8, 4×8, 2×16, 4×16, 8×16, etc.

Intra prediction may be performed considering an intra prediction mode for a target block. The number of intra prediction modes that the target block may have may be a predefined fixed value, and may be a value that is differently determined according to the properties of the prediction block. For example, the attribute of the prediction block may include the size of the prediction block, the type of the prediction block, and the like. Furthermore, the attribute of the prediction block may indicate the coding parameters for the prediction block.

For example, the number of intra prediction modes may be fixed to N regardless of the size of the prediction block. Alternatively, the number of intra prediction modes may be, for example, 3, 5, 9, 17, 34, 35, 36, 65, 67, or 95.

The intra prediction mode may be a non-directional mode or a directional mode.

For example, the intra prediction modes may include two non-directional modes and 65 directional modes corresponding to numbers 0 to 66 shown in fig. 7.

For example, in the case of using a specific intra prediction method, the intra prediction modes may include two non-directional modes and 93 directional modes corresponding to numbers-14 to 80 shown in fig. 7.

The two non-directional modes may include a DC mode and a planar mode.

The direction mode may be a prediction mode having a specific direction or a specific angle. The direction mode may also be referred to as an "angle mode".

The intra prediction mode may be represented by at least one of a mode number, a mode value, a mode angle, and a mode direction. In other words, the terms "intra prediction mode (mode) number", "intra prediction mode (mode) value", "intra prediction mode (mode) angle", and "intra prediction mode (mode) direction" may be used to have the same meaning and may be used interchangeably with each other.

The number of intra prediction modes may be M. The value of M may be 1 or greater. In other words, the number of intra prediction modes may be M, where M includes the number of non-directional modes and the number of directional modes.

The number of intra prediction modes may be fixed to M regardless of the size and/or color components of the block. For example, the number of intra prediction modes may be fixed to any one of 35 and 67 regardless of the size of the block.

Alternatively, the number of intra prediction modes may be different according to the shape, size, and/or type of color component of the block.

For example, in fig. 7, the direction prediction mode as shown by the dotted line may be applied only to prediction for non-square blocks.

For example, the larger the block size, the greater the number of intra prediction modes. Alternatively, the larger the block size, the fewer the number of intra prediction modes. When the size of the block is 4×4 or 8×8, the number of intra prediction modes may be 67. When the block size is 16×16, the number of intra prediction modes may be 35. When the block size is 32×32, the number of intra prediction modes may be 19. When the size of the block is 64×64, the number of intra prediction modes may be 7.

For example, the number of intra prediction modes may be different depending on whether a color component is a luminance signal or a chrominance signal. Alternatively, the number of intra prediction modes corresponding to the luminance component block may be greater than the number of intra prediction modes corresponding to the chrominance component block.

For example, in a vertical mode with a mode value of 50, prediction may be performed in a vertical direction based on pixel values of reference samples. For example, in a horizontal mode with a mode value of 18, prediction may be performed in the horizontal direction based on the pixel value of the reference sample.

Even in the direction modes other than the above-described modes, the encoding apparatus 100 and the decoding apparatus 200 can perform intra prediction on the target unit using the reference samples according to the angles corresponding to the direction modes.

The intra prediction mode located at the right side with respect to the vertical mode may be referred to as a "vertical-right mode". The intra prediction mode located below the horizontal mode may be referred to as a "horizontal-below mode". For example, in fig. 7, the intra prediction mode in which the mode value is one of 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, and 66 may be a vertical-right mode. The intra prediction mode, in which the mode value is one of 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, and 17, may be a horizontal-down mode.

The non-directional modes may include a DC mode and a planar mode. For example, the value of the DC mode may be 1. The value of the planar mode may be 0.

The direction pattern may include an angle pattern. Among the plurality of intra prediction modes, the remaining modes other than the DC mode and the plane mode may be directional modes.

When the intra prediction mode is a DC mode, a prediction block may be generated based on an average value of pixel values of a plurality of reference pixels. For example, the value of a pixel of the prediction block may be determined based on an average of pixel values of a plurality of reference pixels.

The number of intra prediction modes and the mode values of the respective intra prediction modes described above are merely exemplary. The number of intra prediction modes described above and the mode values of the respective intra prediction modes may be defined differently according to embodiments, implementations, and/or requirements.

In order to perform intra prediction on a target block, a step of checking whether a sample included in a reconstructed neighboring block can be used as a reference sample of the target block may be performed. When a sample that cannot be used as a reference sample of the target block exists among samples in the neighboring blocks, a value generated via interpolation and/or copying using at least one sample value among samples included in the reconstructed neighboring blocks may replace a sample value of a sample that cannot be used as a reference sample. When a value generated via copying and/or interpolation replaces a sample value of an existing sample, the sample may be used as a reference sample for the target block.

When intra prediction is used, a filter may be applied to at least one of the reference samples and the prediction samples based on at least one of the size of the target block and the intra prediction mode.

The type of filter to be applied to at least one of the reference sample point and the prediction sample point may be different according to at least one of an intra prediction mode of the target block, a size of the target block, and a shape of the target block. The type of filter may be classified according to one or more of the length of the filter taps, the values of the filter coefficients, and the filter strength. The length of the filter taps may represent the number of filter taps. Furthermore, the number of filter taps may represent the length of the filter.

When the intra prediction mode is a planar mode, a sample value of the prediction target block may be generated according to a position of a prediction target sample point in the prediction block when the prediction block of the target block is generated, using a weighted sum of an upper reference sample point of the target block, a left reference sample point of the target block, an upper right reference sample point of the target block, and a lower left reference sample point of the target block.

When the intra prediction mode is a DC mode, an average value of a reference sample above the target block and a reference sample to the left of the target block may be used in generating a prediction block of the target block. Furthermore, filtering using the values of the reference samples may be performed on a particular row or a particular column in the target block. The particular row may be one or more upper rows adjacent to the reference sample point. The particular column may be one or more left columns adjacent to the reference sample point.

When the intra prediction mode is a directional mode, a prediction block may be generated using an upper reference sample, a left reference sample, an upper right reference sample, and/or a lower left reference sample of the target block.

To generate the above-described predicted samples, real-based interpolation may be performed.

The intra prediction mode of the target block may be predicted from the intra prediction modes of neighboring blocks adjacent to the target block, and information for prediction may be entropy encoded/entropy decoded.

For example, when intra prediction modes of a target block and a neighboring block are identical to each other, a predefined flag may be used to signal that the intra prediction modes of the target block and the neighboring block are identical.

For example, an indicator for indicating the same intra prediction mode as that of the target block among the intra prediction modes of the plurality of neighboring blocks may be signaled.

When intra prediction modes of a target block and neighboring blocks are different from each other, information regarding the intra prediction modes of the target block may be encoded and/or decoded using entropy encoding and/or entropy decoding.

Fig. 8 is a diagram illustrating reference samples used in an intra prediction process.

The reconstructed reference points for intra-prediction of the target block may include a lower left reference point, a left reference point, an upper left corner reference point, an upper reference point, and an upper right reference point.

For example, the left reference sample point may represent a reconstructed reference pixel adjacent to the left side of the target block. The upper reference sample may represent a reconstructed reference pixel adjacent to the top of the target block. The upper left corner reference sample point may represent a reconstructed reference pixel located at the upper left corner of the target block. The lower left reference sample point may represent a reference sample point located below a left sample point line composed of left reference sample points among sample points located on the same line as the left sample point line. The upper right reference sample point may represent a reference sample point located to the right of an upper sample point line composed of upper reference sample points among sample points located on the same line as the upper sample point line.

When the size of the target block is n×n, the number of lower left reference samples, upper reference samples, and upper right reference samples may be N.

By performing intra prediction on a target block, a prediction block may be generated. The process of generating the prediction block may include determining values of pixels in the prediction block. The target block and the prediction block may be the same size.

The reference points for intra-predicting the target block may be changed according to the intra-prediction mode of the target block. The direction of the intra prediction mode may represent a dependency relationship between the reference sample point and the pixels of the prediction block. For example, the value of the specified reference sample point may be used as the value of one or more specified pixels in the prediction block. In this case, the one or more specified pixels in the specified reference sample and prediction block may be samples and pixels located on a straight line along a direction of the intra prediction mode. In other words, the value of the specified reference sample point may be copied as a value of a pixel located in a direction opposite to the direction of the intra prediction mode. Alternatively, the value of a pixel in the prediction block may be a value of a reference sample located in the direction of the intra prediction mode with respect to the position of the pixel.

In an example, when the intra prediction mode of the target block is a vertical mode, the upper reference sample may be used for intra prediction. When the intra prediction mode is a vertical mode, the value of a pixel in the prediction block may be the value of a reference sample vertically above the position of the pixel. Thus, the upper reference sample adjacent to the top of the target block may be used for intra prediction. Furthermore, the values of the pixels in a row of the prediction block may be the same as the values of the pixels of the upper reference sample.

In an example, when the intra prediction mode of the target block is a horizontal mode, the left reference sample may be used for intra prediction. When the intra prediction mode is a horizontal mode, the value of a pixel in the prediction block may be the value of a reference sample horizontally located to the left of the position of the pixel. Thus, a left reference sample adjacent to the left side of the target block may be used for intra prediction. Furthermore, the values of pixels in a column of the prediction block may be the same as the values of pixels of the left reference sample point.

In an example, when a mode value of an intra prediction mode of the current block is 34, at least some of the left reference samples, the upper left reference samples, and at least some of the upper reference samples may be used for intra prediction. When the mode value of the intra prediction mode is 34, the value of a pixel in the prediction block may be the value of a reference sample point diagonally located at the upper left corner of the pixel.

Further, in the case of an intra prediction mode in which the mode value is a value ranging from 52 to 66, at least a part of the upper right reference sample may be used for intra prediction.

Further, in the case of an intra prediction mode in which the mode value is a value ranging from 2 to 17, at least a part of the lower left reference sample may be used for intra prediction.

Furthermore, in the case of an intra prediction mode in which the mode value is a value ranging from 19 to 49, the upper left corner reference sample may be used for intra prediction.

The number of reference samples used to determine the pixel value of one pixel in the prediction block may be 1 or 2 or more.

As described above, the pixel values of the pixels in the prediction block may be determined according to the positions of the pixels and the positions of the reference samples indicated by the direction of the intra prediction mode. When the position of the pixel and the position of the reference sample point indicated by the direction of the intra prediction mode are integer positions, the value of one reference sample point indicated by the integer position may be used to determine the pixel value of the pixel in the prediction block.

When the position of the pixel and the position of the reference sample point indicated by the direction of the intra prediction mode are not integer positions, an interpolation reference sample point based on two reference sample points closest to the position of the reference sample point may be generated. The values of the interpolated reference samples may be used to determine pixel values for pixels in the prediction block. In other words, when the position of a pixel in the prediction block and the position of a reference sample indicated by the direction of the intra prediction mode indicate the position between two reference samples, interpolation based on the values of the two samples may be generated.

The prediction block generated via prediction may be different from the original target block. In other words, there may be a prediction error, which is a difference between the target block and the prediction block, and there may also be a prediction error between the pixels of the target block and the pixels of the prediction block.

Hereinafter, the terms "difference", "error" and "residual" may be used to have the same meaning and may be used interchangeably with each other.

For example, in the case of directional intra prediction, the longer the distance between the pixels of the prediction block and the reference sample, the greater the prediction error that may occur. Such prediction errors may result in discontinuities between the generated prediction block and neighboring blocks.

In order to reduce prediction errors, a filtering operation for the prediction block may be used. The filtering operation may be configured to adaptively apply the filter to regions of the prediction block that are considered to have large prediction errors. For example, the region considered to have a large prediction error may be a boundary of a prediction block. In addition, regions considered to have a large prediction error in a prediction block may differ according to an intra prediction mode, and characteristics of a filter may also differ according to an intra prediction mode.

As shown in fig. 8, for intra prediction of a target block, at least one of reference line 0 to reference line 3 may be used.

Each reference line in fig. 8 may indicate a reference sample line including one or more reference sample points. When the number of reference lines is smaller, reference sample lines closer to the target block may be indicated.

The samples in segments a and F may be obtained by padding using the sample closest to the target block in segments B and E, instead of from the reconstructed neighboring block.

Index information indicating a reference sample line to be used for intra prediction of a target block may be signaled. The index information may indicate a reference sample line to be used for intra prediction of the target block among a plurality of reference sample lines. For example, the index information may have a value corresponding to any one of 0 to 3.

When the upper boundary of the target block is the boundary of the CTU, only the reference sample line 0 may be available. Therefore, in this case, the index information may not be signaled. When an additional reference sample line other than the reference sample line 0 is used, filtering of a prediction block, which will be described later, may not be performed.

In the case of inter-color intra prediction, a prediction block of a target block of a second color component may be generated based on a corresponding reconstructed block of the first color component.

For example, the first color component may be a luminance component and the second color component may be a chrominance component.

To perform inter-color intra prediction, parameters of a linear model between the first color component and the second color component may be derived based on a template.

The template may comprise a reference sample above the target block (upper reference sample) and/or a reference sample to the left of the target block (left reference sample), and may comprise an upper reference sample and/or a left reference sample of the reconstructed block of the first color component corresponding to the reference sample.

For example, the parameters of the linear model may be derived using 1) the value of the sample of the first color component having the largest value among the samples in the template, 2) the value of the sample of the second color component corresponding to the sample of the first color component, 3) the value of the sample of the first color component having the smallest value among the samples in the template, and 4) the value of the sample of the second color component corresponding to the sample of the first color component.

When deriving parameters of the linear model, a prediction block of the target block may be generated by applying the corresponding reconstructed block to the linear model.

Depending on the image format, sub-sampling may be performed on samples adjacent to the reconstructed block of the first color component and the corresponding reconstructed block of the first color component. For example, when one sample of the second color component corresponds to four samples of the first color component, one corresponding sample may be calculated by performing sub-sampling on the four samples of the first color component. When sub-sampling is performed, derivation of parameters of the linear model and inter-color intra prediction may be performed based on the corresponding samples that are sub-sampled.

Information regarding whether to perform inter-color intra prediction and/or the range of templates may be signaled in intra prediction mode.

The target block may be partitioned into two or four sub-blocks in the horizontal direction and/or the vertical direction.

The sub-blocks generated by the partition may be reconstructed sequentially. That is, when intra prediction is performed on each sub-block, a sub-prediction block of the sub-block may be generated. Further, when inverse quantization (inverse quantization) and/or inverse transformation is performed on each sub-block, a sub-residual block for the corresponding sub-block may be generated. The reconstructed sub-block may be generated by adding the sub-prediction block to the sub-residual block. The reconstructed sub-block may be used as a reference sample for intra prediction of the sub-block with the next priority.

A sub-block may be a block that includes a certain number (e.g., 16) or more samples. For example, when the target block is an 8×4 block or a 4×8 block, the target block may be partitioned into two sub-blocks. Further, when the target block is a 4×4 block, the target block cannot be partitioned into sub-blocks. When the target block has another size, the target block may be partitioned into four sub-blocks.

Information on whether to perform intra prediction based on these sub-blocks and/or information on a partition direction (horizontal direction or vertical direction) may be signaled.

Such sub-block based intra prediction may be limited such that it is performed only when the reference sample line 0 is used. When sub-block-based intra prediction is performed, filtering of a prediction block, which will be described below, may not be performed.

The final prediction block may be generated by performing filtering on a prediction block generated through intra prediction.

The filtering may be performed by applying a specific weight to a filtering target sample point, a left reference sample point, an upper reference sample point, and/or an upper left reference sample point, which are targets to be filtered.

The weights for filtering and/or the reference samples (e.g., range of reference samples, location of reference samples, etc.) may be determined based on at least one of the block size, intra prediction mode, and location of the filtering target samples in the prediction block.

For example, filtering may be performed only in a particular intra prediction mode (e.g., DC mode, planar mode, vertical mode, horizontal mode, diagonal mode, and/or adjacent diagonal mode).

The adjacent diagonal pattern may be a pattern having a number obtained by adding k to the number of the diagonal pattern, and may be a pattern having a number obtained by subtracting k from the number of the diagonal pattern. In other words, the number of the adjacent diagonal pattern may be the sum of the number of the diagonal pattern and k, or may be the difference between the number of the diagonal pattern and k. For example, k may be a positive integer of 8 or less.

Intra-prediction modes of the target block may be derived using intra-prediction modes of neighboring blocks that occur around the target block, and such derived intra-prediction modes may be entropy encoded and/or entropy decoded.

For example, when the intra prediction mode of the target block is identical to the intra prediction mode of the neighboring block, specific flag information may be used to signal information indicating that the intra prediction mode of the target block is identical to the intra prediction mode of the neighboring block.

Further, for example, indicator information of neighboring blocks having the same intra prediction mode as that of the target block among the intra prediction modes of the plurality of neighboring blocks may be signaled.

For example, when the intra prediction mode of the target block is different from the intra prediction modes of the neighboring blocks, entropy encoding and/or entropy decoding may be performed on information regarding the intra prediction mode of the target block by performing entropy encoding and/or entropy decoding based on the intra prediction modes of the neighboring blocks.

Fig. 9 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle shown in fig. 9 may represent an image (or screen). Further, in fig. 9, an arrow may indicate a prediction direction. An arrow pointing from the first picture to the second picture indicates that the second picture references the first picture. That is, each image may be encoded and/or decoded according to a prediction direction.

The image may be classified into an intra picture (I picture), a single predicted picture or a predictive coded picture (P picture), and a bi-predicted picture or a bi-predictive coded picture (B picture) according to the type of encoding. Each picture may be encoded and/or decoded according to the type of encoding of each picture.

When the target image that is the target to be encoded is an I picture, the target image may be encoded using data included in the image itself without inter prediction with reference to other images. For example, an I picture may be encoded via intra prediction only.

When the target image is a P picture, the target image may be encoded via inter prediction using a reference picture existing in one direction. Here, the one direction may be a forward direction or a backward direction.

When the target image is a B picture, the image may be encoded via inter prediction using reference pictures existing in both directions, or may be encoded via inter prediction using reference pictures existing in one of a forward direction and a backward direction. Here, the two directions may be a forward direction and a backward direction.

P-pictures and B-pictures encoded and/or decoded using reference pictures may be considered as pictures using inter-prediction.

Hereinafter, inter prediction in inter mode according to an embodiment will be described in detail.

Inter prediction or motion compensation may be performed using the reference image and the motion information.

In the inter mode, the encoding apparatus 100 may perform inter prediction and/or motion compensation on the target block. The decoding apparatus 200 may perform inter prediction and/or motion compensation on the target block corresponding to the inter prediction and/or motion compensation performed by the encoding apparatus 100.

The motion information of the target block may be derived by the encoding apparatus 100 and the decoding apparatus 200 alone during inter prediction. The motion information may be derived using the motion information of the reconstructed neighboring block, the motion information of the col block, and/or the motion information of the block adjacent to the col block.

For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and/or motion compensation by using motion information of spatial candidates and/or temporal candidates as motion information of a target block. The target block may represent a PU and/or a PU partition.

The spatial candidates may be reconstructed blocks spatially adjacent to the target block.

The temporal candidate may be a reconstructed block corresponding to the target block in a previously reconstructed co-located picture (col picture).

In the inter prediction, the encoding apparatus 100 and the decoding apparatus 200 may improve encoding efficiency and decoding efficiency by using motion information of spatial candidates and/or temporal candidates. The motion information of the spatial candidate may be referred to as "spatial motion information". The motion information of the temporal candidates may be referred to as "temporal motion information".

Next, the motion information of the spatial candidate may be motion information of a PU including the spatial candidate. The motion information of the temporal candidate may be motion information of a PU including the temporal candidate. The motion information of the candidate block may be motion information of a PU including the candidate block.

Inter prediction may be performed using a reference picture.

The reference picture may be at least one of a picture preceding the target picture and a picture following the target picture. The reference picture may be an image for prediction of the target block.

In the inter prediction, a region in the reference picture may be specified using a reference picture index (or refIdx) for indicating the reference picture, a motion vector to be described later, or the like. Here, the region specified in the reference picture may indicate a reference block.

Inter prediction may select a reference picture, and may also select a reference block corresponding to the target block from the reference picture. Furthermore, inter prediction may use the selected reference block to generate a prediction block for the target block.

Motion information may be derived by each of the encoding apparatus 100 and the decoding apparatus 200 during inter prediction.

The spatial candidates may be 1) blocks that exist in the target picture 2) have been previously reconstructed via encoding and/or decoding and 3) are adjacent to or located at corners of the target block. Here, the "block located at the corner of the target block" may be a block vertically adjacent to an adjacent block horizontally adjacent to the target block, or a block horizontally adjacent to an adjacent block vertically adjacent to the target block. Further, "a block located at a corner of a target block" may have the same meaning as "a block adjacent to a corner of a target block". The meaning of "a block located at a corner of a target block" may be included in the meaning of "a block adjacent to the target block".

For example, the spatial candidate may be a reconstructed block located to the left of the target block, a reconstructed block located above the target block, a reconstructed block located in the lower left corner of the target block, a reconstructed block located in the upper right corner of the target block, or a reconstructed block located in the upper left corner of the target block.

Each of the encoding apparatus 100 and the decoding apparatus 200 may identify a block existing in a location in the col picture that spatially corresponds to the target block. The position of the target block in the target picture and the position of the identified block in the col picture may correspond to each other.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine col blocks existing at predefined relevant locations for the identified blocks as time candidates. The predefined relevant locations may be locations that exist inside and/or outside the identified block.

For example, the col blocks may include a first col block and a second col block. When the coordinates of the identified block are (xP, yP) and the size of the identified block is expressed by (nPSW, nPSH), the first col block may be a block located at the coordinates (xp+ nPSW, yp+ nPSH). The second col block may be a block located at coordinates (xp+ (nPSW > > 1), yp+ (nPSH > > 1)). The second col block may be selectively used when the first col block is not available.

The motion vector of the target block may be determined based on the motion vector of the col block. Each of the encoding apparatus 100 and the decoding apparatus 200 may scale the motion vector of the col block. The scaled motion vector of the col block may be used as the motion vector of the target block. Further, the motion vector of the motion information of the temporal candidate stored in the list may be a scaled motion vector.

The ratio of the motion vector of the target block to the motion vector of the col block may be the same as the ratio of the first temporal distance to the second temporal distance. The first temporal distance may be a distance between a reference picture and a target picture of the target block. The second temporal distance may be a distance between the reference picture and a col picture of the col block.

The scheme for deriving motion information may vary according to the inter prediction mode of the target block. For example, as an inter prediction mode applied to inter prediction, there may be an Advanced Motion Vector Predictor (AMVP) mode, a merge mode, a skip mode, a merge mode with a motion vector difference, a sub-block merge mode, a triangle partition mode, an inter-intra combined prediction mode, an affine inter mode, a current picture reference mode, and the like. The merge mode may also be referred to as a "motion merge mode". The respective modes will be described in detail below.

1) AMVP mode

When the AMVP mode is used, the encoding apparatus 100 may search for similar blocks in a neighboring area of the target block. The encoding apparatus 100 may acquire a prediction block by performing prediction on a target block using motion information of the found similar block. The encoding apparatus 100 may encode a residual block, which is a difference between the target block and the prediction block.

1-1) Creating a list of predicted motion vector candidates

When AMVP mode is used as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may create a list of prediction motion vector candidates using a motion vector of a spatial candidate, a motion vector of a temporal candidate, and a zero vector. The predicted motion vector candidate list may include one or more predicted motion vector candidates. At least one of a motion vector of a spatial candidate, a motion vector of a temporal candidate, and a zero vector may be determined and used as a predicted motion vector candidate.

Hereinafter, the terms "predicted motion vector (candidate)" and "motion vector (candidate)" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the terms "prediction motion vector candidate" and "AMVP candidate" may be used to have the same meaning and may be used interchangeably with each other.

Hereinafter, the terms "prediction motion vector candidate list" and "AMVP candidate list" may be used to have the same meaning and may be used interchangeably with each other.

The spatial candidates may include reconstructed spatial neighboring blocks. In other words, the motion vectors of the reconstructed neighboring blocks may be referred to as "spatial prediction motion vector candidates".

The temporal candidates may include col blocks and blocks adjacent to the col blocks. In other words, the motion vector of the col block or the motion vector of the block adjacent to the col block may be referred to as a "temporal prediction motion vector candidate".

The zero vector may be a (0, 0) motion vector.

The predicted motion vector candidates may be motion vector predictors for predicting motion vectors. Further, in the encoding apparatus 100, each predicted motion vector candidate may be an initial search position for a motion vector.

1-2) Searching for motion vectors using a list of predicted motion vector candidates

The encoding apparatus 100 may determine a motion vector to be used for encoding the target block within the search range using the list of predicted motion vector candidates. Further, the encoding apparatus 100 may determine a predicted motion vector candidate to be used as a predicted motion vector of the target block among the predicted motion vector candidates existing in the predicted motion vector candidate list.

The motion vector to be used for encoding the target block may be a motion vector that may be encoded at a minimum cost.

Further, the encoding apparatus 100 may determine whether to encode the target block using the AMVP mode.

1-3) Transmission of inter prediction information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The decoding apparatus 200 may perform inter prediction on the target block using inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether AMVP mode is used, 2) a prediction motion vector index, 3) a Motion Vector Difference (MVD), 4) a reference direction, and 5) a reference picture index.

Hereinafter, the terms "prediction motion vector index" and "AMVP index" may be used to have the same meaning and may be used interchangeably with each other.

In addition, the inter prediction information may include a residual signal.

When the mode information indicates that AMVP mode is used, the decoding apparatus 200 may acquire a prediction motion vector index, MVD, reference direction, and reference picture index from the bitstream through entropy decoding.

The prediction motion vector index may indicate a prediction motion vector candidate to be used for predicting the target block among prediction motion vector candidates included in the prediction motion vector candidate list.

1-4) Inter prediction in AMVP mode using inter prediction information

The decoding apparatus 200 may derive a predicted motion vector candidate using the predicted motion vector candidate list, and may determine motion information of the target block based on the derived predicted motion vector candidate.

The decoding apparatus 200 may determine a motion vector candidate for the target block among the predicted motion vector candidates included in the predicted motion vector candidate list using the predicted motion vector index. The decoding apparatus 200 may select the predicted motion vector candidate indicated by the predicted motion vector index from among the predicted motion vector candidates included in the predicted motion vector candidate list as the predicted motion vector of the target block.

The encoding apparatus 100 may generate an entropy-encoded prediction motion vector index by applying entropy encoding to the prediction motion vector index, and may generate a bitstream including the entropy-encoded prediction motion vector index. The entropy-encoded prediction motion vector index may be signaled from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may extract the entropy-encoded prediction motion vector index from the bitstream, and may acquire the prediction motion vector index by applying entropy decoding to the entropy-encoded prediction motion vector index.

The motion vector that will actually be used for inter prediction of the target block may not match the predicted motion vector. In order to indicate the difference between the motion vector that will actually be used for inter prediction of the target block and the predicted motion vector, MVD may be used. The encoding apparatus 100 may derive a prediction motion vector similar to a motion vector that will be actually used for inter prediction of a target block in order to use as small MVD as possible.

The Motion Vector Difference (MVD) may be a difference between a motion vector of the target block and a predicted motion vector. The encoding apparatus 100 may calculate an MVD, and may generate an entropy-encoded MVD by applying entropy encoding to the MVD. The encoding apparatus 100 may generate a bitstream including the entropy-encoded MVD.

The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may extract an entropy-encoded MVD from the bitstream, and may acquire the MVD by applying entropy decoding to the entropy-encoded MVD.

The decoding apparatus 200 may derive a motion vector of the target block by summing the MVD and the predicted motion vector. In other words, the motion vector of the target block derived by the decoding apparatus 200 may be the sum of the MVD and the motion vector candidates.

Further, the encoding apparatus 100 may generate entropy-encoded MVD resolution information by applying entropy encoding to the calculated MVD resolution information, and may generate a bitstream including the entropy-encoded MVD resolution information. The decoding apparatus 200 may extract entropy-encoded MVD resolution information from the bitstream, and may acquire the MVD resolution information by applying entropy decoding to the entropy-encoded MVD resolution information. The decoding apparatus 200 may adjust the resolution of the MVD using the MVD resolution information.

In addition, the encoding apparatus 100 may calculate MVDs based on affine models. The decoding apparatus 200 may derive an affine control motion vector of the target block from the sum of the MVD and affine control motion vector candidates, and may derive a motion vector of the sub-block using the affine control motion vector.

The reference direction may indicate a list of reference pictures to be used for predicting the target block. For example, the reference direction may indicate one of the reference picture list L0 and the reference picture list L1.

The reference direction only indicates a reference picture list to be used for prediction of the target block, and may not mean that the direction of the reference picture is limited to a forward direction or a backward direction. In other words, each of the reference picture list L0 and the reference picture list L1 may include pictures in a forward direction and/or a backward direction.

The reference direction being unidirectional may mean that a single reference picture list is used. The reference direction being bi-directional may mean that two reference picture lists are used. In other words, the reference direction may indicate one of a case where only the reference picture list L0 is used, a case where only the reference picture list L1 is used, and a case where two reference picture lists are used.

The reference picture index may indicate a reference picture for predicting the target block among reference pictures existing in the reference picture list. The encoding apparatus 100 may generate an entropy-encoded reference picture index by applying entropy encoding to the reference picture index, and may generate a bitstream including the entropy-encoded reference picture index. The entropy-encoded reference picture index may be signaled from the encoding device 100 to the decoding device 200 through a bitstream. The decoding apparatus 200 may extract an entropy-encoded reference picture index from the bitstream, and may acquire the reference picture index by applying entropy decoding to the entropy-encoded reference picture index.

When two reference picture lists are used to predict a target block, a single reference picture index and a single motion vector may be used for each of the reference picture lists. Further, when two reference picture lists are used to predict a target block, two prediction blocks may be specified for the target block. For example, an average or weighted sum of two prediction blocks for a target block may be used to generate a (final) prediction block for the target block.

The motion vector of the target block may be derived by predicting the motion vector index, MVD, reference direction, and reference picture index.

The decoding apparatus 200 may generate a prediction block for the target block based on the derived motion vector and the reference picture index. For example, the prediction block may be a reference block indicated by a derived motion vector in a reference picture indicated by a reference picture index.

Since the prediction motion vector index and the MVD are encoded and the motion vector of the target block is not itself encoded, the number of bits transmitted from the encoding apparatus 100 to the decoding apparatus 200 may be reduced and the encoding efficiency may be improved.

For the target block, motion information of the reconstructed neighboring block may be used. In a specific inter prediction mode, the encoding apparatus 100 may not separately encode actual motion information of the target block. The motion information of the target block is not encoded, but may encode additional information that enables the motion information of the target block to be derived using the reconstructed motion information of the neighboring blocks. Since the additional information is encoded, the number of bits transmitted to the decoding apparatus 200 may be reduced and encoding efficiency may be improved.

For example, as an inter prediction mode in which motion information of a target block is not directly encoded, a skip mode and/or a merge mode may exist. Here, each of the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and/or index of a unit, of which motion information is to be used as motion information of a target unit, among reconstructed neighboring units.

2) Merge mode

As a scheme for deriving motion information of a target block, there is merging. The term "merge" may mean merging the motion of multiple blocks. "merge" may mean that motion information of one block is also applied to other blocks. In other words, the merge mode may be a mode in which motion information of a target block is derived from motion information of neighboring blocks.

When the merge mode is used, the encoding apparatus 100 may predict motion information of the target block using motion information of spatial candidates and/or motion information of temporal candidates. The spatial candidates may include reconstructed spatially neighboring blocks that are spatially adjacent to the target block. The spatial neighboring blocks may include a left neighboring block and an upper neighboring block. The temporal candidates may include col blocks. The terms "spatial candidate" and "spatial merge candidate" may be used to have the same meaning and may be used interchangeably with each other. The terms "temporal candidates" and "temporal merging candidates" may be used to have the same meaning and may be used interchangeably with each other.

The encoding apparatus 100 may acquire a prediction block via prediction. The encoding apparatus 100 may encode a residual block, which is a difference between the target block and the prediction block.

2-1) Creation of merge candidate list

When the merge mode is used, each of the encoding apparatus 100 and the decoding apparatus 200 may create a merge candidate list using motion information of spatial candidates and/or motion information of temporal candidates. The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction may be unidirectional or bidirectional. The reference direction may represent an inter prediction indicator.

The merge candidate list may include merge candidates. The merge candidate may be motion information. In other words, the merge candidate list may be a list storing a plurality of pieces of motion information.

The merge candidate may be motion information of a plurality of temporal candidates and/or spatial candidates. In other words, the merge candidate list may include motion information of temporal candidates and/or spatial candidates, and the like.

Further, the merge candidate list may include new merge candidates generated by combining merge candidates already existing in the merge candidate list. In other words, the merge candidate list may include new motion information generated by combining pieces of motion information previously existing in the merge candidate list.

Further, the merge candidate list may include history-based merge candidates. The history-based merge candidate may be motion information of a block that was encoded and/or decoded prior to the target block.

Further, the merge candidate list may include merge candidates based on an average of two merge candidates.

The merge candidate may be a specific mode in which inter prediction information is derived. The merge candidate may be information indicating a specific mode in which the inter prediction information is derived. Inter prediction information of the target block may be derived according to a specific mode indicated by the merge candidate. Further, the particular mode may include a process of deriving a series of inter prediction information. This particular mode may be an inter prediction information derivation mode or a motion information derivation mode.

The inter prediction information of the target block may be derived from a mode indicated by a merge candidate selected from among merge candidates in the merge candidate list by the merge index.

For example, the motion information derivation mode in the merge candidate list may be at least one of 1) a motion information derivation mode for a sub-block unit and 2) an affine motion information derivation mode.

Further, the merge candidate list may include motion information of a zero vector. The zero vector may also be referred to as a "zero merge candidate".

In other words, the pieces of motion information in the merge candidate list may be at least one of 1) motion information of spatial candidates, 2) motion information of temporal candidates, 3) motion information generated by combining pieces of motion information previously existing in the merge candidate list, and 4) zero vector.

The motion information may include 1) a motion vector, 2) a reference picture index, and 3) a reference direction. The reference direction may also be referred to as an "inter prediction indicator". The reference direction may be unidirectional or bidirectional. The unidirectional reference direction may indicate either L0 prediction or L1 prediction.

The merge candidate list may be created before prediction in the merge mode is performed.

The number of merging candidates in the merging candidate list may be defined in advance. Each of the encoding apparatus 100 and the decoding apparatus 200 may add the merge candidates to the merge candidate list according to a predefined scheme and a predefined priority such that the merge candidate list has a predefined number of merge candidates. The merge candidate list of the encoding device 100 and the merge candidate list of the decoding device 200 may be made identical to each other using a predefined scheme and a predefined priority.

Merging may be applied based on a CU or PU. When merging is performed based on a CU or PU, the encoding apparatus 100 may transmit a bitstream including predefined information to the decoding apparatus 200. For example, the predefined information may include 1) information indicating whether to perform merging for each block partition, and 2) information about blocks to be performed to merge among blocks that are spatial candidates and/or temporal candidates for a target block.

2-2) Searching for motion vectors using merge candidate list

The encoding apparatus 100 may determine a merge candidate to be used for encoding the target block. For example, the encoding apparatus 100 may perform prediction on the target block using the merge candidates in the merge candidate list, and may generate a residual block for the merge candidates. The encoding apparatus 100 may encode the target block using a merge candidate that generates the minimum cost in the encoding of the prediction and residual blocks.

Further, the encoding apparatus 100 may determine whether to encode the target block using the merge mode.

2-3) Transmission of inter prediction information

The encoding apparatus 100 may generate a bitstream including inter prediction information required for inter prediction. The encoding apparatus 100 may generate entropy-encoded inter prediction information by performing entropy encoding on the inter prediction information, and may transmit a bitstream including the entropy-encoded inter prediction information to the decoding apparatus 200. The entropy-encoded inter prediction information may be signaled by the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may extract entropy-encoded inter prediction information from the bitstream, and may acquire the inter prediction information by applying entropy decoding to the entropy-encoded inter prediction information.

The decoding apparatus 200 may perform inter prediction on the target block using inter prediction information of the bitstream.

The inter prediction information may include 1) mode information indicating whether a merge mode is used, 2) a merge index, and 3) correction information.

In addition, the inter prediction information may include a residual signal.

The decoding apparatus 200 may acquire the merge index from the bitstream only when the mode information indicates that the merge mode is used.

The mode information may be a merge flag. The unit of mode information may be a block. The information about the block may include mode information, and the mode information may indicate whether a merge mode is applied to the block.

The merge index may indicate a merge candidate to be used for predicting the target block among the merge candidates included in the merge candidate list. Alternatively, the merge index may indicate a block to be merged with the target block among neighboring blocks spatially or temporally adjacent to the target block.

The encoding apparatus 100 may select a merge candidate having the highest encoding performance among the merge candidates included in the merge candidate list, and may set a value of a merge index to indicate the selected merge candidate.

The correction information may be information for correcting a motion vector. The encoding apparatus 100 may generate correction information. The decoding apparatus 200 may correct the motion vector of the merge candidate selected by the merge index based on the correction information.

The correction information may include at least one of information indicating whether correction is to be performed, correction direction information, and correction size information. The prediction mode of correcting the motion vector based on the signaled correction information may be referred to as a "merge mode with motion vector difference".

2-4) Inter prediction using a merge mode of inter prediction information

The decoding apparatus 200 may perform prediction on the target block using the merge candidate indicated by the merge index among the merge candidates included in the merge candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the merge index, the reference picture index, and the reference direction.

3) Skip mode

The skip mode may be a mode in which motion information of a spatial candidate or motion information of a temporal candidate is applied to a target block without change. In addition, the skip mode may be a mode in which a residual signal is not used. In other words, when the skip mode is used, the reconstructed block may be identical to the predicted block.

The difference between the merge mode and the skip mode is whether to transmit or use a residual signal. That is, the skip mode may be similar to the merge mode except that the residual signal is not transmitted or used.

When the skip mode is used, the encoding apparatus 100 may transmit information on a block whose motion information is to be used as motion information of a target block among blocks that are spatial candidates or temporal candidates to the decoding apparatus 200 through a bitstream. The encoding apparatus 100 may generate entropy-encoded information by performing entropy encoding on the information, and may signal the entropy-encoded information to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may extract entropy-encoded information from the bitstream, and may acquire the information by applying entropy decoding to the entropy-encoded information.

Further, when the skip mode is used, the encoding apparatus 100 may not transmit other syntax information (such as MVD) to the decoding apparatus 200. For example, when the skip mode is used, the encoding apparatus 100 may not signal syntax elements related to at least one of the MVD, the encoded block flag, and the transform coefficient level to the decoding apparatus 200.

3-1) Creation of merge candidate list

The skip mode may also use a merge candidate list. In other words, the merge candidate list may be used in both the merge mode and the skip mode. In this regard, the merge candidate list may also be referred to as a "skip candidate list" or a "merge/skip candidate list".

Alternatively, the skip mode may use an additional candidate list different from the candidate list of the merge mode. In this case, in the following description, the merge candidate list and the merge candidate may be replaced with a skip candidate list and a skip candidate, respectively.

The merge candidate list may be created before prediction in the skip mode is performed.

3-2) Searching for motion vectors using merge candidate list

The encoding apparatus 100 may determine a merge candidate to be used for encoding the target block. For example, the encoding apparatus 100 may perform prediction on the target block using the merge candidates in the merge candidate list. The encoding apparatus 100 may encode the target block using the merge candidate that generates the minimum cost in prediction.

Further, the encoding apparatus 100 may determine whether to encode the target block using the skip mode.

3-3) Transmission of inter prediction information

The inter prediction information may include 1) mode information indicating whether a skip mode is used and 2) a skip index.

The skip index may be the same as the merge index described above.

When the skip mode is used, the target block may be encoded without using a residual signal. The inter prediction information may not include a residual signal. Alternatively, the bitstream may not include a residual signal.

The decoding apparatus 200 may acquire the skip index from the bitstream only when the mode information indicates that the skip mode is used. As described above, the merge index and the skip index may be identical to each other. The decoding apparatus 200 may acquire the skip index from the bitstream only when the mode information indicates that the merge mode or the skip mode is used.

The skip index may indicate a merge candidate to be used for predicting the target block among the merge candidates included in the merge candidate list.

3-4) Inter prediction in skip mode using inter prediction information

The decoding apparatus 200 may perform prediction on the target block using the merge candidate indicated by the skip index among the merge candidates included in the merge candidate list.

The motion vector of the target block may be specified by the motion vector of the merge candidate indicated by the skip index, the reference picture index, and the reference direction.

4) Current picture reference mode

The current picture reference mode may represent a prediction mode using a previously reconstructed region in the target picture to which the target block belongs.

A motion vector for specifying a previously reconstructed region may be used. The reference picture index of the target block may be used to determine whether the target block has been encoded in the current picture reference mode.

A flag or index indicating whether the target block is a block encoded in the current picture reference mode may be signaled by the encoding apparatus 100 to the decoding apparatus 200. Alternatively, it may be inferred from the reference picture index of the target block whether the target block is a block encoded in the current picture reference mode.

When a target block is encoded in a current picture reference mode, the current picture may exist at a fixed position or at an arbitrary position in a reference picture list for the target block.

For example, the fixed position may be a position or a last position where the value of the reference picture index is 0.

When a target picture exists at an arbitrary position in the reference picture list, an additional reference picture index indicating such an arbitrary position may be signaled by the encoding apparatus 100 to the decoding apparatus 200.

5) Sub-block merge mode

The sub-block merging mode may be a mode in which motion information is derived from sub-blocks of the CU.

When the sub-block merge mode is applied, a sub-block merge candidate list may be generated using motion information of a co-located sub-block (col-sub-block) of a target sub-block (i.e., sub-block-based temporal merge candidate) in the reference image and/or affine control point motion vector merge candidates.

6) Triangle partition mode

In the triangle division mode, the target block may be divided in a diagonal direction, and the sub-target block generated by the division may be generated. For each sub-target block, motion information of the corresponding sub-target block may be derived, and the derived motion information may be used to derive prediction samples for each sub-target block. The prediction samples of the target block may be derived by a weighted sum of the prediction samples of the sub-target block generated via the partition.

7) Combining inter-intra prediction modes

The combined inter-intra prediction mode may be a mode in which a prediction sample of the target block is derived using a weighted sum of a prediction sample generated via inter prediction and a prediction sample generated via intra prediction.

In the above mode, the decoding apparatus 200 may autonomously correct the derived motion information. For example, the decoding apparatus 200 may search for motion information having a minimum Sum of Absolute Differences (SAD) in a specific region based on a reference block indicated by the derived motion information, and may derive the found motion information as corrected motion information.

In the above mode, the decoding apparatus 200 may use the optical stream to compensate for prediction samples derived via inter prediction.

In the AMVP mode, merge mode, skip mode, and the like described above, the motion information to be used for predicting the target block among the pieces of motion information in the list may be specified using the index information of the list.

In order to improve encoding efficiency, the encoding apparatus 100 may signal only an index of an element generating the minimum cost in inter prediction of a target block among elements in a list. The encoding apparatus 100 may encode the index and may signal the encoded index.

Therefore, it is necessary that the above-described list (i.e., the predicted motion vector candidate list and the merge candidate list) can be derived based on the same data by the encoding apparatus 100 and the decoding apparatus 200 using the same scheme. Here, the same data may include a reconstructed picture and a reconstructed block. Furthermore, in order to specify elements using indexes, the order of elements in the list must be fixed.

Fig. 10 illustrates spatial candidates according to an embodiment.

In fig. 10, the positions of the spatial candidates are shown.

The large block at the center of the graph may represent the target block. Five tiles may represent spatial candidates.

The coordinates of the target block may be (xP, yP), and the size of the target block may be represented by (nPSW, nPSH).

The spatial candidate a ₀ may be a block adjacent to the lower left corner of the target block. A ₀ may be a block that occupies pixels located at coordinates (xP-1, yP+nPSH).

Spatial candidate a ₁ may be a block adjacent to the left side of the target block. A ₁ may be the lowest block among the blocks adjacent to the left side of the target block. Alternatively, a ₁ may be a block adjacent to the top of a ₀. A ₁ may be a block that occupies a pixel located at coordinates (xP-1, yP+nPSH-1).

Spatial candidate B ₀ may be a block adjacent to the upper right corner of the target block. B ₀ may be a block that occupies a pixel located at coordinates (xp+ nPSW, yP-1).

Spatial candidate B ₁ may be a block adjacent to the top of the target block. B ₁ may be the rightmost block among the blocks adjacent to the top of the target block. Alternatively, B ₁ may be the block adjacent to the left side of B ₀. B ₁ can be a block that occupies a pixel located at coordinates (xP+ nPSW-1, yP-1).

The spatial candidate B ₂ may be a block adjacent to the upper left corner of the target block. B ₂ may be a block that occupies a pixel located at coordinates (xP-1, yP-1).

Determination of availability of spatial and temporal candidates

In order to include motion information of a spatial candidate or motion information of a temporal candidate in a list, it is necessary to determine whether the motion information of the spatial candidate or the motion information of the temporal candidate is available.

Hereinafter, the candidate block may include a spatial candidate and a temporal candidate.

The determination may be performed, for example, by sequentially applying the following steps 1) to 4).

Step 1) when a PU including a candidate block is located outside the boundary of a picture, the availability of the candidate block may be set to "false". The expression "availability is set to false" may have the same meaning as "set to unavailable".

Step 2) when the PU including the candidate block is located outside the boundary of the stripe, the availability of the candidate block may be set to "false". When the target block and the candidate block are located in different stripes, the availability of the candidate block may be set to "false".

Step 3) when the PU including the candidate block is located outside the boundary of the parallel block, the availability of the candidate block may be set to "false". When the target block and the candidate block are located in different parallel blocks, the availability of the candidate block may be set to "false".

Step 4) when the prediction mode of the PU including the candidate block is an intra prediction mode, the availability of the candidate block may be set to "false". When the PU including the candidate block does not use inter prediction, the availability of the candidate block may be set to "false".

Fig. 11 illustrates a sequence of adding motion information of a spatial candidate to a merge list according to an embodiment.

As shown in fig. 11, when pieces of motion information of space candidates are added to the merge list, the order of a ₁、B₁、B₀、A₀ and B ₂ may be used. That is, pieces of motion information of available spatial candidates may be added to the merge list in the order of a ₁、B₁、B₀、A₀ and B ₂.

Method for deriving merge list in merge mode and skip mode

As described above, the maximum number of merge candidates in the merge list may be set. "N" may be used to indicate the maximum number of settings. The set number may be transmitted from the encoding apparatus 100 to the decoding apparatus 200. The head of the tape may comprise N. In other words, the maximum number of merge candidates in the merge list for the target block of the stripe may be set by the stripe header. For example, the value of N may be substantially 5.

The pieces of motion information (i.e., merge candidates) may be added to the merge list in the order of the following steps 1) to 4).

Step 1) among the spatial candidates, the available spatial candidates may be added to the merge list. The pieces of motion information of the available spatial candidates may be added to the merge list in the order shown in fig. 10. Here, when the motion information of the available spatial candidate overlaps with other motion information already existing in the merge list, the motion information of the available spatial candidate may not be added to the merge list. The operation of checking whether the corresponding motion information overlaps with other motion information present in the list may be simply referred to as "overlap check".

The maximum number of motion information added may be N.

Step 2) when the number of pieces of motion information in the merge list is less than N and a time candidate is available, motion information of the time candidate may be added to the merge list. Here, when the motion information of the available time candidate overlaps with other motion information already existing in the merge list, the motion information of the available time candidate may not be added to the merge list.

Step 3) when the number of pieces of motion information in the merge list is less than N and the type of the target slice is "B", combined motion information generated by combining bi-prediction (bi-prediction) may be added to the merge list.

The target stripe may be a stripe that includes a target block.

The combined motion information may be a combination of L0 motion information and L1 motion information. The L0 motion information may be motion information referring to only the L0 reference picture list. The L1 motion information may be motion information referring to only the L1 reference picture list.

In the merge list, there may be one or more pieces of L0 motion information. Further, in the merge list, there may be one or more pieces of L1 motion information.

The combined motion information may include one or more pieces of combined motion information. When generating the combined motion information, L0 motion information and L1 motion information to be used for the step of generating the combined motion information among the one or more pieces of L0 motion information and the one or more pieces of L1 motion information may be predefined. One or more pieces of combined motion information may be generated in a predefined order via combined bi-prediction using a pair of different motion information in the merge list. One piece of motion information of the pair of different motion information may be L0 motion information, and the other piece of motion information of the pair of different motion information may be L1 motion information.

For example, the combined motion information added with the highest priority may be a combination of L0 motion information having a merge index 0 and L1 motion information having a merge index 1. When the motion information having the merge index 0 is not L0 motion information or when the motion information having the merge index 1 is not L1 motion information, the combination motion information may be neither generated nor added. Next, the combined motion information added with the next priority may be a combination of L0 motion information having a merge index 1 and L1 motion information having a merge index 0. The detailed combinations that follow may be consistent with other combinations in the video encoding/decoding arts.

Here, when the combined motion information overlaps with other motion information already existing in the merge list, the combined motion information may not be added to the merge list.

Step 4) when the number of pieces of motion information in the merge list is less than N, motion information of a zero vector may be added to the merge list.

The zero vector motion information may be motion information in which the motion vector is a zero vector.

The number of zero vector motion information may be one or more. The reference picture indexes of one or more pieces of zero vector motion information may be different from each other. For example, the value of the reference picture index of the first zero vector motion information may be 0. The value of the reference picture index of the second zero vector motion information may be 1.

The number of zero vector motion information pieces may be the same as the number of reference pictures in the reference picture list.

The reference direction of the zero vector motion information may be bi-directional. Both motion vectors may be zero vectors. The number of zero vector motion information may be the smaller one of the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1. Alternatively, when the number of reference pictures in the reference picture list L0 and the number of reference pictures in the reference picture list L1 are different from each other, the reference direction as one direction may be used for a reference picture index applicable to only a single reference picture list.

The encoding apparatus 100 and/or the decoding apparatus 200 may then add zero vector motion information to the merge list while changing the reference picture index.

When the zero vector motion information overlaps with other motion information already present in the merge list, the zero vector motion information may not be added to the merge list.

The order of steps 1) to 4) described above is merely exemplary and may be changed. Furthermore, some of the above steps may be omitted according to predefined conditions.

Method for deriving a list of predicted motion vector candidates in AMVP mode

The maximum number of predicted motion vector candidates in the predicted motion vector candidate list may be predefined. The predefined maximum number may be indicated with N. For example, the predefined maximum number may be 2.

A plurality of pieces of motion information (i.e., predicted motion vector candidates) may be added to the predicted motion vector candidate list in the order of the following steps 1) to 3).

Step 1) may add an available spatial candidate among the spatial candidates to the predicted motion vector candidate list. The spatial candidates may include a first spatial candidate and a second spatial candidate.

The first spatial candidate may be one of a ₀、A₁, scaled a ₀, and scaled a ₁. The second spatial candidate may be one of B ₀、B₁、B₂, scaled B ₀, scaled B ₁, and scaled B ₂.

The plurality of pieces of motion information of the available spatial candidates may be added to the predicted motion vector candidate list in the order of the first spatial candidate and the second spatial candidate. In this case, when the motion information of the available spatial candidate overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information of the available spatial candidate may not be added to the predicted motion vector candidate list. In other words, when the value of N is 2, if the motion information of the second spatial candidate is the same as the motion information of the first spatial candidate, the motion information of the second spatial candidate may not be added to the predicted motion vector candidate list.

The maximum number of motion information added may be N.

Step 2) when the number of pieces of motion information in the predicted motion vector candidate list is less than N and the temporal candidate is available, the motion information of the temporal candidate may be added to the predicted motion vector candidate list. In this case, when the motion information of the available time candidate overlaps with other motion information already existing in the predicted motion vector candidate list, the motion information of the available time candidate may not be added to the predicted motion vector candidate list.

Step 3) zero vector motion information may be added to the predicted motion vector candidate list when the number of pieces of motion information in the predicted motion vector candidate list is less than N.

The zero vector motion information may include one or more pieces of zero vector motion information. The reference picture indexes of the one or more pieces of zero vector motion information may be different from each other.

The encoding apparatus 100 and/or the decoding apparatus 200 may sequentially add a plurality of pieces of zero vector motion information to the predicted motion vector candidate list while changing the reference picture index.

When the zero vector motion information overlaps with other motion information already present in the predicted motion vector candidate list, the zero vector motion information may not be added to the predicted motion vector candidate list.

The description of zero vector motion information made in connection with the merge list above is also applicable to zero vector motion information. A repetitive description thereof will be omitted.

The order of steps 1) to 3) described above is merely exemplary and may be changed. Furthermore, some of the steps may be omitted according to predefined conditions.

Fig. 12 illustrates a transform and quantization process according to an example.

As shown in fig. 12, the level of quantization may be generated by performing a transform and/or quantization process on the residual signal.

The residual signal may be generated as a difference between the original block and the predicted block. Here, the prediction block may be a block generated via intra prediction or inter prediction.

The residual signal may be transformed into a signal in the frequency domain by a transformation process that is part of the quantization process.

The transform kernels for the transforms may include various DCT kernels, such as Discrete Cosine Transform (DCT) type 2 (DCT-II) and Discrete Sine Transform (DST) kernels.

These transform kernels may perform separable transforms or two-dimensional (2D) non-separable transforms on the residual signal. The separable transform may be a transform indicating that a one-dimensional (1D) transform is performed on the residual signal in each of the horizontal direction and the vertical direction.

The DCT types and DST types adaptively used for 1D transformation may include DCT-V, DCT-VIII, DST-I, and DST-VII in addition to DCT-II, as shown in each of the following tables 3 and 4.

TABLE 3 Table 3

TABLE 4 Table 4

Transform set	Transform candidates
		0	DST-VII,DCT-VIII,DST-I
1	DST-VII,DST-I,DCT-VIII
		2	DST-VII,DCT-V,DST-I

As shown in tables 3 and 4, when deriving the DCT type or DST type to be used for transformation, a transformation set may be used. Each transformation set may include a plurality of transformation candidates. Each transform candidate may be of the DCT type or the DST type.

Table 5 below shows examples of a transform set to be applied to a horizontal direction and a transform set to be applied to a vertical direction according to an intra prediction mode.

TABLE 5

In table 5, the numbers of the vertical transform set and the horizontal transform set to be applied to the horizontal direction of the residual signal according to the intra prediction mode of the target block are shown.

As illustrated in fig. 4 and 5, a set of transforms to be applied to the horizontal direction and the vertical direction may be predefined according to an intra prediction mode of a target block. The encoding apparatus 100 may perform transformation and inverse transformation on the residual signal using the transforms included in the transform set corresponding to the intra prediction mode of the target block. Further, the decoding apparatus 200 may perform inverse transformation on the residual signal using transforms included in a transform set corresponding to the intra prediction mode of the target block.

In the transformation and inverse transformation, as illustrated in tables 3, 4, and 5, a transformation set to be applied to the residual signal may be determined and may not be signaled. The transformation indicating information may be signaled from the encoding apparatus 100 to the decoding apparatus 200. The transform instruction information may be information indicating which one of a plurality of transform candidates included in a transform set to be applied to the residual signal is used.

For example, when the size of the target block is 64×64 or less, a total of three transform sets may be configured according to the intra prediction mode. The optimal transformation method may be selected from a total of nine transformation methods generated by a combination of three transformations in the horizontal direction and three transformations in the vertical direction. By such an optimal transformation method, the residual signal may be encoded and/or decoded, and thus encoding efficiency may be improved.

Here, the information indicating which of the plurality of transforms belonging to each transform set has been used for at least one of the vertical transform and the horizontal transform may be entropy encoded and/or entropy decoded. Here, truncated unary binarization may be used to encode and/or decode such information.

As described above, a method using various transforms may be applied to a residual signal generated via intra prediction or inter prediction.

The transformation may include at least one of a first transformation and a secondary transformation. The transform coefficients may be generated by performing a first transform on the residual signal, and the secondary transform coefficients may be generated by performing a secondary transform on the transform coefficients.

The first transformation may be referred to as a "primary transformation". Further, the first transform may also be referred to as an "adaptive multi-transform (AMT) scheme". As described above, AMT may represent the application of different transformations to the respective 1D directions (i.e., vertical and horizontal directions).

The secondary transform may be a transform for increasing the energy concentration of transform coefficients generated by the first transform. Similar to the first transformation, the secondary transformation may be a separable transformation or a non-separable transformation. Such an inseparable transformation may be an inseparable secondary transformation (NSST).

The first transformation may be performed using at least one of a predefined plurality of transformation methods. For example, the predefined plurality of transform methods may include Discrete Cosine Transform (DCT), discrete Sine Transform (DST), karhunen-Loeve transform (KLT), and the like.

Further, the first transform may be a transform having various types according to a kernel function defining a Discrete Cosine Transform (DCT) or a Discrete Sine Transform (DST).

For example, the transform type may be determined based on at least one of 1) a prediction mode of the target block (e.g., one of intra prediction and inter prediction), 2) a size of the target block, 3) a shape of the target block, 4) an intra prediction mode of the target block, 5) a component of the target block (e.g., one of a luma component and a chroma component), and 6) a partition type (e.g., one of a quadtree, a binary tree, and a trigeminal tree) applied to the target block.

For example, according to the transform kernel presented in Table 6 below, the first transform may include transforms such as DCT-2, DCT-5, DCT-7, DST-1, DST-8, and DCT-8. In table 6 below, various transformation types and transformation kernel functions for transformation selection (MTS) are illustrated.

MTS may refer to the selection of a combination of one or more DCT and/or DST kernels to transform the residual signal in the horizontal and/or vertical directions.

TABLE 6

In Table 6, i and j may be integer values equal to or greater than 0 and less than or equal to N-1.

The secondary transform may be performed on transform coefficients generated by performing the first transform.

As in the first transform, a set of transforms may also be defined in the secondary transform. The method for deriving and/or determining the set of transforms described above may be applied not only to the first transform but also to the secondary transform.

The first transform and the secondary transform may be determined for a particular objective.

For example, the first transform and the secondary transform may be applied to signal components corresponding to one or more of a luminance (luma) component and a chrominance (chroma) component. Whether to apply the first transform and/or the secondary transform may be determined according to at least one of the encoding parameters for the target block and/or the neighboring blocks. For example, whether to apply the first transform and/or the secondary transform may be determined according to the size and/or shape of the target block.

In the encoding apparatus 100 and the decoding apparatus 200, transformation information indicating a transformation method to be used for a target may be derived by using the specification information.

For example, the transform information may include a transform index to be used for the primary transform and/or the secondary transform. Alternatively, the transformation information may indicate that the primary transformation and/or the secondary transformation is not used.

For example, when the target of the primary transform and the secondary transform is a target block, a transform method indicated by the transform information to be applied to the primary transform and/or the secondary transform may be determined according to at least one of the encoding parameters for the target block and/or blocks adjacent to the target block.

Alternatively, transformation information indicating a transformation method for a specific target may be signaled from the encoding apparatus 100 to the decoding apparatus 200.

For example, for a single CU, whether to use the primary transform, an index indicating the primary transform, whether to use the secondary transform, and an index indicating the secondary transform may be derived as transform information by the decoding apparatus 200. Alternatively, for a single CU, transformation information may be signaled indicating whether a primary transformation is used, an index indicating the primary transformation, whether a secondary transformation is used, and an index indicating the secondary transformation.

Quantized transform coefficients (i.e., quantized levels) may be generated by performing quantization on a result generated by performing the first transform and/or the secondary transform or performing quantization on a residual signal.

Fig. 13 illustrates a diagonal scan according to an example.

Fig. 14 illustrates a horizontal scan according to an example.

Fig. 15 illustrates a vertical scan according to an example.

The quantized transform coefficients may be scanned via at least one of (upper right) diagonal scan, vertical scan, and horizontal scan according to at least one of an intra prediction mode, a block size, and a block shape. The block may be a Transform Unit (TU).

Each scan may be initiated at a particular start point and may terminate at a particular end point.

For example, quantized transform coefficients may be changed into a 1D vector form by scanning coefficients of a block using the diagonal scan of fig. 13. Alternatively, the horizontal scan of fig. 14 or the vertical scan of fig. 15 may be used according to the size of a block and/or an intra prediction mode, instead of using a diagonal scan.

The vertical scanning may be an operation of scanning the 2D block type coefficients in the column direction. The horizontal scanning may be an operation of scanning the 2D block type coefficients in the row direction.

In other words, which of the diagonal scan, the vertical scan, and the horizontal scan is to be used may be determined according to the size of the block and/or the inter prediction mode.

As shown in fig. 13, 14, and 15, the quantized transform coefficients may be scanned in a diagonal direction, a horizontal direction, or a vertical direction.

The quantized transform coefficients may be represented by a block shape. Each block may include a plurality of sub-blocks. Each sub-block may be defined according to a minimum block size or a minimum block shape.

In the scanning, a scanning order according to the type or direction of scanning may be first applied to the sub-blocks. Further, a scan order according to a scan direction may be applied to quantized transform coefficients in each sub-block.

For example, as shown in fig. 13, 14 and 15, when the size of the target block is 8×8, quantized transform coefficients may be generated by first transforming, secondary transforming and quantizing the residual signal of the target block. Thus, one of three types of scan orders may be applied to four 4×4 sub-blocks, and quantized transform coefficients may also be scanned for each 4×4 sub-block according to the scan order.

The encoding apparatus 100 may generate entropy-encoded quantized transform coefficients by performing entropy encoding on the scanned quantized transform coefficients, and may generate a bitstream including the entropy-encoded quantized transform coefficients.

The decoding apparatus 200 may extract the entropy-encoded quantized transform coefficients from the bitstream, and may generate the quantized transform coefficients by performing entropy decoding on the entropy-encoded quantized transform coefficients. The quantized transform coefficients may be arranged in the form of 2D blocks via inverse scanning. Here, as a method of inverse scanning, at least one of upper right diagonal scanning, vertical scanning, and horizontal scanning may be performed.

In the decoding apparatus 200, inverse quantization may be performed on the quantized transform coefficients. The secondary inverse transform may be performed on a result generated by performing the inverse quantization according to whether the secondary inverse transform is performed. Further, the first inverse transform may be performed on a result generated by performing the secondary inverse transform according to whether the first inverse transform is to be performed. The reconstructed residual signal may be generated by performing a first inverse transform on a result generated via performing a secondary inverse transform.

For a luma component reconstructed via intra prediction or inter prediction, inverse mapping with dynamic range may be performed before loop filtering.

The dynamic range may be divided into 16 equal segments and the mapping function of the corresponding segments may be signaled. Such mapping functions may be signaled at the stripe level or parallel block group level.

An inverse mapping function for performing the inverse mapping may be derived based on the mapping function.

Loop filtering, storage of reference pictures, and motion compensation may be performed in the inverse mapping region.

A prediction block generated via inter prediction may be transformed into a mapping region by mapping using a mapping function, and the transformed prediction block may be used to generate a reconstructed block. However, since intra prediction is performed in the mapping region, a prediction block generated via intra prediction may be used to generate a reconstructed block without mapping and/or inverse mapping.

For example, when the target block is a residual block of a chrominance component, the residual block may be transformed to an inverse mapping region by scaling the chrominance component of the mapping region.

Whether scaling is available may be signaled at the stripe level or parallel block group level.

For example, scaling may be applied only to the case where the mapping is available for the luma component and the partitions of the chroma component follow the same tree structure.

Scaling may be performed based on an average of values of samples in the luma prediction block corresponding to the chroma prediction block. Here, when the target block uses inter prediction, the luminance prediction block may represent a mapped luminance prediction block.

The value required for scaling may be derived by referencing a look-up table using the index of the segment to which the average of the sample values of the luma prediction block belongs.

The residual block may be transformed to the inverse mapping region by scaling the residual block using the finally derived value. Thereafter, for blocks of the chrominance component, reconstruction, intra prediction, inter prediction, loop filtering, and storage of reference pictures may be performed in the inverse mapping region.

For example, information indicating whether a mapping and/or inverse mapping of the luma component and the chroma component is available may be signaled through the sequence parameter set.

A prediction block of the target block may be generated based on the block vector. The block vector may indicate a displacement between the target block and the reference block. The reference block may be a block in the target image.

In this way, the prediction mode in which the prediction block is generated by referring to the target image may be referred to as an "Intra Block Copy (IBC) mode".

The IBC mode may be applied to CUs having a specific size. For example, the IBC mode may be applied to mxn CUs. Here, M and N may be less than or equal to 64.

IBC mode may include skip mode, merge mode, AMVP mode, and the like. In the case of the skip mode or the merge mode, a merge candidate list may be configured and a merge index may be signaled, and thus a single merge candidate may be designated among the merge candidates existing in the merge candidate list. The block vector of the specified merge candidate may be used as the block vector of the target block.

In the case of AMVP mode, differential block vectors may be signaled. Further, a prediction block vector may be derived from a left neighboring block and an upper neighboring block of the target block. Furthermore, an index indicating which neighboring block is to be used may be signaled.

The prediction block in IBC mode may be included in the target CTU or left CTU and may be limited to a block within the previously reconstructed region. For example, the value of the block vector may be limited such that the predicted block of the target block is located in a specific region. The specific region may be a region defined by three 64×64 blocks encoded and/or decoded before the 64×64 blocks including the target block. Limiting the values of the block vectors in this way may therefore reduce memory consumption and device complexity caused by implementation of IBC mode.

Fig. 16 is a configuration diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1600 may correspond to the encoding apparatus 100 described above.

The encoding apparatus 1600 may include a processing unit 1610, a memory 1630, a User Interface (UI) input device 1650, a UI output device 1660, and a storage 1640 in communication with each other through a bus 1690. The encoding device 1600 may also include a communication unit 1620 connected to the network 1699.

The processing unit 1610 may be a Central Processing Unit (CPU) or semiconductor device for executing processing instructions stored in memory 1630 or memory 1640. The processing unit 1610 may be at least one hardware processor.

The processing unit 1610 may generate and process signals, data, or information input to the encoding apparatus 1600, output from the encoding apparatus 1600, or used in the encoding apparatus 1600, and may perform checking, comparing, determining, etc. related to the signals, data, or information. In other words, in embodiments, the generation and processing of data or information, as well as the checking, comparing, and determining related to the data or information, may be performed by the processing unit 1610.

The processing unit 1610 may include an inter prediction unit 110, an intra prediction unit 120, a switcher 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy coding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

At least some of the inter prediction unit 110, the intra prediction unit 120, the switcher 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy encoding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may be program modules and may communicate with external devices or systems. The program modules may be included in the encoding device 1600 in the form of an operating system, application program modules, or other program modules.

The program modules may be physically stored in various types of well known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the encoding apparatus 1600.

Program modules may include, but are not limited to, routines, subroutines, programs, objects, components, and data structures for performing functions or operations according to embodiments or for implementing abstract data types according to embodiments.

The program modules may be implemented using instructions or code executed by at least one processor of the encoding apparatus 1600.

The processing unit 1610 may run instructions or codes in the inter prediction unit 110, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, the entropy coding unit 150, the inverse quantization unit 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190.

The memory unit may represent the memory 1630 and/or the storage 1640. Each of memory 1630 and storage 1640 may be any of various types of volatile or non-volatile storage media. For example, memory 1630 may include at least one of Read Only Memory (ROM) 1631 and Random Access Memory (RAM) 1632.

The storage unit may store data or information for operation of the encoding apparatus 1600. In an embodiment, data or information of the encoding apparatus 1600 may be stored in a storage unit.

For example, the storage unit may store pictures, blocks, lists, motion information, inter prediction information, bitstreams, and the like.

The encoding apparatus 1600 may be implemented in a computer system that includes a computer readable storage medium.

The storage medium may store at least one module required for operation of the encoding apparatus 1600. The memory 1630 may store at least one module and may be configured to cause the at least one module to be executed by the processing unit 1610.

Functions related to communication of data or information of the encoding apparatus 1600 may be performed by the communication unit 1620.

For example, the communication unit 1620 may transmit a bit stream to a decoding apparatus 1700 to be described later.

Fig. 17 is a configuration diagram of a decoding apparatus according to an embodiment.

The decoding apparatus 1700 may correspond to the decoding apparatus 200 described above.

Decoding apparatus 1700 may include a processing unit 1710, a memory 1730, a User Interface (UI) input device 1750, a UI output device 1760, and a memory 1740 in communication with each other via a bus 1790. Decoding device 1700 may also include a communication unit 1720 connected to a network 1799.

The processing unit 1710 may be a Central Processing Unit (CPU) or a semiconductor device for executing processing instructions stored in the memory 1730 or the memory 1740. The processing unit 1710 may be at least one hardware processor.

The processing unit 1710 may generate and process a signal, data, or information input to the decoding apparatus 1700, output from the decoding apparatus 1700, or used in the decoding apparatus 1700, and may perform checking, comparing, determining, or the like related to the signal, data, or information. In other words, in an embodiment, the generation and processing of data or information, as well as the checking, comparing, and determining related to the data or information, may be performed by the processing unit 1710.

The processing unit 1710 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transformation unit 230, an intra prediction unit 240, an inter prediction unit 250, a switch 245, an adder 255, a filter unit 260, and a reference picture buffer 270.

At least some of the entropy decoding unit 210, the inverse quantization unit 220, the inverse transformation unit 230, the intra prediction unit 240, the inter prediction unit 250, the adder 255, the switch 245, the filter unit 260, and the reference picture buffer 270 of the decoding apparatus 200 may be program modules and may communicate with external devices or systems. The program modules may be included in the decoding device 1700 in the form of an operating system, application program modules, or other program modules.

Program modules may be physically stored in various types of well known storage devices. Furthermore, at least some of the program modules may also be stored in a remote storage device capable of communicating with the decoding apparatus 1700.

The program modules may be implemented using instructions or code executed by at least one processor of decoding device 1700.

The processing unit 1710 may run instructions or codes in the entropy decoding unit 210, the inverse quantization unit 220, the inverse transformation unit 230, the intra prediction unit 240, the inter prediction unit 250, the switch 245, the adder 255, the filter unit 260, and the reference picture buffer 270.

The memory unit may represent the memory 1730 and/or the memory 1740. Each of memory 1730 and memory 1740 may be any of various types of volatile or non-volatile storage media. For example, memory 1730 may include at least one of ROM 1731 and RAM 1732.

The storage unit may store data or information for the operation of the decoding apparatus 1700. In an embodiment, data or information of the decoding apparatus 1700 may be stored in a storage unit.

Decoding device 1700 may be implemented in a computer system that includes a computer-readable storage medium.

The storage medium may store at least one module required for the operation of the decoding apparatus 1700. The memory 1730 may store at least one module and may be configured to cause the at least one module to be executed by the processing unit 1710.

Functions related to communication of data or information of the decoding apparatus 1700 may be performed by the communication unit 1720.

For example, the communication unit 1720 may receive a bitstream from the encoding device 1700.

Hereinafter, the processing unit may represent the processing unit 1610 of the encoding device 1600 and/or the processing unit 1710 of the decoding device 1700. For example, with respect to the prediction related functionality, the processing unit may represent switch 115 and/or switch 245. Regarding functions related to inter prediction, the processing unit may represent the inter prediction unit 110, the subtractor 125, and the adder 175, and may represent the inter prediction unit 250 and the adder 255. Regarding functions related to intra prediction, the processing unit may represent the intra prediction unit 120, the subtractor 125, and the adder 175, and may represent the intra prediction unit 240 and the adder 255. Regarding functions related to transformation, the processing unit may represent the transformation unit 130 and the inverse transformation unit 170, and may represent the inverse transformation unit 230. Regarding functions related to quantization, the processing unit may represent the quantization unit 140 and the inverse quantization unit 160, and may instruct the inverse quantization unit 220. With respect to functions related to entropy encoding and/or entropy decoding, the processing unit may represent entropy encoding unit 150 and/or entropy decoding unit 210. With respect to functions related to filtering, the processing unit may represent the filter unit 180 and/or the filter unit 260. Regarding functions related to reference pictures, the processing unit may instruct the reference picture buffer 190 and/or the reference picture buffer 270.

Prediction using mixed reference lines (BRLs)

Fig. 18 is a flowchart illustrating a target block prediction method and a bitstream generation method according to an embodiment.

The target block may be one of the blocks described in the embodiments. Alternatively, the target block may be one of a sequence, a slice, a parallel block, a tree, a macroblock, and a block having a specific size.

The target block prediction method and the bit stream generation method according to the embodiment may be performed by the encoding apparatus 1600. Embodiments may be part of a target block encoding method or a video encoding method.

In step 1810, the processing unit 1610 may determine prediction information to be applied to encoding of a target block.

The prediction information may include information for prediction described in the embodiment. For example, the prediction information may include inter prediction information. For example, the prediction information may include intra prediction information.

For example, the prediction information may include coding parameters. The prediction information may include BRL information, a number of an intra prediction mode, a block size of a target block, a direction of the intra prediction mode, a reference line index, and the like.

In step 1820, the processing unit 1610 may perform prediction on the target block using the information about the target block and the determined prediction information.

The prediction block may be generated by prediction of the target block.

A residual block, which is a difference between the target block and the prediction block, may be generated. Information about the target block may be generated by applying a transform and quantization to the residual block.

The information about the target block may include transform and quantization coefficients for the target block. The information about the target block may include prediction information.

Furthermore, a reconstructed block may be generated that is a sum of the prediction block and the reconstructed residual block.

In step 1830, the processing unit 1610 may generate a bitstream.

The bitstream may include information about the target block. Furthermore, the bitstream may include the information described in the embodiments. For example, the bitstream may include encoding parameters.

The information included in the bitstream may be generated in step 1830 or may be generated at least partially in steps 1810 and 1820.

The processing unit 1610 may store the generated bit stream in the memory 1640. Alternatively, the communication unit 1620 may transmit the bit stream to the decoding apparatus 1700.

The bitstream may include encoded information about the target block. The processing unit 1610 may generate encoded information about the target block by performing entropy encoding on the information about the target block.

Fig. 19 is a flowchart of a target block prediction method using a bitstream according to an embodiment.

The target block prediction method using a bitstream according to an embodiment may be performed by the decoding apparatus 1700. Embodiments may be part of a target block decoding method or a video decoding method.

In step 1910, the communication unit 1720 may obtain a bit stream. The communication unit 1720 may receive the bitstream from the encoding device 1600.

The bitstream may include information about the target block.

Furthermore, the bitstream may include the information described in the embodiments. For example, the bitstream may include encoding parameters.

The computer-readable storage medium may include a bitstream, and prediction and decoding of the target block may be performed using information about the target block included in the bitstream.

The bitstream may include encoded information about the target block. The processing unit 1710 may generate information on the target block by performing entropy decoding on the encoded information on the target block.

The processing unit 1710 may store the obtained bit stream in the memory 1740.

In step 1920, processing unit 1710 may determine the decoded prediction information to be applied to the target block.

The processing unit 1710 may determine prediction information using the methods described in the embodiments.

The processing unit 1710 may determine prediction information for the target block based on prediction method related information obtained from the bitstream.

The prediction information may include information for prediction described in the embodiment. For example, the prediction information may include inter prediction information. The prediction information may include intra prediction information.

In step 1930, the processing unit 1710 may perform prediction on the target block using the information about the target block and the determined prediction information.

In step 1930, a prediction block may be generated by performing prediction on the target block using the prediction information.

Fig. 20 is a flowchart of a prediction method in encoding according to an embodiment.

In step 2010, the processing unit 1610 of the encoding device 1600 may derive and/or select whether to use a hybrid reference line.

The processing unit 1610 may derive and/or select whether to use the hybrid reference line to predict the target block based on the encoding parameters. For example, the processing unit 1610 may derive and/or select whether to use the mixed reference line to predict the target block based on at least one of 1) BRL information, 2) a number of intra prediction modes, 3) a block size of the target block, and 4) a direction of the intra prediction modes.

In an embodiment, "mode number of intra prediction mode" may refer to "number of intra prediction mode". "number of a specific direction" or "mode number of a specific direction" may refer to "number indicating an intra prediction mode of a specific direction". For example, the "specific direction" may be a vertical direction or a horizontal direction.

In an embodiment, the block size of a block may be a value determined based on one or more attributes of the block, such as 1) a width of the block, 2) a height of the block, 3) a product of the width of the block and the height of the block, 4) a sum of the width of the block and the height of the block, 5) a larger of the width of the block and the height of the block, and 6) a smaller of the width of the block and the height of the block. But it may be the output value of a function that uses one or more properties of the block as input. Here, a function may include one or more functions, operations, formulas, and equations described in the embodiments.

The BRL information may be information indicating whether to use a mixed reference line. In an embodiment, the intra_luma_ BRL _flag may be a flag indicating BRL information. Alternatively, the intra_luma_ BRL _flag may be a syntax element that specifies BRL information. In an embodiment, the intra_luma_ BRL _flag may be replaced with BRL information.

In step 2020, the processing unit 1610 of the encoding device 1600 may derive and/or select a hybrid reference line.

The processing unit 1610 may derive and/or select a hybrid reference line based on the encoding parameters. For example, the processing unit 1610 may derive and/or select a hybrid reference line based on 1) the reference line index and 2) the BRL information.

A blended reference line may be generated when two or more reference lines are blended. The derivation and/or selection of a hybrid reference line may mean derivation and/or selection of two or more reference lines to be mixed.

In an embodiment, the reference lines to be mixed may refer to reference lines used as inputs for generating the mixed reference lines.

The processing unit 1610 may derive and/or select two or more reference lines to be mixed based on the encoding parameters. For example, the processing unit 1610 may derive and/or select two or more reference lines to be mixed based on 1) the reference line index and/or 2) BRL information.

The reference line index may indicate a reference line for prediction of the target block among a plurality of reference lines for the target block. In this sense, the reference line may be referred to as an intra prediction reference line, and the reference line index may be referred to as an intra prediction reference line index.

The reference line index may indicate two or more reference lines to be mixed. In an embodiment, intra_luma_ref_idx may refer to a reference line index. Alternatively, intra_luma_ref_idx may be a syntax element indicating a reference line index. In an embodiment, intra_luma_ref_idx may be replaced with a reference hint.

At step 2030, processing unit 1610 may generate a hybrid reference line sample.

A hybrid reference line sample may refer to one or more samples on a hybrid reference line.

The processing unit 1610 may generate a mixed reference line sample based on the encoding parameters and two or more reference lines to be mixed.

The processing unit 1610 may generate a mixed reference line sample based on at least one of 1) a distance between two or more reference lines to be mixed, 2) a number of intra prediction modes, 3) a number of two or more reference lines to be mixed, 4) a direction of the intra prediction modes, 5) a position of each sample on the mixed reference line, 6) whether the mixed direction is a horizontal direction, 7) whether the mixed direction is a vertical direction, and 8) a weight.

At step 2040, the processing unit 1610 may use the hybrid reference line to perform prediction on the target block. The processing unit may generate the prediction block by performing prediction on the target block using the hybrid reference line.

The prediction may include intra prediction. The prediction may be a prediction using intra prediction. In addition, the predictions may include other predictions described in the embodiments.

The processing unit 1610 may perform prediction on the target block using the hybrid reference line based on the encoding parameters.

The processing unit 1610 may perform prediction on the target block using the mixed reference line based on at least one of 1) the mixed reference line, 2) the direction of the intra prediction mode, 3) the position of each sample on the mixed reference line, 4) the number of the intra prediction mode, 5) the distance between the target sample and the mixed reference line, and 6) the distance between the upper left sample of the target block and the mixed reference line.

In an embodiment, the distance between the target sample point and the hybrid reference line may include one or more of 1) an x-axis distance between the target sample point and the hybrid reference line, and 2) a y-axis distance between the target sample point and the hybrid reference line.

In an embodiment, the x-axis distance may refer to the difference between the x-coordinates. The x-axis distance between the particular targets may be the distance between the x-coordinates of the particular targets. The distance may be a pixel distance. In other words, the x-axis distance may be the difference between the x-coordinates of the pixels in a particular target.

In an embodiment, the y-axis distance may refer to the difference between the y-coordinates. The y-axis distance between the particular objects may be the distance between the y-coordinates of the particular objects. The distance may be a pixel distance. In other words, the y-axis distance may be the difference between the y-coordinates of the pixels in a particular object.

In an embodiment, the upper left sample of the target block may indicate the upper left-most sample of the samples in the target block.

In an embodiment, the distance between the top left sample point of the target sample point and the hybrid reference line may include at least one of 1) an x-axis distance between the top left sample point of the target block and the hybrid reference line, and 2) a y-axis distance between the top left sample point of the target block and the hybrid reference line. The x-axis distance may refer to the difference between the x-coordinates. The y-axis distance may refer to the difference between the y-coordinates.

At least one of steps 2010, 2020, 2030 and 2040 may be selectively performed.

It may be determined whether to perform at least one of steps 2010, 2020, 2030 and 2040 based on the encoding parameters. For example, it may be determined whether to perform at least one of steps 2010, 2020, 2030 and 2040 based on 1) information about a picture, 2) information about a slice, 3) information about a parallel block, 4) a Quantization Parameter (QP), 5) a Coded Block Flag (CBF), 6) a block size of a block, 7) a depth of a block, 8) a shape of a block, 9) an entropy encoding/decoding method, 10) an intra prediction mode of a neighboring block, and 11) a temporal level.

In an embodiment, the block may be one of the blocks described in the embodiment, for example, a target block.

In an embodiment, the block may be a block having a specific size. The blocks may be blocks of a particular size and/or particular properties described in embodiments.

The shape of the block may include 1) whether the block is square, 2) whether the block is rectangular other than square, and 3) whether the block is non-square.

The order of steps 2010, 2020, 2030 and 2040 may be changed and at least one of steps 2010, 2020, 2030 and 2040 may be skipped.

Fig. 21 is a flowchart of a prediction method in decoding according to an embodiment.

At step 2110, the processing unit 1710 of the decoding apparatus 1700 may derive and/or select whether to use a hybrid reference line.

The processing unit 1710 may derive and/or select whether to use the hybrid reference line to predict the target block based on the encoding parameters. For example, the processing unit 1710 may derive and/or select whether to use the hybrid reference line to predict the target block based on at least one of 1) BRL information, 2) a number of intra-prediction modes, 3) a block size of the target block, and 4) a direction of the intra-prediction modes.

In an embodiment, the block size of a block may be a value determined based on one or more attributes of the block, such as 1) the width of the block, 2) the height of the block, 3) the product of the width of the block and the height of the block, 4) the sum of the width of the block and the height of the block, 5) the larger of the width of the block and the height of the block, and 6) the smaller of the width of the block and the height of the block, but it may be an output value of a function using one or more attributes of the block as input. Here, a function may include one or more functions, operations, formulas, and equations described in the embodiments.

At step 2120, the processing unit 1710 of the encoding device 1600 may derive and/or select a hybrid reference line.

The processing unit 1710 may derive and/or select a hybrid reference line based on the encoding parameters. For example, the processing unit 1710 may derive and/or select a hybrid reference line based on 1) the reference line index and 2) the BRL information.

In an embodiment, the reference line to be mixed may refer to a reference line used as an input for generating the mixed reference line.

The processing unit 1710 may derive and/or select two or more reference lines to be mixed based on the encoding parameters. For example, the processing unit 1710 may derive and/or select two or more reference lines to be mixed based on 1) the reference line index and/or 2) BRL information.

The reference line index may indicate two or more reference lines to be mixed. In an embodiment, intra_luma_ref_idx may refer to a reference line index. Alternatively, intra_luma_ref_idx may be a syntax element indicating a reference line index. In an embodiment, intra_luma_ref_idx may be replaced with a reference hint reference.

In step 2130, processing unit 1710 may generate a hybrid reference line sample.

The processing unit 1710 may generate a mixed reference line sample based on the encoding parameters and two or more reference lines to be mixed.

The processing unit 1710 may generate a mixed reference line sample based on at least one of 1) a distance between two or more reference lines to be mixed, 2) a number of intra prediction modes, 3) a number of two or more reference lines to be mixed, 4) a direction of the intra prediction modes, 5) a position of each sample on the mixed reference line, 6) whether the mixed direction is a horizontal direction, 7) whether the mixed direction is a vertical direction, and 8) a weight.

At step 2140, the processing unit 1710 may use the hybrid reference line to perform prediction on the target block. The processing unit may generate the prediction block by performing prediction on the target block using the hybrid reference line.

The processing unit 1710 may perform prediction on the target block using the hybrid reference line based on the encoding parameters.

The processing unit 1710 may perform prediction on the target block using the mixed reference line based on at least one of 1) the mixed reference line, 2) the direction of the intra prediction mode, 3) the position of each sample on the mixed reference line, 4) the number of the intra prediction mode, 5) the distance between the target sample and the mixed reference line, and 6) the distance between the upper left sample of the target block and the mixed reference line.

At least one of steps 2110, 2120, 2130 and 2140 may be optionally performed.

It may be determined whether to perform at least one of steps 2110, 2120, 2130 and 2140 based on the encoding parameters. For example, it may be determined whether to perform at least one of steps 2110, 2120, 2130, and 2140 based on 1) information about a picture, 2) information about a slice, 3) information about a parallel block, 4) a Quantization Parameter (QP), 5) a Coded Block Flag (CBF), 6) a block size of a block, 7) a depth of a block, 8) a shape of a block, 9) an entropy encoding/decoding method, 10) an intra prediction mode of a neighboring block, and 11) a temporal level.

In an embodiment, the block may be one of the blocks described in the embodiment, such as a target block.

The order of steps 2110, 2120, 2130 and 2140 may be changed and at least one of steps 2110, 2120, 2130 and 2140 may be skipped.

Definition of hybrid reference line (BRL)

Refer again to fig. 8.

The reference samples for intra prediction may not be limited to the samples just adjacent to the target block. For prediction of a target block, multiple Reference Lines (MRLs) may be used.

Intra-coding may use neighboring reference lines around a target block to generate a prediction signal. The adjacent reference lines may include reference line 0 through reference line 3.

Here, the adjacent reference lines around the target block may include reference lines 0 adjacent to the left side, upper left side, and upper side of the target block. The reference lines adjacent to the left, upper left, and upper side of each adjacent reference line of the target block may also be adjacent reference lines of the target block.

Alternatively, each adjacent reference line of the target block may be a reference line having an x-distance from the target block and a y-distance from the target block shorter than M. M may be the number of reference lines. That is, reference line 0 through reference line M-1 may be used for prediction of the target block.

As shown in fig. 8, for prediction of a target block, at least one of reference line 0 to reference line 3 may be used. Alternatively, in order to perform prediction on the target block, one of the reference line 0, the reference line 2, and the reference line 3 may be selected, and a prediction block of the target block may be generated using the selected reference line.

Each reference line of fig. 8 may include one or more reference points. Since the reference lines are numbered smaller, the corresponding reference lines may be lines closer to the reference sample point of the target block. Reference line 0 may be a line of reference points immediately adjacent to the target block. When the upper left coordinate of the target block is (X, Y), the horizontal length of the target block is W, and the vertical length thereof is H, the reference sample point on the reference line 0 may be a sample point whose X coordinate is X-1 or whose Y coordinate is Y-1. Here, the Y-coordinate of the reference sample point whose X-coordinate is X-1 may be Y-1 to y+2h. The X-coordinate of the reference sample point whose X-coordinate is Y-1 may be X-1 to X+2W. The reference sample point on reference line A may be a sample point whose X-coordinate is X-A-1 or whose Y-coordinate is Y-A-1. Here, the Y-coordinates of the reference sample points whose X-coordinates are X-A-1 may be Y-A-1 to Y+2H+A. The X-coordinate of the reference sample point whose X-coordinate is Y-A-1 may be X-A-1 to X+2W+A. A may be 1,2 or 3. Further, as described above, even when a is an integer of 4 or more, a sample point on the reference line a can be defined.

The samples in segment a and segment F may be derived by padding using the closest samples to segment B and segment E, respectively, rather than being obtained from reconstructed neighboring blocks.

The reference line index may indicate a reference line for intra prediction of the target block among a plurality of reference lines. For example, the reference line index may have one value from 0 to 3. Alternatively, the reference line index may indicate one of the reference line 0, the reference line 2, and the reference line 3. The reference line index may be signaled.

The available reference lines may be limited according to the encoding parameters and the properties of the target block. For example, when the top boundary of the target block is the boundary of the CTU, only reference line 0 may be available. Therefore, in this case, the reference line index may not be signaled, and the reference line 0 may be used.

In an embodiment, prediction using a hybrid reference line may generate a prediction block of a target block by mixing two or more neighboring reference lines of the target block.

In an embodiment, a hybrid reference line may mean that the hybrid reference line is derived/generated using a plurality of pieces of information about the reference line.

In an embodiment, deriving/generating the hybrid reference line by mixing the reference lines may mean deriving/generating the hybrid reference line using a weighted sum of the reference lines. In other words, mixing may mean applying a weighted sum.

For example, a prediction block of the target block may be generated using a mixed reference line generated by mixing two or more of the reference lines 0, 1, 2, and 3.

Derivation/selection of whether to use mixed reference lines

Fig. 22 illustrates directions of intra prediction modes of a target block according to an example.

In step 2010 and step 2110, whether the target block uses a hybrid reference line may be derived/selected. The prediction may be intra prediction or a prediction comprising intra prediction.

In an embodiment, "the target block uses the hybrid reference line" may mean that the hybrid reference line is used for prediction of the target block. Furthermore, "using a hybrid reference line" may mean that the hybrid reference line is used for prediction of the target block.

In an embodiment, "the target block does not use the hybrid reference line" may mean that the hybrid reference line is not used for prediction of the target block. Furthermore, "not using the hybrid reference line" may mean that the hybrid reference line is not used for prediction of the target block.

In an embodiment, "whether the target block uses the hybrid reference line" may mean "whether the hybrid reference line is used for prediction of the target block". Further, "whether a hybrid reference line is used" and "whether a hybrid reference line is used" may mean "whether a hybrid reference line is used for prediction of a target block".

Whether to use a hybrid reference line may be signaled by syntax elements of the bitstream. The syntax element may be BRL information. intra_luma_ BRL _flag may indicate BRL information.

The intra_luma_ brl _flag may specify whether to use a mixed reference line. Whether to use the hybrid reference line for the target block may be derived/selected by intra_luma_ brl _flag.

When the value of intra_luma_ brl _flag is the first value, the target block may use a mixed reference line.

When the value of intra_luma_ brl _flag is the second value, the target block may not use the mixed reference line.

When the intra_luma_ brl _flag is not present (in the bitstream), is not inserted (into the bitstream), or is not extracted (from the bitstream), it can be inferred that the target block uses the mixed reference line. Alternatively, when the intra_luma_ brl _flag is not present (in the bitstream), not inserted (into the bitstream), or not extracted (from the bitstream), it may be inferred that the target block does not use the mixed reference line.

In an embodiment, whether to use the hybrid reference line may be derived/selected based on at least one of the intra prediction mode and the block size of the target block.

In an embodiment, when a difference value indicating a difference between a direction θ of an intra prediction mode of a target block and a specific direction θ _s is less than a specific threshold Th, a hybrid reference line may be derived/selected for the target block. The threshold Th may be differently defined according to a block size of the target block.

That is, when the following equation 1 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 1]

|θ-θ_s|<Th

In an embodiment, the hybrid reference line may be derived/selected for the target block when the smaller of the difference between the direction θ of the intra prediction mode of the target block and the horizontal direction (i.e., 180 °) or the difference between the direction θ of the intra prediction mode of the target block and the vertical direction (i.e., 90 °) is less than a certain threshold Th. The threshold Th may be differently defined according to a block size of the target block.

That is, when the following equation 2 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 2]

min(|θ-90|,|θ-180|)<Th

In an embodiment, min (a, b) may be a function of the smaller of outputs a and b.

In an embodiment, when the difference between the number m of intra prediction modes of the target block and the specific intra prediction mode number m _s is less than the specific threshold Th, the target block may be derived/selected to use the hybrid reference line. The threshold Th may be differently defined according to a block size of the target block.

That is, when the following equation 3 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 3]

|m-m_s|<Th

In an embodiment, when the smaller of the difference between the intra prediction mode number m of the target block and the horizontal mode number m _h and the difference between the intra prediction mode number m of the target block and the vertical mode number m _v is less than the specific threshold Th, the target block may be derived/selected to use the hybrid reference line. The threshold Th may be differently defined according to a block size of the target block.

That is, when the following equation 4 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 4]

min(|m-m_h|,|m-m_v|)<Th

In an embodiment, the threshold Th may be determined according to a block size of the target block.

For example, the threshold Th may be differently defined according to block sizes, as shown in table 7 below.

TABLE 7

Block size (width x height)	4×4	8×8	16×16	32×32
					Th	T₁	T₂	T₃	T₄

Here, T _i may be a value equal to or greater than 0 (1.ltoreq.i.ltoreq.4)

For example, the threshold Th may be defined differently according to the product of the width and the height of the target block, as shown in table 8 below.

TABLE 8

Block size (width + height)	16	64	128	256
					Th	T₁	T₂	T₃	T₄

Here, T _i may be a value equal to or greater than 0 (1.ltoreq.i.ltoreq.4).

In an embodiment, the hybrid reference line may be derived/selected for the target block when the smaller of the difference between the direction θ of the intra prediction mode of the target block and the horizontal direction (i.e., 180 °) and the difference between the direction θ of the intra prediction mode of the target block and the vertical direction (i.e., 90 °) is less than the threshold Th according to the block size of the target block described in table 9 below.

TABLE 9

Block size (width x height)	4×4	8×8	16×16	32×32
					Th	0°	2°	5°	10°

That is, when the following equation 5 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 5]

min(|θ-90|,|θ-180|)<Th

For example, when the block size of the target block is 8×8 and the direction of the intra prediction mode of the target block is 95 °, it is possible to derive that the target block does not use the mixed reference line.

For example, when the block size of the target block is 32×32 and the direction of the intra prediction mode of the target block is 85 °, the target block use mixed reference line may be derived.

In an embodiment, when the smaller of the difference between the intra prediction mode number m of the target block and the mode number in the horizontal direction (i.e., m _h =18) and the difference between the intra prediction mode number m of the target block and the mode number in the vertical direction (i.e., m _v =50) is smaller than the threshold Th according to the block size of the target block, as described in table 10 below, the target block use mixed reference line may be derived/selected. The threshold Th may be differently defined according to a block size of the target block.

TABLE 10

That is, when the following equation 6 is established, the target block may be derived/selected to use the mixed reference line.

[ Equation 6]

min(|m-m_h|,|m-m_v|)<Th

For example, when the product of the width and the height of the target block is 64 and the intra prediction mode number of the target block is 15, it may be derived that the target block does not use the mixed reference line.

For example, when the product of the width and the height of the target block is 256 and the intra prediction mode number of the target block is 54, the target block use hybrid reference line may be derived.

Derivation/selection of hybrid reference lines

Fig. 23 shows N adjacent reference lines of a target block according to an example.

At steps 2020 and 2120, a hybrid reference line for the target block may be derived/selected.

Information indicating the hybrid reference line and/or information about the hybrid reference line may be signaled by syntax elements of the bitstream.

First embodiment of derivation/selection of Mixed reference lines

In an embodiment, intra_luma_ref_idx may specify a reference line index. The reference line of the target block can be derived/selected by intra_luma_ref_idx.

As shown in fig. 23, N reference lines adjacent to the target block may be referred to as first to nth reference lines. Here, the reference line 0 described above with reference to fig. 8 may correspond to the first reference line. Reference line N-1 of fig. 8 may correspond to the nth reference line in the embodiment.

The reference line of the target block may be determined from the value of intra_luma_ref_idx. For example, the first, second, third, and fourth values of intra_luma_ref_idx may be 0,1, 2, and 3, respectively.

When the value of intra_luma_ref_idx is the first value, a first reference line (e.g., reference line 0 of fig. 8 or the first reference line of fig. 23) may be used as the reference line of the target block.

When the value of intra_luma_ref_idx is the second value, a second reference line (e.g., reference line 2 of fig. 8 or the third reference line of fig. 23) may be used as the reference line of the target block.

When the value of intra_luma_ref_idx is the third value, a third reference line (e.g., reference line 3 of fig. 8 or the fourth reference line of fig. 23) may be used as the reference line of the target block.

When the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the p ₁ th reference line, the p ₂ th reference line, the..and the p _t th reference line may be used as the reference line of the target block.

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t), the following equation may be established. That is, p _i may be one of integers from 1 to N. t may be an integer equal to or greater than 1 and less than N. Alternatively, t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 7]

P _i = 1,2, or n.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line and the second reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the second reference line and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line, the third reference line, and the fourth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line, the second reference line, the third reference line, and the fourth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line, the third reference line, the fourth reference line, and the fifth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the third reference line, the fourth reference line, the fifth reference line, and the sixth reference line may be used as the reference line of the target block.

Second embodiment of derivation/selection of Mixed reference lines

The reference line of the target block may be determined from the value of intra_luma_ref_idx. For example, the first, second, third, fourth, and fifth values of intra_luma_ref_idx may be 0,1, 2, 3, and 4, respectively.

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t), the following equation 8 may be established. That is, p _i may be one of integers from 1 to N. t may be an integer equal to or greater than 1 and less than N. Alternatively, t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 8]

P _i = 1,2, or n.

When the value of intra_luma_ref_idx is the fifth value, a mixed reference line derived by mixing the q ₁ th reference line, the q ₂ th reference line, the..the q _u th reference line may be used as the reference line of the target block.

Here, for i equal to or greater than 1 and less than or equal to u (i.e., 1.ltoreq.i.ltoreq.u), the following equation 9 may be established. That is, q _i may be one of integers from 1 to N. Here, u may be an integer equal to or greater than 1 and less than N, alternatively u may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 9]

Q _i = 1,2, or N

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line and the second reference line may be used as the reference line of the target block. When the value of intra_luma_ref_idx is the fifth value, a mixed reference line derived by mixing the second reference line and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line and the second reference line may be used as the reference line of the target block. When the value of intra_luma_ref_idx is the fifth value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the second reference line and the third reference line may be used as the reference line of the target block. When the value of intra_luma_ref_idx is the fifth value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the fourth value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be used as the reference line of the target block. When the value of intra_luma_ref_idx is the fifth value, a mixed reference line derived by mixing the second reference line, the third reference line, and the fourth reference line may be used as the reference line of the target block.

Third embodiment for derivation/selection of Mixed reference lines

As shown in fig. 23, N reference lines adjacent to the target block may be referred to as first to nth reference lines. Here, the reference line 0 described above with reference to fig. 8 may correspond to the first reference line. The reference line N-1 of fig. 8 may correspond to the nth reference line.

The reference line of the target block may be determined from the value of intra_luma_ref_idx.

When the value of intra_luma_ref_idx is the j-th value, a mixed reference line derived by mixing the j-th reference line, the p ₁ -th reference line, the p ₂ -th reference line, the..the., the p _t -th reference line may be used as the reference line of the target block. Here, j may be equal to or greater than 1 and less than or equal to N (i.e., 1+.j+.n).

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t), the following equations 10 and 11 can be established. Here, t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 10]

P _i = 1,2, or n.

[ Equation 11]

P _i +.j (i.e., j is excluded from 1to N in equation 10)

For example, when the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line and the second reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the second value, a mixed reference line derived by mixing the second reference line and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line and the first reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the second value, a mixed reference line derived by mixing the second reference line, the third reference line, and the first reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the second value, a mixed reference line derived by mixing the second reference line, the third reference line, and the fourth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the second reference line, and the first reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the second reference line, and the fourth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the fourth reference line, and the fifth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line, the second reference line, the third reference line, and the fourth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the second value, a mixed reference line derived by mixing the second reference line, the third reference line, the fourth reference line, and the fifth reference line may be used as the reference line of the target block.

For example, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the fourth reference line, the fifth reference line, and the sixth reference line may be used as the reference line of the target block.

Fourth embodiment of derivation/selection of Mixed reference lines

When the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the j-th value, a mixed reference line derived by mixing the p ₁ -th reference line, the p ₂ -th reference line, the p _t -th reference line may be used as the reference line of the target block. Here, j may be equal to or greater than 1 and less than or equal to N (i.e., 1+.j+.n).

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1+.i+.t), the following equations 12 and 13 can be established. Here, t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 12]

P _i = 1,2, or n.

[ Equation 13]

P _i +.j (i.e., j is excluded from 1 to N in equation 12)

When the value of intra_luma_ brl _flag is the second value and the value of intra_luma_ref_idx is the j-th value, the j-th reference line may be used as a reference line of the target block. Here, j may be a value equal to or greater than 1 and less than or equal to N (i.e., 1+.j+.n).

For example, when the value of intra_luma_ brl _flag is the second value and the value of intra_luma_ref_idx is the first value, a first reference line (e.g., reference line 0 of fig. 8 or the first reference line of fig. 23) may be used as the reference line of the target block.

For example, when the value of intra_luma_ brl _flag is the second value and the value of intra_luma_ref_idx is the second value, the second reference line (e.g., reference line 2 of fig. 8 or the third reference line of fig. 23) may be used as the reference line of the target block.

For example, when the value of intra_luma_ brl _flag is the second value and the value of intra_luma_ref_idx is the third value, a third reference line (e.g., reference line 3 of fig. 8 or the fourth reference line of fig. 23) may be used as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line and the second reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the second reference line and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the first reference line and the second reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the second reference line and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the first reference line and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the first reference line and the second reference line may be taken as the reference line of the target block.

For example, in case that the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the second reference line and the third reference line may be taken as the reference line of the target block.

For example, in case that the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the first reference line and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the second reference line, the third reference line, and the fourth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the second reference line, the third reference line, and the fourth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the first reference line, the second reference line, and the third reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the second reference line, the third reference line, and the fourth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the fourth reference line, and the fifth reference line may be taken as the reference line of the target block.

Fifth embodiment of derivation/selection of Mixed reference lines

When the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the j-th value, a mixed reference line derived by mixing the j-th reference line, the p ₁ -th reference line, the p ₂ -th reference line, the..and the p _t -th reference line may be used as the reference line of the target block. Here, j may be a value equal to or greater than 1 and less than or equal to N (i.e., 1+.j+.n).

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1+.i+.t), the following equations 14 and 15 can be established. t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 14]

P _i = 1,2, or n.

[ Equation 15]

P _i +.j (i.e., j is excluded from 1 to N of equation 14)

For example, in case that the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a third value, a mixed reference line derived by mixing the third reference line and the first reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the second reference line, the third reference line, and the first reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the second reference line, and the first reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the second reference line, and the fourth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the first value, a mixed reference line derived by mixing the first reference line, the second reference line, the third reference line, and the fourth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is a first value and the value of intra_luma_ref_idx is a second value, a mixed reference line derived by mixing the second reference line, the third reference line, the fourth reference line, and the fifth reference line may be taken as the reference line of the target block.

For example, in the case where the value of intra_luma_ brl _flag is the first value and the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the fourth reference line, the fifth reference line, and the sixth reference line may be taken as the reference line of the target block.

Sixth embodiment of derivation/selection of Mixed reference lines

When the value of intra_luma_ref_idx is an r-th value, an r-th reference line may be used as a reference line of the target block. Here, j may be equal to or greater than 2 and less than or equal to N (i.e., 1+.r+.n, r+.1).

When the value of intra_luma_ref_idx is the j-th value, a mixed reference line derived by mixing the j-th reference line, the p ₁ -th reference line, the p ₂ -th reference line, the..once, and the p _t -th reference line may be used as the reference line of the target block. Here, j may be a value equal to or greater than 1 and less than or equal to N (i.e., 1+.j+.n, j+.1).

Here, for i equal to or greater than 1 and equal to or less than t (i.e., 1+.i+.t), the following equations 16, 17, and 18 may be established. Here, t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 16]

P _i = 1,2, or n.

[ Equation 17]

P _i +.j (i.e., j is excluded from 1 to N in equation 16)

[ Equation 18]

P _i +.r (i.e., r is excluded from 1 to N in equation 16)

Further, when the value of intra_luma_ref_idx is the second value, the second reference line may be used as the reference line of the target block.

Further, when the value of intra_luma_ref_idx is a third value, the third reference line may be used as a reference line of the target block.

For example, when the value of intra_luma_ref_idx is a first value, the first reference line may be used as a reference line for the target block.

Furthermore, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line and the first reference line may be used as the reference line of the target block.

Further, when the value of intra_luma_ref_idx is the third value, a mixed reference line derived by mixing the third reference line, the first reference line, and the second reference line may be used as the reference line of the target block.

Generation of mixed reference line samples

When a hybrid reference line is selected as the reference line for the target block in at least one of steps 2010 and 2110 and steps 2020 and 2120, a hybrid reference line sample may be generated.

In the embodiments, "p ₁" and "p1" may be used as the same meaning and may be used interchangeably with each other in the description, equations, etc. related to the reference lines. Furthermore, "p _i" and "pi" may be used in the same sense and may be used interchangeably with each other.

The hybrid reference line (BRL) may be generated by mixing at least two of the p ₁ th reference line RL _p1, the p ₂ th reference line RL _p2, the..and the p _t th reference line RL _pt, as shown in the following equation 19.

[ Equation 19]

BRL=w₁×RL_p1+w₂×RP_p2+…+w_t×RL_pt

W _i may be the weight value in the weighted sum. i may be an integer equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t).

The sum of the weights in the weighted sum may be 1.

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1+.i+.t), the following equation 20 can be established. t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 20]

P _i = 1,2, or n.

The sample value BRL (x, y) of the sample, which is located (x, y) in the mixed reference line, can be generated by mixing samples on at least two of the p ₁ th reference line RL _p1, the p ₂ th reference line RL _p2, the..and the p _t th reference line RL _pt, as shown in the following equation 21.

In an embodiment, the mixing of the samples may refer to a weighted sum of the samples.

[ Equation 21]

BRL(x,y)＝w₁×RL_p1(x_p1,y_p1)+w₂×RP_p2(x_p2,y_p2)+…+w_t×RL_pt(x_pt,y_pt)

The sum of the weights in the weighted sum may be 1.

Here, for i equal to or greater than 1 and less than or equal to t (i.e., 1+.i+.t), the following equation 22 can be established. t may be an integer equal to or greater than 1 and less than or equal to N.

[ Equation 22]

P _i = 1,2, or n.

RL _pi(x_pi,y_pi) can be the sample value of the sample whose position in the p _i reference line is (x _pi,y_pi).

The (x _pi,y_pi) in equation 21 may indicate the position of the sample point on the p _i th reference line RL _pi corresponding to the coordinates (x, y) as shown in equations 23 and 24. Here, i may be an integer equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t).

[ Equation 23]

x_pi＝f_pi(x,d^x _pi,d^y _pi,θ)

[ Equation 24]

y_pi＝g_pi(y,d^x _pi,d^y _pi,θ)

In equation 23, d ^x _pi can be the x-axis distance between the hybrid reference line and the p _i th reference line RL _pi. θ may be the direction of the intra prediction mode. The function f _pi(??) may calculate a value of x _pi corresponding to x by using at least one of x, d ^x _pi、d^y _pi, and θ.

In equation 24, d ^y _pi can be the y-axis distance between the hybrid reference line and the p _i th reference line RL _pi. θ may be the direction of the intra prediction mode. Function g _pi(??) may calculate a value of y _pi corresponding to y by using at least one of y, d ^x _pi、d^y _pi, and θ.

Determination of reference position of hybrid reference line

Determining the reference position of the hybrid reference line may refer to determining where the hybrid reference line is located. In other words, the determination of the reference position of the hybrid reference line may be a determination of the position of the hybrid reference line among the positions of the plurality of reference lines of the target block.

BRL may be determined to lie on the p _k th reference line RL _Pk. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x _pk,y_pk) of the p _k th reference line RL _Pk. That is, the following equation 25 may be established.

[ Equation 25]

(x,y)=(x_pk,y_pk)

In an embodiment, a case where the first coordinates of the BRL and the second coordinates of the specific reference line correspond to each other may mean that the sample point at the first coordinates of the BRL and the sample point at the second coordinates of the specific reference line correspond to each other. Alternatively, in an embodiment, a case where the first coordinate of the BRL and the second coordinate of the specific reference line correspond to each other may mean that a sample point at the first coordinate of the BRL replaces a sample point at the second coordinate of the specific reference line. Alternatively, a case where the first coordinate of the BRL and the second coordinate of the specific reference line correspond to each other may mean that a sample point at the first coordinate of the BRL is regarded as a sample point at the second coordinate of the specific reference line.

In other words, x and x _pk may be the same as each other, and y _pk may be the same as each other. Here, k may be a value equal to or greater than 1 and less than or equal to t (i.e., 1+.k+.t).

When x _pk falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 26 is established, equation 27 may be established.

[ Equation 26]

a≤x_pk≤b

[ Equation 27]

a≤x≤b

When y _pk falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 28 is established, equation 29 is established.

[ Equation 28]

c≤y_pk≤d

[ Equation 29]

c≤y≤d

Fig. 24 shows a case where the mixed reference line is located on the first reference line.

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 30 may be established.

[ Equation 30]

(x,y)=(x₁,y₁)

In other words, x and x ₁ may be the same as each other, and y ₁ may be the same as each other.

When x ₁ falls within a specific range, x may fall within the same range as the specific range. That is, when the following equation 31 is established, the equation 32 may be established.

[ Equation 31]

x_left≤x₁≤x_right

[ Equation 32]

x_left≤x≤x_right

When y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 33 is established, the equation 34 may be established.

[ Equation 33]

y_top≤y₁≤y_bottom

[ Equation 34]

y_top≤y≤y_bottom

Generating mixed reference line samples using horizontal or vertical mixing

Referring to equation 21, the location of a sample point on a reference line corresponding to the location of a sample point on the BRL may be determined.

When the following conditions 1 and 2 are satisfied, the position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x, y-d _top). That is, x _pi and x may be the same as each other, and y _pi and y-d _top may be the same as each other. Here, d _top may be the y-axis distance between BRL and the p _i th reference line RL _pi. d _top may be 0 or greater.

The distance between the [ condition 1] brl and the target block is less than or equal to the distance between the p _i th reference line RL _pi and the target block.

Condition 2 as shown in fig. 25, coordinates (x, y) are positions of the sample points on the horizontal line.

When the following conditions 3 and 4 are satisfied, the position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x, y+d _{i_y}). That is, x _pi and x may be the same as each other, and y _pi and y+d _{i_y} may be the same as each other. Here, d _{i_y} may be the y-axis distance between BRL and the p _i th reference line RL _pi. d _{i_y} may be equal to or greater than 0.

The distance between the [ condition 3] brl and the target block is equal to or greater than the distance between the p _i th reference line RL _pi and the target block.

Condition 4 as shown in fig. 25, the coordinates (x, y) are the positions of the sample points on the horizontal line.

The position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x-d _left, y) when the following conditions 5 and 6 are satisfied. That is, x _pi and x-d _left may be identical to each other, and y _pi and y may be identical to each other. Here, d _left may be the x-axis distance between BRL and the p _i th reference line RL _pi. d _left may be equal to or greater than 0.

The distance between the [ condition 5] brl and the target block is less than or equal to the distance between the p _i th reference line RL _pi and the target block.

Condition 6 as shown in fig. 25, coordinates (x, y) are positions of the sample points on the vertical line.

When the following conditions 7 and 8 are satisfied, the position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x+d _{i_x}, y). That is, x _pi and x+d _{i_x} may be the same as each other, and y _pi and y may be the same as each other. Here, d _{i_x} may be the x-axis distance between BRL and the p _i th reference line RL _pi. d _{i_x} may be equal to or greater than 0.

The distance between the [ condition 7] brl and the target block is equal to or greater than the distance between the p _i th reference line RL _pi and the target block.

Condition 8 as shown in fig. 25, coordinates (x, y) are positions of the sample points on the vertical line.

When the following conditions 9 and 10 are satisfied, the position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x-d _left,y-d_top). That is, x _pi and x-d _left may be identical to each other, and y _pi and y-d _top may be identical to each other. Here, d _left may be the x-axis distance between BRL and the p _i th reference line RL _pi. d _top may be the y-axis distance between BRL and the p _i th reference line RL _pi. d _top may be equal to or greater than 0.d _left may be equal to or greater than 0.

[ Condition 9] the distance between BRL and the target block is less than or equal to the distance between the p _i th reference line RL _pi and the target block.

Condition 10 as shown in fig. 25, the coordinates (x, y) are the positions of the sample points in the upper left corner.

When the following conditions 11 and 12 are satisfied, the position (x _pi,y_pi) of the sample point on the p _i th reference line RL _pi, which corresponds to the position (x, y) of the sample point on the BRL, may be (x+d _{i_x},y+d_{i_y}). That is, x _pi and x+d _{i_x} may be the same as each other, and y _pi and y+d _{i_y} may be the same as each other. Here, d _{i_x} may be the x-axis distance between BRL and the p _i th reference line RL _pi. d _{i_y} may be the y-axis distance between BRL and the p _i th reference line RL _pi. d _{i_x} may be equal to or greater than 0.d _{i_y} may be equal to or greater than 0.

The distance between the [ condition 11] brl and the target block is equal to or greater than the distance between the p _i th reference line RL _pi and the target block.

Condition 12 as shown in fig. 25, (x, y) is the position of the sample point in the upper left corner.

Generating mixed reference line samples using a mix according to directionality of intra prediction modes

Fig. 26 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the first direction.

Fig. 27 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the second direction.

Fig. 28 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the third direction.

Fig. 29 shows positions of samples on the reference line and positions of samples on the reference line mixed when the direction of the intra prediction mode is the fourth direction.

Referring to equation 21, the location of a sample point on the reference line corresponding to the location of each sample point on the BRL may be determined.

In fig. 26, 27, 28, and 29, a target pixel may indicate a target sample that is a target of encoding and/or decoding. In other words, a target pixel or target sample may refer to a pixel of a target of pixels in a target block that is processed as encoding and/or decoding.

In the embodiment, as shown in fig. 26, 27, 28, and 29, assuming that there is a straight line in the direction θ of the intra prediction mode from the position of the target sample point, the sample point position (x, y) of the sample point on the straight line among the sample points on the BRL and the sample point position (x _pi,y_pi) of the sample point on the straight line among the sample points on the p _i th reference line RL _pi may correspond to each other. The spots at the corresponding spot positions may correspond to each other. In other words, the corresponding samples on the BRL and p _i th reference lines RL _pi may refer to samples on a straight line in the direction θ of the intra prediction mode from the position of the target sample.

As described in equation 21, the samples on the BRL may be generated by mixing RP _pi(x_pi,y_pi). Here, i may be one of values equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t).

For i equal to or greater than 1 and less than or equal to t, the following equation 35 may be established. t may be less than or equal to N.

[ Equation 35]

P _i = 1,2, or n.

RP _pi(x_pi,y_pi) may be the p _i th reference line RL _pi sample value at the sample position (x _pi,y_pi) of the p _i th reference line RL _pi corresponding to the sample position (x, y) of BRL. In other words, RP _pi(x_pi,y_pi) may be a sample point whose coordinates are (x _pi,y_pi) among the sample points on the reference line RP _pi.

In the embodiment, as shown in fig. 26, 27, 28, and 29, assuming that there is a straight line in the direction θ of the intra prediction mode from the position of the target sample point, RP _pi(x_pi,y_pi) may be the sample value of the sample point on the straight line among the samples on the p _i -th reference line RL _pi. In other words, RP _pi(x_pi,y_pi) may be the p _i reference line sample value at the sample point (x _pi,y_pi).

As described in equation 21, the samples on BRL may be generated by mixing the sample values RP _pi(x_pi,y_pi). Here, i may be one of values equal to or greater than 1 and less than or equal to t (i.e., 1.ltoreq.i.ltoreq.t).

For i equal to or greater than 1 and less than or equal to t, the following equation 36 may be established. t may be a value less than or equal to N.

[ Equation 36]

P _i = 1,2, or n.

In an embodiment, when the direction of the intra prediction mode is θ and at least one of the following conditions 13, 14, and 15 is satisfied, the sample position (x _pi,y_pi) of the p _i th reference line RL _pi corresponding to the sample position (x, y) of BRL may be (x+d _{i_x_θ},y-d_top). In other words, the following equations 37 and 38 are satisfied.

The distance between the [ condition 13] brl and the target block is less than or equal to the distance between the p _i th reference line RL _pi and the target block.

As shown in fig. 26, it is assumed that there is a straight line in the direction θ of the intra prediction mode from the coordinates (x, y) that intersects with a sample point on the horizontal line of the p _i th reference line RL _pi.

Condition 15 as shown in fig. 27, it is assumed that coordinates (x, y) indicate the sample positions of the samples on the horizontal line, and that there is a straight line in the direction θ of the intra prediction mode from the coordinates (x, y), which intersects the samples at the corner of the p _i th reference line RL _pi.

[ Equation 37]

x_pi＝x+d_{i_x_θ}

[ Equation 38]

y_pi＝y-d_top

Here, the y-axis distance between BRL and the p _i th reference line RL _pi is d _top, and the following equation 39 can be established.

[ Equation 39]

d_{i_x_θ}＝d_top/tanθ

Here, d _top may be equal to or greater than 0.

In an embodiment, when the direction of the intra prediction mode is θ and at least one of the following conditions 16, 17, and 18 is satisfied, the sample position (x _pi,y_pi) of the p _i th reference line RL _pi corresponding to the sample position (x, y) of BRL may be (x-d _{i_x_θ},y+d_top). In other words, the following equations 40 and 41 may be established.

The distance between the [ condition 16] brl and the target block is equal to or greater than the distance between the p _i th reference line RL _pi and the target block.

Condition 17 it is assumed that there is a straight line from the coordinates (x, y) in the direction opposite to the direction θ of the intra prediction mode, which intersects with a sample point on the horizontal line of the p _i th reference line RL _pi.

Condition 18 it is assumed that coordinates (x, y) indicate a sample point of the sample points on the horizontal line and that there is a straight line from the coordinates (x, y) in a direction opposite to the direction θ of the intra prediction mode, the straight line intersecting the sample points at the corner of the p _i th reference line RL _pi.

[ Equation 40]

x_pi＝x-d_{i_x_θ}

[ Equation 41]

y_pi＝y+d_top

Here, when the direction of the intra prediction mode is θ and the y-axis distance between BRL and the p _i th reference line RL _pi is d _top, the following equation 42 is established.

[ Equation 42]

d_{i_x_θ}＝d_top/tanθ

Here, d _top may be equal to or greater than 0.

In an embodiment, when the direction of the intra prediction mode is θ and at least one of the following conditions 19, 20, and 21 is satisfied, the sample position (x _pi,y_pi) of the p _i th reference line RP _pi corresponding to the sample position (x, y) of the BRL may be (x-d _left,y+d_{i_y_θ}). In other words, the following equations 43 and 44 can be established.

The distance between the [ condition 19] brl and the target block is less than or equal to the distance between the p _i th reference line RL _pi and the target block.

As shown in fig. 28, it is assumed that there is a straight line in the direction θ of the intra prediction mode from the coordinates (x, y) that intersects with a sample point on the perpendicular line to the p _i th reference line RL _pi.

Condition 21 as shown in fig. 29, it is assumed that coordinates (x, y) represent the sample positions of the samples on the vertical line, and that there is a straight line in the direction θ of the intra prediction mode from the coordinates (x, y), which intersects the samples at the corner of the p _i th reference line RL _pi.

[ Eq.43 ]

x_pi＝x-d_left

[ Equation 44]

y_pi＝y+d_{i_y_θ}

Here, when the x-axis distance between BRL and the p _i th reference line RL _pi is d _left, the following equation 45 can be established.

[ Equation 45]

d_{i_y_θ}＝d_left×tanθ

Here, d _left may be equal to or greater than 0.

In an embodiment, when the direction of the intra prediction mode is θ and at least one of the following conditions 22, 23, and 24 is satisfied, the sample position (x _pi,y_pi) of the p _i th reference line RL _pi corresponding to the sample position (x, y) of BRL may be (x+d _left,y-d_{i_y_θ}). In other words, the following equations 46 and 47 can be established.

The distance between the [ condition 22] brl and the target block is equal to or greater than the distance between the p _i th reference line RL _pi and the target block.

Condition 23 it is assumed that there is a straight line from the coordinates (x, y) in the direction opposite to the direction θ of the intra prediction mode, which intersects with a sample point on the perpendicular line to the p _i th reference line RL _pi.

Condition 24 it is assumed that coordinates (x, y) indicate the positions of the sample points on the vertical line, and that there is a straight line in a direction opposite to the direction θ of the intra prediction mode from the coordinates (x, y), which intersects the sample points at the corner of the p _i th reference line RL _pi.

[ Equation 46]

x_pi＝x+d_left

[ Equation 47]

y_pi＝y-d_{i_y_θ}

Here, when the direction of the intra prediction mode is θ and the x-axis distance between BRL and the p _i th reference line RL _pi is d _left, the following equation 48 may be established.

[ Equation 48]

d_{i_y_θ}＝d_left×tanθ

Here, d _left may be equal to or greater than 0.

Generating mixed reference line samples based on mixing in horizontal and vertical directions

Mixing of two reference lines

BRL may be generated by mixing the first reference line and the second reference line of fig. 23.

As shown in fig. 24, BRL is determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 49 may be established.

[ Equation 49]

(x,y)=(x₁,y₁)

In other words, the following equations 50 and 51 may be established.

[ Equation 50]

x=x₁

[ Equation 51]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 52 is established, the following equation 53 may be established.

[ Equation 52]

x_left≤x₁≤x_right

[ Eq.53 ]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 54 is satisfied, the following equation 55 may be established.

[ Equation 54]

y_top≤y₁≤y_bottom

[ Equation 55]

y_top≤y≤y_bottom

In an embodiment, (x ₂,y₂) corresponding to (x, y) of equation 21 may be (x, y-1) when the following condition 25 is satisfied. That is, when the condition 25 is satisfied, the following equations 56 and 57 may be established.

Condition 25 as shown in fig. 25, (x, y) is the position of the sample point on the horizontal line.

[ Equation 56]

x₂=x

[ Equation 57]

y₂=y-1

When condition 25 is satisfied, then equations 56 and 57 are established, a hybrid reference line sample point BRL (x, y) may be generated by one or more of the following equations 58, 59, 60 and 61.

[ Equation 58]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x,y-1)

[ Equation 59]

BRL(x,y)=(RL₁(x,y)+RL₂(x,y-1))>>1

"> >" May represent a right shift operation.

[ Equation 60]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x,y-1)

[ Equation 61]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x,y-1))>>2

In an embodiment, (x ₂,y₂) corresponding to (x, y) of equation 21 may be (x-1, y) when the following condition 26 is satisfied. That is, when the condition 26 is satisfied, the following equations 62 and 63 may be established.

Condition 26 as shown in fig. 25, (x, y) represents the position of the sample point on the vertical line.

[ Equation 62]

x₂=x-1

[ Equation 63]

y₂=y

When the condition 26 is satisfied, and then equations 62 and 63 are established, the hybrid reference line sample point BRL (x, y) may be generated by one or more of the following equations 64, 65, 66 and 67.

[ Equation 64]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x-1,y)

[ Equation 65]

BRL(x,y)=(RL₁(x,y)+RL₂(x-1,y))>>1

"> >" May represent a right shift operation.

[ Equation 66]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x-1,y)

[ Equation 67]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x-1,y))>>2

In an embodiment, (x ₂,y₂) corresponding to (x, y) of equation 21 may be (x-1, y-1) when the following condition 27 is satisfied. That is, when the condition 27 is satisfied, the following equations 68 and 69 may be established.

Condition 27 as shown in fig. 25, (x, y) is the position of the sample point in the upper left corner.

[ Equation 68]

x₂=x-1

[ Equation 69]

y₂=y-1

When condition 27 is satisfied, and then equations 68 and 69 are established, a hybrid reference line sample point BRL (x, y) may be generated by one or more of the following equations 70, 71, 72, and 73.

[ Equation 70]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x-1,y-1)

[ Equation 71]

BRL(x,y)=(RL₁(x,y)+RL₂(x-1,y-1))>>1

"> >" May represent a right shift operation.

[ Equation 72]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x-1,y-1)

[ Equation 73]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x-1,y-1))>>2

Mixing of three reference lines

BRL may be generated by mixing the first reference line, the third reference line, and the fourth reference line of fig. 23.

As shown in fig. 24, BRL may be determined to be on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 74 may be established.

[ Equation 74]

(x,y)=(x₁,y₁)

In other words, the following equations 75 and 76 may be established.

[ Equation 75]

x=x₁

[ Equation 76]

y=y₁

When x ₁ falls within a specific range, x may fall within the same range as the specific range. That is, when the following equation 77 is established, the equation 78 may be established.

[ Equation 77]

x_left≤x₁≤x_right

[ Equation 78]

x_left≤x≤x_right

Further, when y ₁ falls within a specific range, y may fall within the same range as the specific range. That is, when the following equation 79 is established, the equation 80 may be established.

[ Equation 79]

y_top≤y₁≤y_bottom

[ Equation 80]

y_top≤y≤y_bottom

In an embodiment, (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x, y-2) when the following condition 28 is satisfied. That is, when the condition 28 is satisfied, the following equations 81 and 82 may be established.

Condition 28 as shown in fig. 25, (x, y) is the position of the sample point on the horizontal line.

[ Equation 81]

x₃=x

[ Equation 82]

y₃=y-2

Further, (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x, y-3) when the following condition 29 is satisfied. That is, when the condition 29 is satisfied, the following equations 83 and 84 can be established.

Condition 29 as shown in fig. 25, (x, y) is the position of the sample point on the horizontal line.

[ Equation 83]

x₄=x

[ Equation 84]

y₄=y-3

When conditions 28 and 29 are satisfied, a hybrid reference line sample may be generated by one or more of equations 85, 86, and 87 below.

[ Equation 85]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x,y-2)+1/3×RL₄(x,y-3)

[ Equation 86]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x,y-2)+1/4×RL₄(x,y-3)

[ Equation 87]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x,y-2)+RL₄(x,y-3))>>2

"> >" May be a right shift operation.

In an embodiment, (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x-2, y) when the following condition 30 is satisfied. That is, when the condition 30 is satisfied, the following equations 88 and 89 can be established.

Condition 30 as shown in fig. 25, (x, y) is the position of the sample point on the vertical line.

[ Equation 88]

x₃=x-2

[ Equation 89]

y₃=y

Also, (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x-3, y) when the following condition 31 is satisfied. That is, when the condition 31 is satisfied, the following equations 90 and 91 may be established.

Condition 31 as shown in fig. 25, (x, y) may be the position of the sample point on the vertical line.

[ Equation 90]

x₄=x-3

[ Equation 91]

y₄=y

When conditions 30 and 31 are satisfied, a hybrid reference line sample may be generated by one or more of the following equations 92, 93, and 94.

[ Equation 92]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x-2,y)+1/3×RL₄(x-3,y)

[ Equation 93]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x-2,y)+1/4×RL₄(x-3,y)

[ Equation 94]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x-2,y)+RL₄(x-3,y))>>2

"> >" May be a right shift operation.

In an embodiment, (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x-2, y-2) when the following condition 32 is satisfied. That is, when the condition 32 is satisfied, the following equations 95 and 96 may be established.

Condition 32 as shown in fig. 25, (x, y) is the position of the sample point in the upper left corner.

[ Equation 95]

x₃=x-2

[ Equation 96]

y₃=y-2

Further, (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x-3, y-3) when the following condition 33 is satisfied. That is, when the condition 31 is satisfied, the following equations 97 and 98 can be established.

Condition 33 as shown in fig. 25, (x, y) may be the position of the sample point on the vertical line.

[ Equation 97]

x₄=x-3

[ Equation 98]

y₄=y-3

When conditions 32 and 33 are satisfied, a hybrid reference line sample may be generated by one or more of the following equations 99, 100, and 101.

[ Equation 99]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x-2,y-2)+1/3×RL₄(x-3,y-3)

[ Equation 100]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x-2,y-2)+1/4×RL₄(x-3,y-3)

[ Equation 101]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x-2,y-2)+RL₄(x-3,y-3))>>2

"> >" May be a right shift operation.

Case of mixing two reference lines and the direction of intra prediction mode is 45 deg

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. In this case, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 102 may be established.

[ Equation 102]

(x,y)=(x₁,y₁)

In other words, the following equations 103 and 104 may be established.

[ Equation 103]

x=x₁

[ Equation 104]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 105 is established, the equation 106 may be established.

[ Equation 105]

x_left≤x₁≤x_right

[ Equation 106]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 107 is established, the equation 108 may be established.

[ Equation 107]

y_top≤y₁≤y_bottom

[ Equation 108]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 45 °, the (x ₂,y₂) corresponding to (x, y) of equation 21 may be (x+1, y-1). That is, when the direction of the intra prediction mode is 45 °, the following equations 109 and 110 may be established.

[ Equation 109]

x₂=x+1

[ Equation 110]

y₂=y-1

When equations 109 and 110 are established, the hybrid reference line samples may be generated by one or more of equations 111, 112, 113, and 114 below.

[ Equation 111]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x+1,y-1)

[ Equation 112]

BRL(x,y)=(RL₁(x,y)+RL₂(x+1,y-1))>>1

"> >" May be a right shift operation.

[ Equation 113]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x+1,y-1)

[ Equation 114]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x+1,y-1))>>2

Case of mixing two reference lines and the direction of intra prediction mode is 135 deg

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 115 may be established.

[ Equation 115]

(x,y)=(x₁,y₁)

In other words, the following equations 116 and 117 may be established.

[ Equation 116]

x=x₁

[ Equation 117]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when equation 118 is established, equation 119 may be established.

[ Equation 118]

x_left≤x₁≤x_right

[ Equation 119]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 120 is established, the equation 121 may be established.

[ Equation 120]

y_top≤y₁≤y_bottom

[ Equation 121]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 135 °, the (x ₂,y₂) corresponding to (x, y) of equation 21 may correspond to (x-1, y-1). That is, when the direction of the intra prediction mode is 135 °, the following equations 122 and 123 may be established.

[ Equation 122]

x₂=x-1

[ Equation 123]

y₂=y-1

When equations 122 and 123 are established, the hybrid reference line samples may be generated by one or more of equations 124, 125, 126, and 127 below.

[ Equation 124]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x-1,y-1)

[ Equation 125]

BRL(x,y)=(RL₁(x,y)+RL₂(x-1,y-1))>>1

"> >" May represent a right shift operation.

[ Equation 126]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x-1,y-1)

[ Equation 127]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x-1,y-1))>>2

Case of mixing two reference lines and the direction of intra prediction mode is 225 deg

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 128 may be established.

[ Equation 128]

(x,y)=(x₁,y₁)

In other words, the following equations 129 and 130 may be established.

[ Equation 129]

x=x₁

[ Equation 130]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 131 is established, the equation 132 may be established.

[ Equation 131]

x_left≤x₁≤x_right

[ Equation 132]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 133 is established, the equation 134 may be established.

[ Equation 133]

y_top≤y₁≤y_bottom

[ Equation 134]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 225 °, the (x ₂,y₂) corresponding to (x, y) of equation 21 may be (x-1, y-1). That is, when the direction of the intra prediction mode is 225 °, the following equations 135 and 136 may be established.

[ Equation 135]

x₂=x-1

[ Equation 136]

y₂=y+1

When equations 135 and 136 are established, the hybrid reference line samples may be generated by one or more of equations 137, 138, 139, and 140 below.

[ Equation 137]

BRL(x,y)=1/2×RL₁(x,y)+1/2×RL₂(x-1,y+1)

[ Equation 138]

BRL(x,y)=(RL₁(x,y)+RL₂(x-1,y+1))>>1

"> >" May represent a right shift operation.

[ Equation 139]

BRL(x,y)=3/4×RL₁(x,y)+1/4×RL₂(x-1,y+1)

[ Equation 140]

BRL(x,y)=(3×RL₁(x,y)+RL₂(x-1,y+1))>>2

Case of mixing three reference lines and the direction of the intra prediction mode is 45 deg

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 141 may be established.

[ Equation 141]

(x,y)=(x₁,y₁)

In other words, the following equations 142 and 143 may be established.

[ Equation 142]

x=x₁

[ Equation 143]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 144 is established, the following equation 145 may be established.

[ Equation 144]

x_left≤x₁≤x_right

[ Equation 145]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 146 is established, the following equation 147 may be established.

[ Equation 146]

y_top≤y₁≤y_bottom

[ Equation 147]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 45 °, the (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x+2, y-2). That is, when the direction of the intra prediction mode is 45 °, the following equations 148 and 149 may be established.

[ Equation 148]

x₃=x+2

[ Equation 149]

y₃=y-2

In an embodiment, when the direction of the intra prediction mode is 45 °, the (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x+3, y-3). That is, when the direction of the intra prediction mode is 45 °, the following equations 150 and 151 may be established.

[ Equation 150]

x₄=x+3

[ Equation 151]

y₄=y-3

When equations 150 and 151 are established, a hybrid reference line sample may be generated by one or more of equations 152, 153 and 154 below.

[ Equation 152]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x+2,y-2)+1/3×RL₄(x+3,y-3)

[ Equation 153]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x+2,y-2)+1/4×RL₄(x+3,y-3)

[ Equation 154]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x+2,y-2)+RL₄(x+3,y-3))>>2

"> >" May represent a right shift operation.

Three reference lines are mixed and the direction of the intra prediction mode is 135 °

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 155 may be established.

[ Equation 155]

(x,y)=(x₁,y₁)

In other words, the following equations 156 and 157 may be established.

[ Equation 156]

x=x₁

[ Equation 157]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 158 is established, equation 159 may be established.

[ Equation 158]

x_left≤x₁≤x_right

[ Equation 159]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 160 is established, the equation 161 may be established.

[ Equation 160]

y_top≤y₁≤y_bottom

[ Equation 161]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 135 °, the (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x-2, y-2). That is, when the direction of the intra prediction mode is 135 °, the following equations 162 and 163 may be established.

[ Equation 162]

x₃=x-2

[ Equation 163]

y₃=y-2

In an embodiment, when the direction of the intra prediction mode is 135 °, the (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x-3, y-3). That is, when the direction of the intra prediction mode is 135 °, the following equations 164 and 165 may be established.

[ Equation 164]

x₄=x-3

[ Equation 165]

y₄=y-3

When equations 164 and 165 are established, a hybrid reference line sample may be generated by one or more of the following equations 166, 167, and 168.

[ Equation 166]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x-2,y-2)+1/3×RL₄(x-3,y-3)

[ Equation 167]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x-2,y-2)+1/4×RL₄(x-3,y-3)

[ Equation 168]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x-2,y-2)+RL₄(x-3,y-3))>>2

"> >" May represent a right shift operation.

Case of mixing three reference lines and the direction of intra prediction mode is 225 deg

As shown in fig. 24, BRL may be determined to be located on the first reference line RL ₁ of fig. 23. Here, the coordinates (x, y) of the BRL may correspond to the coordinates (x ₁,y₁) of the first reference line. That is, the following equation 169 may be established.

[ Equation 169]

(x,y)=(x₁,y₁)

In other words, the following equations 170 and 171 may be established.

[ Equation 170]

x=x₁

[ Equation 171]

y=y₁

When x ₁ falls within a specific range, x may also fall within the same range as the specific range. That is, when the following equation 172 is established, the equation 173 may be established.

[ Equation 172]

x_left≤x₁≤x_right

[ Equation 173]

x_left≤x≤x_right

In addition, when y ₁ falls within a specific range, y may also fall within the same range as the specific range. That is, when the following equation 174 is established, the equation 175 may be established.

[ Equation 174]

y_top≤y₁≤y_bottom

[ Equation 175]

y_top≤y≤y_bottom

In an embodiment, when the direction of the intra prediction mode is 225 °, the (x ₃,y₃) corresponding to (x, y) of equation 21 may be (x-2, y+2). That is, when the direction of the intra prediction mode is 225 °, the following equations 176 and 177 may be established.

[ Equation 176]

x₃=x-2

[ Equation 177]

y₃=y-2

In an embodiment, when the direction of the intra prediction mode is 225 °, the (x ₄,y₄) corresponding to (x, y) of equation 21 may be (x-3, y+3). That is, when the direction of the intra prediction mode is 225 °, the following equations 178 and 179 may be established.

[ Equation 178]

x₄=x-3

[ Equation 179]

y₄=y+3

When equations 178 and 179 are established, a hybrid reference line sample may be generated by one or more of equations 180, 181 and 182 below.

[ Equation 180]

BRL(x,y)=1/3×RL₁(x,y)+1/3×RL₂(x-2,y+2)+1/3×RL₄(x-3,y+3)

[ Equation 181]

BRL(x,y)=1/2×RL₁(x,y)+1/4×RL₂(x-2,y+2)+1/4×RL₄(x-3,y+3)

[ Equation 182]

BRL(x,y)=(2×RL₁(x,y)+RL₂(x-2,y+2)+RL₄(x-3,y+3))>>2

"> >" May represent a right shift operation.

Generating prediction blocks using mixed reference lines

By utilizing the hybrid reference line samplings generated based on at least one of steps 2010 and 2110, steps 2020 and 2120, and steps 2030 and 2130, a prediction block for the target block may be generated at step 2040 or 2050.

The mixed reference line sample BRL (x, y) may be used to generate a predicted pixel P (x ₀,y₀) for a pixel whose coordinates in the target block are (x ₀,y₀).

Pixel P (x ₀,y₀) with coordinates (x ₀,y₀) in the prediction block may be generated by the following equation 183.

[ Equation 183]

P(x₀,y₀)＝BRL(f(x₀),g(y₀))

Here, f (??) may be a function of calculating the x-axis coordinates of the mixed reference line sample point corresponding to the x-axis coordinates (i.e., x ₀) of the target pixel according to the intra-prediction direction.

Here, g (??) may be a function of calculating the y-axis coordinates of the mixed reference line sample point corresponding to the y-axis coordinates of the target pixel (i.e., y ₀) according to the intra-prediction direction.

In an embodiment, when the block size of the target block is m×n, the upper left coordinate of the target block is (x _{o_left},y_{o_top}), the direction of the intra prediction mode is θ, and prediction according to a sample on a horizontal line of the hybrid reference line is performed, the prediction pixel P (x ₀,y₀) may be generated by performing prediction on a pixel whose coordinate in the target block is (x ₀,y₀) using the hybrid reference line.

Pixel P (x ₀,y₀) with coordinates (x ₀,y₀) in the prediction block may be generated by the following equation 184.

[ Equation 184]

P(x₀,y₀)＝BRL(f(x₀),g(y₀))

Here, x ₀ may be represented by x _{o_left} +i. That is, the following equation 185 may be established. i may be an integer equal to or greater than 0 and less than M (i.e., 0.ltoreq.i < M).

[ Equation 185]

x_o＝x_{o_left}+i

Here, y _o may be represented by y _{o_top} +j. That is, the following equation 186 may be established. i may be an integer equal to or greater than 0 and less than N (i.e., 0.ltoreq.i < N).

[ Equation 186]

y_o＝y_{o_top}+j

When the y-axis distance between the target sample point, whose coordinates in the target block are (x ₀,y₀), and the BRL is d _y, f (x 0) and g (y 0) can be defined by the following equations 187 and 188.

[ Equation 187]

f(x₀)＝x₀+d_y/Tanθ

[ Equation 188]

g(y₀)＝y₀-d_y

Equations 187 and 188 can be represented by equations 189 and 190, respectively, below.

[ Equation 189]

f(x_o)＝x_{o_left}+i+d_y/tanθ

[ Equation 190]

g(y_o)＝y_{o_top}+j-d_y

When the y-axis distance between the upper left coordinate (x _{o_left},y_{o_top}) of the target block and the BRL is d _{o_y}, d _y and d _{o_y} +j may be the same as each other. That is, the following equation 191 may be established.

[ Equation 191]

d_y＝d_{o_y}+j

Equations 189 and 190 can be represented by equations 192 and 193, respectively, below.

[ Equation 192]

f(x_o)＝x_{o_left}+i+(d_{o_y}+j)/tanθ

[ Equation 193]

g(y_o)＝y_{o_top}-d_{o_y}

In an embodiment, when the block size of the target block is m×n and the upper left coordinate of the target block is (x _{o_left},y_{o_top}), the direction of the intra prediction mode is θ, and prediction according to a sample on a vertical line of the hybrid reference line is performed, the prediction pixel P (x _o,y_o) may be generated by performing prediction for a pixel whose coordinate in the target block is (x _o,y_o) using the hybrid reference line.

Pixel P (x _o,y_o) with coordinates (x _o,y_o) in the prediction block may be generated by the following equation 194.

[ Equation 194]

P(x_o,y_o)＝BRL(f(x_o),g(y_o))

Here, x _o may be represented by x _{o_left} +i. That is, the following equation 195 may be established. i may be an integer equal to or greater than 0 and less than M (i.e., 0.ltoreq.i < M).

[ Equation 195]

x_o＝x_{o_left}+i

Here, y _o may be represented by y _{o_top} +j. That is, the following equation 196 may be established. i may be an integer equal to or greater than 0 and less than N (i.e., 0.ltoreq.i < N).

[ Equation 196]

y_o＝y_{o_top}+j

When the y-axis distance between the target sample point, whose coordinates in the target block are (x _o,y_o), and BRL is d _y, f (x _o) and g (y _o) can be defined by the following equations 197 and 198, respectively.

[ Equation 197]

f(x_o)＝x_o-d_x

[ Equation 198]

g(y_o)＝y_o+d_x×tanθ

Equations 197 and 198 can be represented by equations 199 and 200, respectively, below.

[ Equation 199]

f(x_o)＝x_{o_left}+i-d_x

[ Equation 200]

g(y_o)＝y_{o_top}+j+d_x×tanθ

When the x-axis distance between the upper left coordinate (x _{o_left},y_{o_top}) of the target block and the BRL is d _{o_x}, d _x and d _{o_x} +j may be the same as each other. That is, the following equation 201 may be established.

[ Equation 201]

d_x＝d_{o_x}+j

Equations 199 and 200 can be represented by equations 202 and 203, respectively, below.

[ Equation 202]

f(x_o)＝x_{o_left}-d_{o_x}

[ Equation 203]

g(y_o)＝y_{o_top}+j+(d_{o_x}+i)×tanθ

Examples of using hybrid reference line prediction

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is the horizontal prediction mode, and 4) the x-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to the BRL is 2, prediction of the prediction signal P (17, 16) may be performed using a sample on the vertical line of the BRL. The prediction signal P (17, 16) may indicate a prediction sample of coordinates (17, 16) in the prediction block. P (17, 16) may be set as represented by the following equation 204.

[ Equation 204]

P(17,16)=BRL(14,16)

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is the vertical prediction mode, and 4) the y-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to the BRL is 2, prediction of the prediction signal P (17, 16) may be performed using a sample on the horizontal line of the BRL. The prediction signal P (17, 16) may indicate that the coordinates in the prediction block are prediction samples of (17, 16). P (17, 16) may be set as represented by equation 205 below.

[ Equation 205]

P(17,16)=BRL(17,14)

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is a prediction mode of 45 °, and 4) the x-axis distance or y-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to the BRL is 2, prediction of the prediction signal P (17, 16) may be performed using a sample on the horizontal line of the BRL. The prediction signal P (17, 16) may indicate that the coordinates in the prediction block are prediction samples of (17, 16). P (17, 16) may be set as represented by equation 206 below.

[ Equation 206]

P(17,16)=BRL(19,14)

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is a prediction mode of 135 °, 4) the x-axis distance or y-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to BRL is 2, and 5) the position of the pixel of the target block is (18, 16), prediction of the prediction signal P (18, 16) may be performed using a sample point on the horizontal line of BRL. The prediction signal P (18, 16) may indicate that the coordinates in the prediction block are prediction samples of (18, 16). P (18, 16) may be set as represented by equation 207 below.

[ Equation 207]

P(18,16)=BRL(16,14)

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is a prediction mode of 135 °, 4) the x-axis distance or y-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to BRL is 2, and 5) the pixel position of the target block is (16, 18), prediction of the prediction signal P (16, 18) may be performed using the samples on the vertical line of BRL. The prediction signal P (16, 18) may indicate that the coordinates in the prediction block are prediction samples of (16, 18). P (16, 18) may be set as represented by equation 208 below.

[ Equation 208]

P(16,18)=BRL(14,16)

For example, when 1) the block size of the target block is 16×8, 2) the upper left coordinate of the target block is (16, 16), 3) the intra prediction mode of the target block is a prediction mode of 225 °, and 4) the x-axis distance or y-axis distance from the upper left pixel of the target block (i.e., the pixel having the coordinate (16, 16) to the BRL is 2, prediction of the prediction signal P (17, 16) may be performed using a sample on the vertical line of the BRL. The prediction signal P (17, 16) may indicate that the coordinates in the prediction block are prediction samples of (17, 16). P (17, 16) may be set as represented by equation 209 below.

[ Equation 209]

P(17,16)=BRL(14,19)

Multi-reference line list

FIG. 30 illustrates target blocks, reference lines, and fragments according to an example.

The multi-reference line list may be configured using a plurality of available reference lines in the vicinity of the target block.

For example, as shown in fig. 23, N reference lines may exist near the target block. When the N reference lines are ordered in order of proximity to the target block, a multi-reference line list may be configured as represented by equation 210 below.

[ Equation 210]

{ A-th reference line, B-th reference line, C-th reference line, D-th reference line, E-th reference line, F-th reference line }

Here, N may be a positive integer.

In other words, the multi-reference list may include N reference lines.

Next, a multi-reference line list of n=6 is described.

A. Each of B, C, D, E and F may be at least one of positive integers.

Furthermore, at least two of A, B, C, D, E and F may have different values.

For A, B, C, D, E and F, the relationship in equation 211 below can be established.

[ Equation 211]

A<B<C<D<E<F

Here, "<" may indicate that the value to the right of each of the unequally signs is greater than the value to the left of the corresponding unequally sign.

Furthermore, at least two of A, B, C, D, E and F may have the same value.

For A, B, C, D, E and F, the relationship in equation 212 below can be established.

[ Equation 212]

A≤B≤C≤D≤E≤F

Here, "+.ltoreq.may indicate that the value to the right of each of the inequality signs is equal to or greater than the value to the left of the corresponding inequality sign.

For example A, B, C, D, E and F can be 1,2, 4, 6, 8 and 13, respectively. However, in the embodiment, A, B, C, D, E and F are not limited to the above values.

In an embodiment, the value of I in the I-th reference line may start from 0.

Intra_luma_ref_idx may specify a reference line of the target block among reference lines in the multi-reference line list. The reference line of the target block may be a reference line for prediction of the target block.

For example, when configuring the multi-reference line list, as represented by the following equation 213, the reference line of the target block may be specified using intra_luma_ref_idx as shown in the following table 11.

[ Equation 213]

{ First reference line, second reference line, fourth reference line, sixth reference line, eighth reference line, thirteenth reference line }

TABLE 11

intra_luma_ref_idx	Reference line of target block
		0	First reference line
1	Second reference line
		2	Fourth reference line
3	Sixth reference line
		4	Eighth reference line
5	Thirteenth reference line

Derivation/selection of hybrid reference lines

The reference line information may indicate a reference line RL _x for prediction of the target block.

The reference line information may be signaled from the encoding device 1600 to the decoding device 1700 through a bitstream.

At steps 2020 and 2120, an additional reference line RL _y to be mixed may be derived from the signaled reference line information. In other words, the reference line RL _x indicated by the reference line information by the signaled reference line information and the derived additional reference line RL _y are mixed, and thus a mixed reference line can be generated.

Prediction of the target block may be performed using reference line RL _x signaled by reference line information and RL _y derived from reference line RL _x.

In an embodiment, the additional reference line RL _y to be mixed can be derived from intra_luma_ref_idx.

For example, the reference line RL _y indicated by the sum of intra_luma_ref_idx and α may be derived as an additional reference line to be mixed. Here, α may be one of integers such as-2, -1, and 2.

For example, the sum of intra_luma_ref_idx and α may indicate an additional reference line RL _y to be mixed.

For example, the reference line indicated by intra_luma_ref_idx+1 may be derived as the additional reference line RL _y to be mixed.

For example, the reference line indicated by intra_luma_ref_idx-1 may be derived as an additional reference line RL _y to be mixed.

In an embodiment, the multi-reference list may be configured as shown in equation 214 below.

[ Equation 214]

When the multi-reference line list may be configured as shown in equation 214 and the reference line of the target block is indicated by intra_luma_ref_idx, the reference line may be derived from the value of intra_luma_ref_idx as follows, as shown in table 7.

For example, when the value of intra_luma_ref_idx is 0, a second reference line indicated by the sum of intra_luma_ref_idx and 1 may be derived as the additional reference line RL _y to be mixed. In other words, according to the following equation 215 and the table in table 7, the second reference line, which is the reference line when the value of intra_luma_ref_idx is 1, may be selected.

[ Equation 215]

intra_luma_ref_idx+1=1

For example, when the value of intra_luma_ref_idx is 1, a fourth reference line indicated by the sum of intra_luma_ref_idx and 1 may be derived as the additional reference line RL _y to be mixed. In other words, according to the following equation 216 and the table in table 7, the fourth reference line, which is the reference line when the value of intra_luma_ref_idx is 2, may be selected.

[ Equation 216]

intra_luma_ref_idx+1=2

In an embodiment, at steps 2020 and 2120, an additional reference line RL _y to be mixed may be derived from the signaled reference line information.

For example, a reference line having the highest similarity with the reference line RL _x signaled by the reference line information may be derived as the reference line RL _y to be mixed. In other words, the derived reference line RL _y may be the reference line having the highest similarity other than the reference line RL _x among the reference lines in the multi-reference line list. Here, the similarity of the specific reference line may be a similarity between the reference line RL _x and the specific reference line. Prediction of the target block may be performed using the reference line RL _x and an additional reference line RL _y having the highest similarity to the reference line RL _x.

The multiple reference lines to be mixed may be referred to as candidate reference lines. The candidate reference lines may include candidate reference lines signaled by reference line information and candidate reference lines in a multi-reference line list.

The similarity between the signaled candidate reference line and the candidate reference lines in the multi-reference line list may be defined by a cost function value in a cost function of pixels belonging to the signaled candidate reference line and corresponding pixels belonging to the candidate reference lines in the multi-reference line list.

The candidate reference line with the lowest cost function value may be selected as the reference line RL _y to be mixed.

The cost function may be Sum of Absolute Differences (SAD), sum of Squared Differences (SSD), etc.

When the value of intra_luma_ref_idx is x, the reference line RL _x may be indicated by intra_luma_ref_idx. Further, RL _c may indicate candidate reference lines in the multi-reference line list. For RL _c, the following equations 217 and 218 may be established.

[ Equation 217]

c∈{0,1,...,N-1}

[ Equation 218]

c≠x

Here, N may be the total number of elements in the multi-reference list.

Each of the following equations 219 and 220 may indicate a cost function between candidate reference line RL _c and candidate reference line RL _x.

[ Equation 219]

[ Equation 220]

Here, "(i, j)" may represent coordinates of a corresponding pixel. "RL _x (i, j)" may be the pixel value of the reference line RL _x at position (i, j). "(i, j) ∈rl _x" may indicate the coordinates of the pixel belonging to the reference line RL _x. "(i+α _c,j+β_c)" may be the coordinates of the pixel on the candidate reference line RL _c. The pixel on candidate reference line RL _c may correspond to the coordinate (i, j) of reference line RL _x. "RL _c(i+α_c,j+β_c)" may be the pixel value of the pixel at the position (i+α _c,j+β_c) on the reference line RL _c.

For example, when RL _x is the second reference line and RL _c is the fourth reference line, the coordinates of the pixel on RL _c that corresponds to the location (i, j) of RL _x that is located around the top of the target block may be (i, j-2).

For example, when RL _x is the second reference line and RL _c is the fourth reference line, the coordinates of the pixel on RL _c that corresponds to the location (i, j) of RL _x that is located around above the target block may be (i-2, j).

For example, when RL _x is the second reference line and RL _c is the fourth reference line, the coordinates of the pixel on RL _c that corresponds to the location (i, j) of RL _x that is located around above the target block may be (i-2, j-2).

The cost function value may be calculated by means of a cost function for similarity as described above. The candidate reference line RL _c with the lowest cost function value may be determined as the to-be-blended reference line RL _y.

For example, a reference line having a prediction signal having the highest similarity with the prediction signal of the reference line RL _x signaled by the reference line information may be derived as the additional reference line RL _y to be mixed. The prediction signal may refer to a prediction block or a prediction sample.

In other words, the derived reference line RL _y may be the reference line having the highest similarity other than the reference line RL _x among the reference lines in the multi-reference line list. Here, the similarity of the specific reference line may be a similarity between the predicted signal of the reference line RL _x and the predicted signal of the specific reference line. Prediction of the target block may be performed using the reference line RL _x and an additional reference line RL _y having the highest similarity to the reference line RL _x.

The similarity between the signaled candidate reference line and the candidate reference lines in the multi-reference line list may be defined by a cost function value of a cost function between the first prediction signal and the second prediction signal. The first prediction signal may be a prediction signal generated by using signaled candidate reference lines and prediction of an intra prediction mode. The second prediction signal may be a prediction signal generated by prediction using a candidate reference line in the multi-reference line list and the same intra prediction mode.

The cost function may be SAD, SSD, etc.

When the value of intra_luma_ref_idx is x, the reference line RL _x may be indicated by intra_luma_ref_idx. Further, RL _c may indicate candidate reference lines in the multi-reference line list.

For RL _c, the following equations 221 and 222 may be established.

[ Equation 221]

c∈{0,1,...,N-1}

[ Equation 222]

c≠x

Here, N may be the total number of elements in the multi-reference list.

Each of the following equations 223 and 224 may represent a cost function of the similarity between the candidate reference line RL _c and the candidate reference line RL _x.

[ Equation 223]

SAD(RL_c)＝∑_(i,)∈CB|P_RL_x(i,)-P_RL_c(i,)|

[ Equation 224]

SSD(RL_c)＝∑_(i,)eCB((P_RL_x(i,j)-P_RL_c(i,))²

Here, "CB" may represent a target block. "(i, j)" may indicate coordinates of pixels in the target block CB. "(i, j)" may be a prediction signal at the position p_rl _x (i, j) generated by prediction using the reference line RL _x and the intra prediction mode. The prediction signal may be a pixel value of a pixel in the prediction block. "p_rl _c (i, j)" may be a prediction signal at the position (i, j) generated by prediction using the candidate reference line RL _c and the same intra prediction mode.

The cost function value may be calculated by a cost function such as described above for similarity. The candidate reference line RL _c with the lowest cost function value may be determined as the to-be-blended reference line RL _y.

Generation of mixed reference line samples

The signaled reference line RL _x and the reference line RL _y derived from the signaled reference line information at steps 2020 and 2120 can be given. Here, at steps 2030 and 2130, RL _x and RL _y can be mixed, and thus mixed reference line samples can be generated. Prediction of the target block may be performed using the hybrid reference line.

Generating prediction blocks using mixed reference lines

When the signaled reference line RL _x and the reference line RL _y derived from the signaled reference line information in steps 2020 and 2120 are obtained, a (final) prediction signal for the target block may be generated by generating a weighted sum of the first prediction signal and the second prediction signal.

The first prediction signal may be a prediction signal generated by prediction using the reference line RL _x and an intra prediction mode. The second prediction signal may be a prediction signal generated by prediction using the reference line RL _y and the same intra prediction mode.

For example, a weighted sum of the prediction signal p_rl _x (i, j) and the prediction signal p_rl _y (i, j) may be used as the (final) prediction signal for the target block.

The prediction signal p_rl _x (i, j) may be a prediction signal generated by prediction using RL _x and an intra prediction mode. The prediction signal p_rl _y (i, j) may be a prediction signal generated by prediction using the reference line RL _y derived from the reference line information and the same intra prediction mode.

The prediction signal P _cb (i, j) of the target block CB can be derived by the following equation 225.

[ Equation 225]

P_cb(i,j)＝w_x×P_RL_x(i,j)+w_y×P_RL_y(i,j)

Here, w _x may be the weight of p_rl _x (i, j). w _y may be the weight of P_RL _y (i, j).

In an example, the weights may be used as shown in equation 226 below.

[ Equation 226]

P_cb(i,j)＝(3×P_RL_x(i,j)+1×P_RL_y(i,j)+bitdepth>>1)>>2

Here bitdepth may be the bit depth of the input image or the target image.

Template-based multi-reference line intra prediction (TMRL) mode

In an embodiment, TMRL mode may be provided.

The provided pattern may combine the reference lines with the prediction pattern and template matching may be used to construct a list of candidate combinations.

The provided pattern may be used to construct a list of candidate combinations using a template matching method. In the list of candidate combinations, each combination may include a reference line number and an intra prediction mode.

The index of the candidate combination list may be encoded to indicate which reference line and which prediction mode to use when encoding the target block.

The index of the candidate combination list may be encoded to indicate which particular combination of reference lines and prediction modes in the candidate list is used for encoding the target block.

Conventional multi-reference lines (MRLs) for non-template-based intra mode derived (non-TIMD) portions may be replaced with modes according to embodiments.

The MRL may correspond to multiple reference lines as described in the embodiments.

The MRL candidate list may correspond to the multi-reference line list described in the embodiments.

The MRL candidate list may further include two additional reference lines.

The extended MRL may allow more reference lines to be used for prediction.

For example, for conventional intra prediction, the MRL candidate list may be extended from {1,2} to {1,3,5,7,12}. The numbers in the MRL candidate list may indicate reference lines included in the MRL candidate list. The number N in the MRL candidate list may indicate the n+1th reference line or reference line N.

For example, for TIMD intra prediction, the MRL candidate list may be extended from {1,2} to {1,3}.

TMRL mode according to an embodiment may include the following aspects 1,2, 3 and 4.

[ Aspect 1] extended reference line candidate list and intra prediction mode candidate list

The extended reference line candidate list may be {1,3,5,7,12}. The restrictions on the upper column of CTUs may not change.

The size of the intra prediction mode candidate list may be 10.

The construction of the intra prediction mode candidate list may be similar to the Most Probable Mode (MPM).

In an embodiment, only one difference between them is that the plane mode is excluded from the proposed intra prediction mode candidate list.

In an embodiment, the differences may include the following differences 1,2, and 3.

[ Difference 1] the plane mode may be excluded from the intra prediction mode candidate list.

[ Difference 2] when the DC mode is not included after the modes of five neighboring Prediction Units (PUs) and the DIM mode, the DC mode may be included.

[ Difference 3] an angle pattern having a delta angle ranging from + -1 to + -4 can be added. The angle mode may be compared with existing angle modes in the intra prediction mode candidate list.

[ Aspect 2] configuration of TMRL combination list.

For a block, there may be 50 combinations of extended reference lines and allowed intra prediction modes (5×10=50).

The extended reference line may start from reference line 1. Thus, the area covered by reference line 0 may be used for template matching.

The SAD cost of the template region may be calculated between prediction (generated by 50 combinations) and reconstruction.

In an embodiment, only the first 20 combinations with the lowest cost may be pruned from (50) combinations in ascending order to configure the combination candidate list.

In an embodiment, the first 20 combinations with the lowest cost may be selected in ascending order to form the TMRL candidate list.

[ Aspect 3] Signaling of TMRL

The MRL may always be combined with the main MPM non-planar intra prediction mode.

When selecting extended reference lines, up to six context-encoded bins and three Equal Probability (EP) bins may be required to represent RML and MPM indexes.

To indicate which combination of reference lines and prediction modes to use to encode the target block, instead of encoding the reference lines and intra modes directly, the indices of the TMRL candidate list may be encoded.

In TMRL mode according to an embodiment, truncated Columbus-rice codes with divisor 4 may be employed to select combinations from the combination list. Such binarization processing and codewords are shown in the following table 12.

TABLE 12

Indexing of	Binary string (prefix)	Binary string (suffix)
			0	0	00
1	0	01
			2	0	10
3	0	11
			4	10	00
...	...	...
			18	1111	10
19	1111	11

[ Aspect 4] encoder-side modification

Encoder-side modifications may be selectively tested to further improve coding efficiency.

For intra blocks greater than 8 x 8 size, after comparing the Sum of Absolute Transform Differences (SATD) cost of the selected intra prediction mode to a threshold, more passes in full Rate Distortion Optimization (RDO) may be allowed for the selected intra prediction mode.

For intra blocks greater than 8 x 8, no TMRL mode is selected by SATD comparison, additional TMRL RDO may be performed.

Weighted combination of multiple predictors generated from different reference lines

The intra prediction method may use a weighted combination of a plurality of predictors generated from different reference lines to derive a prediction sample.

An alternative approach may combine two reference lines into one line and then may derive the intra-predictor from the combined reference lines.

The intra prediction fusion method according to an embodiment may use a plurality of predictors generated from different modes/reference lines.

In an embodiment, a plurality of intra predictors may be generated and then the plurality of intra predictors generated by weighted average may be fused. The processing of the derived predictors used in the fusion process may include the following processing 1, processing 2, and processing 3.

Process 1 for angular intra prediction modes including TIMD and single mode cases of decoder side intra mode derivation (DIMD), a method according to an embodiment may derive intra prediction based on weighting intra prediction obtained from multiple reference lines.

Intra prediction may be performed by the following equation 227.

[ Equation 227]

p_fusion＝w₀p_line+w₁p_line+1

Here, p _line may be intra prediction from a default reference line. p _line+1 may be intra prediction from a reference line above the default reference line.

Weights may be set as represented by equations 228 and 229 below.

[ Equation 228]

w₀=3/4

[ Equation 229]

w₁=1/4

Process 2 for TIMD modes with mixing, p _line can be used for primary mode (w ₀＝1,w₁＝0).p_line+1 can be used for secondary mode (w ₀＝0,w₁ =1).

Process 3 for DIMD modes with blending, the number of predictors selected for weighted averaging can be increased from 3 to 6.

In an embodiment, instead of predicting a block, intra prediction fusion may be performed on a reference line. For intra prediction fusion, two reference lines may be used. The two reference lines may be referred to as r _line and r _line+1. During the fusion process, deltaInt of the corresponding intra prediction angles may be considered. Each value of the fused reference line may be derived from equation 230.

[ Equation 230]

r_fusion[i]＝(3??r_line[i]+r_line+1[i+DeltaInt])>>2

R _fusion [ i ] can indicate each value of the fused reference line.

When the angular intra mode has a non-integer slope and the block size of the target block is greater than 16, the intra prediction fusion according to the embodiment may be applied to a specific block.

Interpolation of reference samples may be required when the angular intra mode has a non-integer slope.

The block size of the target block may refer to one or more of the width and height of the target block.

The specific block may be a luminance block.

Intra prediction fusion according to an embodiment may be used with MRL.

Intra prediction fusion according to an embodiment may not be applied to blocks encoded using intra sub-partitions (ISPs).

In an embodiment, a position-dependent intra prediction combination (PDPC) may be applied to an intra prediction mode using one of the reference lines closest to the target block.

The same method may be used by the encoding device 1600 and the decoding device 1700 to perform the embodiments. Further, at least one of the embodiments or at least one combination thereof may be used to encode/decode an image.

The order of application of the embodiments of the encoding device 1600 and the decoding device 1700 may be different from each other, and the order of application of the embodiments of the encoding device 1600 and the decoding device 1700 may be (at least partially) the same as each other.

The embodiments may be performed for each of the luminance signal and the chrominance signal, and the embodiments may be equally performed for the luminance signal and the chrominance signal.

The form of the blocks of the application embodiments may have square or non-square shapes.

Whether to apply and/or perform at least one of the above-described embodiment target blocks may be determined based on conditions related to the size of the blocks. In other words, at least one of the above-described embodiments may be applied and/or performed when a condition related to the size of a block is satisfied. The conditions include a minimum block size and a maximum block size. The block may be one of the blocks described above in connection with the embodiment and the units described above in connection with the embodiment. The block to which the minimum block size is applied and the block to which the maximum block size is applied may be different from each other.

For example, the above embodiments may be applied and/or performed when the block size is equal to or greater than the minimum block size and/or less than or equal to the maximum block size. The above embodiments may be applied and/or performed when the block size is greater than the minimum block size and/or less than or equal to the maximum block size.

For example, the above-described embodiments may be applied only to the case where the block size is a predefined block size. The predefined block size may be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. The predefined block SIZE may be (2 x sizex) × (2 x SIZE _Y).SIZE_X may be one of integers of 1 or more. SIZE _Y may be one of integers of 1 or more.

For example, the above-described embodiments may be applied only to the case where the block size is equal to or larger than the minimum block size. The above-described embodiments may be applied only to the case where the block size is larger than the minimum block size. The minimum block size may be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. Alternatively, the minimum block SIZE may be (2 x SIZE _{MIN_X})×(2*SIZE_{MIN_Y}).SIZE_{MIN_X} may be one of integers of 1 or more. SIZE _{MIN_Y} may be one of integers of 1 or more.

For example, the above-described embodiments may be applied only to the case where the block size is smaller than or equal to the maximum block size. The above-described embodiments may be applied only to the case where the block size is smaller than the maximum block size. The maximum block size may be 2×2, 4×4, 8×8, 16×16, 32×32, 64×64, or 128×128. Alternatively, the maximum block SIZE may be (2 x SIZE _{MAX_X})×(2*SIZE_{MAX_Y}).SIZE_{MAX_X} may be one of integers of 1 or more. SIZE _{MAX_Y} may be one of integers of 1 or more.

For example, the above-described embodiments may be applied only to the case where the block size is equal to or larger than the minimum block size and smaller than or equal to the maximum block size. The above-described embodiments may be applied only to the case where the block size is greater than the minimum block size and less than or equal to the maximum block size. The above-described embodiments may be applied only to the case where the block size is equal to or larger than the minimum block size and smaller than the maximum block size. The above-described embodiments may be applied only to the case where the block size is larger than the minimum block size and smaller than the maximum block size.

In the above embodiment, the block size may be a horizontal size (width) or a vertical size (height) of the block. The block size may indicate both the horizontal and vertical dimensions of the block. The block size may indicate the area of the block. Each of the area, the minimum block size, and the maximum block size may be one of integers equal to or greater than 1. In addition, the block size may be a result (or value) of a well-known equation using the horizontal size and the vertical size of the block, or a result (or value) of an equation in the embodiment.

Further, in the embodiments, the first embodiment may be applied to the first size, and the second embodiment may be applied to the second size. That is, the embodiments may be applied in combination according to the size.

Embodiments may be applied according to a temporal layer. To identify the temporal layer to which the embodiment is applicable, a separate identifier may be signaled, and the embodiment may be applied to the temporal layer specified by the corresponding identifier. Here, the identifier may be defined as a lowest (bottom) layer and/or a highest (top) layer to which the embodiment is applicable, and may be defined as a specific layer indicating an application embodiment. Furthermore, a fixed temporal layer of the application embodiment may also be defined.

For example, the embodiment may be applied only in the case where the temporal layer of the target image is the lowest layer. For example, the embodiment may be applied only to a case where the temporal layer identifier of the target image is equal to or greater than 1. For example, the embodiment may be applied only in a case where the temporal layer of the target image is the highest layer.

Stripe types or parallel block group types may be defined to which embodiments of the embodiments are applied, and embodiments may be applied according to the corresponding stripe types or parallel block group types.

In the above-described embodiments, it may be explained that, during application of a specific process to a specific target, assuming that a specific condition may be required and the specific process is performed under a specific determination, when it has been described such that a specific condition is satisfied based on a specific encoding parameter determination or a specific determination is made based on a specific encoding parameter, the specific encoding parameter may be replaced with an additional encoding parameter. In other words, the encoding parameters affecting a particular condition or a particular determination may be considered merely exemplary, and it is understood that a combination of one or more additional encoding parameters in addition to the particular encoding parameters is used as the particular encoding parameters.

In the above-described embodiments, although the method has been described based on a flowchart as a series of steps or units, the present disclosure is not limited to the order of the steps, and some steps may be performed in a different order from the order of the steps that have been described or simultaneously with other steps. Moreover, those of skill in the art will understand that the steps illustrated in the flowcharts are not exclusive and that other steps may also be included, and that alternatively one or more steps in the flowcharts may be deleted without departing from the scope of the present disclosure.

The above-described embodiments include examples of various aspects. Although not all possible combinations for indicating various aspects are described, one of ordinary skill in the art will appreciate that other combinations are possible in addition to the combinations explicitly described. Accordingly, it is to be understood that the present disclosure includes other alternatives, variations, and modifications, which fall within the scope of the appended claims.

The embodiments according to the present disclosure described above may be implemented as programs capable of being executed by various computer devices, and may be recorded on computer-readable storage media. The computer readable storage medium may include program instructions, data files, and data structures, alone or in combination. The program instructions recorded on the storage medium may be specially designed and configured for the present disclosure, or may be known or available to those having ordinary skill in the computer software arts.

The computer readable storage medium may include information used in embodiments of the present disclosure. For example, the computer-readable storage medium may include a bitstream, and the bitstream may include the information described above in embodiments of the present disclosure.

The bitstream may include computer executable code and/or programs. The computer executable code and/or program may include a plurality of pieces of information described in the embodiments, and may include syntax elements described in the embodiments. In other words, the pieces of information and syntax elements described in the embodiments may be considered to be computer executable code in the bitstream and may be considered to be at least part of the computer executable code and/or program provided by the bitstream.

The computer-readable storage medium may include a non-transitory computer-readable medium.

Examples of computer readable storage media may include all types of hardware devices that are specially configured for recording and executing program instructions, such as magnetic media (such as hard disks, floppy disks, and magnetic tape), optical media (such as Compact Discs (CD) -ROMs, and Digital Versatile Discs (DVDs)), magneto-optical media (such as floppy disks, ROMs, RAMs, and flash memories). Examples of program instructions include both machine code, such as produced by a compiler, and high-level language code that can be executed by the computer using an interpreter. The hardware devices may be configured to operate as one or more software modules to perform the operations of the present disclosure, and vice versa.

As noted above, although the present disclosure has been described based on specific details (such as detailed components and a limited number of embodiments and figures), the specific details are provided only for ease of understanding the entire disclosure, the present disclosure is not limited to these embodiments, and various changes and modifications will be practiced by those of skill in the art in light of the foregoing description.

It is, therefore, to be understood that the spirit of the present embodiments is not to be limited to the above-described embodiments and that the appended claims and equivalents thereof, and modifications thereto, fall within the scope of the present disclosure.

Claims

1. An image decoding method, comprising:

Choose whether to use mixed guides;

deriving the hybrid reference line;

generating one or more sample points on the hybrid reference line; and

Prediction of a target block is performed using the hybrid reference line.

2 . The image decoding method of claim 1 , wherein whether to use the hybrid reference line is selected based on a direction of an intra prediction mode.

3 . The image decoding method of claim 1 , wherein whether to use the hybrid reference line is selected based on a size of a target block.

4 . The image decoding method of claim 1 , wherein the mixed reference line is generated when two or more reference lines are mixed.

5 . The image decoding method of claim 4 , wherein the two or more reference lines are selected based on a reference line index.

6 . The image decoding method of claim 1 , wherein the samples on the mixed reference line are generated based on the distance between the two or more reference lines.

7 . The image decoding method of claim 1 , wherein the prediction is performed based on a distance between the two or more reference lines.

8. An image encoding method, comprising:

Choose whether to use mixed guides;

deriving the hybrid reference line;

generating one or more sample points on the hybrid reference line; and

Prediction of a target block is performed using the hybrid reference line.

9 . The image encoding method of claim 8 , wherein whether to use the hybrid reference line is selected based on a direction of an intra prediction mode.

10 . The image encoding method of claim 8 , wherein whether to use the hybrid reference line is selected based on a size of the target block.

11. The image encoding method of claim 8, wherein the mixed reference line is generated when two or more reference lines are mixed.

12 . The image encoding method of claim 11 , wherein the two or more reference lines are selected based on a reference line index.

13. The image encoding method of claim 8, wherein the samples on the mixed reference line are generated based on the distance between the two or more reference lines.

14. The image encoding method of claim 8, wherein the prediction is performed based on a distance between the two or more reference lines.

15. A computer-readable storage medium for storing a bit stream for image decoding, wherein:

The bitstream includes coding information about the target block,

Whether to use mixed guides is selected,

The hybrid reference line is derived,

One or more sample points on the hybrid reference line are generated, and

Prediction of a target block using the hybrid reference line is performed.

16. The computer-readable storage medium of claim 15, wherein whether to use the hybrid reference line is selected based on a direction of an intra prediction mode.

17 . The computer-readable storage medium of claim 15 , wherein whether to use the hybrid reference line is selected based on a size of a target block.

18. The computer-readable storage medium of claim 15, wherein the mixed reference line is generated when two or more reference lines are mixed.

19. The computer-readable storage medium of claim 18, wherein the two or more reference lines are selected based on a reference line index.

20. The computer-readable storage medium of claim 15, wherein the sample points on the hybrid reference line are generated based on distances between the two or more reference lines.