RU2803457C2 - Image encoding method based on transformation and device for its implementation - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: processor can be configured to derive a transformation ratio for a current block based on the residual information, to determine whether a significant ration occurs in the second region that excludes the top-left first region of the current block, to obtain, by parsing, an MTS index from a bitmap flow if the significant ratio is not in the second region, and to derive the residual sample by applying the MTS core derived from the MTS index to the transformation ratios of the first region.

EFFECT: increase of the efficiency of video encoding.

12 cl, 23 dwg, 20 tbl

Description

TECHNICAL AREA

[1] The present disclosure relates to an image encoding method and, in particular, to a method and apparatus for encoding an image based on a transform in an image encoding system.

Related technical field

[2] Nowadays, the need for high-resolution and high-quality images/videos, such as ultra-high-definition (UHD) 4K or 8K or higher images/videos, has increased in various fields. Since image/video data is high resolution and high quality, the amount of information or number of bits transmitted is increased compared to normal image data. Therefore, when image data is transmitted using media such as a conventional wired/wireless broadband line, or image/video data is stored using an existing storage medium, transmission costs and storage costs increase.

[3] In addition, currently there is an increasing interest and need for immersive media, such as content or holograms of virtual reality (VR) and artificial reality (AR) or the like, and the broadcast of images/videos having image characteristics other than features of a real image, such as a game image.

[4] Accordingly, there is a need for a high performance image/video compression technology for efficiently compressing, transmitting or storing and reproducing high resolution and high quality image/video information having various features as described above.

Brief description of the invention

[5] A technical aspect of the present disclosure is to provide a method and apparatus for improving image encoding efficiency.

[6] Another technical aspect of the present disclosure is to provide a method and apparatus for improving the efficiency of transform index encoding.

[7] Another technical aspect of the present disclosure is to provide an encoding method and apparatus using LFNST and MTS.

[8] Another technical aspect of the present disclosure is to provide an image encoding method and apparatus for signaling a LFNST index and an MTS index.

[9] According to an embodiment of the present disclosure, an image decoding method performed by a decoding apparatus is provided. The method may include: deriving transformation coefficients for the current block based on the residual information; determining whether a significant coefficient is present in a second region other than the first region from the top left of the current block; obtaining, by parsing, an MTS index from the bitstream when a significant coefficient is not present in the second region, deriving residual samples by applying an MTS kernel derived from the MTS index to the transform coefficients of the first region.

[10] The first region is the top-left 16x16 region of the current block.

[11] Determining whether a significant coefficient is present in the second region comprises deriving a value of a variable indicating whether a significant coefficient is present in the second region during the decoding process of the residual encoding layer, the value of the variable is initially set to 1, and when the significant coefficient is present in the second region , the value of the variable is changed to 0 and the MTS index is not parsed.

[12] The MTS index is analyzed at the coding unit level.

[13] The image decoding method further comprises parsing with respect to a LFNST index indicating a LFNST kernel applied to the current block, and the LFNST index and the MTS index are signaled at the coding unit level, and the MTS index is signaled immediately after the LFNST index signaling.

[14] The three types of the current block are dual luma tree or single tree, and when the LFNST index value is 0, the MTS index is signaled.

[15] According to another embodiment of the present disclosure, an image encoding method performed by an encoding apparatus is provided. The method may include: deriving residual samples for the current block based on the prediction samples; deriving transform coefficients for the current block based on the MTS for the residual samples; zeroing the second area of the current block other than the first area from the top left of the current block; and encoding the residual information output by quantizing the transform coefficients and the MTS index indicating the MTS core.

[16] According to yet another embodiment of the present disclosure, a digital storage medium that stores image data including encoded image information and a bit stream generated in accordance with an image encoding method performed by an encoding apparatus may be provided.

[17] According to another embodiment of the present disclosure, a digital storage medium may be provided that stores image data including encoded image information and a bitstream for causing a decoding apparatus to perform an image decoding method.

[18] According to the present disclosure, the overall efficiency of image/video compression can be improved.

[19] According to the present disclosure, it is possible to improve the efficiency of transform index encoding.

[20] A technical aspect of the present disclosure may provide an encoding method and apparatus using LFNST and MTS.

[21] A technical aspect of the present disclosure may provide a method and apparatus for signaling a LFNST index and an MTS index.

[22] The results that can be obtained from the specific examples of this disclosure are not limited to the results listed above. For example, various technical results may be achieved that one skilled in the art will be able to understand or obtain based on the present disclosure. Accordingly, the specific results of the present disclosure are not limited to those expressly described in the present disclosure and may include various results that can be understood or obtained by the technical features of the present disclosure.

Brief description of drawings

[23] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[24] FIG. 2 is a diagram schematically illustrating the configuration of a video/image encoding apparatus to which the present disclosure is applicable.

[25] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure is applicable.

[26] FIG. 4 schematically illustrates a multiple conversion circuit in accordance with an embodiment of the present document.

[27] FIG. 5 schematically shows the intra-directional modes of 65 prediction directions.

[28] FIG. 6 is a diagram for explaining RST in accordance with an embodiment of the present disclosure.

[29] FIG. 7 is a diagram illustrating a layout sequence of direct primary transform output data into a one-dimensional vector according to an example.

[30] FIG. 8 is a diagram illustrating a layout sequence of direct secondary transform output data into a two-dimensional block according to an example.

[31] FIG. 9 is a diagram illustrating wide-angle intra-prediction modes in accordance with an embodiment of the present document.

[32] FIG. 10 is a diagram illustrating a block shape to which LFNST is applied.

[33] FIG. 11 is a diagram illustrating the layout of forward LFNST output according to an example.

[34] FIG. 12 is a diagram illustrating that the number of outputs for forward LFNST is limited to a maximum of 16 according to the example.

[35] FIG. 13 is a diagram illustrating nulling in a block to which a 4x4 LFNST is applied according to an example.

[36] FIG. 14 is a diagram illustrating nulling in a block to which 8x8 LFNST is applied according to an example.

[37] FIG. 15 is a diagram illustrating nulling in a block to which 8x8 LFNST is applied according to another example.

[38] FIG. 16 is a diagram illustrating an example of a sub-block into which one encoding block is divided.

[39] FIG. 17 is a diagram illustrating another example of a subblock into which one encoding block is divided.

[40] FIG. 18 is a diagram illustrating the symmetry of an M×2 block (M×1) and a 2XM block (1XM) according to an example.

[41] FIG. 19 is a diagram illustrating an example of transposing a 2HM block according to the example.

[42] FIG. 20 shows the scanning order for the 8×2 or 2×8 area according to the example.

[43] FIG. 21 is a flowchart for explaining an image decoding method according to an example.

[44] FIG. 22 is a flowchart for explaining an image encoding method according to an example.

[45] FIG. 23 illustrates the structure of a content streaming system to which the present disclosure is applied.

Description of Exemplary Embodiments

[46] Although the present document is subject to various modifications and includes various embodiments, specific embodiments thereof, shown in the drawings by way of example, will now be described in detail. However, this is not intended to limit the present disclosure to the specific embodiments described herein. The terminology used herein is intended to describe specific embodiments only and is not intended to limit the technical intent of the present disclosure. Singular forms may include plural forms unless the context clearly indicates otherwise. Terms such as "include" and "have" are intended to indicate that the features, numbers, steps, operations, elements, components, or combinations thereof used in the following description exist, and thus are not to be construed as such. that the possibility of the existence or addition of one or more other features, numbers, steps, operations, elements, components or a combination thereof is precluded.

[47] Meanwhile, each component in the drawings described herein is illustrated independently for convenience of description regarding various characteristic functions, however, this does not mean that each component is implemented as separate hardware or software. For example, two or more of these components may be combined to form a single component, and any single component may be divided into multiple components. Embodiments in which components are combined and/or separated will be included within the scope of the patent rights of this disclosure if they do not depart from the spirit of this disclosure.

[48] Next, preferred embodiments of the present disclosure will be explained in more detail with reference to the accompanying drawings. In addition, like reference numerals are used for like components in the drawings, and repeated descriptions of like components will be omitted.

[49] This document relates to video/image coding. For example, the method/example disclosed herein may relate to the VVC (Video Video Coding) standard (ITU-T Rec. H.266), the next generation video/image coding standard after VVC, or other standards related to video coding (eg HEVC (High Efficiency Video Coding) standard (ITU-T Rec. H.265), EVC (Essential Video Coding) standard AVS2 standard, etc.).

[50] Herein, various embodiments related to video/image encoding may be provided, and the embodiments may also be combined with each other and executed.

[51] As used herein, video may refer to a series of images over time. A picture usually means a unit that represents an image in a particular time zone, and a slice/tile is a unit that makes up a part of a picture. The clipping/tile may include one or more coding tree units (CTUs). One picture may consist of one or more cutouts/mosaic elements. One picture can consist of one or more groups of mosaic elements. One tile group may include one or more tiles.

[52] Pixel or pel can mean the smallest unit that makes up one picture (or image). Also, 'sample' can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a luma component pixel/pixel value or only a chrominance component pixel/pixel value. Alternatively, the sample may refer to a pixel value in the spatial domain, or when that pixel value is converted to the frequency domain, it may refer to a frequency domain conversion factor.

[53] The unit may represent a basic image processing unit. The unit may include at least one of a specific area and information related to the area. One unit may include one luma block and two chrominance blocks (eg, cb, cr). A unit and a term such as a block, area or the like may be used interchangeably as appropriate. In general, an MCN block may include a set (or array) of samples (or arrays of samples) or transform coefficients consisting of M columns and N rows.

[54] In this document, the characters "/" or "," are to be interpreted as indicating "and/or". For example, the expression "A/B" could mean "A and/or B". Additionally, "A, B" may mean "A and/or B". Additionally, "A/B/C" may mean "at least one of A, B, and/or C." Also, "A/B/C" may mean "at least one of A, B and/or C".

[55] Additionally, as used herein, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may include 1) A only, 2) B only, and/or 3) both A and B. In other words, the term "or" as used herein is to be interpreted as indicating "in addition or alternatively" .

[56] In the present disclosure, “at least one of A and B” may mean “A only,” “B only,” or “both A and B.” Also, in the present disclosure, the expression “at least one of A or B” or “at least one of A and/or B” can be interpreted as “at least one of A and B”.

[57] Moreover, in the present disclosure, at least one of "A, B and C" may mean "A only", "B only", "C only" or "any combination of A, B and C". Also, "at least one of A, B or C" or "at least one of A, B and/or C" may mean "at least one of A, B and C".

[58] In addition, parentheses used in this disclosure may mean "for example." Specifically, when "prediction (intra-prediction)" is specified, it may mean that "intra-prediction" is proposed as an example of "prediction". In other words, “prediction” in the present disclosure is not limited to “intra-prediction”, and “intra-prediction” may be offered as an example of “prediction”. Also, when "prediction (ie, intra-prediction)" is specified, it also means that "intra-prediction" is proposed as an example of "prediction".

[59] Technical features separately described in a single drawing in the present disclosure may be implemented separately or may be implemented simultaneously.

[60] FIG. 1 schematically illustrates an example of a video/image coding system to which the present disclosure is applicable.

[61] With reference to FIG. 1, a video/image encoding system may include a first device (a source device) and a second device (a receiving device). A source device may deliver encoded information or video/image data in the form of a file or streaming to a receiving device via a digital storage medium or network.

[62] The source device may include a video source, an encoder, and a transmitter. The receiving device may include a receiver, a decoding device, and a rendering device. The encoding device may be called a video/image encoding device, and the decoding device may be called a video/image decoding device. The transmitter may be included in the encoding device. The receiver may be included in a decoding device. The imaging device may include a display, and the display may be configured as a separate device or an external component.

[63] The video source may obtain video/image through a video/image capture, synthesis, or generation process. The video source may include a video/image capturing device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating device may include, for example, a computer, tablet and smartphone and may (electronically) generate video/images. For example, the virtual video/image may be generated through a computer or the like. In this case, the video/image capture process can be replaced by the associated data generation process.

[64] The encoding device can encode the input video/image. The encoder may perform a series of procedures such as prediction, transform and quantization for compression and encoding efficiency. The encoded data (encoded video/image information) may be output in the form of a bit stream.

[65] A transmitter may transmit encoded information or video/image data output in the form of a bitstream to a receiver of a receiving device via a digital storage medium or network in the form of a file or stream. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like. The transmitter may include an element for generating a media file through a predefined file format and may include an element for transmission over a broadcast/communications network. The receiver may receive/extract the bit stream and transmit the received/extracted bit stream to a decoding device.

[66] A decoding apparatus may decode a video/image by performing a series of procedures such as dequantization, deconversion, and prediction corresponding to an operation of the encoding apparatus.

[67] The renderer can render the decoded video/image. The rendered video/image can be played back via the display.

[68] FIG. 2 is a diagram schematically illustrating the configuration of a video/image encoding apparatus to which embodiments of the present document may be applied. Further, what is referred to as a video encoding device may include an image encoding device.

[69] With reference to FIG. 2, the encoding device 200 includes an image partitioning unit 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-predictor 221 and an intra-predictor 222. The processor The remainder 230 may include a converter 232, a quantizer 233, a dequantizer 234, and an inverse converter 235. The remainder processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or reconstructed block generator. The image partitioning unit 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260, which were described above, may be constituted by one or more hardware components (eg, chipsets or encoder processors) in accordance with an embodiment. In addition, memory 270 may include a decoded picture buffer (DPB) or may be configured with digital storage media. The hardware component may further include memory 270 as an internal/external component.

[70] The image splitter 210 may split an input image (or picture or frame) input to the encoder 200 into one or more processing units. For example, a processing unit may be called a coding unit (CU). In this case, starting from a coding tree unit (CTU) or largest coding unit (LCU), the coding unit can be recursively split according to a quadtree/binary tree/ternary tree (QTBTTT) structure. For example, one encoding unit may be partitioned into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary tree structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and/or ternary tree structure may be applied later. Alternatively, the binary tree structure may be applied first. The encoding procedure in accordance with the present disclosure may be performed based on a final encoding unit that is no longer split. In this case, the largest encoding unit can be used directly as the final encoding unit based on the encoding efficiency according to the characteristic of the image. Alternatively, a coding unit can be recursively split into coding units of greater depth as needed, so that an optimally sized coding unit can be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which will be described later. As an example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be separated or split from the above-mentioned final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for deriving a transform coefficient and/or a unit for deriving a residual signal from the transform coefficient.

[71] Unit may be used interchangeably with terms such as block, area or the like, as appropriate. In general, an MCN block can represent a set of samples or transform coefficients consisting of M columns and N rows. The sample may generally represent a pixel or pixel value, may represent only a luma component pixel/pixel value, or may represent only a chrominance component pixel/pixel value. Sampling can be used as a term corresponding to a single picture (or image) for a pixel or pel.

[72] The subtractor 231 subtracts the prediction signal (predicted block, prediction sample array) output from the inter-predictor 221 or intra-predictor 222 from the input image signal (source block, source sample array) to generate a residual signal (residual block, residual array of samples), and the generated residual signal is transmitted to converter 232. The predictor may perform prediction on a target processing block (hereinafter referred to as a current block), and may generate a predicted block including prediction samples for the current block. The predictor can determine whether intra-prediction or inter-prediction is applied on the current block or on a CU basis. As discussed below in the description of each prediction mode, the predictor may generate various prediction-related information, such as prediction mode information, and transmit the generated information to the entropy encoder 240. The prediction information may be encoded in the entropy encoder 240 and output in the form of a bit stream.

[73] Intra predictor 222 may predict the current block by referring to samples in the current picture. These samples may be located adjacent to or spaced apart from the current block in accordance with the prediction mode. In intra-prediction, prediction modes may include a plurality of undirected modes and a plurality of directed modes. Non-directional modes may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is just an example, and more or less directional prediction modes may be used depending on the setting. Intra predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[74] The inter-predictor 221 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample based on the correlation of motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporary adjacent block may be the same or different. A temporary neighbor block may be called a co-located reference block, a co-located CU (colCU), etc., and a reference picture including a temporary neighboring block may be called a co-located picture (colPic). For example, inter-predictor 221 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to output the motion vector and/or reference picture index of the current block. Inter-prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter-predictor 221 may use the motion information of a neighboring block as the motion information of the current block. In skip mode, unlike combine mode, the residual signal may not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling a motion vector difference.

[75] Predictor 220 may generate a prediction signal based on various prediction methods described below. For example, a predictor may not only apply intra-prediction or inter-prediction to make a prediction on one block, but may also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter- and intra-prediction (CIIP). In addition, the predictor can be based on intra-block copy (IBC) prediction mode or palette mode to perform prediction on a block. The IBC prediction mode or palette mode can be used for encoding image/video game content or the like, such as screen content encoding (SCC). Although IBC primarily performs prediction in the current block, it can perform similar to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter-prediction methods described in the present disclosure.

[76] The prediction signal generated by inter-predictor 221 and/or intra-predictor 222 can be used to generate a reconstructed signal or to generate a residual signal. Converter 232 may generate transform coefficients by applying a transform method to the residual signal. For example, the transform method may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). . Here, GBT means the transformation obtained from a graph when the relation information between pixels is represented by the graph. CNT refers to the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the transformation process can be applied to square blocks of pixels that are the same size, or can be applied to blocks that are variable in size rather than square.

[77] The quantizer 233 may quantize the transform coefficients and transmit them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal in a bit stream. Information about the quantized transform coefficients may be called remainder information. Quantizer 233 may reorder the quantized block type transform coefficients into one-dimensional vector form based on the scanning order of the coefficients, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form. Entropy encoder 240 may perform various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 240 may encode information needed for video/image reconstruction other than quantized transform coefficients (eg, syntax element values, etc.) together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in Network Abstraction Layer (NAL) units in the form of a bitstream. The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. In the present disclosure, information and/or syntax elements transmitted/signaled from an encoding device to a decoding device may be included in video/image information. The video/image information may be encoded by the above-described encoding procedure and included in the bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcast network, a communications network and/or the like, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 240 and/or a storage module (not shown) storing the signal may be configured as an internal/external element of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[78] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients by dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal with the prediction signal output from inter-predictor 221 or intra-predictor 222 so that a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) can be generated. If there is no remainder for the target processing block, as in the case where the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 250 may be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used to intra-predict the next block to be processed in the current picture and, as described below, can be used to inter-predict the next picture by filtering.

[79] Meanwhile, luminance chroma scaling (LMCS) mapping can be used in the process of encoding and/or restoring a picture.

[80] Filter 260 may improve subjective/objective video quality by applying filtering to the reconstructed signal. For example, filter 260 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and may store the modified reconstructed picture in memory 270, specifically in DPB memory 270. Various filtering methods may include, for example, deblocking filtering, adaptive offset samples, adaptive loop filter, bidirectional filter and the like. Filter 260 may generate various filtering-related information and transmit the generated information to entropy encoder 240, as described later in the description of each filtering method. The filtering information may be encoded by entropy encoder 240 and output in the form of a bitstream.

[81] The modified reconstructed picture that has been transferred to the memory 270 can be used as a reference picture in the inter-predictor 221. Through this, the encoding device can prevent prediction inconsistency in the encoding device 200 and the decoding device when inter-prediction is applied, and can also improve coding efficiency.

[82] DPB memory 270 may store a modified reconstructed picture for use as a reference picture in inter-predictor 221. Memory 270 may store motion information of a block in the current picture from which motion information was derived (or encoded) and/or information movement of blocks in the already restored picture. The stored motion information may be transmitted to the inter-predictor 221 for use as neighbor motion information or temporary neighbor motion information. Memory 270 may store reconstructed samples of reconstructed blocks in the current picture and may transmit them to intra-predictor 222.

[83] FIG. 3 is a diagram schematically illustrating the configuration of a video/image decoding apparatus to which the present disclosure can be applied.

[84] With reference to FIG. 3, decoding apparatus 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-predictor 331 and an intra-predictor 332. A residual processor 220 may include includes a dequantizer 321 and an inverse converter 321. The entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 that were described above may be configured by one or more hardware components (e.g., decoder chipsets or processors) in accordance with embodiment. In addition, memory 360 may include a decoded picture buffer (DPB) or may be configured with a digital storage medium. The hardware component may further include memory 360 as an internal/external component.

[85] When a bitstream including video/image information is input, the decoding apparatus 300 can reconstruct the image according to the process by which the video/image information was processed in the encoding apparatus according to FIG. 2. For example, the decoding apparatus 300 may output units/blocks based on the block division related information obtained from the bitstream. The decoding apparatus 300 may perform decoding using a processing unit applied in the encoding apparatus. Therefore, the decoding processing unit may be, for example, an encoding unit, which may be divided according to a quadtree structure, a binary tree structure, and/or a ternary tree structure from an encoding tree unit or a largest encoding unit. One or more transformation units may be derived from a coding unit. And the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by the playback device.

[86] The decoding device 300 may receive a signal output from the encoding device according to FIG. 2 in the form of a bit stream, and the received signal may be decoded by entropy decoder 310. For example, entropy decoder 310 may parse the bit stream to output information (eg, video/image information) necessary for image reconstruction (or picture reconstruction). The video/image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), a video parameter set (VPS), and the like. In addition, the video/image information may further include general restriction information. The decoding apparatus may further decode the picture based on the parameter set information and/or general constraint information. The signaled/received information and/or syntax elements described later in the present disclosure may be decoded through a decoding procedure and obtained from the bitstream. For example, entropy decoder 310 may decode information in the bit stream based on an encoding method such as exponential Golomb coding, CAVLC, or CABAC, and may output the syntax element value required to reconstruct the image and the quantized values of the transform coefficients for the remainder. More specifically, the CABAC entropy decoding method may take a bin corresponding to each syntactic element in a bitstream, determine a context model using information of a target decoding syntax element, decoding information of an adjacent and target decoding block, or information of a symbol/bin decoded in a previous stage, predict the probability generate a bean according to a defined context model and perform arithmetic decoding of the bean to generate a symbol corresponding to the meaning of each syntactic element. Here, the CABAC entropy decoding method can update the context model using the information of the decoded symbol/bin for the context model of the next symbol/bin after determining the context model. Prediction information among the information decoded by the entropy decoder 310 may be provided to the predictor (inter-predictor 332 and intra-predictor 331), and residual values, that is, quantized transform coefficients on which entropy decoding was performed in the entropy decoder 310, and associated parameter information may be input to residual processor 320. Residual processor 320 may output a residual signal (residual block, residual samples, residual sample array). In addition, filtering information among the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) that receives a signal output from the encoding device may be further configured as an internal/external element of the device 300 decoding device, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to the present disclosure may be called a video/image/picture decoding device, and the decoding device may be classified into an information decoder (video/image/picture information decoder) and a decoder sampling (video/image/picture sampling decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse converter 322, an adder 340, a filter 350, a memory 360, an inter-predictor 332, and an intra-predictor 331.

[87] Dequantizer 321 may output transform coefficients by dequantizing the quantized transform coefficients. The dequantizer 321 may reorder the quantized transform coefficients in the form of a two-dimensional block. In this case, reordering may be performed based on the order of coefficient scanning performed in the encoder. The dequantizer 321 may perform dequantization on the quantized transform coefficients using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[88] The inverter 222 obtains the residual signal (residual block, residual sample array) by inversely converting the transform coefficients.

[89] The predictor may perform prediction on the current block and generate a predicted block including prediction samples for the current block. The predictor can determine whether intra-prediction or inter-prediction is applied to the current block based on the prediction information output from the entropy decoder 310, and can determine the intra/inter-prediction mode in particular.

[90] The predictor can generate a prediction signal based on various prediction methods. For example, a predictor may apply intra-prediction or inter-prediction on one block, and may also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter- and intra-prediction (CIIP). In addition, the predictor can perform intra-block copying (IBC) to make predictions on a block. The intra-block copy mode can be used for encoding image/video game content or the like, such as screen content encoding (SCC). Although IBC primarily performs prediction in the current block, it can perform similar to inter-prediction in that it outputs a reference block in the current block. That is, the IBC may use at least one of the inter-prediction methods described in the present disclosure.

[91] Intra predictor 331 may predict the current block by referring to samples in the current picture. These samples may be located adjacent to or spaced apart from the current block in accordance with the prediction mode. In intra-prediction, prediction modes may include a plurality of undirected modes and a plurality of directed modes. Intra predictor 331 may determine the prediction mode applied to the current block using the prediction mode applied to the adjacent block.

[92] Inter-predictor 332 may output a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector in the reference picture. Here, in order to reduce the amount of motion information transmitted in the inter-prediction mode, motion information may be predicted on a block, sub-block, or sample based on the correlation of motion information between an adjacent block and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block existing in the current picture and a temporal neighbor block existing in the reference picture. For example, inter-predictor 332 may configure a motion information candidate list based on neighboring blocks and output a motion vector and/or reference picture index of the current block based on the received candidate selection information. Inter-prediction may be performed based on various prediction modes, and the prediction information may include information indicating an inter-prediction mode for the current block.

[93] The adder 340 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted array of samples) output from the predictor 330. If there is no residual for the target processing block, as in the case when the skip mode is applied, the predicted block can be used as a reconstructed block.

[94] The adder 340 may be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used to intra-predict the next processing target block in the current block and, as described below, can be output by filtering or used to inter-predict the next picture.

[95] Meanwhile, in the picture decoding process, luminance chroma scaling (LMCS) display can be applied.

[96] Filter 350 may improve subjective/objective image quality by applying filtering to the reconstructed signal. For example, filter 350 may generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and may transfer the modified reconstructed picture to memory 260, specifically a DPB of memory 360. Various filtering methods may include, for example, deblocking filtering, adaptive sampling bias, adaptive loop filter, bidirectional filter and the like.

[97] The (modified) reconstructed picture that has been stored in the DPB memory 360 may be used as a reference picture in the inter-predictor 332. The memory 360 may store motion information of a block in the current picture from which motion information is derived (or decoded), and /or information about the movement of blocks in an already restored picture. The stored motion information may be transmitted to the inter-predictor 332 for use as neighbor motion information or temporary neighbor motion information. Memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to intra-predictor 331.

[98] In this specification, the examples described for predictor 330, dequantizer 321, inverse converter 322, and filter 350 of decoder 300 may be similarly or correspondingly applied to predictor 220, dequantizer 234, inverse converter 235, and filter 260 of encoder 200, respectively. .

[99] As described above, prediction is performed to improve compression efficiency when performing video encoding. By this, a predicted block can be generated including prediction samples for the current block, that is, a target encoding block. Here, the predicted block includes prediction samples in a spatial domain (or pixel domain). The predicted block may be output equally in the encoding apparatus and the decoding apparatus, and the encoding apparatus may improve the efficiency of image encoding by signaling to the decoding apparatus not the original sample value of the original block itself, but the residual information (residual information) between the original block and the predicted block. The decoding apparatus can output a residual block including residual samples based on the residual information, generate a reconstructed block including the reconstructed samples by summing the residual block and the predicted block, and generate a reconstructed picture including the reconstructed blocks.

[100] Residual information may be generated through transformation and quantization procedures. For example, the encoding device may output a residual block between an original block and a predicted block, output transform coefficients by performing a transform procedure on the residual samples (residual array of samples) included in the residual block, and output quantized transform coefficients by performing a quantization procedure on the transform coefficients, such as that it can signal the associated remainder information to the decoder (via a bit stream). Here, the remainder information may include value information, location information, a transform circuit, a transform kernel, a quantization parameter, or the like of quantized transform coefficients. The decoding apparatus may perform a dequantization/deconversion procedure and output residual samples (or a residual block of samples) based on the residual information. The decoding apparatus may generate a reconstructed block based on the predicted block and the residual block. The encoding apparatus may output a residual block by dequantizing/deconverting the quantized transform coefficients for the next picture inter-prediction reference and may generate a reconstructed picture based on this.

[101] FIG. 4 schematically illustrates a multiple transform method in accordance with an embodiment of the present disclosure.

[102] With reference to FIG. 4, the converter may correspond to a converter in the encoding device according to FIG. 2, and the inverse converter may correspond to the inverse converter in the encoding device according to FIG. 2 or an inverse converter in the decoding device according to FIG. 3.

[103] The converter can output the (primary) conversion coefficients by performing a primary conversion based on the residual samples (residual sample array) in the residual block (S410). This primary transformation may be referred to as the main (base) transformation. Here, the primary transform may be based on a multiple transform selection (MTS), and when a multiple transform is applied as a primary transform, it may be referred to as a multiple primary transform.

[104] The multiple base transform may represent a transform method further using discrete cosine transform (DCT) type 2 and discrete sine transform (DST) type 7, DCT type 8 and/or DST type 1. That is, the multiple base transform may represent a method transform to transform the spatial domain residual signal (or residual block) into frequency domain transform coefficients (or primary transform coefficients) based on a plurality of transform kernels selected from DCT type 2, DST type 7, DCT type 8 and DST type 1. Here, the coefficients The primary conversion may be referred to as the temporary conversion coefficients from the converter's point of view.

[105] In other words, when a conventional transform method is applied, transform coefficients can be generated by applying a spatial domain to frequency domain transform to a residual signal (or residual block) based on DCT type 2. In contrast, when a multiple fundamental transform is applied, transform coefficients (or primary transform coefficients) can be generated by applying a spatial domain to frequency domain transform to the residual signal (or residual block) based on DCT type 2, DST type 7, DCT type 8 and/or DST type 1. Here, DCT Type 2, DST Type 7, DCT Type 8, and DST Type 1 may be referred to as transform types, transform kernels, or transform bases. These DCT/DST transformation types can be determined based on basis functions.

[106] When multiple core transformation is performed, a vertical transformation kernel and a horizontal transformation kernel for a target block can be selected from the transformation kernels, a vertical transformation can be performed on the target block based on the vertical transformation kernel, and a horizontal transformation can be performed on the target block based on the kernel horizontal transformation. Here, a horizontal transform may indicate a transform on the horizontal components of the target block, and a vertical transform may indicate a transform on the vertical components of the target block. The vertical transform kernel/horizontal transform kernel may be adaptively determined based on the prediction mode and/or transform index for the target block (CU or subblock) including the residual block.

[107] In addition, according to the example, if the primary transformation is performed by applying MTS, the mapping relation for the transformation kernels can be set by setting special basis functions to predefined values and combining the basis functions for use in the vertical transformation or horizontal transformation. For example, when the horizontal transformation kernel is expressed as trTypeHor and the vertical direction transformation kernel is expressed as trTypeVer, the value 0 for trTypeHor or trTypeVer can be set to DCT2, the value 1 for trTypeHor or trTypeVer can be set to DST7, and the value 2 for trTypeHor or trTypeVer can be installed in DCT8.

[108] In this case, the MTS index information may be encoded and signaled to the decoder to indicate any one of a plurality of sets of transform cores. For example, MTS index 0 may indicate that both trTypeHor and trTypeVer are 0, MTS 1 may indicate that both trTypeHor and trTypeVer are 1, MTS index 2 may indicate that trTypeHor is 2 and trTypeVer is 1, MTS index 3 may indicate that trTypeHor is 1 and trTypeVer is 2, and MTS index 4 may indicate that both trTypeHor and trTypeVer are 2.

[109] In one example, sets of transformation kernels according to MTS index information are shown in the following table.

[110] [Table 1]

[112] The converter can output modified (secondary) conversion coefficients by performing secondary conversion based on the (primary) conversion coefficients (S420). The primary transform is a conversion from the spatial domain to the frequency domain, and the secondary transform refers to the conversion to a more compressed expression using the correlation existing between the (primary) transform coefficients. The secondary transformation may include an inseparable transformation. In this case, the secondary transform may be called a non-separable secondary transform (NSST) or a mode-dependent non-separable secondary transform (MDNSST). The inseparable secondary transform may be a transform that generates modified transform coefficients (or secondary transform coefficients) for the residual signal by a secondary transform, based on the inseparable transform matrix of the (primary) transform coefficients derived by the primary transform. However, vertical transformation and horizontal transformation cannot be applied separately (or horizontal and vertical transformations cannot be applied independently) to the (primary) transformation coefficients, but transformations can be applied simultaneously based on the inseparable transformation matrix. In other words, the inseparable secondary transform may represent a transform method that is not separately applied in the vertical direction and horizontal direction for the (primary) transform coefficients, and, for example, the two-dimensional signals (transform coefficients) are reordered into a one-dimensional signal through some specific direction (for example, the direction row-first or column-first directions), and then modified transform coefficients (or secondary transform coefficients) are generated based on the inseparable transform matrix. For example, according to line first order, M×N blocks are arranged in a line in the order of first line, second line, ... and Nth line. According to column first order, MCN blocks are arranged in a line in the order of first column, second column, ... and Nth column. An inseparable secondary transform may be applied to the top-left region of a block configured with (primary) transform coefficients (hereinafter may be referred to as a transform coefficient block). For example, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 8, then an inseparable secondary 8×8 transform may be applied to the top-left 8×8 region of the transform coefficient block. Additionally, if the width (W) and height (H) of the transform coefficient block are equal to or greater than 4, and the width (W) or height (H) of the transform coefficient block is less than 8, then an inseparable 4x4 secondary transform can be applied to the top-left region min(8,W) × min(8,H) of the transform coefficient block. However, the embodiment is not limited to this, and for example, even if one of the conditions that the width (W) or height (H) of the transform coefficient block is equal to or greater than 4 is satisfied, then an inseparable 4×4 secondary transform can be applied to the upper- left area min(8,W) × min(8,H) of the block of transformation coefficients.

[112] Specifically, for example, if a 4x4 input block is used, then the non-separable secondary transform can be performed as follows.

[113] The X 4x4 input block can be represented as follows.

[114] [Equation 1]

[115] If X is represented in vector form, then the vector can be presented as below.

[116] [Equation 2]

[117] In Equation 2, the vector is the one-dimensional vector obtained by reordering the two-dimensional block X in Equation 1 according to row-first order.

[118] In this case, the secondary inseparable transform can be calculated as follows.

[119] [Equation 3]

[120] In this equation, represents a vector of transform coefficients, and T represents a 16x16 (non-separable) transform matrix.

[121] Through the above Equation 3, a 16×1 vector of transform coefficients can be derived , And can be rearranged into a 4x4 block using scan order (horizontal, vertical, diagonal, etc.). However, the above calculation is an example, and a Givens hypercube transform (HyGT) or the like can be used to calculate the inseparable secondary transform to reduce the computational complexity of the inseparable secondary transform.

[122] Meanwhile, in the non-separable secondary transformation, the transformation kernel (transformation base, transformation type) can be selected as mode-dependent. In this case, the mode may include an intra-prediction mode and/or an inter-prediction mode.

[123] As described above, the non-separable secondary transform may be performed based on an 8×8 transform or a 4×4 transform determined based on the width (W) and height (H) of the transform coefficient block. 8×8 transform refers to a transform that is applicable to an 8×8 region included in a block of transform coefficients when both W and H are equal to or greater than 8, and the 8×8 region may be a top-left 8×8 region in the block of conversion coefficients. Similarly, a 4x4 transform refers to a transform that is applicable to a 4x4 region included in a block of transform coefficients when both W and H are equal to or greater than 4, and the 4x4 region may be a top-left region of 4 ×4 in the conversion coefficient block. For example, an 8×8 matrix of a transform kernel may be a 64×64/16×64 matrix, and a 4×4 matrix of a transform kernel may be a 16×16/8×16 matrix.

[124] Here, to select a mode-dependent transform kernel, two non-separable secondary transform kernels per transform set for the non-separable secondary transform can be configured for both the 8×8 transform and the 4×4 transform, and there can be four transform sets. That is, four transform sets can be configured for an 8x8 transform, and four transform sets can be configured for a 4x4 transform. In this case, each of the four transform sets for the 8×8 transform may include two 8×8 transform kernels, and each of the four transform sets for the 4×4 transform may include two 4×4 transform kernels.

[125] However, since the transformation size, that is, the size of the region to which the transformation is applied, may be of a size other than 8×8 or 4×4, for example, the number of sets may be n, and the number of transformation cores in each set may be k.

[126] A set of transformations may be referred to as an NSST set or a LFNST set. A particular set among the transformation sets may be selected, for example, based on the intra-prediction mode of the current block (CU or subblock). Low Frequency Nonseparable Transform (LFNST) may be an example of a shortened nonseparable transform that will be described later, and represents a nonseparable transform for the low frequency component.

[127] For reference, for example, an intra-prediction mode may include two non-directional (or non-angular) intra-prediction modes and 65 directional (or angular) intra-prediction modes. The non-directional intra-prediction modes may include planar intra-prediction mode No. 0 and DC intra-prediction mode No. 1, and the directional intra-prediction modes may include 65 intra-prediction modes No. 2 to 66. However, this is an example , and this document can be applied even if the number of intra-prediction modes differs. Meanwhile, in some cases, the intra-prediction mode No. 67 can be further used, and the intra-prediction mode No. 67 can represent the linear model (LM) mode.

[128] FIG. 5 shows as an example the intra-directional modes of 65 prediction directions.

[129] With reference to FIG. 5, based on the intra-prediction mode 34 having a left-up diagonal prediction direction, the intra-prediction modes can be divided into intra-prediction modes having a horizontal direction and intra-prediction modes having a vertical direction. In fig. 5, H and V denote horizontal directivity and vertical directivity, respectively, and the numbers -32 to 32 indicate offsets in units of 1/32 at a discrete grid position. These numbers can represent an offset for the mode index value. Intra-prediction modes 2 to 33 have a horizontal focus, and intra-prediction modes 34 to 66 have a vertical focus. Strictly speaking, the intra-prediction mode 34 may be considered to be neither horizontal nor vertical, but may be classified as being of horizontal orientation when defining the secondary transform transform set. This is because the input data is transposed to be used for the vertical direction mode symmetrically based on the intra-prediction mode 34, and the input data alignment method for the horizontal mode is used for the intra-prediction mode 34. Transposing the input data means that the rows and columns of a two-dimensional block of M×N data are switched to N×M data. The intra-prediction mode 18 and the intra-prediction mode 50 may represent the horizontal intra-prediction mode and the vertical intra-prediction mode, respectively, and the intra-prediction mode 2 may be referred to as the diagonal right-up intra-prediction mode since the intra-prediction mode 2 -prediction has a left reference pixel and performs prediction in a right-up direction. Likewise, intra-prediction mode 34 may be referred to as a right-down diagonal intra-prediction mode, and intra-prediction mode 66 may be referred to as a left-down diagonal intra-prediction mode.

[130] According to the example, four sets of transformations in accordance with the intra-prediction mode can be displayed, for example, as shown in the following table.

[131] [Table 2]

[132] As shown in Table 2, any one of the four sets of transformations, that is, lfnstTrSetIdx, can be mapped to any of the four indices, that is, from 0 to 3, in accordance with the intra-prediction mode.

[133] When it is determined that a particular set is used for an inseparable transformation, one of the k transformation kernels in the particular set can be selected using the inseparable secondary transformation index. The encoding device may output a non-separable secondary transform index indicating a particular transform kernel based on a rate-distortion (RD) test, and may signal the non-separable secondary transform index to the decoding device. The decoder may select one of the k transform kernels in a particular set based on the inseparable secondary transform index. For example, lfnst index value 0 may refer to a first non-separable secondary transform kernel, lfnst index value 1 may refer to a second non-separable secondary transform kernel, and lfnst index value 2 may refer to a third non-separable secondary transform kernel. Alternatively, an lfnst index value of 0 may indicate that the first non-separable secondary transform is not applied to the target block, and lfnst index values 1 to 3 may indicate three transform cores.

[134] The transformer may perform a non-separable secondary transform based on the selected transform kernel and may obtain modified (secondary) transform coefficients. As described above, the modified transform coefficients can be output as transform coefficients quantized by a quantizer, and can be encoded and signaled to a decoding device and transmitted to a dequantizer/inverse converter in the encoding device.

[135] Meanwhile, as described above, if the secondary transform is omitted, the (primary) transform coefficients that are the output of the primary (separable) transform can be output as transform coefficients quantized by a quantizer as described above, and can be encoded and signaled to the decoder and transmitted to the dequantizer/inverter in the encoder.

[136] The inverse converter can perform a number of procedures in the reverse order of how they were performed in the above-described converter. The inverse converter can receive (dequantized) transform coefficients and output (primary) transform coefficients by performing a secondary (inverse) transform (S450) and can obtain a residual block (residual samples) by performing a primary (inverse) transform on the (primary) transform coefficients (S460 ). In this regard, the primary conversion coefficients may be called modified conversion coefficients from the point of view of the inverse converter. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed picture based on the reconstructed block.

[137] The decoding device may further include a secondary inverse transform application determiner (or an element for determining whether the secondary inverse transform should be applied) and a secondary inverse transform determination (or an element for determining whether the secondary inverse transform is applied). The secondary inverse transform application determiner may determine whether the secondary inverse transform should be applied. For example, the secondary inverse transform may be NSST, RST, or LFNST, and the secondary inverse transform application determiner may determine whether to apply the secondary inverse transform based on the secondary transform flag obtained by parsing the bit stream. In another example, the secondary inverse transform application determiner may determine whether to apply the secondary inverse transform based on the transform coefficient of the residual block.

[138] The secondary inverse transform determinant may determine the secondary inverse transform. In this case, the secondary inverse transform determiner may determine the secondary inverse transform applied to the current block based on the LFNST transform set (NSST or RST) specified in accordance with the intra-prediction mode. In an embodiment, the method for determining the secondary transformation may be determined depending on the method for determining the primary transformation. Various combinations of primary transforms and secondary transforms may be determined in accordance with the intra-prediction mode. Additionally, in the example, the secondary inverse transform determiner may determine the region to which the secondary inverse transform is applied based on the size of the current block.

[139] Meanwhile, as described above, if the secondary (inverse) transform is omitted, the (dequantized) transform coefficients can be received, the primary (separable) inverse transform can be performed, and the residual block (residual samples) can be obtained. As described above, the encoding device and the decoding device can generate a reconstructed block based on the residual block and the predicted block, and can generate a reconstructed picture based on the reconstructed block.

[140] Meanwhile, in the present disclosure, reduced secondary transform (RST), in which the size of the transformation matrix (kernel) is reduced, can be applied in the NSST concept to reduce the amount of computation and memory required for the non-separable secondary transform.

[141] Meanwhile, the transformation kernel, the transformation matrix, and the coefficient forming the transformation kernel matrix, that is, the kernel coefficient or the matrix coefficient described in the present disclosure, can be expressed in 8 bits. This may be a requirement for implementation in the decoding apparatus and the encoding apparatus, and may reduce the amount of memory required to store the conversion kernel, at a performance penalty that may be rationally tolerated, compared to the existing 9-bit or 10-bit cases. Additionally, expressing the kernel matrix in 8 bits may allow the use of a small multiplier and may be more suitable for Single Instruction Multiple Data (SIMD) architecture instructions used for optimal software implementation.

[142] In this specification, the term "RST" may mean a transform that is performed on the residual samples for a target block based on a transform matrix that is reduced in size according to a reduction factor. In the case of performing a reduced transformation, the amount of computation required for the transformation can be reduced due to the reduction in the size of the transformation matrix. That is, RST can be used to solve the problem of computational complexity encountered in non-separable transformation or large block size transformation.

[143] RST may be defined by various terms such as shorthand transform, shortened secondary transform, shorthand transform, simplified transform, simple transform, and the like, and the name RST is not limited to these examples. Alternatively, since RST is primarily performed in the low-frequency region involving a non-zero coefficient in the transform block, it may be referred to as low-frequency non-separable transform (LFNST). The transformation index may be referred to as the LFNST index.

[144] Meanwhile, when the secondary inverse transform is performed based on the RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse shortened secondary transformer that outputs modified transform coefficients based on the inverse RST transform coefficients, and an inverse primary converter that outputs residual samples for the target block based on the inverse primary transform of the modified transform coefficients. The inverse of the primary transformation refers to the inverse of the primary transformation applied to the remainder. In the present disclosure, deriving a transformation coefficient based on a transformation may refer to deriving a transformation coefficient by applying a transformation.

[145] FIG. 6 is a diagram illustrating an RST in accordance with an embodiment of the present disclosure.

[146] In the present disclosure, a "target block" may refer to a current block to be encoded, a residual block, or a transform block.

[147] In RST, according to the example, an N-dimensional vector can be mapped to an R-dimensional vector located in another space, so that a reduced transformation matrix can be defined, where R is less than N. N can mean the square of the side length of the block , to which the transformation is applied, or the total number of transformation factors corresponding to the block to which the transformation is applied, and the shortened factor may indicate the R/N value. A shortened factor (ratio) may be referred to as a shortened factor, a shortened factor, a simplified factor, a simple factor, or defined by various other terms. Meanwhile, R may be referred to as a short ratio, but depending on the circumstances, the short ratio may mean R. Moreover, depending on the circumstances, the short ratio may mean N/R value.

[148] In an example, a shortened factor or shortened coefficient may be signaled by a bit stream, but the example is not limited to this. For example, a predetermined value for a shortened factor or shortened coefficient may be stored in each of the encoding device 200 and decoding device 300, in which case, the shortened factor or shortened coefficient may not be signaled separately.

[149] The size of the reduced transform matrix according to the example may be R×N smaller than the N×N size of the normal transform matrix, and may be determined as in Equation 4 below.

[150] [Equation 4]

[151] The matrix T in the reduced transform block shown in FIG. 6(a) may mean the T _R×N matrix of Equation 4. As shown in FIG. 6(a), when the reduced transform matrix T _R×N is multiplied by the residual samples for the target block, the transform coefficients for the target block can be output.

[152] In the example, if the block size to which the transformation is applied is 8×8 and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 6(a) can be expressed as a matrix operation as shown in Equation 5 below. In this case, the memory and multiplication calculation can be reduced to about 1/4 by means of a shortening factor.

[153] In the present disclosure, a matrix operation can be understood as the operation of multiplying a column vector by a matrix located to the left of the column vector to obtain a column vector.

[154] [Equation 5]

[155] In Equation 5, r ₁ to r ₆₄ may represent residual samples for the target block and may specifically be transform coefficients generated by applying the primary transform. As a result of computing Equation 5, the conversion coefficients ci for the target block can be derived, and the process of deriving ci can be as in Equation 6.

[156] [Equation 6]

[157] As a result of computing Equation 6, the conversion coefficients from ₁ to c _R for the target block can be derived. That is, when R=16, the conversion coefficients from ₁ to c ₁₆ for the target block can be derived. If, instead of an RST, a regular transform is applied, and a 64x64 (NxN) transformation matrix is multiplied by the 64x1 (Nx1) residual samples, then only 16 (R) transform coefficients are output for the target block as applied RST, although 64(N) transform coefficients are derived for the target block. Since the total number of transform coefficients for the target block is reduced from N to R, the amount of data transmitted by the encoding device 200 to the decoding device 300 is reduced, so that the transmission efficiency between the encoding device 200 and the decoding device 300 can be improved.

[158] When considered in terms of the size of the transformation matrix, the size of the regular transformation matrix is 64×64 (N×N), but the size of the reduced transformation matrix is reduced to 16×64 (R×N), so the memory usage in case of performing RST may be reduced in R/N ratio compared to the case of performing a regular conversion. In addition, compared with the number of N×N multiplication calculations in the case of using a regular transformation matrix, using a reduced transformation matrix can reduce the number of multiplication calculations in the ratio R/N (R×N).

[159] In an example, transformer 232 of encoder 200 may output transform coefficients for a target block by performing a primary transform and an RST-based secondary transform on the residual samples for the target block. These transform coefficients may be transmitted to the inverse transformer of the decoding apparatus 300, and the inverse transformer 322 of the decoding apparatus 300 may output modified transform coefficients based on the inverse reduced secondary transform (RST) for the transform coefficients, and may output residual samples for the target block based on the inverse primary transform for modified conversion factors.

[160] The size of the inverse RST matrix T _R×N according to the example is N×R smaller than the N×N size of the regular inverse transform matrix, and is in a transposition relationship with the reduced transform matrix T _R×N shown in Equation 4.

[161] The matrix T ^t in the reduced inverse transform block shown in FIG. 6(b) may denote the inverse RST matrix T _R×N ^T (the superscript T denotes transposition). When the inverse RST matrix T _R×N ^T is multiplied by the transform coefficients for the target block, as shown in FIG. 6(b), modified transform coefficients for the target block or residual samples for the current block can be output. The inverse RST matrix T _R×N ^T can be expressed as (T _R×N ^T ) _NxR .

[162] More specifically, when the inverse RST is applied as a secondary inverse transform, modified transform coefficients for the target block can be output when the inverse RST matrix T _R×N ^T is multiplied by the transform coefficients for the target block. Meanwhile, the inverse RST can be applied as an inverse primary transform, in which case, residual samples for the target block can be output when the inverse RST matrix T _R×N ^T is multiplied by the transform coefficients for the target block.

[163] In the example, if the block size to which the inverse transform is applied is 8×8 and R=16 (ie, R/N=16/64=1/4), then RST according to FIG. 6(b) can be expressed as a matrix operation as shown in Equation 7 below.

[164] [Equation 7]

[165] In Equation 7, ₁ to ₁₆ may represent the conversion coefficients for the target block. As a result of computing Equation 7, r _i representing modified transform coefficients for the target block or residual samples for the target block can be output, and the derivation process for r _i can be as in Equation 8.

[166] [Equation 8]

[167] As a result of computing Equation 8, r ₁ to r _N representing the modified transform coefficients for the target block or the residual samples for the target block can be output. When considered in terms of the size of the inverse transform matrix, the size of the regular inverse transform matrix is 64×64 (N×N), but the size of the reduced inverse transform matrix is reduced to 64×16 (R×N), so that the memory usage in case of performing the inverse The RST can be reduced in the R/N ratio compared to the case of performing a regular inverse transform. Additionally, compared with the number of N×N multiplication calculations in the case of using a regular inverse transform matrix, using a reduced inverse transform matrix can reduce the number of multiplication calculations in the ratio R/N (N×R).

[168] The transform set configuration shown in Table 2 can be applied to 8x8 RST. That is, 8x8 RST can be applied according to the transform set in Table 2. Since one transform set includes two or three transforms (kernels) according to the intra-prediction mode, it can be configured to select one of up to four transformations, including the case where a secondary transformation is not applied. In a transformation where a secondary transformation is not applied, this can be taken into account to apply the identity matrix. Assuming that indexes 0, 1, 2 and 3 are respectively assigned to the four transforms (for example, index 0 could be allocated for the case where the identity matrix is applied, that is, the case where the secondary transform is not applied), the index transform or index lfnst as syntax element may be signaled for each block of transform coefficients, thereby indicating the transform applied. That is, for a top-left 8x8 block, using the transform index, you can specify 8x8 RST in the RST configuration or specify 8x8 lfnst when LFNST is applied. 8×8 lfnst and 8×8 RST refer to transformations applicable to an 8×8 area included in a transform coefficient block when the W and H of the target block to be transformed are equal to or greater than 8, and the 8×8 area can be top-left 8x8 region in the transform coefficient block. Similarly, 8x8 lfnst and 8x8 RST refer to transformations applicable to an 8x8 region included in a block of transform coefficients when the W and H of the target block are equal to or greater than 4, and the 4x4 region may be top- left 4x4 region in the transform coefficient block.

[169] According to an embodiment of the present disclosure, for transformation in the encoding process, only 48 pieces of data can be selected, and a maximum 16x48 matrix of transformation kernels can be applied to them, instead of applying a 16x64 matrix of transformation kernels to 64 pieces of data , forming an 8x8 area. Here, "maximum" means that m has a maximum value of 16 in the m×48 matrix of the transformation kernel for generating m coefficients. That is, when RST is performed by applying an m×48 matrix of transformation kernels (m≤16) to an 8×8 region, 48 pieces of data are input and m coefficients are generated. When m is 16, 48 pieces of data are input and 16 coefficients are generated. That is, assuming that 48 pieces of data form a 48x1 vector, the 16x48 matrix and the 48x1 vector are sequentially multiplied, thereby generating a 16x1 vector. Here, 48 pieces of data forming an 8x8 area can be properly arranged to form a 48x1 vector. For example, a 48×1 vector may be formed based on the 48 pieces of data forming a region excluding the bottom-right 4×4 region among the 8×8 regions. Here, when the matrix operation is performed by applying a maximum 16×48 matrix of transformation kernels, 16 modified transformation coefficients are generated, and the 16 modified transformation coefficients can be arranged in the upper-left region of 4×4 according to the scanning order, and the upper-right region of 4 The ×4 and bottom-left 4x4 area can be filled with zeros.

[170] For inverse transformation in the decoding process, a transposed matrix of the above-described transformation kernel matrix can be used. That is, when the inverse RST or LFNST is performed in the inverse transform process by the decoding device, the input coefficient data to which the inverse RST is applied is configured into a one-dimensional vector according to a predetermined arrangement order, and a vector of modified coefficients obtained by multiplying the one-dimensional vector and the corresponding inverse matrix The RST to the left of the one-dimensional vector can be arranged into a two-dimensional block according to a predefined arrangement order.

[171] Finally, in the transformation process, when RST or LFNST is applied to the 8×8 area, the matrix operation is applied to the 48 transform coefficients in the upper-left, upper-right and lower-left regions of the 8×8 area, excluding the lower-right area among the transform coefficients in an 8x8 region and a 16x48 matrix of transform kernels. For matrix operation, 48 transformation coefficients are entered in a one-dimensional array. When the matrix operation is performed, 16 modified transform coefficients are output, and the modified transform coefficients can be arranged in the top-left region in the 8×8 region.

[172] In contrast, in the inverse transform process, when the inverse RST or LFNST is applied to the 8×8 region, 16 transform coefficients corresponding to the top-left region in the 8×8 region among the transform coefficients in the 8×8 region can be input in a one-dimensional array according to the scanning order and can undergo matrix operation with a 48x16 matrix transformation kernel. That is, the matrix operation can be expressed as (48×16 matrix)*(16×1 vector of transform coefficients)=(48×1 vector of modified transform coefficients). Here, an n×1 vector can be interpreted to have the same value as an n×1 matrix, and can thus be expressed as an n×1 column vector. Also, * means matrix multiplication. When the matrix operation is performed, 48 modified transform coefficients can be output, and the 48 modified transform coefficients can be arranged in the top-left, top-right, and bottom-left regions in the 8×8 area excluding the bottom-right region.

[173] When the secondary inverse transform is based on RST, the inverse transformer 235 of the encoding device 200 and the inverse transformer 322 of the decoding device 300 may include an inverse shortened secondary transformer for deriving modified transform coefficients based on the inverse RST on the transform coefficients and an inverse primary transformer for deriving residual samples for the target block based on the inverse primary transform on the modified transform coefficients. The inverse of the primary transformation refers to the inverse of the primary transformation applied to the remainder. In the present disclosure, deriving a transformation coefficient based on a transformation may refer to deriving a transformation coefficient by applying a transformation.

[174] The above-described non-partitioned transform, LFNST, will be described in detail as follows. The LFNST may include a forward transform by an encoding device and a reverse transform by a decoding device.

[175] The encoder takes the result (or part of the result) output after applying the primary transform as input and applies the forward secondary transform (secondary transform).

[176] [Equation 9]

[177] In Equation 9, x and y are the inputs and outputs of the secondary transform, respectively, and G is a matrix representing the secondary transform, and the transform basis vectors are formed by column vectors. In the case of inverse LFNST, when the dimension of the transformation matrix G is expressed as [number of rows X number of columns], in the case of direct LFNST, the transposition of the matrix G becomes the dimension G ^T .

[178] For the inverse LFNST, the dimensions of matrix G are [48×16], [48×8], [16×16], [16×8], and matrix [48×8] and matrix [16×8] are partial matrices , obtained by discretizing 8 basis vectors of the left transformation of the [48×16] matrix and the [16×16] matrix, respectively.

[179] On the other hand, for direct LFNST, the dimensions of matrix G ^T are [16×48], [8×48], [16×16], [8×16], and matrix [8×48] and matrix [ 8×16] are partial matrices obtained by discretizing 8 basis vectors of the transformation on top of the [16×48] matrix and the [16×16] matrix, respectively.

[180] Therefore, in the case of direct LFNST, the [48×1] vector or the [16×1] vector is possible as the x input, and the [16×1] vector or the [8×1] vector is possible as the y output. In video encoding and decoding, the output of the direct primary transform is two-dimensional (2D) data, so to generate a [48×1] vector or a [16×1] vector as input x, a 1D vector must be formed by properly arranging the 2D data, which is direct conversion output.

[181] FIG. 7 is a diagram illustrating a layout sequence of direct primary transform output data into a one-dimensional vector according to an example. Left diagrams (a) and (b) in Fig. 7 shows the sequence for generating the vector [48×1], and the right diagrams (a) and (b) in FIG. 7 shows the sequence for generating the [16×1] vector. In the case of LFNST, a 1D vector x can be obtained by sequentially arranging the 2D data in the same order as in (a) and (b) in FIG. 7.

[182] The layout direction of the forward primary transform output data may be determined in accordance with the intra-prediction mode of the current block. For example, when the intra prediction mode of the current block is in the horizontal direction with respect to the diagonal direction, the forward primary transform output data can be arranged in the order of FIG. 7(a), and when the intra-prediction mode of the current block is in the vertical direction relative to the diagonal direction, the forward primary transform output data can be arranged in the order of FIG. 7(b).

[183] According to the example, a different arrangement order from the arrangement order in FIG. 7(a) and (b), and to obtain the same result (vector y) as obtained by applying the layout orders in FIG. 7(a) and (b), the column vectors of matrix G can be rearranged according to the layout order. That is, you can rearrange the column vectors of G so that each element composing the vector x is always multiplied by the same basis vector of the transformation.

[184] Since the output y obtained by Equation 9 is a one-dimensional vector, when two-dimensional data is required as input in the process of using the result of direct secondary transformation as input, for example, in the process of performing quantization or remainder encoding, the output vector y in Equation 9 must be properly reassembled as 2D data.

[185] FIG. 8 is a diagram illustrating a layout sequence of the output of the direct secondary transform into a two-dimensional vector in accordance with the example.

[186] In the case of LFNST, the output values can be arranged into a 2D block according to a predefined scan order. Fig. 8(a) shows that when the output y is a [16×1] vector, the output values are arranged at 16 positions of the 2D block according to the diagonal scan order. Fig. 8(b) shows that when the output y is the vector [8×1], the output values are arranged in 8 positions of the 2D block according to the diagonal scan order, and the remaining 8 positions are filled with zeros. X in fig. 8(b) indicates that this position is filled with zero.

[187] According to another example, since the order in which the output vector y is processed when performing quantization or remainder encoding may be preset, the output vector y may not be arranged into a 2D block as shown in FIG. 8. However, in the case of remainder encoding, the data encoding may be performed in 2D block units (eg 4x4) such as CG (coefficient group), in which case, the data is arranged according to a specific order, such as diagonal order scanning in Fig. 8.

[188] Meanwhile, the decoding apparatus can configure the one-dimensional input vector y by arranging the two-dimensional data output by a dequantization process or the like according to a predetermined scan order for inverse transformation. The input vector y can be output as the output vector x through the following equation.

[189] [Equation 10]

x=Gy

[190] In the case of inverse LFNST, the output vector x can be derived by multiplying the input vector y, which is a [16×1] vector or an [8×1] vector, by the matrix G. For inverse LFNST, the output vector x can be either a vector [48×1], or vector [16×1].

[191] The output vector x is arranged into a two-dimensional block according to the order shown in FIG. 7, and is assembled as two-dimensional data, and this two-dimensional data becomes the input data (or part of the input data) of the inverse primary transform.

[192] Accordingly, the inverse secondary transformation is generally the opposite process of the direct secondary transformation, and in the case of the inverse transformation, unlike the forward direction, the inverse secondary transformation is first applied and then the inverse primary transformation is applied.

[193] In the inverse LFNST, one of 8 matrices [48×16] and 8 matrices [16×16] can be selected as the transformation matrix G. Whether to use a [48x16] matrix or a [16x16] matrix depends on the size and shape of the block.

[194] In addition, 8 matrices can be derived from four sets of transformations, as shown in Table 2 above, and each set of transformations can consist of two matrices. Which transform set to use among the 4 transform sets is determined according to the intra-prediction mode, and more specifically, the transform set is determined based on the value of the wide-angle intra-prediction (WAIP) extended intra-prediction mode. Which matrix to choose from the two matrices that make up the selected set of transformations is inferred using index signaling. More specifically, 0, 1, and 2 are possible as the passed index value, 0 may indicate that LFNST is not applied, and 1 and 2 may indicate any one of the two transformation matrices constituting the transformation set selected based on the intra-prediction mode value.

[195] FIG. 9 is a diagram illustrating wide-angle intra-prediction modes in accordance with an embodiment of the present document.

[196] A normal intra-prediction mode value may have values from 0 to 66 and 81 to 83, and an intra-prediction mode value expanded due to WAIP may have a value from -14 to 83, as shown. Values from 81 to 83 indicate the CCLM mode (Cross Component Linear Model), and values from -14 to -1 and values from 67 to 80 indicate the intra-prediction mode extended due to the application of WAIP.

[197] When the width of the current prediction block is greater than the height, the top reference pixels are typically closer to positions within the prediction block. Therefore, it would be more accurate to predict in the lower-left direction than in the upper-right direction. In contrast, when a block's height is greater than its width, the left reference pixels tend to be closer to positions inside the predicted block. Therefore, it may be more accurate to predict in the top-right direction than in the bottom-left direction. Therefore, it may be preferable to apply remapping, i.e. modification of the mode index to the wide-angle intra-prediction mode index.

[198] When wide-angle intra-prediction is applied, information about the existing intra-prediction may be signaled, and after the information is parsed, the information may be re-mapped to the wide-angle intra-prediction mode index. Therefore, the total number of intra-prediction modes for a particular block (eg, a non-square block of a particular size) may not change, and the total number of intra-prediction modes is 67, and the encoding of the intra-prediction mode for a specific block may not change.

[199] Table 3 below shows the process of deriving the modified intra-mode by re-mapping the intra-prediction mode to the wide-angle intra-prediction mode.

[200] [Table 3]

The inputs to this process are:
- variable predModeIntra, specifying the intra-prediction mode,
- variable nTbW, specifying the width of the conversion block,
- variable nTbH, specifying the height of the transformation block,
- variable cIdx, specifying the color component of the current block.
The output of this process is a modified intra-prediction mode predModeIntra.
The variables nW and nH are output as follows:
- If IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not equal to 0, then the following applies:
nW=nTbW
nH=nTbH
- Otherwise (IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, and cIdx is equal to 0), then the following applies:
nW=nCbW
nH=nCbH
The whRatio variable is set to Abs(Log2(nW/nH)).
For non-square blocks (nW is not equal to nH), the intra-prediction mode predModeIntra is modified as follows:
- If all of the following conditions are true, then predModeIntra is set to (predModeIntra+65),
-- nW is greater than nH
-- predModeIntra is greater than or equal to 2
-- predModeIntra is less than (whRatio > 1)?(8+2* whRatio) : 8
- Otherwise, if all of the following conditions are true, then predModeIntra is set to (predModeIntra - 67),
-- nH is greater than nW
-- predModeIntra is less than or equal to 66
-- predModeIntra is greater than (whRatio > 1)?(60-2* whRatio): 60

[201] In Table 3, the value of the extended intra-prediction mode is finally stored in the variable predModeIntra, and ISP_NO_SPLIT indicates that the CU is not subdivided by the intra-subdivision (ISP) method currently adopted in the VVC standard, and the value is 0 1 and 2 of the cIdx variable indicate the case of the brightness components, Cb and Cr, respectively. The Log2 function, shown in Table 3, returns the log value in base 2, and the Abs function returns the absolute value.

[202] A predModeIntra variable indicating the intra prediction mode, and the height and width of the transform block, etc. are used as input values of the wide-angle intra-prediction mode display process, and the output value is the modified intra-prediction mode predModeIntra. The height and width of the transform block or encoding block may be the height and width of the current block to re-display the intra-prediction mode. In this case, the whRatio variable, reflecting the ratio of width to height, can be set to Abs(Log2(nW/nH)).

[203] For a non-square block, the intra-prediction mode can be divided into two cases and modified.

[204] First, if all conditions (1)~(3) are satisfied, (1) the width of the current block is greater than the height, (2) the intra-prediction mode before modification is equal to or greater than 2, (3) the mode intra prediction is less than the value derived from (8+2*whRatio) when whRatio is greater than 1, and less than 8 when whRatio is less than or equal to 1 [predModeIntra less than (whRatio>1)? (8+2*whRatio):8], intra prediction mode is set to 65 greater than intra prediction mode [predModeIntra is set to (predModeIntra+65)].

[205] If, contrary to the above, that is, the following conditions (1)~(3) are satisfied: (1) the height of the current block is greater than the width, (2) the intra-prediction mode before modification is less than or equal to 66, (3 ) intra prediction mode is greater than the value derived from (60−2*whRatio) when whRatio is greater than 1, and greater than 60 when whRatio is less than or equal to 1 [predModeIntra is greater than (whRatio>1 )? (60−2*whRatio):60], intra prediction mode is set to 67 less than intra prediction mode [predModeIntra is set to (predModeIntra−67)].

[206] Table 2 above shows how a set of transforms is selected based on the intra-prediction mode value extended by WAIP in LFNST. As shown in FIG. 9, modes 14 to 33 and modes 35 to 80 are symmetrical with respect to the prediction direction around mode 34. For example, mode 14 and mode 54 are symmetrical with respect to the direction corresponding to mode 34. Therefore, the same set of transformations applies to modes located in mutually symmetrical directions , and this symmetry is also reflected in Table 2.

[207] Meanwhile, it is assumed that the forward LFNST input data for mode 54 is symmetrical with the forward LFNST input data for mode 14. For example, for mode 14 and mode 54, two-dimensional data is reordered into one-dimensional data according to the arrangement order shown in FIG. . 7(a) and FIG. 7(b), respectively. Moreover, it can be seen that the patterns are in the order shown in FIG. 7(a) and FIG. 7(b) are symmetrical with respect to the direction (diagonal direction) indicated by mode 34.

[208] Meanwhile, as described above, which transformation matrix from the [48×16] matrix and the [16×16] matrix is applied to the LFNST is determined by the size and shape of the transformation target block.

[209] FIG. 10 is a diagram illustrating a block shape to which LFNST is applied. Fig. 10(a) shows 4x4 blocks, (b) shows 4x8 and 8x4 blocks, (c) shows 4×N blocks or Nx4 blocks in which N is 16 or more, (d) shows 8x blocks 8, (e) shows M×N blocks, where M≥8, N≥8 and N>8 or M>8.

[210] In FIG. 10, blocks with thick borders indicate the regions to which LFNST is applied. For blocks according to Fig. 10(a) and (b), LFNST is applied to the top-left 4x4 region, and for the block according to FIG. 10(c), LFNST is applied individually to two top-left 4x4 regions that are adjacent. In fig. 10(a), (b) and (c), since the LFNST is applied in units of 4x4 areas, this LFNST will be referred to as "4x4 LFNST" hereinafter. As the corresponding transformation matrix, [16×16] matrix or [16×8] matrix can be applied based on the dimension of matrix G in Equations 9 and 10.

[211] More specifically, the [16×8] matrix is applied to a 4×4 unit (TU 4×4 or CU 4×4) according to FIG. 10(a), and the [16×16] matrix is applied to the blocks in FIG. 10(b) and (c). This is intended to regulate the worst case computational complexity to 8 multiplications per sample.

[212] Referring to FIG. 10(d) and (e), LFNST is applied to the top-left 8×8 region, and this LFNST is hereinafter referred to as “8×8 LFNST”. The corresponding transformation matrix can be a [48×16] matrix or a [48×8] matrix. In the case of forward LFNST, since the [48×1] vector (vector x in Equation 9) is input as input, the values of all samples of the top-left 8×8 region are not used as input values of forward LFNST. That is, as can be seen in the left layout in FIG. 7(a) or the left layout in FIG. 7(b), the [48×1] vector can be generated based on the samples belonging to the remaining 3 4×4 blocks, leaving the bottom right 4×4 block as it is.

[213] The [48×8] matrix may be applied to an 8×8 unit (TU 8×8 or CU 8×8) in FIG. 10(d), and the [48×16] matrix can be applied to the 8×8 block in FIG. 10(e). This is also intended to regulate the worst case computational complexity to 8 multiplications per sample.

[214] Depending on the block shape, when the corresponding forward LFNST (4×4 LFNST or 8×8 LFNST) is applied, 8 or 16 outputs are generated (the y vector in Equation 9, the [8×1] vector or [16×1 ]). In direct LFNST, the number of outputs is equal to or less than the number of inputs due to the characteristics of the matrix G ^T .

[215] FIG. 11 is a diagram illustrating an arrangement of forward LFNST output data according to an example, and shows a block in which forward LFNST output data is arranged according to a block shape.

[216] The shaded area of the top-left block shown in FIG. 11 corresponds to the region where the forward LFNST output data is located, the positions labeled 0 indicate the samples filled with 0 values, and the remaining region represents the regions not modified by the forward LFNST. In a region not modified by LFNST, the forward primary transform output remains unchanged.

[217] As described above, since the dimension of the applied transformation matrix varies according to the block shape, the number of output data also varies. As shown in FIG. 11, the forward LFNST output may not completely fill the top-left 4x4 block. In the case of FIG. 11(a) and (d), the [16×8] matrix and the [48×8] matrix are applied to the block indicated by the thick line, or the partial region within the block, respectively, and the [8×1] vector is generated as the output of the forward LFNST. That is, according to the scanning order shown in FIG. 8(b), only 8 output data can be filled as shown in FIG. 11(a) and (d), and 0 can be filled in the remaining 8 positions. In the case of the applied LFNST block according to FIG. 10(d), as shown in FIG. 11(d), the two 4x4 blocks at the top-right and bottom-left of the top-left 4x4 block are also filled with 0 values.

[218] As described above, generally by signaling the LFNST index, whether LFNST is to be applied and the transformation matrix to be applied is specified. As shown in FIG. 11, when LFNST is applied, since the number of output data of forward LFNST may be equal to or less than the number of input data, an area filled with zero values occurs as follows.

[219] 1) As shown in FIG. 11(a), samples from the 8th and subsequent positions in the scanning order in the top-left 4x4 block, that is, samples from the 9th to the 16th.

[220] 2) As shown in FIG. 11(d) and (e), when a [16×48] matrix or an [8×48] matrix is used, two 4×4 blocks adjacent to the top-left 4×4 block, or the second and third 4×4 blocks in scan order .

[221] Therefore, if non-zero data exists when checking areas 1) and 2), then it is clear that LFNST is not applied, so the corresponding LFNST index signaling can be omitted.

[222] According to an example, for example, in the case of LFNST adopted in the VVC standard, since LFNST index signaling is performed after encoding the remainder, the encoder can know whether there are non-zero data (significant coefficients) for all positions in the TU or CU by coding the remainder. Accordingly, the encoding apparatus may determine whether to perform signaling on the LFNST index based on the existence of non-zero data, and the decoding apparatus may determine whether the LFNST index is analyzed. When non-zero data does not exist in the area specified in 1) and 2 above), LFNST index signaling is performed.

[223] Since a truncated unary code is used as a binarization method for the LFNST index, the LFNST index consists of up to two bins, and 0, 10, and 11 are assigned as the binary codes for possible values 0, 1, and 2 of the LFNST index, respectively. In the case of LFNST currently adopted for VVC, context-based CABAC coding is applied to the first bin (regular coding) and bypass coding is applied to the second bin. The total number of contexts for the first bin is 2 when (DCT-2, DCT-2) is applied as a primary transform pair for the horizontal and vertical directions, and the luma component and the chrominance component are dual-tree encoded, one context is extracted and the other context is applied for the remaining cases. The LFNST index coding is shown in the table as follows.

[224] [Table 4]

[225] Meanwhile, for the adopted LFNST, the following simplification methods can be applied.

[226] (i) According to the example, the number of outputs for direct LFNST can be limited to a maximum of 16.

[227] In the case of FIG. 10(c), 4×4 LFNST can be applied to two 4×4 regions adjacent top-left, respectively, and in this case, a maximum of 32 LFNST outputs can be generated. When the number of output data for direct LFNST is limited to a maximum of 16, in case of 4×N/N×4 (N≥16) blocks (TU or CU), 4×4 LFNST is applied to only one 4×4 area from top-left, only one LFNST can be applied times to all blocks in Fig. 10. With this, the implementation of image encoding can be simplified.

[228] FIG. 12 shows that the number of outputs for direct LFNST is limited to a maximum of 16 according to the example. As shown in FIG. 12, when LFNST is applied to the top-left 4x4 region in a 4×N block or an NC4 block in which N is 16 or more, the output of the forward LFNST becomes 16 pieces.

[229] (ii) According to the example, nulling may additionally be applied to an area for which LFNST does not apply. As used herein, nulling may mean filling the values of all positions belonging to a particular region with the value 0. That is, nulling may be applied to a region that is not changed due to LFNST and retains the result of the direct primary transformation. As described above, since the LFNST is divided into 4x4 LFNST and 8x8 LFNST, the nulling can be divided into two types ((ii)-(A) and (ii)-(B)) as follows.

[230] (ii)-(A) When 4x4 LFNST is applied, the area to which 4x4 LFNST is not applied can be set to zero. Fig. 13 is a diagram illustrating nulling in a block to which a 4x4 LFNST is applied according to an example.

[231] As shown in FIG. 13, with respect to the block to which the 4x4 LFNST is applied, that is, for all blocks in FIG. 11(a), (b) and (c), the entire region to which LFNST does not apply can be filled with zeros.

[232] On the other hand, FIG. 13(d) shows that when the maximum value of the number of forward LFNST output data is limited to 16, as shown in FIG. 12, zeroing is performed on the remaining blocks to which the 4x4 LFNST does not apply.

[233] ii)-(B) When 8x8 LFNST is applied, the area to which 8x8 LFNST is not applied can be set to zero. Fig. 14 is a diagram illustrating nulling in a block to which 8x8 LFNST is applied according to an example.

[234] As shown in FIG. 14, with respect to the block to which the 8x8 LFNST is applied, that is, for all blocks in FIG. 11(d) and (e), the entire region to which LFNST does not apply can be filled with zeros.

[235] iii) Due to the nulling presented in (ii) above, the area filled with zeros may not be the same when LFNST is applied. Accordingly, it is possible to check whether non-zero data exists, in accordance with the nulling proposed in (ii), in a wider region than in the case of LFNST according to FIG. eleven.

[236] For example, when (ii)-(B) is applied, after checking whether non-zero data exists, where the region is filled with zero values in FIG. 11(d) and (e) in addition to the area filled with 0 additionally in FIG. 14, signaling for the LFNST index can only be performed when non-zero data does not exist.

[237] Of course, even if the nulling proposed in (ii) is applied, it is possible to check whether non-zero data exists in the same way as the existing LFNST index signaling. That is, after checking whether non-zero data exists in the block filled with zeros in FIG. 11, LFNST index signaling may be applied. In this case, only the encoding device performs zeroing, and the decoding device does not assume zeroing, that is, by only checking whether non-zero data exists only in the region explicitly marked as 0 in FIG. 11 can parse against the index LFNST.

[238] Alternatively, according to another example, zeroing may be performed as shown in FIG. 15. Fig. 15 is a diagram illustrating nulling in a block to which 8x8 LFNST is applied according to another example.

[239] As shown in FIG. 13 and 14, nulling may be applied to all regions other than the region to which LFNST is applied, or nulling may only be applied to a partial region, as shown in FIG. 15. Nulling only applies to areas other than the top-left 8x8 area in FIG. 15, nulling cannot be applied to the bottom-right 4x4 block within the top-left 8x8 region.

[240] Various embodiments can be obtained that employ combinations of simplification methods ((i), (ii)-(A), (ii)-(B), (iii)) for LFNST. Of course, combinations of the above simplification methods are not limited to the following embodiments, and any combination may be applied to LFNST.

[241] Embodiment

[242] - Limit the number of outputs for direct LFNST to a maximum of 16 → (i)

[243] - When 4x4 LFNST is applied, all areas to which 4x4 LFNST is not applied are reset to zero → (ii)-(A)

[244] - When 8x8 LFNST is applied, all areas to which 8x8 LFNST is not applied are reset to zero → (ii)-(B)

[245] - After checking whether there are non-zero data, existing areas filled with zero values, and areas filled with zeros due to additional zeros ((ii)-(A), (ii)-(B)), the LFNST index is signaled, only when non-null data does not exist → (iii)

[246] In the case of an embodiment where LFNST is applied, the region in which non-zero output data can exist is limited within the top-left 4x4 region. More specifically, in the case of FIG. 13(a) and FIG. 14(a), the 8th position in the scan order is the last position where non-zero data can exist. In the case of FIG. 13(b) and (c) and FIG. 14(b), the 16th position in the scan order (i.e., the position of the top-right edge of the top-left 4x4 block) is the last position where data other than 0 can exist.

[247] Therefore, when LFNST is applied, after checking whether non-zero data exists at a position where the remainder encoding process is not enabled (at a position beyond the last position), it can be determined whether the LFNST index is signaled.

[248] In the case of the nulling method proposed in (ii), since the number of data ultimately generated when both the primary transform and LFNST are applied, the amount of computation required to perform the entire transform process can be reduced. That is, when LFNST is applied, since nulling is applied to the output of the forward primary transform existing in an area in which LFNST is not applied, there is no need to generate data for the area that becomes null during the execution of the forward primary transform. Accordingly, the amount of computation required to generate the corresponding data can be reduced. Additional results of the nulling method proposed in (ii) are formulated as follows.

[249] First, as described above, the amount of computation required to perform the entire conversion process is reduced.

[250] In particular, when (ii)-(B) is applied, the worst-case computation amount is reduced so that the conversion process can be simplified. In other words, in general, a large amount of computation is required to perform a large-scale primary transformation. By applying (ii)-(B), the number of data output as a result of executing direct LFNST can be reduced to 16 or less. In addition, as the size of the entire block (TU or CU) increases, the effect of reducing the size of the conversion operation further increases.

[251] Secondly, the amount of computation required for the entire conversion process can be reduced, thereby reducing the power consumption required to perform the conversion.

[252] Third, the latency associated with the conversion process is reduced.

[253] A secondary transform, such as LFNST, adds computation to the existing primary transform, thereby increasing the overall latency associated with performing the transform. In particular, in the case of intra-prediction, since the reconstructed data of neighboring blocks is used in the prediction process during encoding, the increase in latency due to the secondary transformation results in an increase in the latency before reconstruction. This may result in an increase in the overall intra-prediction encoding latency.

[254] However, if the nulling proposed in (ii) is applied, the delay time for performing the primary transformation can be noticeably reduced; when LFNST is applied, the delay time for the entire transformation is maintained or shortened, so that the encoding device can be implemented more simply.

[255] Meanwhile, in conventional intra-prediction, the target encoding block is considered as one encoding unit, and encoding is performed without dividing it. However, ISP (intra-subdivision) encoding refers to performing intra-prediction encoding with the encoding target block divided in the horizontal direction or vertical direction. In this case, a reconstructed block can be generated by performing encoding/decoding in units of divided blocks, and the reconstructed block can be used as a reference block of the next divided block. According to an example, in ISP encoding, one encoding block may be divided into two or four sub-blocks and encoded, and in ISP, intra-prediction is performed on one sub-block with reference to the reconstructed pixel value of the sub-block located adjacent to its left or top side. Further, the term "encoding" can be used as a concept that includes both encoding performed by an encoding device and decoding performed by a decoding device.

[256] Table 5 shows an example of sub-blocks divided according to block sizes when ISP is applied, and sub-divisions divided according to ISP may be referred to as transform units (TU).

[257] [Table 5]

Unit size (CU) Number of splits 4x4 Not available 4x8, 8x4 2 All other cases 4

[258] The ISP divides the block predicted as luminance intra-prediction into two or four subdivisions in the vertical direction or horizontal direction according to the block size. For example, the minimum block size that ISP can apply to is 4x8 or 8x4. If the block size is larger than 4x8 or 8x4, the block is split into four subdivisions.

[259] FIG. 16 and 17 show an example of a sub-block into which one coding block is divided, and more specifically, FIG. 16 is a partitioning example for the case in which the encoding block (width (W) x height (H)) is a 4×8 block or an 8×4 block, and FIG. 17 shows an example of partitioning for a case in which the encoding block is neither a 4×8 block, nor an 8×4 block, nor a 4×4 block.

[260] When ISP is applied, sub-blocks are sequentially encoded according to the partition type, such as horizontal or vertical, left-to-right, or top-to-bottom, and encoding for the next sub-block can be performed after execution up to the reconstruction process through inverse transform and intra- prediction for one subblock. For the leftmost or topmost subblock, as in a conventional intra-prediction method, a reconstructed pixel of an encoding block that has already been encoded is referenced. Additionally, if the previous sub-block is not adjacent to each side of the inner sub-block that follows it, to output reference pixels adjacent to the corresponding side, as in a conventional intra-prediction method, reference is made to the reconstructed pixel of the already encoded adjacent encoding block.

[261] In ISP encoding mode, all sub-blocks can be encoded using the same intra-prediction mode, and a flag indicating whether or not to use ISP encoding and a flag indicating in which direction (horizontal or vertical) can be signaled. partitioning must be performed. As in fig. 16 and 17, the number of sub-blocks can be adjusted to 2 or 4 depending on the shape of the block, and when the size (width x height) of one sub-block is less than 16, partitioning cannot be allowed for the corresponding sub-blocks, or the application of the ISP encoding itself may be limited .

[262] Meanwhile, in the case of the ISP prediction mode, one coding unit is divided into two or four partitioning blocks, that is, sub-blocks, and predicted, and the same intra-prediction mode is applied to the two or four partitioning blocks thus divided.

[263] As described above, as the horizontal direction (if an M×N encoding unit having a horizontal length and a vertical length of M and N, respectively, is divided in the horizontal direction, it is divided into M×(N/2) blocks when divided by two and into M×(N/4) blocks when divided by four) and the vertical direction (if an M×N coding unit is divided in the vertical direction, it is divided into (M/2)×N blocks when divided by two and divided by blocks (M/4)×N when divided by four) are possible as partitioning directions. When partitioning in the horizontal direction, the partition blocks are encoded in order from top to bottom, and when partitioning in the vertical direction, the partition blocks are encoded in order from left to right. The current split block to be encoded may be predicted with reference to the reconstructed pixel values of the top (left) split block in the case of splitting in the horizontal (vertical) direction.

[264] The transformation may be applied to the residual signal generated by the ISP prediction method in units of partitioning blocks. MTS (Multiple Transform Selection) technology, based on the combination of DST-7/DCT-8 as well as the existing DCT-2, can be applied to primary transform (main transform or primary transform) based on forward direction, and LFNST (low frequency inseparable transform) can be applied to the transform coefficient generated in accordance with the primary transform to generate the final modified transform coefficient.

[265] That is, LFNST can also be applied to partition blocks divided by applying the ISP transform mode, and the same intra-prediction mode is applied to the divided transform blocks as described above. Accordingly, when selecting a set of LFNSTs derived based on the intra-prediction mode, the derived set of LFNSTs can be applied to all partitioning blocks. That is, the same intra-prediction mode is applied to all partition blocks, and the same set of LFNST can be applied to all partition blocks.

[266] Meanwhile, according to the example, LFNST can only be applied to transform blocks having horizontal and vertical lengths of 4 or more. Therefore, when the horizontal or vertical length of the partition block divided according to the ISP prediction method is less than 4, LFNST is not applied and the LFNST index is not signaled. Accordingly, when LFNST is applied to each partition block, the corresponding partition block can be considered as one transform block. Of course, when the ISP prediction method is not applied, LFNST may be applied to the coding block.

[267] The application of LFNST to each partition block is described in detail as follows.

[268] According to the example, after applying direct LFNST to an individual partitioning block, and after only 16 coefficients (8 or 16) remain in the top-left 4x4 region according to the scanning order of the transform coefficients, padding zeroing may be applied all remaining positions and areas with a value of 0.

[269] Alternatively, according to the example, when the length of one side of the partition block is 4, LFNST is applied only to the top-left 4x4 region, and when the length of all sides of the partition block, that is, the width and height, are 8 or more ,LFNST can be applied to the remaining 48 coefficients except the 4x4 bottom-right region in the 8x8 top-left region.

[270] Alternatively, according to the example, to adjust the worst-case computational complexity to 8 multiplications per sample, when each partition block is 4x4 or 8x8, only 8 transform coefficients can be derived after applying forward LFNST. That is, if the partition block is 4x4, an 8x16 matrix can be used as a transformation matrix, and if the partition block is 8x8, an 8x48 matrix can be used as a transformation matrix.

[271] Meanwhile, in the current VVC standard, LFNST index signaling is performed in coding units. Accordingly, when the ISP prediction mode is used and LFNST is applied to all partition blocks, the same LFNST index value can be applied to the corresponding partition blocks. That is, when a LFNST index value is transmitted at the coding unit level, the corresponding LFNST index can be applied to all partitioning blocks in the coding unit. As described above, the LFNST index value can have the values 0, 1, and 2; 0 indicates the case in which LFNST is not applied, 1 and 2 indicate two transformation matrices present in the same LFNST set when LFNST is applied.

[272] As described above, the set of LFNST is determined by the intra-prediction mode, and since all partitioning blocks in a coding unit are predicted in the same intra-prediction mode in the case of the ISP prediction mode, the partitioning blocks can refer to the same set of LFNST.

[273] As another example, LFNST index signaling is still performed in coding units, but in the case of ISP prediction mode, without determining whether or not LFNST should be applied uniformly to all partitioning blocks, whether the LFNST index value signaled at the level should be applied coding units, whether or not to apply LFNST to each partitioning block can be determined by a separate condition. Here, a separate condition may be signaled in the form of a flag for each partitioning block by the bitstream, and when the flag value is 1, the index value LFNST signaled at the coding unit level may be applied, and when the flag value is 0, LFNST may not be applied.

[274] Meanwhile, in a coding unit to which the ISP mode is applied, an application example of LFNST when the length of one side of a splitting block is less than 4 is discussed as follows.

[275] First, when the partition block size is N×2 (2×N), LFNST can be applied to the top-left region M×2 (2×M) (here, M≤N). For example, when M=8, the corresponding top-left region becomes 8×2 (2×8), and thus, the region in which 16 residual signals exist can be the input to the forward LFNST, and the R× forward transform matrix can be applied 16 (R≤16).

[276] Here, the direct LFNST matrix may not be the matrix included in the current VVC standard, but a separate additional matrix. In addition, for comprehensive worst-case adjustment, an 8x16 matrix obtained by sampling only the top 8 row vectors of a 16x16 matrix can be used for transformation. The method for adjusting the difficulty will be described in detail below.

[277] Secondly, when the partition block size is N×1 (1×N), LFNST can be applied to the top-left region M×1 (1×M) (here, M≤N). For example, when M=16, the corresponding top-left region becomes 16×1 (1×16), and thus, the region in which 16 residual signals exist may be the forward LFNST input, and an R×16 matrix (R ≤16) direct conversion.

[278] Here, the matrix of the corresponding direct LFNST may not be the matrix included in the current VVC standard, but a separate additional matrix. Additionally, to adjust the worst-case complexity, the 8x16 matrix obtained by sampling only the top 8 row vectors of the 16x16 matrix can be used for the transformation. The method for adjusting the difficulty will be described in more detail below.

[279] The first embodiment and the second embodiment may be used simultaneously, or any one of the two embodiments may be used. Particularly in the case of the second embodiment, since a one-dimensional transform is considered in LFNST, it was found through experiments that the improvement in compression performance that could be obtained from the existing LFNST was not relatively large compared with the cost of signaling the LFNST index. However, in the case of the first embodiment, an improvement in compression performance similar to that obtained from conventional LFNST was observed, that is, in the case of ISP, it can be verified by experiments that the application of LFNST for 2×N and N×2 contributes to the actual compression performance.

[280] In LFNST in the current VVC, symmetry between inter-prediction modes is applied. The same set of LFNST is applied to the two directional modes centered on mode 34 (prediction in the lower-right 45-degree diagonal direction), and, for example, the same set of LFNST is applied to mode 18 (horizontal direction prediction mode) and mode 50 (vertical direction prediction mode). However, in modes 35-66, when direct LFNST is applied, the input data is transposed and then LFNST is applied.

[281] Meanwhile, VVC supports wide-angle intra-prediction (WAIP) mode, and the LFNST set is derived based on the modified intra-prediction mode taking into account the WAIP mode. For modes extended by WAIP, the LFNST set is also determined using symmetry, as in the general directed intra-prediction mode. For example, mode -1 is symmetrical to mode 67, so the same set of LFNSTs applies, and mode -14 is symmetrical to mode 80, so the same set of LFNSTs applies. Modes 67-80 apply the LFNST transformation after transposing the input data before applying direct LFNST.

[282] In the case of LFNST applied to a top-left M×2 (M×1) block, the above-described symmetry for LFNST cannot be applied because the block to which LFNST is applied is non-square. Therefore, instead of applying symmetry based on intra-prediction mode as in LFNST in Table 2, symmetry between an M×2 block (M×1) and a 2×M block (1×M) can be applied.

[283] FIG. 18 is a diagram illustrating the symmetry between an M×2 block (M×1) and a 2×M block (1×M) according to an example.

[284] As shown in FIG. 16, mode 2 in block M×2 (M×1) can be considered symmetrical to mode 66 in block 2×M (1×M), and thus the same set of LFNST can be applied to block 2×M (1 ×M) and block M×2 (M×1).

[285] Here, in order to apply the LFNST set that was applied to the M×2 (M×1) block to the 2×M (1×M) block, the LFNST set is selected based on mode 2 instead of mode 66. That is, before applying direct LFNST, you can apply LFNST after transposing the input data of the 2×M (1×M) block.

[286] FIG. 19 is a diagram illustrating an example of transposing a 2×M block according to the example.

[287] FIG. 19A is a diagram illustrating that LFNST can be applied by reading input data in column first order relative to a 2×M block, and FIG. 19B is a diagram illustrating that LFNST is applied by reading input data in row first order relative to an M×2 (M×1) block. The method of applying LFNST to a top-left M×2 (M×1) block or a 2×M (M×1) block is summarized as follows.

[288] 1. First, as shown in FIG. 19A and 19B, the input data is ordered to form a forward input vector LFNST. For example, with reference to FIG. 18, for the M×2 block predicted in mode 2, the order according to FIG. 19(b), and for a 2×M block predicted in mode 66, after ordering the input data according to the order shown in FIG. 19(a), the LFNST set for mode 2 may be used.

[289] 2. For a block M×2 (M×1), a set of LFNST is determined based on a modified intra-prediction mode taking into account WAIP. As described above, a predefined mapping relationship is established between the intra-prediction mode and the LFNST set, and can be represented by a mapping table as shown in Table 2.

[290] For a 2×M (1×M) block, after obtaining a symmetrical mode relative to the prediction mode (mode 34 in the case of the VVC standard) in a 45 degree right-down diagonal direction from the WAIP-modified intra-prediction mode, the LFNST set is determined based on the corresponding symmetrical mode and mapping table. The symmetric mode (y) with respect to mode 34 can be derived using the following equation. The mapping table is described in more detail below.

[291] [Equation 11]

if 2≤x≤66, y=68-x,

otherwise (x≤-1 or x≥67), y=66-x

[292] 3. When direct LFNST is applied, the conversion coefficient can be output by multiplying the input data prepared by process 1 by the LFNST kernel. The LFNST core may be selected from the set of LFNSTs determined in process 2 and the predefined LFNST index.

[293] For example, if M=8 and a 16x16 matrix is used as the LFNST kernel, then 16 transform coefficients can be generated by multiplying the matrix by 16 inputs. The generated transform coefficients can be arranged in a top-left 8x2 or 2x8 region according to the scan order used in the VVC standard.

[294] FIG. 20 shows the scanning order for the 8×2 or 2×8 area according to the example.

[295] All regions other than the top-left 8x2 or 2x8 region may be filled with zero values (nulled), or the existing transform factor to which the primary transform was applied may be maintained as is. The predefined LFNST index may be one of the (0, 1, 2) LFNST index values checked in the RD cost calculation when the LFNST index value changes during the encoding process.

[296] For a configuration that results in worst-case computational complexity below a certain level (e.g., 8 multiplications per sample), for example, after generating only eight transform coefficients by multiplying an 8x16 matrix obtained by taking only the top eight rows of a 16x16 matrix , the eight transform coefficients can be arranged according to the scanning order, as shown in FIG. 20, and zeroing can be applied to the remaining coefficient areas. Worst case difficulty adjustment will be described below.

[297] 4. When applying the inverse LFNST, a predefined number (for example, 16) of transform coefficients is set as the input vector, and after selecting a LFNST kernel (for example, a 16x16 matrix), derived from the set of LFNST obtained from process 2 and the LFNST index obtained through parsing, the output vector can be derived by multiplying the LFNST kernel with the input vector.

[298] In the case of an M×2 (M×1) block, the output vector may be ordered according to row first order, as shown in FIG. 19B, and in the case of a 2×M (1×M) block, the output vector may be ordered according to column first order as shown in FIG. 19A.

[299] For the remaining region excluding the region where the corresponding output vector is located in the upper-left region M×2 (M×1) or 2×M (M×2), and the region other than the upper-left region M×2 (M×1) or 2HM (M×2) in the partitioning block, all of which can be configured to be filled with zero values (zeroed) or to store the transform coefficient recovered by the residual encoding and dequantization processes as is.

[300] As in step 3, when generating the input vector, the input data may be ordered according to the scanning order of FIG. 20, and the input vector may be generated with a reduced number of inputs (eg, 8 instead of 16) to adjust the worst-case computational complexity to a certain level or less.

[301] For example, when using 8 inputs and when M=8, only the left 16x8 matrix can be taken from the corresponding 16x16 matrix, and multiplied, thereby obtaining 16 outputs. Worst case difficulty adjustment will be described below.

[302] While the above-described embodiment represents a case in which symmetry is applied between an M×2 (M×1) block and a 2XM (1XM) block when applying LFNST, different sets of LFNST may be applied to the two block shapes, respectively, in according to another example.

[303] Next, various configuration examples of the LFNST set for the ISP mode and a mapping method using the intra-prediction mode will be described.

[304] In the case of ISP mode, the LFNST set configuration may be different from the existing LFNST set configuration. In other words, kernels different from existing LFNST kernels may be used, and a mapping table different from the mapping table between the intra-prediction mode index applied to the current VVC standard and the LFNST set may be used. The mapping table applied to the current VVC standard can be shown in Table 2.

[305] In Table 2, the preModeIntra value means the intra prediction mode value modified to take into account WAIP, and the lfnstTrSetIdx value is an index value indicating a particular LFNST set. Each LFNST set is configured with two LFNST cores.

[306] When the ISP prediction mode is applied, and when the horizontal length and vertical length of each partitioning block are equal to or greater than 4, the same cores as the LFNST cores used in the current VVC standard can be used, and the mapping table can also be used , as it is. Of course, it is also possible to use a different mapping table and LFNST kernels that differ from the current VVC standard.

[307] When the ISP prediction mode is applied and when the horizontal length or vertical length of each partitioning block is less than 4, a LFNST table and kernels different from the current VVC standard may be used. Next, Tables 7-9 show the mapping tables between the intra-prediction mode value (the intra-prediction mode value modified by WAIP) and the LFNST set, which can be applied to an M×2 block (M×1) or a 2HM block (1HM) .

[308] [Table 6]

[309] [Table 7]

[310] [Table 8]

[311] The first mapping table of Table 6 includes seven sets of LFNST, and the mapping table of Table 7 includes four sets of LFNST, and the mapping table of Table 8 includes two sets of LFNST. As another example, when configured with a single LFNST set, the value of lfnstTrSetIdx may be fixed to 0 relative to the value of preModeIntra.

[312] Next, a method for maintaining worst-case computational complexity when LFNST is applied to ISP mode will be described.

[313] In the case of ISP mode, in order to maintain the number of multiplications per sample (or per coefficient or per position) at a certain value or less when LFNST is applied, the application of LFNST may be limited. Depending on the size of the partitioning block, the number of multiplications per sample (or per coefficient or per position) can be kept to 8 or less by applying LFNST as follows.

[314] 1. When the horizontal length and vertical length of the partitioning block are equal to or greater than 4, the same method as the worst-case calculation complexity adjustment method for LFNST in the current VVC standard can be applied.

[315] That is, when the partitioning block is a 4x4 block, instead of a 16x16 matrix, in the forward direction, an 8x16 matrix obtained by sampling the top 8 rows from a 16x16 matrix can be used, and in the reverse direction, a 16x8 matrix can be used, obtained by sampling the left 8 columns from a 16x16 matrix. Additionally, when the partitioning block is an 8x8 block, in the forward direction, instead of a 16x48 matrix, an 8x48 matrix obtained by sampling the top 8 rows of a 16x48 matrix can be used, and in the reverse direction, instead of a 48x16 matrix , a 48×8 matrix can be used, obtained by sampling the left 8 columns from a 48×16 matrix.

[316] In the case of a 4×N or N×4 block (N>4), when direct transformation is performed, the 16 coefficients generated after applying the 16×16 matrix only to the top-left 4×4 block are arranged in the top-left region 4x4, and other regions can be filled with 0 values. Additionally, when performing the inverse transform, the 16 coefficients located in the top-left 4x4 block can be arranged in scan order to configure the input vector, and then the 16 output data can be generated by multiplying with a 16x16 matrix. The generated output data can be ordered in the top-left 4x4 region, and the remaining regions except the top-left 4x4 region can be filled with zeros.

[317] In the case of an 8×N or N×8 block (N>8), when direct transformation is performed, 16 coefficients are generated after applying the 16×48 matrix only to the ROI region in the top-left 8×8 block (the remaining regions, excluding the bottom-right 4x4 block from the top-left 8x8 block), can be ordered in the top-left 4x4 region, and the other regions can be filled with 0 values. Additionally, when performing the inverse transformation, the 16 coefficients located in the top-left 4x4 block, can be arranged in scan order to configure the input vector, and then 48 outputs can be generated by multiplying with a 48x16 matrix. The generated output can be filled in the ROI region, and other regions can be filled with 0 values.

[318] 2. When the partition block size is N×2 or 2×N and LFNST is applied to the top-left region M×2 or 2×M (M≤N), the matrix to which sampling has been applied according to the value can be applied N.

[319] When M=8, for a partition block N=8, that is, an 8x2 or 2x8 block, in the case of direct conversion, instead of a 16x16 matrix, an 8x16 matrix obtained by sampling the top 8 rows can be used from a 16×16 matrix, and in the case of an inverse transformation, instead of a 16×16 matrix, a 16×8 matrix can be used, obtained by sampling the left 8 columns from a 16×16 matrix.

[320] When N is greater than 8, in the case of direct conversion, the 16 output data generated after applying the 16x16 matrix to the top-left 8x2 or 2x8 block can be arranged in the top-left 8x2 block or 2x8, and the remaining regions can be filled with 0 values. When performing the inverse transform, the 16 coefficients located in the top-left block of 8x2 or 2x8 can be arranged in scan order to configure the input vector, and then 16 outputs can be generated by multiplying with a 16x16 matrix. The generated output can be arranged in a top-left 8x2 or 2x8 block, and all other areas can be filled with 0 values.

[321] 3. When the partition block size is N×1 or 1×N and LFNST is applied to the top-left region M×1 or 1×M (M≤N), the matrix to which sampling has been applied according to value N.

[322] When M=16, for a partition block N=16, that is, a 16x1 or 1x16 block, in the case of direct conversion, instead of a 16x16 matrix, an 8x16 matrix obtained by sampling the top 8 rows can be used from a 16×16 matrix, and in the case of an inverse transformation, instead of a 16×16 matrix, a 16×8 matrix can be used, obtained by sampling the left 8 columns from a 16×16 matrix.

[323] When N is greater than 16, in the case of direct conversion, the 16 output data generated after applying the 16×16 matrix to the top-left 16×1 or 1×16 block can be arranged in the top-left 16×1 block or 1x16, and the remaining regions can be filled with 0 values. When performing the inverse transform, the 16 coefficients located in the top-left 16x1 or 1x16 block can be arranged in scan order to configure the input vector, and then 16 outputs can be generated by multiplying with a 16x16 matrix. The generated output can be arranged in a top-left 16x1 or 1x16 block, and all other regions can be filled with 0 values.

[324] As another example, to maintain the number of multiplications per sample (or per coefficient or per position) at some value or less, the number of multiplications per sample (or per coefficient or per position) is based on the ISP encoding unit size rather than ISP partitioning block size can be supported at 8 or less. If there is only one block among the ISP partitioning blocks that satisfies the condition under which LFNST is applied, the LFNST worst-case complexity calculation may be applied based on the corresponding coding unit size rather than the partitioning block size. For example, when a luminance coding block for a certain coding unit is divided into 4 partition blocks of size 4x4 and encoded by the ISP, and when a non-zero transform coefficient exists for two partition blocks among them, the other two partition blocks can be set accordingly to generate 16 coefficients conversions instead of 8 (encoder based).

[325] Next, a method for signaling the LFNST index in the case of the ISP mode will be described.

[326] As described above, the LFNST index can have the values 0, 1, and 2, where 0 indicates that LFNST is not applied, and 1 and 2, respectively, indicate any one of the two LFNST core matrices included in the selected LFNST set. LFNST is applied based on the LFNST kernel matrix selected by the LFNST index. The transmission method of the LFNST index in the current VVC standard will be described as follows.

[327] 1. The LFNST index may be transmitted once for each coding unit (CU), and in the case of a dual tree, individual LFNST indexes may be signaled for a luma block and a chrominance block, respectively.

[328] 2. When the LFNST index is not signaled, the LFNST index value is assumed to be the default value 0. The case where the LFNST index value is assumed to be 0 is as follows.

[329] A. In case of a mode in which no conversion is applied (for example, skip conversion, BDPCM, lossless encoding, etc.)

[330] B. When the primary transformation is not DCT-2 (DST7 or DCT8), that is, when the horizontal direction transformation or the vertical direction transformation is not DCT-2

[331] C. When the horizontal length or vertical length for a luminance block of a coding unit exceeds the size of the transformable maximum luminance transform, for example, when the size of the transformable maximum luminance transform is 64, and when the size for the luminance block of a coding unit is 128×16, LFNST cannot be applied.

[332] In the case of a dual tree, it is determined whether or not the size of the maximum luminance conversion for each of the coding unit for the luminance component and the coding unit for the chrominance component is exceeded. That is, for a luma block, it is checked whether the size of the maximum luma transform transform is exceeded, and for a chroma block, it is checked whether the horizontal/vertical length of the corresponding luma block for the color format and the size of the transformable maximum luma transform are exceeded. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding lumina block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding lumina block is equal to twice the size of the corresponding chroma block. As another example, when the color format is 4:4:4, the horizontal/vertical length and transform size of the corresponding luminance block are the same as the dimensions of the corresponding chrominance block.

[333] A transform of length 64 or a transform of length 32 may mean a transform applied to a width or height having a length of 64 or 32, respectively, and "transform size" may mean 64 or 32 as the corresponding length.

[334] In the case of a single tree, after checking whether the horizontal length or the vertical length of the luminance block exceeds the maximum size of the converted luminance transform block, if it exceeds it, then the LFNST index signaling may be omitted.

[335] D. The LFNST index can be transmitted only when the horizontal length and vertical length of the coding unit are equal to or greater than 4.

[336] In the case of a dual tree, the LFNST index may be signaled only when both the horizontal and vertical lengths for the corresponding component (i.e., luma or chrominance component) are equal to or greater than 4.

[337] In the case of a single tree, the LFNST index may be signaled when both the horizontal and vertical lengths for the luma component are equal to or greater than 4.

[338] E. If the position of the last non-zero coefficient is not the DC position (top-left position of the block), and if the position of the last non-zero coefficient is not the DC position, in the case of a dual-type tree luma block, then the index LFNST is transmitted. In the case of a dual-type tree chromaticity block, if any one of the last non-zero coefficient position for Cb and the last non-zero coefficient position for Cr is not a DC position, the corresponding LNFST index is transmitted.

[339] In the case of a single type tree, if the position of the last non-zero coefficient of any one of the luminance component, the Cb component and the Cr component is not a DC position, the index LFNST is transmitted.

[340] Here, if the value of the coded block flag (CBF) indicating whether or not a transform coefficient exists for one transform block is 0, the position of the last non-zero coefficient for the corresponding transform block is not checked to determine whether or not LFNST is signaled. That is, when the value of the corresponding CBF is 0, since the transformation is not applied to the corresponding block, the position of the last non-zero coefficient may not be taken into account when checking the condition for signaling the LFNST index.

[341] For example, 1) in the case of a dual type tree and a luma component, if the value of the corresponding CBF is 0, the LFNST index is not signaled, 2) in the case of a dual type tree and a chrominance component, if the CBF value for Cb is 0 and the CBF value for Cr is 1, then only the position of the last non-zero coefficient for Cr is checked, and the corresponding LFNST index is passed. 3) In the case of a single tree type, the position of the last non-zero coefficient is checked only for components having a CBF value of 1 for each of luma, Cb and Cr.

[342] F. When it is confirmed that a transform coefficient exists at a position other than a position where a transform coefficient LFNST may exist, the LFNST index alarm may be omitted. In the case of a 4x4 transform block and an 8x8 transform block, the transform coefficients LFNST may exist at eight positions from the DC position according to the scanning order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, when the 4×4 transform block and the 8×8 transform block are absent, the transform coefficients LFNST may exist at sixteen positions from the DC position according to the scanning order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros.

[343] Accordingly, if non-zero transform coefficients exist in a region that should be filled with a zero value after the remainder encoding is advanced, the LFNST index signaling may be omitted.

[344] Meanwhile, the ISP mode can also be applied to only the luma block, or can be applied to both the luma block and the chrominance block. As described above, when ISP prediction is applied, the corresponding coding unit may be divided into two or four partition blocks and predicted, and a transform may be applied to each of the partition blocks. Therefore, when defining the condition for signaling the LFNST index in coding units, it is necessary to take into account the fact that LFNST may apply to the corresponding partitioning blocks. In addition, when the ISP prediction mode is applied only to a specific component (eg, a luminance block), the LFNST index must be signaled in consideration of the fact that only that component is divided into partitioning blocks. The LFNST signaling methods available in ISP mode are summarized as follows.

[345] 1. The LFNST index may be signaled once for each coding unit (CU), and in the case of a dual tree, separate LFNST indexes may be signaled for the luma block and the chrominance block, respectively.

[346] 2. When the LFNST index is not signaled, the LFNST index value is assumed to be the default value 0. The case where the LFNST index value is assumed to be 0 is as follows.

[347] A. In the case of a mode in which no conversion is applied (for example, skip conversion, BDPCM, lossless encoding, etc.)

[348] B. When the horizontal length or vertical length for a luminance block of a coding unit exceeds the size of the transformable maximum luminance transform, for example, when the size of the transformable maximum luminance transform is 64, and when the size for the luminance block of a coding unit is 128×16, LFNST cannot be applied.

[349] Whether or not the LFNST index should be signaled may be determined based on the partitioning block size instead of the coding unit. That is, if the horizontal or vertical length of the partition block for the corresponding luminance block exceeds the size of the converted maximum luminance transform, the LFNST index signaling may be omitted and the LFNST index value may be assumed to be 0.

[350] In the case of a dual tree, it is determined whether or not the size of the maximum luma transform for each of the coding unit or splitter for the luma component and the coding unit or splitter for the chrominance component is exceeded. That is, if the horizontal and vertical lengths of a luminance coding unit or splitter are compared to the maximum luminance transform size, respectively, and at least one of them is greater than the maximum luminance transform size, then LFNST does not apply, and in the case of a coding unit or the chroma partition block, the horizontal/vertical length of the corresponding luminance block for the color format, and the size of the maximum luminance transform being converted are compared. For example, when the color format is 4:2:0, the horizontal/vertical length of the corresponding lumina block is equal to twice the length of the corresponding chroma block, and the transform size of the corresponding lumina block is equal to twice the size of the corresponding chroma block. As another example, when the color format is 4:4:4, the horizontal/vertical length and conversion size of the corresponding luminance block are the same as those for the corresponding chrominance block.

[351] In the case of a single tree, after checking whether or not the horizontal length or vertical length for a luminance block (coding unit or partitioning block) exceeds the maximum size of the luminance transform block being converted, if it exceeds, then the LFNST index alarm may be omitted.

[352] C. If the LFNST included in the current VVC standard is used, the LFNST index can only be transmitted when both the horizontal length and the vertical length of the splitter are equal to or greater than 4.

[353] If LFNST is applied for a 2×M (1×M) or M×2 (M×1) block in addition to the LFNST included in the current VVC standard, the LFNST index can only be transmitted when the split block size is equal to or greater than than a 2CHM (1×M) or M×2 (M×1) block. Here, the expression “block PxQ is equal to or greater than block R×S” means that P≥R and Q≥S.

[354] Thus, the LFNST index can be transmitted only when the partitioning block is equal to or greater than the minimum size to which LFNST applies. In the case of a dual tree, the LFNST index may be signaled only when the splitter for the luma or chrominance component is equal to or greater than the minimum size to which LFNST applies. In the case of a single tree, the LFNST index may be signaled only when the split block for the luma component is equal to or greater than the minimum size to which LFNST applies.

[355] Herein, the expression “M×N block is greater than or equal to K×L block” means that M is greater than or equal to K and N is greater than or equal to L. The expression “M×N block is greater than K×L block” means that M is greater than or equal to K and N is greater than or equal to L, and that M is greater than K or N is greater than L. The expression "M×N block is less than or equal to K×L block" means that M is less than or is equal to K and N is less than or equal to L, while the expression "a block M×N is less than a block K×L" means that M is less than or equal to K and N is less than or equal to L, and that M is less than K, or N is less than L.

[356] D. If the position of the last non-zero coefficient is not the DC position (top-left position of the block) and if the position of the last non-zero coefficient is not the DC position in any one of all partition blocks, in the case of a dual-type tree luma block, then the index is LFNST transmitted. In the case of a dual type tree and a chroma block, if at least one position of the last non-zero coefficient of all partition blocks for Cb (if ISP mode is not applied to the chroma component, the number of partition blocks is considered to be one) and the position of the last non-zero coefficient of all partition blocks for Cr (if the ISP mode is not applied to the chrominance component, the number of partitioning blocks is considered to be one) is not a DC position, the corresponding LNFST index may be transmitted.

[357] In the case of a single type tree, if the position of the last non-zero coefficient of any one of all partitioning blocks for the luma component, the Cb component, and the Cr component is not a DC position, the corresponding LFNST index may be transmitted.

[358] Here, if the value of the coded block flag (CBF) indicating whether a transform coefficient exists for each partition block is 0, the position of the last non-zero coefficient for the corresponding partition block is not checked to determine whether the LFNST index is signaled or not. That is, when the corresponding CBF value is 0 because no transformation is applied to the corresponding block, the position of the last non-zero coefficient for the corresponding partition block is not taken into account when checking the condition for LFNST index signaling.

[359] For example, 1) in the case of a dual type tree and a luma component, if the corresponding CBF value for each partitioning block is 0, the partitioning block is excluded when determining whether or not to signal the LFNST index, 2) in the case of a dual type tree and a component chromaticity, if the CBF value of Cb is 0 and the CBF value of Cr is 1 for each partition block, then only the position of the last non-zero coefficient for Cr is checked to determine whether or not the LFNST index should be signaled, 3) in the case of a single type tree, it is possible to determine whether or not the LFNST index should be signaled by checking the position of the last non-zero coefficient only for blocks having a CBF value of 1 for all luma component, Cb component, and Cr component partitioning blocks.

[360] In the case of the ISP mode, the image information may also be configured such that the position of the last non-zero coefficient is not checked, and an embodiment thereof is as follows.

[361] i. In the case of ISP mode, LFNST index signaling may be enabled without checking the position of the last non-zero coefficient for the luma block and the chrominance block. That is, even if the position of the last non-zero coefficient for all partitioning blocks is a DC position or the corresponding CBF value is 0, LFNST index signaling can be enabled.

[362] ii. In the case of the ISP mode, checking the position of the last non-zero coefficient only for the luma block may be omitted, and in the case of the chrominance block, checking the position of the last non-zero coefficient can be performed in the manner described above. For example, in the case of a dual type tree and a luma block, signaling of the LFNST index is enabled without checking the position of the last non-zero coefficient, and in the case of a dual type tree and a chrominance block, whether or not the corresponding LFNST index is signaled can be checked by checking whether or not a DC exists position for the position of the last non-zero coefficient, as described above.

[363] iii. In case of ISP mode and single tree type, method i or ii can be used. That is, in the case of ISP mode and when option i is applied to a single type tree, it is possible to omit the check for the position of the last non-zero coefficient for both the luma and chrominance blocks and enable LFNST index signaling. Alternatively, when applying option ii, for partition blocks for the luma component, the check for the position of the last non-zero coefficient is omitted, and for partition blocks for the chrominance component (if ISP is not applied for the chrominance component, the number of partition blocks can be considered equal to 1), the position of the last non-zero coefficient is checked in the manner described above, thereby determining whether or not the LFNST index should be signaled.

[364] E. When it is confirmed that a transform coefficient exists in a position other than a position where a transform coefficient LFNST may exist even for one partition block among all partition blocks, the LFNST index signaling may be omitted.

[365] For example, in the case of a 4x4 partitioner and an 8x8 partitioner, the transform coefficients LFNST may exist at eight positions from the DC position according to the scanning order of the transform coefficients in the VVC standard, and the remaining positions are filled with zeros. Additionally, if it is equal to or greater than 4x4, and is neither a 4x4 partition block nor an 8x8 partition block, then the transform coefficients LFNST may exist at 16 positions from the DC position according to the scan order of the transform coefficients in VVC standard, and all other positions are filled with zeros.

[366] Accordingly, if non-zero transform coefficients exist in a region that should be filled with a zero value after the remainder encoding is advanced, the LFNST index signaling may be omitted.

[367] If LFNST can be applied even when the partitioning block is 2×M (1×M) or M×2 (M×1), the area in which the LFNST transform coefficients can be located can be specified as follows. If a region other than the region where transform coefficients may be located may be filled with 0, and if a non-zero transform coefficient exists in the region that should be filled with 0 when LFNST is assumed to be applied, the LFNST index signaling may be omitted.

[368] i. If LFNST can be applied to a 2xM or Mx2 block and M=8, only 8 LFNST transform coefficients can be generated for a 2x8 or 8x2 partition block. When the transform coefficients are arranged in scan order as shown in FIG. 20, 8 conversion coefficients are arranged in scan order from DC position, and the remaining 8 positions can be filled with zeros.

[369] For a 2×N or N×2 (N>8) partitioning block, 16 transform coefficients LFNST can be generated, and when the transform coefficients are arranged in scan order, as shown in FIG. 20, 16 transform coefficients are arranged in scan order from DC position, and the rest of the region can be filled with zeros. That is, in a 2xN or Nx2 (N>8) partition block, an area other than the top-left 2x8 or 8x2 block may be filled with zeros. Even for a 2x8 or 8x2 partition block, 16 transform coefficients can be generated instead of 8 transform coefficients LFNST, in which case, the region that needs to be filled with zeros does not occur. As described above, when LFNST is applied, and when it is detected that a non-zero transform coefficient exists in a region determined to be filled with 0 in at least one partition block, the LFNST index signaling may be omitted and the LFNST index may be assumed to be 0.

[370] ii. If LFNST can be applied to a 1xM or Mx1 block, and M=16, only 8 LFNST transform coefficients can be generated for a 1x16 or 16x1 partition block. When the transform coefficients are arranged in scan order from left to right or top to bottom, the 8 transform coefficients are arranged in the corresponding scan order from the DC position, and the remaining 8 positions can be filled with zeros.

[371] For a 1×N or N×1 (N>16) partitioning block, 16 transform coefficients LFNST can be generated, and when the transform coefficients are arranged in scan order from left to right or top to bottom, the 16 transform coefficients are arranged in the corresponding scan order from DC positions, and the rest of the area can be filled with zeros. That is, in a 1×N or N×1 (N>16) partition block, an area other than the top-left 1×16 or 16×1 block may be filled with zeros.

[372] Even for a 1×16 or 16×1 partition block, 16 transform coefficients may be generated instead of 8 transform coefficients LFNST, in which case, an area that should be filled with zeros does not occur. As described above, when LFNST is applied, and when it is detected that a non-zero transform coefficient exists in a region determined to be filled with 0 in at least one partitioning block, the LFNST index signaling may be omitted and the LFNST index may be assumed to be 0.

[373] Meanwhile, in the case of ISP mode, the length condition is independently considered for the horizontal direction and vertical direction, and DST-7 is applied instead of DCT-2 without signaling for the MTS index. It is determined whether or not the horizontal or vertical length is greater than or equal to 4 and less than or equal to 16, and the primary transformation kernel is determined according to the determination result. Accordingly, in the case of ISP mode where LFNST can be applied, the following transformation combination configuration is possible.

[374] 1. When the LFNST index is 0 (including the case in which the LFNST index is implied to be 0), the ISP-time primary conversion decision condition included in the current VVC standard can be followed. In other words, it can be checked whether the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied or not independently for the horizontal and vertical directions, respectively, and if it is satisfied, then DST-7 can be applied instead of DCT -2 for the primary conversion, and if this is not satisfied, then DCT-2 can be applied.

[375] 2. For the case in which the LFNST index is greater than 0, the following two configurations are possible as the primary transformation.

[376] A. DCT-2 can be used in both horizontal and vertical directions.

[377] B. You can follow the ISP primary conversion decision condition included in the current VVC standard. In other words, it can be checked whether the length condition (equal to or greater than 4, or equal to or less than 16) is satisfied or not independently for the horizontal and vertical directions, respectively, and if it is satisfied, then DST-7 can be applied instead of DCT -2, and if it is not satisfied, then DCT-2 can be applied.

[378] In ISP mode, picture information can be configured such that the LFNST index is transmitted for each partitioning block rather than for each coding unit. In this case, in the above-described LFNST index signaling method, it can be considered that only one partition block exists in the unit in which the LFNST index is transmitted, and it can be determined whether or not to signal the LFNST index.

[379] Meanwhile, the signaling order of the LFNST index and the MTS index will be described below.

[380] According to an example, the LFNST index signaled in the remainder encoding may be encoded after the encoding position for the position of the last non-zero coefficient, and the MTS index may be encoded immediately after the LFNST index. In the case of this configuration, the LFNST index may be signaled for each conversion unit. Alternatively, even if not signaled in the remainder encoding, the index LFNST may be encoded after the encoding for the position of the last significant coefficient, and the index MTS may be encoded after the index LFNST.

[381] The syntax for encoding the remainder according to the example is as follows.

[382] [Table 9]

[383]

[384] The values of the main variables shown in Table 9 are as follows.

[385] 1. cbWidth, cbHeight: width and height of the current encoding block

[386] 2. log2TbWidth, log2TbHeight: The base-2 logarithm value of the width and height of the current transform block, it can be reduced, by reflecting nulling, to the top-left region in which a non-zero coefficient can exist.

[387] 3. sps_lfnst_enabled_flag: A flag indicating whether LFNST is enabled or not, if the flag value is 0 it indicates that LFNST is not enabled and if the flag value is 1 it indicates LFNST is enabled. This is defined in the Sequence Parameter Set (SPS).

[388] 4. CuPredMode[chType][x0][y0]: The prediction mode corresponding to the variable chType and position (x0, y0), chType can have the values 0 and 1, with 0 indicating the luma component and 1 indicating the chrominance component. Position (x0, y0) indicates the position in the picture, and MODE_INTRA (intra-prediction) and MODE_INTER (inter-prediction) are possible as the value of CuPredMode[chType][x0][y0].

[389] 5. IntraSubPartitionsSplit[x0][y0]: The contents of position (x0, y0) are the same as in step 4. This indicates which ISP split is applied at position (x0, y0), ISP_NO_SPLIT indicates that The coding unit corresponding to the position (x0, y0) is not divided into partitioning blocks.

[390] 6. intra_mip_flag[x0][y0]: The contents of position (x0, y0) are the same as in point 4 above. intra_mip_flag is a flag indicating whether or not the matrix-based prediction (MIP) mode is applied. If the flag value is 0, it indicates that MIP is not enabled, and if the flag value is 1, it indicates that MIP is enabled.

[391] 7. cIdx: A value of 0 indicates luma, and values of 1 and 2 indicate Cb and Cr, which are the chrominance components respectively.

[392] 8. treeType: indicates single tree and dual tree, etc. (SINGLE_TREE: single tree, DUAL_TREE_LUMA: dual tree for luma components, DUAL_TREE_CHROMA: dual tree for chroma components).

[393] 9. tu_cbf_cb[x0][y0]: The position contents (x0, y0) are the same as in point 4. This indicates the coded block flag (CBF) for the Cb component. If its value is 0, it indicates that non-zero coefficients are not present in the corresponding transformation unit for the Cb component, and if its value is 1, it indicates that non-zero coefficients are present in the corresponding transformation unit for the Cb component.

[394] 10. lastSubBlock: Indicates the position in the scanning order of the subblock (coefficient group (CG)) in which the last non-zero coefficient is located. 0 indicates a subblock in which the DC component is included, and if the value is greater than 0, it is not a subblock in which the DC component is included.

[395] 11. lastScanPos: Indicates the position where the last significant coefficient is found in the scan order within one sub-block. If one subblock includes 16 positions, values from 0 to 15 are possible.

[396] 12. lfnst_idx[x0][y0]: The LFNST index syntax element to be parsed. If it is not parsed, it is assumed to have a value of 0. That is, the default value is set to 0, indicating that LFNST is not applied.

[397] 13. LastSignificantCoeffX, LastSignificantCoeffY: indicate the x and y coordinates where the last significant coefficient is located in the transformation block. The x-coordinate starts at 0 and increases from left to right, and the y-coordinate starts at 0 and increases from top to bottom. If the values of both variables are 0, it means that the last significant coefficient is located in DC.

[398] 14. cu_sbt_flag: A flag indicating whether the subblock transform (SBT) included in the current VVC standard is enabled or disabled. If the flag value is 0, it indicates that SBT is not enabled, and if the flag value is 1, it indicates that SBT is enabled.

[399] 15. sps_explicit_mts_inter_enabled_flag, sps_explicit_mts_intra_enabled_flag: Flags indicating whether or not explicit MTS applies to inter-CU and intra-CU, respectively. If the value of the corresponding flag is 0, it indicates that MTS is not allowed for inter-CU or intra-CU, and if the value of the corresponding flag is 1, it indicates that MTS is allowed.

[400] 16. tu_mts_idx[x0][y0]: MTS index syntax element to be parsed. If not parsed, it is assumed to have a value of 0. That is, the default value is set to 0, indicating that DCT-2 is enabled for both horizontal and vertical directions.

[401] As shown in Table 9, in the case of a single tree, it is possible to determine whether or not the LFNST index should be signaled using only the position condition of the last significant coefficient for the brightness. That is, if the position of the last significant coefficient is not DC and the last significant coefficient exists in a top-left subblock (CG), for example, a 4x4 block, then the index LFNST is signaled. In this case, in the case of a 4x4 transform block and an 8x8 transform block, the LFNST index is signaled only when the last significant coefficient exists at positions 0-7 in the top-left subblock.

[402] In the case of a dual tree, the LFNST index is signaled independently of each of luma and chrominance, and in the case of chrominance, the LFNST index may be signaled by applying the last significant coefficient position condition to the Cb component only. The corresponding condition may not be checked for the Cr component, and if the CBF value for Cb is 0, the LFNST index may be signaled by applying the last significant coefficient position condition to the Cr component.

[403] 'Min(log2TbWidth, log2TbHeight)>=2' in Table 9 may be expressed as "Min(tbWidth, tbHeight)>=4", and 'Min(log2TbWidth, log2TbHeight)>=4' may be expressed as "Min( tbWidth, tbHeight)>= 16”.

[404] In Table 9, log2ZoTbWidth and log2ZoTbHeight denote the values of logarithms with base width and height equal to 2 (base 2) for the top-left region where the last significant coefficient can exist, when set to zero, respectively.

[405] As shown in Table 9, the log2ZoTbWidth and log2ZoTbHeight values can be updated in two places. The first is before the value of the MTS index or LFNST index is analyzed, and the second is after the MTS index is analyzed.

[406] The first update is performed before parsing the MTS index value (tu_mts_idx[x0][y0]), so that log2ZoTbWidth and log2ZoTbHeight can be set independently of the MTS index value.

[407] After the MTS index is analyzed, log2ZoTbWidth and log2ZoTbHeigh are set for the MTS index greater than 0 (DST-7/DCT-8 combination). When DST-7/DCT-8 is applied independently in each of the horizontal and vertical directions in the primary transform, there can be up to 16 significant coefficients per row or column in each direction. That is, after applying DST-7/DCT-8 with a length of 32 or more, up to 16 conversion factors can be output for each row or column on the left or top. Accordingly, in a 2D block, when DST-7/DCT-8 is applied in both the horizontal direction and the vertical direction, significant coefficients can only exist in the top-left 16×16 region.

[408] Additionally, when DCT-2 is independently applied in each of the horizontal and vertical directions in the current primary transform, there can be up to 32 significant coefficients per row or column in each direction. That is, when using DCT-2 with a length of 64 or more, up to 32 conversion factors can be output for each row or column on the left or top. Accordingly, in a 2D block, when DCT-2 is applied in both the horizontal direction and the vertical direction, significant coefficients can only exist in the upper-left region 32×32.

[409] Additionally, when DST-7/DCT-8 is applied on one side and DCT-2 is applied on the other side to the horizontal and vertical directions, 16 significant coefficients can exist in the former direction, and 32 significant coefficients can exist in the latter direction. For example, in the case of a 64x8 transform block, if DCT-2 is applied in the horizontal direction and DST-7 is applied in the vertical direction (a situation may arise where an implicit MTS is applied), a significant coefficient may exist in the region up to the top-left region 32 ×8.

[410] If, as shown in Table 9, log2ZoTbWidth and log2ZoTbHeight are updated in two places, that is, before parsing the MTS index, the ranges last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be determined by log2ZoTbWidth and log2ZoTbHeight, as shown in the table below.

[411] [Table 10]

7.4.9.11 Remainder encoding semantics
...
last_sig_coeff_x_prefix specifies the column position prefix of the last significant coefficient in scan order in the transform block. Last_sig_coeff_x_prefix values must be in the range 0 to (log2ZoTbWidth << 1)-1, inclusive.
When last_sig_coeff_x_prefix is missing, it is assumed to be 0.
last_sig_coeff_y_prefix specifies the row position prefix of the last significant coefficient in scan order in the transform block. Last_sig_coeff_y_prefix values must be in the range 0 to (log2ZoTbHeight << 1)-1, inclusive.
When last_sig_coeff_y_prefix is missing, it is assumed to be 0.
...

[412] Additionally, in this case, the maximum values of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix can be set by mapping the values of log2ZoTbWidth and log2ZoTbHeight in the binarization process for last_sig_coeff_x_prefix and last_sig_coeff_y_prefix.

[413] [Table 11]

[414] According to the example, in the case where ISP mode and LFNST are applied, when Table 9 signaling is applied, the specification text can be configured as shown in Table 12. Compared with Table 9, the LFNST index signaling condition is only in the case excluding the ISP mode (IntraSubPartitionsSplit[x0][y0]==ISP_NO_SPLIT in Table 9) is removed.

[415] In a single tree, when the LFNST index transmitted for the luma component (cIdx=0) is reused for the chroma component, the LFNST index transmitted for the first ISP partition block in which a significant coefficient exists can be applied to the transform block. Alternatively, even in a single tree, the index LFNST may be signaled for the chroma component separately from the index for the luma component. The descriptions of the variables in Table 12 are the same as in Table 9.

[416] [Table 12]

[417] According to another example, in Table 12, when the last significant coefficient is allowed to be in the DC position for the ISP, the condition for parsing with respect to the LFNST index can be changed as follows.

[418] [Table 13]

[419] According to an example, the LFNST index and/or the MTS index may be signaled at the coding unit level. As described above, the LFNST index can have three values 0, 1 and 2, where 0 indicates that LFNST is not applied, and 1 and 2 respectively indicate the first candidate and second candidate of the two LFNST core candidates included in the selected LFNST set. The LFNST index is encoded using truncated unitary binarization, and the values 0, 1, and 2 can be encoded as bin strings 0, 10, and 11, respectively.

[420] According to the example, LFNST can only be applied when DCT-2 is applied in both the horizontal direction and the vertical direction in the primary transform. Therefore, if the MTS index is signaled after the LFNST index signaling, the MTS index can be signaled only when the LFNST index is 0, and the primary conversion can be performed by applying DCT-2 in both the horizontal direction and the vertical direction, without signaling the MTS index when LFNST index is not 0.

[421] The MTS index may have the values 0, 1, 2, 3 and 4, where 0, 1, 2, 3 and 4 may indicate that DCT-2/DCT-2, DST-7/DST-7, DCT- 8/DST-7, DST-7/DCT-8, DCT-8/DCT-8 are used in horizontal and vertical directions, respectively. In addition, the MTS index can be encoded by truncated unitary binarization and the values 0, 1, 2, 3 and 4 can be encoded as bin strings 0, 10, 110, 1110 and 1111, respectively.

[422] The LFNST index and the MTS index may be signaled at the coding unit level, and the MTS index may then be encoded after the LFNST index at the coding unit level. The encoding unit syntax table for this is as follows.

[423] [Table 14]

[424] Comparing Table 14 with Table 13, the condition for checking whether the value of tu_mts_idx[x0][y0] is 0 (i.e., checking whether DCT-2 is applied in the horizontal direction and in the vertical direction), in the condition for The lfnst_idx[x0][y0] alarm has been changed to a condition to check if transform_skip_flag[x0][y0] is equal to 0 (!transform_skip_flag[x0][y0]). transform_skip_flag[x0][y0] indicates whether the coding unit is encoded in transform skip mode, in which the transform is skipped and a flag is signaled before the MTS index and LFNST index. That is, since lfnst_idx[x0][y0] is signaled before the value tu_mtx_idx[x0][y0] is signaled, only the condition for the transform_skip_flag[x0][y0] value can be checked.

[425] As shown in Table 14, a variety of conditions are checked when encoding tu_mts_idx[x0][y0], and tu_mts_idx[x0][y0] is signaled only when the value of lfnst_idx[x0][y0] is 0, as described above.

[426] tu_cbf_LUMA[x0][y0] is a flag indicating whether a significant coefficient exists for the luma component, and cbWidth and cbHeight indicate the width and height of the coding unit for the luma component, respectively.

[427] In Table 14, (IntraSubPartitionsSplit[x0][y0]== ISP_NO_SPLIT) indicates that ISP mode is not applied, and (!cu_sbt_flag) indicates that SBT is not applied.

[428] According to Table 14, when both the width and height of the coding unit for the luma component are 32 or less, tu_mts_idx[x0][y0] is signaled, that is, whether MTS is applied is determined by the width and height of the coding unit for the component brightness

[429] According to another example, when transform unit (TU) tiling occurs (eg, when the maximum transform size is set to 32, a 64×64 coding unit is divided into four 32×32 transform units and encoded), the MTS index may be signaled based on the size of each conversion block. For example, when both the width and height of a transform block are 32 or less, the same MTS index value may be applied to all transform blocks in a coding unit, thereby applying the same primary transform. In addition, when tiling of a transform block occurs, the value of tu_cbf_LUMA[x0][y0] in Table 14 may be the CBF value for the top-left transform block or may be set to 1 when the CBF value for even one transform block among all conversion blocks is equal to 1.

[430] According to the example, when the ISP mode is applied to the current block, LFNST may be applied, in which case Table 14 may be changed as shown in Table 15.

[431] [Table 15]

[432] As shown in Table 15, even in ISP mode, (IntraSubPartitionsSplitType!=ISP_NO_SPLIT)lfnst_idx[x0][y0] can be configured to be signaled, and the same LFNST index value can be applied to all ISP partitioning blocks.

[433] Additionally, as shown in Table 15, since tu_mts_idx[x0][y0] can only be signaled in the non-ISP mode, the coding portion of the MTS index is the same as in Table 14.

[434] As shown in Table 14 and Table 15, when the MTS index is signaled immediately after the LFNST index, the primary transform information cannot be known when performing remainder encoding. That is, the MTS index is signaled after the remainder is encoded. Accordingly, in the remainder encoding portion, the portion at which zeroing is performed leaving only 16 coefficients for DST-7 or DCT-8 having a length of 32 may be changed as shown below in Table 16.

[435] [Table 16]

residual_coding( x0, y0, log2TbWidth, log2TbHeight, cIdx ) { Descriptor if( ( sps_mts_enabled_flag && cu_sbt_flag && log2TbWidth < 6 && log2TbHeight < 6 ) && cIdx == 0 && log2TbWidth > 4 ) log2ZoTbWidth=4 Else log2ZoTbWidth=Min( log2TbWidth, 5 ) MaxCcbs=2 * ( 1 << log2TbWidth ) * ( 1<< log2TbHeight ) if( ( sps_mts_enabled_flag && cu_sbt_flag && log2TbWidth < 6 && log2TbHeight < 6 ) && cIdx == 0 && log2TbHeight > 4 ) log2ZoTbHeight=4 Else log2ZoTbHeight=Min( log2TbHeight, 5 )

[436]

if( log2TbWidth > 0 ) last_sig_coeff_x_prefix ae(v) if( log2TbHeight > 0 ) last_sig_coeff_y_prefix ae(v) if( last_sig_coeff_x_prefix > 3 ) last_sig_coeff_x_suffix ae(v) if( last_sig_coeff_y_prefix > 3 ) last_sig_coeff_y_suffix ae(v) log2TbWidth=log2ZoTbWidth log2TbHeight=log2ZoTbHeight remBinsPass1=( ( 1 << ( log2TbWidth+log2TbHeight ) ) * 7 ) >> 2 log2SbW=( Min( log2TbWidth, log2TbHeight ) < 2 ? 1 : 2 ) log2SbH=log2SbW if( log2TbWidth+log2TbHeight > 3 ) { if( log2TbWidth < 2 ) { log2SbW=log2TbWidth log2SbH=4 − log2SbW } else if( log2TbHeight < 2 ) { log2SbH=log2TbHeight log2SbW=4 − log2SbH } } numSbCoeff=1 << ( log2SbW+log2SbH ) lastScanPos=numSbCoeff lastSubBlock=( 1 << ( log2TbWidth+log2TbHeight − ( log2SbW+log2SbH ) ) ) - 1 do { if( lastScanPos== 0 ) { lastScanPos=numSbCoeff lastSubBlock−− } lastScanPos−− xS=DiagScanOrder[ log2TbWidth − log2SbW ][ log2TbHeight − log2SbH ] [ lastSubBlock ][ 0 ] yS=DiagScanOrder[ log2TbWidth − log2SbW ][ log2TbHeight − log2SbH ] [ lastSubBlock ][ 1 ] xC=( xS << log2SbW ) + DiagScanOrder[ log2SbW ][ log2SbH ][ lastScanPos ][ 0 ] yC=( yS << log2SbH ) + DiagScanOrder[ log2SbW ][ log2SbH ][ lastScanPos ][ 1 ] } while( ( xC != LastSignificantCoeffX ) || ( yC != LastSignificantCoeffY ) ) if( lastSubBlock== 0 &&log2TbWidth >= 2 && log2TbHeight >= 2 && !transform_skip_flag[ x0 ][ y0 ] && lastScanPos > 0 ) LfnstDcOnly= 0 if( ( lastSubBlock > 0 && log2TbWidth >= 2 &&log2TbHeight >= 2 ) || ( lastScanPos > 7 && ( log2TbWidth== 2 || log2TbWidth== 3 ) && log2TbWidth == log2TbHeight ) ) LfnstZeroOutSigCoeffFlag=0

[437] As shown in Table 16, during the determination process

log2ZoTbWidth and log2ZoTbHeight (where log2ZoTbWidth and log2ZoTbHeight respectively denote the base 2 logarithm values of the width and height of the top-left region remaining after performing zeroing), checking the value of tu_mts_idx[x0][y0] can be omitted.

[438] The binarization of last_sig_coeff_x_prefix and last_sig_coeff_y_prefix in Table 16 can be determined based on log2ZoTbWidth and log2ZoTbHeight, as shown in Table 11.

[439] Additionally, as shown in Table 16, the sps_mts_enable_flag test condition may be added when defining log2ZoTbWidth and log2ZoTbHeight in the residual encoding.

[440] The TR in Table 11 indicates the truncated Rice binarization method, and the last significant coefficient information may be binarized based on cMax and cRiceParam defined in Table 11 according to the method described in the following table.

[441] According to an example, when information about the position of the last significant coefficient of the luma transform block is recorded in the remainder encoding process, the MTS index may be signaled as shown in Table 17.

[442] [Table 17]

[443] In Table 17, LumaLastSignificantCoeffX and LumaLastSignificantCoeffY indicate the X coordinate and Y coordinate of the position of the last significant coefficient for the luminance conversion block, respectively. The condition that LumaLastSignificantCoeffX and LumaLastSignificantCoeffY must be less than 16 has been added to Table 17. When either one is 16 or more, DCT-2 is applied in both the horizontal and vertical directions, so that the alarm for tu_mts_idx[x0][y0] is omitted and may imply that DCT-2 applies to both horizontal and vertical directions.

[444] When both LumaLastSignificantCoeffX and LumaLastSignificantCoeffY are less than 16, it means that the last significant coefficient exists in the top-left 16×16 region. Additionally, this indicates that when DST-7 or DCT-8 length 32 is applied in the current VVC standard, there is a possibility that nulling is applied, which leaves only 16 conversion coefficients on the left or top. Accordingly, tu_mts_idx[x0][y0] may be signaled to indicate the transformation kernel used for the primary transformation.

[445] In another example, the encoding unit syntax table and the remainder encoding syntax table are as follows.

[446] [Table 18]

[447] [Table 19]

[448] In Table 18, MtsZeroOutSigCoeffFlag is initially set to 1, and this value can be changed in the residual coding in Table 19. The value of the variable MtsZeroOutSigCoeffFlag changes from 1 to 0 when a significant coefficient exists in the region (LastSignificantCoeffX > 15 || LastSignificantCoeffY > 15) , to be filled with 0 by zeroing, in which case the MTS index is not signaled, as shown in Table 19.

[449] Meanwhile, the conversion unit level syntax table according to the example is shown as follows.

[450] [Table 20]

[451] As shown in Table 20, MtsZeroOutSigCoeffFlag can be set to 1 when tu_cbf_LUMA[x0][y0] is 1, and when tu_cbf_LUMA[x0][y0] is 0, the existing value of MtsZeroOutSigCoeffFlag can be maintained. Accordingly, when tu_cbf_LUMA[x0][y0] is 0 and the value of MtsZeroOutSigCoeffFlag is maintained at 0, the encoding of mts_idx[x0][y0] may be omitted. That is, if the CBF value of the luma component is 0 because the MTS index is insignificant since no transformation is applied, the encoding of the MTS index may be omitted.

[452] The following drawings have been prepared to explain a specific example of this specification. While the names of specific devices described in the drawings or the names of specific signals/messages/fields are presented by way of example, the technical features of this specification are not limited to the specific names used in the following drawings.

[453] FIG. 21 is a flowchart showing operation of a video decoding apparatus according to an embodiment of the present document.

[454] Each step disclosed in FIG. 21 is based on some of the content described above in FIG. 4-20. Therefore, the explanation for specific content repeating the content described above in FIG. 3-20 will be omitted or given briefly.

[455] According to an embodiment, the decoding apparatus 300 may receive residual information from the bitstream (S2110).

[456] More specifically, the decoding apparatus 300 can decode quantized transform coefficient information for the current block from the bit stream, and can output quantized transform coefficients for the target block based on the quantized transform coefficient information for the current block. Information about the quantized transform coefficients for the target block may be included in a sequence parameter set (SPS) or slice header and may include at least one of information about whether or not a shortened transform (RST) is applied, shortened coefficient information, information about the minimum transform size to which the reduced transform is applied, information about the maximum transform size to which the reduced transform is applied, and information about a transform index indicating any one of the reduced transform size and the transform kernel matrix included in the transform set.

[457] In addition, the decoding device may further receive information about the intra-prediction mode for the current block and information about whether or not ISP is applied to the current block. The decoding apparatus can output whether or not a current block is divided into a predetermined number of subdivision transform blocks by applying and parsing flag information indicating whether or not to apply ISP encoding or ISP mode. Here, the current block may be an encoding block. Additionally, the decoding apparatus may output the size and number of divided subdivision blocks through flag information indicating in which direction the current block will be divided.

[458] The decoding apparatus 300 may output transform coefficients by dequantizing residual information about the current block, that is, quantized transform coefficients (S2120).

[459] The output transform coefficients may be ordered according to a reverse diagonal scan order in units of 4x4 blocks, and the transform coefficients within a 4x4 block may also be ordered according to a reverse diagonal scan order. That is, the transform coefficients that have been dequantized can be ordered in the reverse scanning order of a video codec, such as VVC or HEVC.

[460] The transform coefficient derived from this residual information may be a dequantized transform coefficient as described above, and may also be a quantized transform coefficient. That is, the transform coefficient can be any data capable of checking whether or not it is non-zero data in the current block, regardless of whether it is quantized or not.

[461] The decoder may output residual samples by applying an inverse transform to the quantized transform coefficients.

[462] As described above, the decoding apparatus may output residual samples by applying LFNST, which is an inseparable transform, or MTS, which is a separable transform, and such transforms may be performed based on the LFNST index indicating the LFNST kernel, that is, the matrix LFNST, and an MTS index indicating the MTS core, respectively.

[463] The decoding apparatus may receive and parse at least one of a LFNST index or an MTS index at the coding unit level, and may parse an LFNST index indicating a LFNST core before, that is, immediately before, the index MTS, indicating the MTS core.

[464] The MTS index may be parsed according to a specific condition for the LFNST index, for example, when the LFNST index is 0. Also, according to an example when the tree type of the current block is a dual luma tree or a single tree, and the value of the LFNST index is 0, parsing can be performed against the MTS index.

[465] When the value of the LFNST index is 0, the remainder encoding layer may include syntax for the last significant coefficient position information, and the LFNST index may be obtained by parsing after parsing the last significant coefficient position information.

[466] According to an example, when the current block is a luma block and the index LFNST is 0, parsing may be performed with respect to the MTS index. That is, when the current block is a luma block and if the LFNST index is greater than 0, parsing with respect to the MTS index may not be performed.

[467] According to an example, when the tree type of the current block is a dual tree, the index LFNST for each of the luma block and the chrominance block can be obtained by parsing.

[468] Meanwhile, when outputting a transform coefficient, the width and height for the top-left region in which the last significant coefficient may exist in the current block may be output by zeroing, and the width and height for the top-left region may be output before parsing in relation to the MTS Index.

[469] Meanwhile, the position of the last significant coefficient can be output by the width and height for the top-left region, and the position information of the last significant coefficient can be binarized based on the width and height for the top-left region.

[470] Alternatively, according to an example, when the tree type of the current block is a single tree type, the decoding apparatus may perform residual encoding on the luma block and chrominance block of the current block and may then parse with respect to the index LFNST.

[471] When parsing is performed on the LFNST index at the coding unit level after performing residual encoding not at the transform block level or the residual coding level, the LFNST index displaying the positions of the complete transform coefficients for the luma block and the chrominance block, and nulling information may be received, involved in the conversion process, rather than the conversion coefficient information over either the luma block or the chrominance block.

[472] When the tree type of the current block is a dual tree type and the chrominance component is encoded, the decoding apparatus may perform remainder encoding on the Cb component and the Cr component of the chroma block and then may parse with respect to the index LFNST.

[473] When parsing is performed on an index LFNST at the coding unit level after performing residue encoding not at the transform block level or the residue coding level, the LFNST index mapping the positions of the complete transform coefficients for the Cb component and the Cr component of the chrominance block, and nulling information is received , involved in the conversion process, rather than the conversion coefficient information on any one of the Cb component and the Cr component of the chrominance block.

[474] When the current block is divided into a plurality of subdivision blocks, the decoding apparatus may perform remainder encoding across the plurality of subdivision blocks and may then parse with respect to the index LFNST.

[475] Similar to the above description, when parsing with respect to the LFNST index at the coding unit level after performing remainder encoding not at the transform block level or the remainder encoding level, an LFNST index representing the positions of the complete transform coefficients for all subdivision blocks may be received, and the nulling information involved in the conversion process, rather than the conversion coefficient information over some or individual subdivision blocks.

[476] According to an example, when a current block is divided into a plurality of subdivision blocks, parsing with respect to the index LFNST may be performed regardless of whether a transform coefficient exists in the region excluding the DC position of each of the plurality of subdivision blocks. That is, if the last significant coefficient is only allowed to be in the DC position for all blocks of a subdivision when ISP is applied to the current block, LFNST index signaling can be enabled.

[477] During the remainder encoding process, the decoding apparatus may output a first variable indicating whether a transform coefficient exists in a region excluding the DC position of the current block, and may output a second variable indicating whether a transform coefficient exists in a second region excluding the upper-left the first area of the current block or subdivision block into which the current block is divided.

[478] The decoding apparatus may parse with respect to the LFNST index when a transform coefficient exists in a region excluding the DC position and a transform coefficient does not exist in a second region.

[479] Specifically, to determine whether to parse the LFNST index, the decoding apparatus may output a first variable indicating whether a transform coefficient, that is, a significant coefficient, exists in a region excluding the DC position of the current block.

[480] The first variable may be an LfnstDcOnly variable, which may be output during the remainder encoding process. The first variable may be output as 0 when the index of the sub-block including the last significant coefficient in the current block is 0, and the position of the last significant coefficient in the sub-block is greater than 0, and parsing may be carried out with respect to the index LFNST when the first variable is 0 A sub-block refers to a 4x4 block used as a coding unit in residue coding, and can also be referred to as a coefficient group (CG). A subblock index of 0 indicates the top-left 4x4 subblock.

[481] The first variable may be initially set to 1 and may be stored as 1 or may be changed to 0 depending on whether there is a significant coefficient in the region excluding the DC position.

[482] The variable LfnstDcOnly indicates whether a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, may be equal to 0 when a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, and may be equal to 1 when a non-zero coefficient does not exist at the position of the non-DC component for all transform blocks within one coding unit.

[483] The decoding apparatus may output a second variable indicating whether a significant coefficient exists in a second region excluding the top-left first region of the current block, thereby checking whether a nulling has been performed on the second region.

[484] The second variable may be a LfnstZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when LFNST is applied. The second variable may be initially set to 1 and may be changed to 0 when a significant coefficient exists in the second region.

[485] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last non-zero coefficient exists is greater than 0, and the width and height of the transform block are equal to or greater than 4, or when the position of the last non-zero coefficient in the subblock, in where there is a last non-zero coefficient greater than 7, and the transform block size is 4x4 or 8x8. A sub-block refers to a 4x4 block used as a coding unit in residue coding, and may also be referred to as a coefficient group (CG). A subblock index of 0 indicates the top-left 4x4 subblock.

[486] That is, when a non-zero coefficient is output in a region excluding the top-left region where a transform coefficient LFNST may exist in a transform block, or a non-zero coefficient exists beyond the eighth position in the scanning order for a 4x4 block and an 8x8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[487] According to an example, when ISP is applied to a coding unit, if it is identified that a transform coefficient exists in a position excluding a position where a transform coefficient LFNST may exist for even one subdivision block among all subdivision blocks, the LFNST index signaling may be omitted. That is, when zeroing is not performed in one subdivision block and a significant coefficient exists in a second region, the LFNST index is not signaled.

[488] The first region may be inferred based on the size of the current block.

[489] For example, when the size of the current block is 4x4 or 8x8, the first region may be from the top-left of the current block to the position of the eighth sample in the scanning direction. When the current block is split and the size of the subdivision block is 4x4 or 8x8, the first region may be from the top-left of the subdivision block to the position of the eighth sample in the scan direction.

[490] When the size of the current block is 4x4 or 8x8, eight pieces of data are output by forward LFNST, and therefore the eight transform coefficients received by the decoding device can be ordered from the top-left of the current block to the position of the eighth sample in scanning direction, as shown in FIG. 13(a) and fig. 14(a).

[491] When the size of the current block is not 4x4 or 8x8, the first region may be the top-left 4x4 region of the current block. When the size of the current block is not 4x4 or 8x8, 16 pieces of data are output by forward LFNST, and therefore 16 transform coefficients received by the decoding device can be arranged in the top-left 4x4 region of the current block, as shown in fig. 13(b)-(d) and FIG. 14(b).

[492] Transform coefficients that can be ordered in the first region can be ordered according to the diagonal scanning direction, as shown in FIG. 8.

[493] As described above, when the current block is divided into subdivision blocks, if the transform coefficient does not exist in all separate areas for a plurality of subdivision blocks, the decoding apparatus may parse with respect to the LFNST index. When a transform factor exists in the second region for any one subdivision block, no parsing is performed with respect to the LFNST index.

[494] As described above, LFNST may be applied to a subdivision block having a width and height of 4 or more, and the LFNST index for the current block, which is an encoding block, may be applied to multiple subdivision blocks.

[495] Since the nulling mapped to LFNST (including all nullings affected by the LFNST application) applies to the subdivision block as is, the first area is equally applied to the subdivision block. That is, when the divided subdivision block is a 4x4 block or an 8x8 block, LFNST can be applied from the top-left of the subdivision block to the eighth transform coefficient in the scanning direction, and when the subdivision block is not a 4x4 block or an 8x8 block ,LFNST can be applied to the transform coefficients in the ,top-left region of the 4x4 subdivision block.

[496] In addition, in the remainder encoding process, the decoding apparatus can determine whether a transform coefficient is present in an area other than the upper-left 16×16 area of the current block (S2130), and can parse with respect to the MTS index when the coefficient transformation does not exist in the area except for the 16×16 area (S2140).

[497] To this end, the decoding apparatus may output a third variable indicating whether a transform coefficient exists in a region other than the top-left 16×16 region.

[498] The third variable may be a MtsZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when MTS is applied. The variable MtsZeroOutSigCoeffFlag indicates whether there is a conversion coefficient in the top-left region where the last significant coefficient may exist, by zeroing after performing MTS, that is, the region excluding the 16x16 top-left region, and the value of the third variable may initially be set to 1 and may change from 1 to 0 when the transform coefficient exists in a region other than the top-left 16×16 region. When the value of the third variable is 0, the MTS index is not signaled.

[499] The decoding apparatus may output residual samples by applying at least one of an LFNST performed based on the LFNST index or an MTS performed based on the MTS index, and according to an example, the residual samples may be output by applying an MTS core output to based on the MTS index, to the conversion coefficients of the top-left region 16x16 (S2150).

[500] Next, the decoding apparatus 300 can generate reconstructed samples based on the residual samples for the current block and the prediction samples for the current block (S2160).

[501] The following drawings have been prepared to explain a specific example of the present disclosure. Since the names of specific devices described in the drawings or the names of specific signals/messages/fields are presented by way of example, the technical features of the present description are not limited to the specific names used in the following drawings.

[502] FIG. 22 is a flowchart showing operation of a video encoding apparatus according to an embodiment of the present document.

[503] Each step disclosed in FIG. 22 is based on some of the content described above with reference to FIG. 4-20. Therefore, an explanation for specific content repeating the content described above with reference to FIG. 2 and 4-20 will be omitted or given briefly.

[504] The encoding device 200 according to an embodiment can output prediction samples for the current block based on the intra-prediction mode applied to the current block (S2210).

[505] The encoder may perform prediction for each sub-block of the transform split in the case where ISP is applied to the current block.

[506] The encoding apparatus can determine whether or not ISP encoding or ISP mode should be applied to the current block, that is, the encoding block, and according to the result of the determination, it is possible to determine in which direction the current block should be divided, and output the size and number subblocks to be divided.

[507] The encoder 200 may output residual samples for the current block based on the prediction samples (S2220).

[508] The encoder 200 may output transform coefficients for the current block by applying at least one of LFNST or MTS to the residual samples, and may arrange the transform coefficients according to a predetermined scan order, according to an example, the transform coefficients for the current block may be output based on MTS for residual samples (S2230).

[509] The primary transform may be performed by a plurality of transform kernels, such as MTS, in which case, the transform kernel may be selected based on the intra-prediction mode.

[510] After deriving the transform coefficients by applying the MTS, the encoder may zero out the remaining areas of the current block except for a specific area at the top-left of the current block, for example, a 16×16 area (S2240).

[511] According to an example, the encoder may perform nulling when the MTS is applied in the primary transform of the current block. The encoding apparatus may perform zero-filling of a region excluding the top-left 16×16 region of the current block or sub-block of the zero-splitting region, and may encode the MTS index according to a third variable indicating whether a transform coefficient exists in the nulling region.

[512] The third variable may be a MtsZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when MTS is applied. The variable MtsZeroOutSigCoeffFlag indicates whether there is a conversion coefficient in the upper-left region where the last significant coefficient may exist, by zeroing after performing MTS, that is, the region excluding the upper-left 16x16 region, and the value of the third variable may initially be set to 1 and may change from 1 to 0 when the transform coefficient exists in a region other than the top-left 16×16 region. When the value of the third variable is 0, the MTS index is not encoded or signaled.

[513] According to an example, the encoding device may output the position of the last significant coefficient based on the width and height of the top-left region, and may binarize the position information of the last significant coefficient.

[514] According to the example, the width and height of the top-left region may be output before the MTS index signaling.

[515] Also, encoder 200 can determine whether or not to perform a secondary transform, or non-separable transform, specifically LFNST, on the transform coefficients for the current block, and can apply LFNST to the transform coefficients to output modified transform coefficients.

[516] LFNST is an inseparable transform that applies the transform without splitting the coefficients in a particular direction, as opposed to a primary transform that splits and transforms vertically or horizontally the coefficients that are the targets of the transform. Such a non-separable transform may be a low-frequency non-separable transform that applies the transform only to the low-frequency region rather than to the entire target block that is the target of the transform.

[517] When the ISP is applied to the current block, the encoder may determine whether or not LFNST can be applied to the height and width of the divided sub-block of the partition.

[518] The encoder may determine whether or not LFNST can be applied to the height and width of the divided sub-divider. In this case, the decoding device can parse the LFNST index when the height and width of the partitioning subblock are equal to or greater than 4.

[519] The encoding device can configure and output image information such that at least one of the LFNST index and the MTS index is signaled at the encoding unit level, and the MTS index is signaled immediately after the LFNST index signaling, and the remainder is output by quantizing the transform coefficient information, can be coded (S2250).

[520] The MTS index may be encoded according to a specific condition for the LFNST index, for example, when the LFNST index is 0. Also, according to an example, when the tree type of the current block is a dual luma tree or a single tree and the value of the LFNST index is 0, MTS index can be encoded.

[521] According to an example, when the current block is a luminance block and the index LFNST indicates 0, the encoder may encode the MTS index.

[522] According to an example, when the tree type of the current block is a dual tree, the encoding apparatus may encode an index LFNST for each of the luma block and the chrominance block.

[523] Alternatively, according to an example, when the tree type of the current block is a single tree type, the encoding device may output all transform coefficients for the luma block and chrominance block of the current block and may then encode the LFNST index at the encoding unit level.

[524] When the LFNST index is encoded at the coding unit level after outputting all the transform coefficients, rather than at the transform block level or the remainder coding level, the LFNST index representing the positions of the complete transform coefficients for the luma block and the chrominance block, and the nulling information affected in the process transforms, rather than transform coefficient information per luma block or chrominance block, may be encoded.

[525] When the tree type of the current block is a dual tree type and a chroma component is encoded, the encoding device can output all transform coefficients of the Cb component and the Cr component of the chroma block and can then encode the LFNST index at the encoding unit level.

[526] When the LFNST index is encoded at the coding unit level after outputting all transform coefficients not at the transform block level or at the remainder coding level, the LFNST index representing the positions of the complete transform coefficients for the Cb component and the Cr component of the chrominance block, and the nulling information affected in during the conversion process, rather than the conversion coefficient information of any one of the Cb component and the Cr component of the chrominance block, may be encoded.

[527] When the current block is divided into a plurality of subdivision blocks, the encoder may output all transform coefficients for the plurality of subdivision blocks and may then encode the LFNST index at the encoding unit level.

[528] Similar to the above description, when the LFNST index is analyzed at the coding unit level after deducing all transform coefficients not at the transform block level or at the remainder coding level, the LFNST index displaying the positions of the complete transform coefficients for all subdivision blocks, and the nulling information affected during the transformation process, rather than the information of the transformation coefficients of some or individual subdivision blocks, may be encoded.

[529] When the current block is divided into a plurality of subdivision blocks, the encoder may encode the index LFNST regardless of whether a transform coefficient exists in the region excluding the DC position of each of the plurality of subdivision blocks. That is, if the last significant coefficient is only allowed to be in the DC position for all blocks of a subdivision when ISP is applied to the current block, LFNST index signaling can be enabled.

[530] In the process of deriving transform coefficients, the encoding device may output a first variable indicating whether a transform coefficient exists in a region excluding the DC position of the current block, and a second variable indicating whether a transform coefficient exists in a second region excluding the top-left first the area of the current block or subdivision block into which the current block is divided.

[531] The encoding device can encode the LFNST index when a transform coefficient exists in a region excluding the DC position, and a transform coefficient does not exist in a second region.

[532] Specifically, the first variable may be a LfnstDcOnly variable and may be output as 0 when the index of a sub-block including the last significant coefficient in the current block is 0, and the position of the last significant coefficient in the sub-block is greater than 0, and the index LFNST may be encoded when the first variable is 0.

[533] The first variable may be initially set to 1 and may be stored as 1 or may be changed to 0 depending on whether there is a significant coefficient in the region excluding the DC position.

[534] The LfnstDcOnly variable indicates whether a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, may be equal to 0 when a non-zero coefficient exists at a non-DC component position for at least one transform block within one coding unit, and may be equal to 1 when a non-zero coefficient does not exist at the position of the non-DC component for all transform blocks within one coding unit.

[535] The encoder may zero out a second region of the current block where modified transform coefficients do not exist after performing LFNST, and may output a second variable indicating whether a transform coefficient exists in the second region.

[536] As shown in FIG. 13 and fig. 14, the remaining area of the current block where modified transform coefficients do not exist can be set to zero. Due to this nulling, it is possible to reduce the amount of calculation required to perform the entire conversion process and reduce the amount of operations required for the entire conversion process, thereby reducing the power consumption required to perform the conversion. In addition, the wait caused by the conversion process can be reduced, thereby improving the efficiency of image encoding.

[537] The second variable may be a LfnstZeroOutSigCoeffFlag variable, which may indicate that zeroing is performed when LFNST is applied. The second variable may be initially set to 1 and may be changed to 0 when a significant coefficient exists in the second region.

[538] The variable LfnstZeroOutSigCoeffFlag may be output as 0 when the index of the subblock in which the last nonzero coefficient exists is greater than 0 and both the width and height of the transform block are equal to or greater than 4, or when the position of the last nonzero coefficient in subblock in which the last non-zero coefficient exists is greater than 7, and the transform block size is 4x4 or 8x8.

[539] That is, when a non-zero coefficient is output in a region excluding the top-left region where a transform coefficient LFNST may exist in a transform block, or a non-zero coefficient exists beyond the eighth position in the scanning order for a 4x4 block and an 8x8 block, the variable LfnstZeroOutSigCoeffFlag is set to 0.

[540] The description of the first region and the description of nulling in the case where ISP is applied are essentially the same as the descriptions given in the decoding method, and thus redundant description will be omitted.

[541] Additionally, the encoding device can output quantized transform coefficients by performing quantization based on the transform coefficients or modified transform coefficients for the current block, and can encode and output image information including information about the quantized transform coefficients.

[542] The encoding device may generate residual information including information about quantized transform coefficients. The residual information may include the above-described transformation-related information/syntactic element. The encoding apparatus may encode image/video information including residual information, and output the encoded image/video information in the form of a bitstream.

[543] More specifically, the encoding device may generate information about quantized transform coefficients and encode information about the generated quantized transform coefficients.

[544] In the present disclosure, at least one of quantization/dequantization and/or transform/inverse transform may be omitted. When quantization/dequantization is omitted, the quantized transform coefficient may be referred to as the transform coefficient. When the transform/inverse transform is omitted, the conversion factor may be referred to as the coefficient or residual coefficient, or may still be referred to as the conversion factor for purposes of expression consistency.

[545] Moreover, in the present disclosure, the quantized transform coefficient and the transform coefficient may be referred to as the transform coefficient and the scaled transform coefficient, respectively. In this case, the residual information may include information about the transform coefficient(s), and the information about the transform coefficient(s) may be signaled by a residual encoding syntax. Transform coefficients can be derived based on the residual information (or information about the transform coefficient(s)), and scaled transform coefficients can be derived by inversely transforming (scaling) the transform coefficients. Residual samples can be inferred based on the inverse transform (transform) of the scaled transform coefficients. These details may also be applied/expressed in other parts of this disclosure.

[546] In the above-described embodiments, the methods are explained on the basis of flowcharts, using a sequence of steps or blocks, but the present disclosure is not limited to this order of steps or blocks, and a particular step may be performed in an order other than described above, or simultaneously with another stage. Additionally, one skilled in the art will appreciate that the steps shown in the flowchart above are not exclusive, that another step may be included, or that one or more steps in the flowchart may be deleted without impact to the scope of this disclosure.

[547] The above-described methods according to the present disclosure may be implemented in the form of software, and the encoding device and/or decoding device according to the present disclosure may be included in an image processing device such as a television, computer, smartphone, set-top box , display device, etc.

[548] When embodiments of the present disclosure are implemented in software, the above-described methods may be implemented as modules (processes, functions, etc.) to perform the above-described functions. Modules can be stored in memory and can be executed by the processor. The memory may be internal or external to the processor and may be coupled to the processor by various well known means. The processor may include an application specific integrated circuit (ASIC), other chipsets, logic circuitry, and/or data processing apparatus. The memory may include read only memory (ROM), random access memory (RAM), flash memory, memory card, storage medium, and/or other storage device. That is, the embodiments described in the present disclosure may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional blocks shown in each drawing may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[549] In addition, the decoding device and encoding device to which the present disclosure applies may be included in a multimedia broadcast transceiver, mobile communication terminal, home theater device, digital cinema device, surveillance camera, video talking device, communication device in such as video communication, mobile streaming device, storage medium, camera recorder, video on demand (VoD) device, over-the-top (OTT) video delivery device, internet streaming device, three-dimensional (3D) device ) video, video teleconferencing device and medical video device, and can be used to process video signal and data signal. For example, an OTT video device may include a game console, blue-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), and the like.

[550] Moreover, the processing method to which the present disclosure applies may be in the form of a computer-executable program and may be stored in a computer-readable recording medium. Multimedia data having a data structure in accordance with the present disclosure may also be stored in computer-readable recording media. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. Computer-readable recording media may include, for example, Blu-ray disc (BD), universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical storage device data. Additionally, the computer-readable recording medium includes media implemented in the form of a carrier wave (eg, transmission over the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or transmitted over wired or wireless communication networks. Additionally, embodiments of the present disclosure may be implemented as a computer program product through program codes, and the program codes may be executed on a computer by embodiments of the present disclosure. Program codes may be stored on a computer-readable medium.

[551] FIG. 23 shows an example of a content streaming system to which the present disclosure is applicable.

[552] A content streaming system to which the present disclosure applies may broadly include an encoding server, a streaming server, a web server, a media storage device, a user equipment, and a media input device.

[553] The encoding server functions to compress digital data of content input from multimedia input devices such as a smartphone, camera, camcorder, etc. into digital data to generate a bit stream and transmit it to the streaming server . As another example, when a multimedia input device such as a smartphone, camera, dashcam, etc. directly generates a bitstream, the encoding server may be omitted. The bitstream may be generated by an encoding method or a bitstream generation method to which the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[554] The streaming server transmits media data to the user equipment based on the user's request through a web server, which functions as a tool that informs the user about the service. When a user requests a desired service, the web server forwards the request to the streaming server, and the streaming server transmits the media data to the user. In this regard, the content streaming system may include a separate control server, in which case, the control server functions to manage commands/responses between the respective equipments in the content streaming system.

[555] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined period of time.

[556] For example, user equipment may include a mobile phone, smartphone, laptop, digital broadcast terminal, personal digital assistant (PDA), portable multimedia player (PMP), navigator, slim PC, tablet PC, ultrabook, wearable device (eg , watch type terminal (smart watch), glasses type terminal (smart glasses), head mounted display (HMD)), digital TV, desktop computer, digital pointer and the like. Each of the servers in a content streaming system can operate as a distributed server, in which case, the data received by each server can be processed in a distributed manner.

[557] The claims disclosed herein may be combined in various ways. For example, the technical features of the method claims in the present disclosure may be combined to be implemented or performed in an apparatus, and the technical features of the device claims may be combined to be implemented or performed in a method. In addition, the technical features of the method claims and the device claims can be combined to be implemented or performed in a device, and the technical features of the method claims and device claims can be combined to be implemented or performed in a method.

Clause 1. A method for decoding an image performed by a decoding device, the method comprising:

receiving residual information from the bit stream;

deriving conversion coefficients for the current block based on residual information;

determining whether a significant coefficient is present in a second region other than the first region from the top left of the current block;

obtaining, by parsing, an MTS index from the bitstream when a significant coefficient is not present in the second region;

deriving residual samples by applying the MTS kernel derived from the MTS index to the transform coefficients of the first region; And

generating a reconstructed image based on residual samples.

Position 2: A method for decoding an image according to Position 1, the first area being a top-left 16×16 area of the current block.

Statement 3: A method for decoding an image according to Position 1, wherein determining whether a significant coefficient is present in a second region comprises deriving a value of a variable indicating whether a significant coefficient is present in the second region in a residual encoding layer decoding process,

wherein the value of the variable is initially set to 1, and

wherein, when a significant coefficient is present in the second region, the value of the variable is changed to 0 and no parsing is performed with respect to the MTS index.

Statement 4: A method for decoding a picture according to Position 3, wherein parsing with respect to the MTS index is performed at the coding unit level.

Statement 5: A picture decoding method according to Statement 1, further comprising parsing with respect to a LFNST index indicating a LFNST kernel applied to the current block, and

wherein the LFNST index and the MTS index are signaled at the coding unit level, and the MTS index is signaled immediately after the LFNST index signaling.

Position 6. A method for decoding an image according to Position 5, wherein the tree type of the current block is a dual luma tree or a single tree, and wherein the MTS index is signaled when the index value LFNST is 0.

Claim 7: An image encoding method performed by an image encoding device, the method comprising:

outputting prediction samples for the current block;

deriving residual samples for the current block based on the prediction samples;

deriving transform coefficients for the current block based on the MTS for the residual samples;

zeroing the second area of the current block other than the first area from the top left of the current block; And

encoding the residual information output by quantizing the transform coefficients, and the MTS index indicating the MTS core.

Position 8: A method for encoding an image according to Position 7, the first area being a top-left 16×16 area of the current block.

Provision 9. An image encoding method according to Provision 7, wherein the MTS index is signaled at the coding unit level.

Provision 10. A picture encoding method according to Proposition 9, further comprising signaling an LFNST index indicating a LFNST kernel applied to the current block, the MTS index being signaled immediately after the LFNST index signaling.

Statement 11: A picture encoding method according to Statement 10, wherein the tree type of the current block is a dual luma tree or a single tree, and wherein when the index value LFNST is 0, the MTS index is signaled.

Claim 12: A computer-readable digital storage medium that stores indication information to cause execution of an image decoding method, wherein the image decoding method comprises:

receiving residual information from the bit stream;

deriving conversion coefficients for the current block based on residual information;

determining whether a significant coefficient is present in a second region other than the first region from the top left of the current block;

obtaining, by parsing, an MTS index from the bitstream when a significant coefficient is not present in the second region;

deriving residual samples by applying the MTS kernel derived from the MTS index to the transform coefficients of the first region; And

generating a reconstructed image based on residual samples.

Claims (29)

1. A decoding device for decoding images, comprising: memory and at least one processor coupled to the memory, wherein the at least one processor is configured to: obtaining residual information from the bit stream, deriving conversion coefficients for the current block based on residual information, determining whether a significant coefficient is present in a second region other than the first region from the top left of the current block, obtaining, by parsing, an MTS index from the bitstream based on the fact that a significant coefficient is not present in the second region, deriving the residual samples by applying the MTS kernel derived from the MTS index to the transform coefficients of the first region, and generating a reconstructed image based on residual discrete samples. 2. The decoding device according to claim 1, wherein the first region is a top-left 16×16 region of the current block. 3. The decoding apparatus of claim 1, wherein determining whether a significant coefficient is present in the second region comprises deriving a value of a variable associated with whether a significant coefficient is present in the second region in a residual encoding layer decoding process, wherein the value of the variable is initially set to 1, and based on the fact that a significant coefficient is present in the second region, the value of the variable is changed to 0 and no parsing is performed with respect to the MTS index. 4. The decoding apparatus of claim 3, wherein parsing regarding the MTS index is performed at the encoding unit level. 5. The decoding apparatus of claim 1, wherein the at least one processor is further configured to parse a LFNST index associated with an LFNST kernel applied to the current block, the LFNST index and the MTS index being signaled at the coding unit level, and the MTS index is signaled immediately after the LFNST index is signaled. 6. The decoding apparatus as set forth in claim 5, wherein the tree type of the current block is a dual luminance tree or a single tree, and based on the index value LFNST being 0, the MTS index is signaled. 7. An encoding device for encoding images, comprising: memory and at least one processor coupled to the memory, wherein the at least one processor is configured to: outputting discrete prediction samples for the current block, deriving residual samples for the current block based on the prediction samples; deriving conversion coefficients for the current block based on the MTS for residual sampled samples, zeroing out a second area of the current block other than the first area from the top-left of the current block, and encoding the residual information output by quantizing the transform coefficients, and an MTS index associated with the MTS core. 8. The encoding device according to claim 7, wherein the first region is a top-left 16×16 region of the current block. 9. The encoding device according to claim 7, wherein the MTS index is signaled at the level of the encoding unit. 10. The encoding apparatus of claim 9, wherein the at least one processor is further configured to signal an LFNST index associated with a LFNST kernel applied to the current block, the MTS index being signaled immediately after signaling the LFNST index. 11. The encoding apparatus as set forth in claim 10, wherein the tree type of the current block is a dual luminance tree or a single tree, and based on the index value LFNST being 0, the MTS index is signaled. 12. An image data transmission device, comprising: at least one processor configured to obtain a bit stream of image information including residual information, wherein the residual information is generated by deriving prediction samples for a current block, deriving residual samples for the current block based on the prediction samples, deriving transformation coefficients for the current block based on the MTS for the residual samples, zeroing out a second region of the current block other than the first top-left region of the current block, and encoding the residual information output by quantizing the transform coefficients and the MTS index associated with the MTS core to generate bitstream; And a transmitter configured to transmit data containing a bit stream of image information.