RU2809518C2 - Method and device for encoding/decoding images using filtering and method for transmitting a bit stream - Google Patents

Abstract

FIELD: image encoding/decoding technologies.

SUBSTANCE: group of inventions is related to equipment for image encoding/decoding using filtering. A method for decoding images is proposed, carried out by means of image decoding equipment containing a decoding module that decodes an image from a bit stream. The method comprises retrieving a target deblocking filter boundary in the reconstructed frame and determining a maximum deblocking filter length for the target boundary. A deblocking filter is applied to the target boundary based on the maximum length. Based on the target boundary being a vertical boundary, the maximum length is determined based on the width of at least one of the first block adjacent to the left side of the target boundary or the second block adjacent to the right side of the target boundary.

EFFECT: increase of efficiency of coding/decoding images.

15 cl, 17 dwg, 2 tbl

Description

Field of technology to which the invention relates

[1] The present disclosure relates to a method and equipment for encoding/decoding images and to a method for transmitting a bit stream, and more specifically, to a method and equipment for encoding/decoding an image using filtering and to a method for transmitting a bit stream generated by the method/equipment encoding images of the present disclosure.

State of the art

[2] Recently, the demand for high-quality, high-definition images, such as high-definition (HD) images and ultra-high-definition (UHD) images, has been increasing in various fields of technology. As the resolution and quality of image data increases, the amount of transmitted information or bits increases relatively compared to existing image data. Increasing the amount of information or bits transmitted results in increased transmission costs and storage costs.

[3] Accordingly, there is a need for high-performance image compression technology for efficiently transmitting, storing and reproducing information regarding high-quality, high-resolution images.

The essence of the invention

Technical problem

[4] The purpose of the present disclosure is to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[5] The purpose of the present disclosure is to provide a method and equipment for encoding/decoding an image using filtering.

[6] Another object of the present disclosure is to provide a method for transmitting a bit stream generated by an image encoding method or equipment according to the present disclosure.

[7] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by an image encoding method or equipment according to the present disclosure.

[8] Another object of the present disclosure is to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[9] The technical problems addressed by the present disclosure are not limited to the above technical problems, and other technical problems not described herein should become apparent to those skilled in the art from the following description.

Technical solution

[10] According to the image encoding/decoding method according to an aspect of the present disclosure, since a situation in which parallel processing of a deblocking filter is not possible due to reduction in the transform block size, or a situation in which overlap between filters occurs due to different target boundaries does not occur, the image encoding/decoding equipment can always perform parallel filtering processing, and the efficiency of image encoding/decoding can be increased.

[11] An image decoding method performed by the image decoding equipment according to an aspect of the present disclosure may comprise extracting a reconstructed block for a current block, extracting a target boundary for the reconstructed block, determining a filter length for a deblocking filter to be applied to the target boundary, and Applying a deblocking filter to the target boundary based on a specific filter length. The length of the filter may be determined based on at least one of the width or height of a transform block adjacent to the target boundary.

[12] In the image decoding method of the present disclosure, the current block may be a block to which an internal subsegment-based (ISP) mode is applied.

[13] The image decoding method of the present disclosure may further comprise determining a boundary type for a target boundary, and the boundary type may be determined to be one of a vertical boundary and a horizontal boundary.

[14] In the image decoding method of the present disclosure, based on the target boundary being a vertical boundary and the transform block width equal to or smaller than the first value, the filter length may be determined as the first length.

[15] In the image decoding method of the present disclosure, based on a transform block width equal to or greater than the second value, the filter length may be determined as the second length.

[16] In the image decoding method of the present disclosure, based on a transform block width larger than the first value and smaller than the second value, the filter length may be determined as the third length.

[17] In the image decoding method of the present disclosure, the first value may be 4, and the first length may be 1.

[18] In the image decoding method of the present disclosure, based on the target boundary being a horizontal boundary and the transform block height equal to or smaller than the first value, the filter length may be determined as the first length.

[19] In the image decoding method of the present disclosure, based on a transform block height equal to or greater than the second value, the filter length may be determined as the second length.

[20] In the image decoding method of the present disclosure, based on a transform block height greater than the first value and less than the second value, the filter length may be determined as the third length.

[21] In the image decoding method of the present disclosure, the first value is 4, and the first length may be 1.

[22] Image decoding equipment according to another aspect of the present disclosure may comprise a storage device and at least one processor.

[23] The at least one processor may retrieve a reconstructed block for the current block, retrieve a target boundary for the reconstructed block, determine a filter length for a deblocking filter to be applied to the target boundary, and apply a deblocking filter to the target boundary based on the determined filter length. The length of the filter may be determined based on at least one of the width or height of a transform block adjacent to the target boundary.

[24] An image encoding method according to another aspect of the present disclosure may comprise extracting a reconstructed block for a current block, extracting a target boundary for the reconstructed block, determining a filter length for a deblocking filter to be applied to the target boundary, and applying a deblocking filter to the target boundaries based on a specific filter length. The length of the filter may be determined based on at least one of the width or height of a transform block adjacent to the target boundary.

[25] In the image encoding method of the present disclosure, the current block may be a block to which an internal subsegment-based (ISP) mode is applied.

[26] In addition, the computer-readable recording medium according to another aspect of the present disclosure can store a bit stream generated by the image encoding equipment or the image encoding method of the present disclosure.

[27] The features briefly summarized above with respect to the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

Advantages of the invention

[28] According to the present disclosure, it is possible to provide an image encoding/decoding method and equipment with improved encoding/decoding efficiency.

[29] In addition, according to the present disclosure, a method and equipment for encoding/decoding an image using filtering can be provided.

[30] In addition, according to the present disclosure, it is possible to provide a method for transmitting a bitstream generated by an image encoding method or equipment according to the present disclosure.

[31] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bitstream generated by the image encoding method or equipment according to the present disclosure.

[32] In addition, according to the present disclosure, it is possible to provide a recording medium storing a bitstream received, decoded, and used to reconstruct an image by the image decoding equipment according to the present disclosure.

[33] Those skilled in the art will appreciate that the advantages that may be achieved through the present disclosure are not limited to those specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description.

Brief description of drawings

[34] FIG. 1 is a view schematically showing a video encoding system to which an embodiment of the present disclosure is applicable.

[35] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[36] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[37] FIG. 4 is a view illustrating an embodiment of a block segmentation type according to a multi-type tree structure.

[38] FIG. 5 is a view illustrating a signaling mechanism for blocking information in a quadtree with a nested multi-type tree structure according to the present disclosure.

[39] FIG. 6 is a view illustrating a method of applying a deblocking filter according to an embodiment.

[40] FIG. 7 is a view illustrating a method for determining filter intensity in a deblocking filter.

[41] FIG. 8 is a view illustrating the internal sub-segment (ISP) mode of internal prediction technologies.

[42] FIG. 9 to 11 are views illustrating a method for determining a filter length for a deblocking filter applied to a target boundary.

[43] FIG. 12 is a view illustrating an image encoding/decoding method according to an embodiment of the present disclosure.

[44] FIG. 13 and 14 are views illustrating a method for determining the length of a deblocking filter according to another embodiment of the present disclosure.

[45] FIG. 15 and 16 are views illustrating a method for determining the length of a deblocking filter according to another embodiment of the present disclosure.

[46] FIG. 17 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

Optimal mode for carrying out the invention

[47] Embodiments of the present disclosure are described in detail hereinafter with reference to the accompanying drawings so that they can be easily implemented by those skilled in the art. However, the present disclosure may be implemented in a variety of forms and is not limited to the embodiments described herein.

[48] In describing the present disclosure, if it is determined that a detailed description of a related known function or construct would cause unnecessary ambiguity to the scope of the present disclosure, the detailed description thereof is omitted. In the drawings, parts not related to the description of the present disclosure are omitted, and like reference numbers are added to like parts.

[49] In the present disclosure, when a component “connects,” “pairs,” or “links” to another component, it may include not only a direct connection-based relationship, but also an indirect connection-based relationship in which there is intermediate component. In addition, when a component “includes” or “has” other components, it means that other components may be additionally included, not the exclusion of other components, unless otherwise noted.

[50] In the present disclosure, the terms “first”, “second”, etc. may be used only for the purpose of distinguishing one component from other components and does not limit the order or importance of the components unless otherwise noted. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a “second component” in another embodiment, and likewise, a second component in one embodiment may be referred to as a “first component” in another embodiment.

[51] In the present disclosure, components that are different from each other are intended to clearly describe each feature and do not mean that the components are necessarily separate. Thus, multiple components may be integrated and implemented in a single hardware or software module, or a single component may be distributed and implemented in multiple hardware or software modules. Therefore, even if not stated otherwise, such embodiments in which components are integrated or a component is distributed are also included within the scope of the present disclosure.

[52] In the present disclosure, the components described in the various embodiments do not necessarily mean essential components, and some components may be optional components. Accordingly, an embodiment consisting of a subset of the components described in the embodiment is also included within the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included within the scope of the present disclosure.

[53] The present disclosure relates to image encoding and decoding, and the terms used in the present disclosure may have a general meaning commonly used in the art to which the present disclosure pertains, unless defined for the first time in the present disclosure.

[54] In the present disclosure, "frame" generally means a unit representing one image at a particular time period, and a slice/tile is an encoding unit constituting part of a frame, and one frame may consist of one or more slices/ tiles In addition, a slice/tile may include one or more coding tree units (CTUs).

[55] In the present disclosure, "pixel" or "pixel" can mean the smallest unit constituting one frame (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel/pixel value of a luma component or only a pixel/pixel value of a chroma component.

[56] In the present disclosure, a "unit" may represent a basic unit of image processing. The unit may include at least one of a specific frame area and information associated with the area. The unit may be used interchangeably with terms such as "sample array", "block" or "zone" in some cases. In general, an MxN block may include samples (or arrays of samples) or a set (or array) of transform coefficients of M columns and N rows.

[57] In the present disclosure, "current block" may mean one of "current encoding block", "current encoding unit", "target encoding block", "target decoding block" or "target processing block". When prediction is performed, "current block" may mean "current prediction block" or "target prediction block". When transform (inverse transform)/quantization (dequantization) is performed, "current block" can mean "current transform block" or "transform target block". When filtering is performed, "current block" can mean "target filtering block".

[58] In the present disclosure, the terms "/" and "" are to be interpreted to indicate "and/or". For example, the expression "A/B" and "A, B" can mean "A and/or B". Additionally, "A/B/C" and "A/B/C" may mean "at least one of A, B and/or C".

[59] In the present disclosure, the term “or” should be interpreted to indicate “and/or.” For example, the expression "A or B" may contain 1) only "A", 2) only "B", and/or 3) "both A and B". In other words, in the present disclosure, the term “or” is to be interpreted as indicating “in addition or alternatively.”

[60] Video Coding System Overview

[61] FIG. 1 is a view showing a video encoding system according to the present disclosure.

[62] The video encoding system according to an embodiment may include encoding equipment 10 and decoding equipment 20. The encoding equipment 10 may deliver encoded information or video and/or image data to the decoding equipment 20 in the form of a file or streaming via a digital storage medium or a network.

[63] The encoding equipment 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding equipment 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a “video/image encoding unit,” and the decoding unit 22 may be referred to as a “video/image decoding unit.” The transmitter 13 may be included in the encoding module 12. The receiver 21 may be included in the decoding module 22. The rendering module 23 may include a display, and the display may be configured as a separate device or an external component.

[64] The video source generator 11 may obtain video/image through a video/image capturing, synthesizing, or shaping process. The video source generator 11 may include a video/image capture device and/or a video/image generation device. The video/image capturing device may include, for example, one or more cameras, video/image archives including previously captured video/images, and the like. The video/image generating apparatus may include, for example, computers, tablet computers and smartphones and may (electronically) generate video/images. For example, a virtual video/image may be generated via a computer or the like. In this case, the video/image capturing process can be replaced by the associated data generation process.

[65] The encoding unit 12 can encode the input video/image. The encoding unit 12 may perform a series of procedures such as prediction, transform and quantization for compression and encoding efficiency. The encoding unit 12 can output encoded data (encoded video/image information) in the form of a bit stream.

[66] The transmitter 13 may transmit encoded information or video/image data output in the form of a bitstream to the receiver 21 of the decoding equipment 20 via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. The transmitter 13 may include an element for generating a multimedia file through a predefined file format and may include an element for transmission through a broadcast/communications network. The receiver 21 may retrieve/receive a bit stream from the storage medium or network and transmit the bit stream to the decoding unit 22.

[67] The decoding unit 22 can decode the video/image by performing a sequence of procedures such as dequantization, deconversion and prediction corresponding to the operation of the encoding unit 12.

[68] The rendering unit 23 may prepare a decoded video/image by rendering. The video/image prepared by rendering can be displayed through the display.

[69] General introduction to image encoding equipment

[70] FIG. 2 is a view schematically showing image encoding equipment to which an embodiment of the present disclosure is applicable.

[71] As shown in FIG. 2, the image encoding equipment 100 may include an image segmentation unit 110, a subtractor 115, a converter 120, a quantizer 130, a dequantizer 140, an inverse converter 150, an adder 155, a filter 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185 and an entropy encoder 190. The inter-prediction module 180 and the intra-prediction module 185 may be collectively referred to as a “prediction module”. Converter 120, quantizer 130, dequantizer 140, and inverse converter 150 may be included in the residual processor. The residual processor may further include a subtractor 115.

[72] All or at least a portion of the plurality of components configuring image encoding equipment 100 may be configured by a single hardware component (eg, an encoder or processor) in some embodiments. In addition, storage device 170 may include a decoded frame buffer (DPB) and may be configured with a digital storage medium.

[73] The image segmentation module 110 may segment an input image (either a frame or a movie frame) input to the image encoding equipment 100 into one more processing unit. For example, the processing unit may be called a "coding unit (CU)". A coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or largest coding unit (LCU) according to a quadtree, binary tree, and ternary tree (QT/BT/TT) structure. For example, one coding unit may be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and/or a ternary structure. To segment a coding unit, a quadtree structure may first be applied, and subsequently a binary tree structure and/or a ternary structure may be applied. The encoding procedure of the present disclosure may be performed based on a final encoding unit that is no longer segmented. The largest coding unit may be used as the final coding unit, or a coding unit of greater depth obtained by segmenting the largest coding unit may be used as the final coding unit. Here, the encoding procedure may include a prediction, transformation and reconstruction procedure, which is described below. As another example, the processing unit of the encoding procedure may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or segmented from the final encoding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for extracting a transformation coefficient and/or a unit for extracting a residual signal from the transformation coefficient.

[74] The prediction module (inter-prediction module 180 or intra-prediction module 185) may perform prediction regarding a block to be processed (the current block) and generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied based on the current block or CU. The prediction module may generate various information related to prediction of the current block and transmit the generated information to the entropy encoder 190. Information regarding the prediction may be encoded in the entropy encoder 190 and output in the form of a bit stream.

[75] The intra prediction module 185 may predict the current block by reference to the samples in the current frame. Samples for reference may be located in the surroundings of the current block or may be spaced apart according to an intra-prediction mode and/or intra-prediction technology. The intra prediction modes may include a plurality of non-directional modes and a plurality of directed modes. The omnidirectional mode may include, for example, DC mode and planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is merely an example; more or less directional prediction modes may be used depending on the setting. The intra prediction module 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the adjacent block.

[76] The inter-prediction unit 180 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. The reference frame including the reference block and the reference frame including the temporary adjacent block may be identical or different. A temporary neighbor block may be referred to as a "co-located reference block", "co-located CU (colCU)", or the like. A reference frame including a temporary adjacent block may be referred to as a "co-located frame (colPic)". For example, the inter-prediction module 180 may configure a list of motion information candidates based on neighboring blocks and generate information indicating which candidate is used to retrieve the motion vector and/or reference frame index of the current block. Mutual prediction can be performed based on different prediction modes. For example, in the case of the skip mode and the merge mode, the inter-prediction unit 180 may use the motion information of a neighboring block as the motion information of the current block. In the case of the skip mode, unlike the combining mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of an adjacent block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[77] The prediction module can generate a prediction signal based on various prediction methods and prediction technologies described below. For example, the prediction module may not only apply intra-prediction or inter-prediction, but also simultaneously apply both intra-prediction and inter-prediction to predict the current block. A prediction method that simultaneously uses both intra prediction and inter prediction to predict the current block may be called "combined inter and intra prediction (CIIP)". In addition, the prediction module can perform intra-block copying (IBC) to predict the current block. In-block copying can be used to encode game image/video content and the like, such as screen content encoding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a location spaced from the current block by a predetermined distance. When IBC is applied, the location of the reference block in the current frame can be encoded as a vector (block vector) corresponding to a predetermined distance.

[78] The prediction signal generated by the prediction module can be used to generate a reconstructed signal or generate a residual signal. The subtractor 115 may generate a residual signal (a residual block or an array of residual samples) by subtracting a prediction signal (a prediction block or an array of prediction samples) output from the prediction module from an input image signal (a source block or an array of original samples). The generated residual signal can be transmitted to the converter 120.

[79] Converter 120 can generate conversion coefficients by applying conversion technology to the residual signal. For example, the transform technology may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally nonlinear transform (CNT). ). Here, GBT means a graph-derived transformation when the relationship information between pixels is represented by the graph. CNT means the transformation obtained from the prediction signal generated using all the previously reconstructed pixels. In addition, the conversion process can be applied to square pixel blocks that are identical in size, or can be applied to blocks that are variable in size rather than square.

[80] Quantizer 130 may quantize the transform coefficients and pass them to entropy encoder 190. Entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bit stream. Information regarding the quantized transform coefficients may be referred to as "residual information". Quantizer 130 may re-arrange the quantized transform coefficients in block form into one-dimensional vector form based on the scan order of the coefficients, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[81] The entropy encoder 190 may implement various encoding techniques, such as, for example, exponential Golomb coding, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. Entropy encoder 190 may encode information needed to reconstruct video/images other than quantized transform coefficients (eg, syntax element values, etc.), together or separately. Encoded information (eg, encoded video/image information) may be transmitted or stored in network abstraction layer (NAL) units in the form of a bitstream. The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The signaling information, transmission information, and/or syntax elements described in the present disclosure may be encoded through the encoding procedure described above and included in the bit stream.

[82] The bitstream may be transmitted over a network or may be stored on a digital storage medium. The network may include a broadcast network and/or a communications network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-Ray, HDD, SSD, and the like. A transmitter (not shown) transmitting the signal output from the entropy encoder 190 and/or a storage module (not shown) storing the signal may be included as an internal/external element of the encoding equipment 100, and alternatively, the transmitter may be included in the entropy encoder 190 .

[83] The quantized transform coefficients output from quantizer 130 can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) may be recovered by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 140 and inverse transformer 150.

[84] An adder 155 adds the reconstructed residual signal with the prediction signal output from the inter-prediction module 180 or the intra-prediction module 185 so as to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If there is no remainder for a block to be processed, for example in a case in which the skip mode is applied, the predicted block can be used as a reconstructed block. The adder 155 may be referred to as a "recovery module" or a "recovered block generator." The generated reconstructed signal can be used to internally predict the next block to be processed in the current frame, and can be used to inter-predict the next frame through filtering, as described below.

[85] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied in the frame encoding process as described below.

[86] Filter 160 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 160 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 170, namely, a DPB of storage device 170. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive contour filter, bilateral filter, etc. The filter 160 may generate various filtering-related information and transmit the generated information to the entropy encoder 190, as described below in the description of each filtering method. Information associated with filtering may be encoded by entropy encoder 190 and output in the form of a bitstream.

[87] The modified reconstructed frame transferred to the storage device 170 can be used as a reference frame in the inter-prediction module 180. When mutual prediction is applied through the image encoding equipment 100, prediction mismatch between the image encoding equipment 100 and the image decoding equipment can be eliminated, and encoding efficiency can be improved.

[88] The DPB of the storage device 170 may store the modified reconstructed frame for use as a reference frame in the inter-prediction module 180 . The storage device 170 may store block motion information from which motion information in the current frame is retrieved (or encoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 180 and used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 170 may store reconstructed samples of reconstructed blocks in the current frame and may transmit reconstructed samples to intra prediction module 185.

[89] General introduction to image decoding equipment

[90] FIG. 3 is a view schematically showing image decoding equipment to which an embodiment of the present disclosure is applicable.

[91] As shown in FIG. 3, the image decoding equipment 200 may include an entropy decoder 210, a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction module 260 and the intra-prediction module 265 may be collectively referred to as a “prediction module”. A dequantizer 220 and an inverse converter 230 may be included in the residual processor.

[92] All or at least a portion of the plurality of components configuring the image decoding equipment 200 may be configured by a hardware component (eg, a decoder or processor) according to an embodiment. In addition, storage device 250 may include a decoded frame buffer (DPB) or may be configured with a digital storage medium.

[93] The image decoding equipment 200, which receives the bit stream including video/image information, can reconstruct the image by performing a process corresponding to the process performed by the image encoding equipment 100 in FIG. 1. For example, the image decoding equipment 200 may perform decoding using a processing unit used in the image encoding equipment. Thus, a decoding processing unit, for example, may be an encoding unit. A coding unit may be obtained by segmenting a coding tree unit or a largest coding unit. The reconstructed image signal decoded and output by the decoding equipment 200 can be reproduced by the playback equipment.

[94] The image decoding equipment 200 may receive a signal output from the image encoding equipment of FIG. 1 in the form of a stream of bits. The received signal may be decoded through entropy decoder 210. For example, entropy decoder 210 may parse the bitstream to extract information (eg, video/image information) needed to reconstruct images (or reconstruct frames). The video/image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a frame parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video/image information may further include general restriction information. The image decoding equipment may further decode the frame based on information regarding the parameter set and/or general constraint information. The signaling/receiving information and/or syntax elements described in the present disclosure may be decoded by a decoding procedure and obtained from the bitstream. For example, entropy decoder 210 decodes information in a bitstream based on an encoding method, such as exponential Golomb, CAVLC, or CABAC encoding, and the output values of the syntax elements required for image reconstruction and the quantized values of the transform coefficients for the remainder. More specifically, the entropy CABAC decoding method may take a resolution element corresponding to each syntax element in the bitstream, determine a context model using information of a target decoding syntax element, decoding information of an adjacent block and a target decoding block, or information of a symbol/resolution element decoded on a previous one. stage, and perform arithmetic decoding for the resolution element by predicting the probability of occurrence of the permission element according to the determined context model and generating a symbol corresponding to the meaning of each syntactic element. In this case, the entropy CABAC decoding method can update the context model by using the decoded symbol/grant element information for the context model of the next symbol/grant element after determining the context model. The prediction-related information from the information decoded by the entropy decoder 210 may be provided to the prediction module (inter-prediction module 260 and intra-prediction module 265), and the residual value for which entropy decoding is performed in the entropy decoder 210, i.e. quantized conversion coefficients and related parameter information may be input to the dequantizer 220. In addition, information regarding filtering from information decoded by the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the equipment image encoding device may be further configured as an internal/external element of image decoding equipment 200, or the receiver may be a component of entropy decoder 210.

[95] Meanwhile, the image decoding equipment according to the present disclosure may be referred to as “video/image/frame decoding equipment.” The image decoding equipment can be classified into an information decoder (video/image/frame information decoder) and a sample decoder (video/image/frame sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse converter 230, an adder 235, a filter 240, a memory 250, an inter-prediction module 260, or an intra-prediction module 265.

[96] Dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 220 may re-arrange the quantized transform coefficients in the form of a two-dimensional block. In this case, rearrangement may be performed based on the order of coefficient scanning performed in the image encoding equipment. Dequantizer 220 may perform dequantization on the quantized transform coefficients by using a quantization parameter (eg, quantization step size information) and obtain the transform coefficients.

[97] The inverse converter 230 may inversely convert the transform coefficients to obtain a residual signal (residual block, residual sample array).

[98] The prediction module may perform prediction for the current block and may generate a predicted block including prediction samples for the current block. The prediction module may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoder 210, and may determine a particular intra/inter prediction mode.

[99] Identical to that described in the prediction module of the image encoding equipment 100, the prediction module can generate a prediction signal based on various prediction methods (techniques), which are described below.

[100] Intra prediction module 265 may predict the current block by reference to samples in the current frame. The description of the intra prediction module 185 applies equally to the intra prediction module 265.

[101] The inter-prediction module 260 may extract a predicted block for the current block based on a reference block (an array of reference samples) indicated by a motion vector for the reference frame. In this case, in order to reduce the amount of motion information transmitted in the inter-prediction mode, the motion information may be predicted in units of blocks, sub-blocks or samples based on the correlation of the motion information between a neighboring block and the current block. The motion information may include a motion vector and a reference frame index. The motion information may further include inter-prediction direction information (L0-prediction, L1-prediction, bi-prediction, etc.). In the case of inter-prediction, the neighbor block may include a spatial neighbor block present in the current frame and a temporal neighbor block present in the reference frame. For example, the inter-prediction module 260 may configure a motion information candidate list based on neighboring blocks and extract the current block's motion vector and/or reference frame index based on the received candidate selection information. The inter-prediction may be performed based on various prediction modes, and the information regarding the prediction may include information indicating the inter-prediction mode for the current block.

[102] The adder 235 may generate a reconstructed signal (recovered frame, reconstructed block, reconstructed sample array) by adding the resulting residual signal with a prediction signal (predicted block, predicted sample array) output from a prediction module (including inter-prediction module 260 and /or internal prediction module 265). The description of adder 155 is equally applicable to adder 235.

[103] Meanwhile, luma conversion with chrominance scaling (LMCS) can be applied in the frame decoding process as described below.

[104] Filter 240 may enhance the subjective/objective quality of images by applying filtering to the reconstructed signal. For example, filter 240 may generate a modified reconstructed frame by applying various filtering techniques to the reconstructed frame and store the modified reconstructed frame in storage device 250, namely, a DPB of storage device 250. Various filtering techniques may include, for example, deblocking filtering. , sampled adaptive bias, adaptive loop filter, bilateral filter, etc.

[105] The (modified) reconstructed frame stored in the DPB of the storage device 250 can be used as a reference frame in the inter-prediction unit 260. Memory 250 may store block motion information from which motion information in the current frame is retrieved (or decoded) and/or motion information of blocks in a frame that have already been reconstructed. The stored motion information may be transmitted to the inter-prediction unit 260 so that it is used as spatial neighbor block motion information or temporal neighbor block motion information. Memory 250 may store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra prediction module 265.

[106] In the present disclosure, the embodiments described in the filter 160, the inter-prediction module 180, and the intra-prediction module 185 of the encoding equipment 100 may be equally or correspondingly applied to the filter 240, the inter-prediction module 260, and the intra-prediction module 265 of the hardware. 200 image decoding.

[107] CTU segmentation overview

[108] As described above, a coding unit can be obtained by recursively segmenting a coding tree unit (CTU) or largest coding unit (LCU) according to a quadtree/binary tree/ternary tree (QT/BT/TT) structure. For example, a CTU may first be segmented into quadtree structures. Thereafter, the leaf nodes of the quadtree structure can be further segmented by a multi-type tree structure.

[109] Quadtree segmentation means that the current CU (or CTU) is segmented equally into four. Through segmentation according to the quadtree, the current CU can be segmented into four CUs having identical width and identical height. When the current CU is no longer segmented into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be segmented and can be used as the final encoding unit described above. Alternatively, the CU corresponding to a leaf node of the quadtree structure may further be segmented by a multi-type tree structure.

[110] FIG. 4 is a view showing an embodiment of a block segmentation type according to a multi-type tree structure. Segmentation according to a multi-type tree structure may include two types of partitioning according to a binary tree structure and two types of partitioning according to a ternary tree structure.

[111] Two types of partitioning according to the binary tree structure may include vertical binary partitioning (SPLIT_BT_VER) and horizontal binary partitioning (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means that the current CU is split equally in half in the vertical direction. As shown in FIG. 4, through vertical binary partitioning, two CUs having a height identical to the height of the current CU and having a width that is half the width of the current CU can be formed. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is split equally in half in the horizontal direction. As shown in FIG. 4, through horizontal binary partitioning, two CUs having a height that is half the height of the current CU and having a width identical to the width of the current CU can be formed.

[112] Two types of partitioning according to the ternary tree structure may include vertical ternary partitioning (SPLIT_TT_VER) and horizontal ternary partitioning (SPLIT_TT_HOR). In vertical ternary splitting (SPLIT_TT_VER), the current CU is split in the vertical direction in a ratio of 1:2:1. As shown in FIG. 4, through vertical ternary partitioning, two CUs having a height identical to the height of the current CU and having a width that is 1/4 of the width of the current CU, and a CU having a height identical to the height of the current CU and having a width that is half the width of the current CU. In horizontal ternary splitting (SPLIT_TT_HOR), the current CU is split horizontally in a ratio of 1:2:1. As shown in FIG. 4, through horizontal ternary partitioning, two CUs having a height that is 1/4 of the height of the current CU and having a width identical to the width of the current CU, and a CU having a height that is half the height of the current CU and having width identical to the width of the current CU.

[113] FIG. 5 is a view showing a mechanism for signaling signaling of partitioning information in a quadtree with a nested multi-type tree structure according to the present disclosure.

[114] Here, the CTU is treated as the root node of a quadtree and is segmented for the first time into a quadtree structure. Information (eg, qt_split_flag) indicating whether or not quadtree splitting is performed for the current CU (CTU or QT_node) is signaled. For example, when qt_split_flag has the first value (for example, "1"), the current CU can be segmented into a quadtree. In addition, when qt_split_flag has a second value (for example, "0"), the current CU is not split into the quadtree, but becomes a leaf node (QT_leaf_node) of the quadtree. Each quadtree leaf node can then be further segmented into multi-type tree structures. Thus, a quadtree leaf node can become a node (MTT_node) of a multi-type tree. In a multi-type tree structure, the first flag (eg, Mtt_split_cu_flag) is signaled to indicate whether or not the current node is further segmented. If the corresponding node is further split (eg, if the first flag is 1), a second flag (eg, Mtt_split_cu_vertical_flag) may be signaled to indicate the direction of the split. For example, the split direction may be a vertical direction if the second flag is 1, and may be a horizontal direction if the second flag is 0. A third flag (for example, Mtt_split_cu_binary_flag) may then be signaled to indicate what represents the split type is a binary partition type or a ternary partition type. For example, the partition type may be a binary partition-based type when the third flag is 1, and may be a ternary partition-based type when the third flag is 0. A node of a multi-type tree obtained by a binary partition or a ternary partition may further be segmented into multi-type tree structures. However, a multi-type tree node may not be segmented into quadtree structures. If the first flag is 0, the corresponding multi-type tree node is no longer split, but becomes a leaf node (MTT_leaf_node) of the multi-type tree. The CU corresponding to a leaf node of a multi-type tree can be used as the final encoding unit described above.

[115] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree split mode (MttSplitMode) of the CU can be extracted as shown in Table 1 below.

[116] Tab. 1

[117]

[118] The CTU may include a luma sample encoding unit (hereinafter referred to as a "luminance signal unit") and two chrominance signal sample encoding units corresponding thereto (hereinafter referred to as "chrominance signal units"). The above-described coding tree-based scheme can be applied equally or separately to the luma signal block and the chrominance signal block of the current CU. In particular, luma and chroma blocks in the same CTU may be segmented into an identical block tree structure, in which case, the tree structure is represented as SINGLE_TREE. Alternatively, blocks of luma and chrominance signals in a single CTU may be segmented into separate block tree structures, in which case, the tree structure may be represented as DUAL_TREE. Thus, when a CTU is segmented into dual trees, a block tree structure for a luma block and a block tree for a chrominance block may be separately present. In this case, the block tree structure for the luminance signal block may be called "DUAL_TREE_LUMA", and the block tree structure for the chrominance signal component may be called "DUAL_TREE_CHROMA". For groups of P- and B-slices/tiles, luma and chrominance blocks in the same CTU may be limited to having identical encoding tree structure. However, for groups of I-slices/tiles, luma and chroma blocks may have a separate block tree structure relative to each other. If a separate block tree structure is used, the luma CTB may be segmented into CUs based on a particular coding tree structure, and the chroma CTB may be segmented into chrominance CUs based on a different coding tree structure. Thus, a CU in an I-slice/tile group to which a separate block tree structure is applied may consist of a luma component block encoding unit or two chroma component encoding units. In addition, CUs in an I-slice/tile group to which an identical block tree structure is applied, and CUs in a P- or B-slice/tile group may be composed of blocks of three color components (a luma component and two chrominance components).

[119] Although a structure in the form of a quadtree-based coding tree with a nested multi-type tree is described, the structure in which the CU is segmented is not limited to this. For example, a BT structure and a TT structure may be interpreted as a concept included in a multi-partitioning tree (MPT) structure, and a CU may be interpreted as segmentable through a QT structure and an MPT structure. In an example in which the CU is segmented across a QT structure and an MPT structure, a syntax element (for example, MPT_split_type) including information regarding how many blocks the leaf node of the QT structure is segmented into, and a syntax element (for example, MPT_split_mode) , including information regarding in which of the vertical and horizontal directions the leaf node of the QT structure is segmented, may be signaled to determine the segmentation structure.

[120] In another example, the CU may be segmented in a manner different from the QT structure, BT structure, or TT structure. Thus, in contrast to a smaller CU being segmented 1/4 of a larger depth CU according to the QT structure, a shallower CU being segmented 1/2 of a larger depth CU according to the BT structure, or a shallower CU being segmented by 1 /4 or 1/2 of a deeper CU according to the TT structure, a shallower CU can be segmented to 1/5, 1/3, 3/8, 3/5, 2/3 or 5/8 of a deeper CU in some cases, and the CU segmentation method is not limited to this.

[121] General overview of filtering

[122] The filtering method according to the present disclosure is described later in this document.

[123] According to some embodiments of the present disclosure, filtering may be performed on a reconstructed frame generated by image encoding/decoding equipment. As a result of performing filtering, a modified reconstructed frame may be generated, and the image decoding equipment may determine the modified reconstructed frame as the final decoded frame. In addition, in the image encoding/decoding equipment, the modified reconstructed frame may be stored in a decoded frame buffer (DPB) or memory, and then may be used as a reference frame when encoding/decoding the frame.

[124] Filtering according to some embodiments of the present disclosure may be used with a meaning identical to that of in-loop filtering. The filter used for filtering may include at least one of a deblocking filter, a sampled adaptive offset (SAO) filter, an adaptive loop filter (SLF), or a two-way filter. At least one of a deblocking filter, an SAO filter, an ALF, and/or a two-way filter may be sequentially applied to the reconstructed frame, thereby generating a modified reconstructed frame. The filter application order may be preset in the image encoding/decoding equipment. For example, after the deblocking filter is applied to the reconstructed frame, the SAO filter may be applied. As another example, after a deblocking filter is applied to the reconstructed frame, ALF may be applied. Filtering according to some embodiments of the present disclosure may be performed by at least one of the filter 160 of FIG. 2 and/or filter 240 of FIG. 3.

[125] For example, a deblocking filter may eliminate aliasing generated at a block boundary of a reconstructed frame. For example, a deblocking filter may extract a target boundary, which is the boundary between blocks in the reconstructed frame, and set a boundary intensity or filtering intensity for the target boundary. The image encoding/decoding equipment may perform deblocking filtering on the target edge by applying filtering based on a predetermined filtering intensity to the target edge. In this case, the filtering intensity may be determined based on at least one of the prediction modes of two blocks adjacent to the target boundary, the difference in motion vectors, whether or not the reference frames are identical, or the presence/absence of a non-zero effective coefficient.

[126] As another example, the SAO filter may compensate for the offset difference between the reconstructed frame and the original frame in units of samples. For example, the SAO filter may be implemented by a filter type such as a bandpass filter or an offset filter. SAO: When applying a filter, samples can be classified into different categories according to the SAO type, and the offset value can be added to each sample based on the category. The SAO filter information may include at least one of information regarding whether or not to apply an SAO filter, SAO filter type information, and/or SAO offset value information. Meanwhile, for example, the SAO filter may be limited to being applied to the reconstructed frame to which the deblocking filter is applied.

[127] As another example, the ALF may be a sample unit filter for applying filter coefficients according to the shape of the filter to the reconstructed frame. The image encoding equipment may signal at least one of whether or not an ALF should be applied, an ALF shape, and/or a filter coefficient by comparing the reconstructed frame with the original frame. Thus, the information regarding the ALF may include at least one of information regarding whether or not to apply the ALF, ALF filter shape information, and/or ALF filter coefficient information. Meanwhile, for example, ALF may be limited to being applied to a reconstructed frame to which a deblocking filter is applied.

[128] FIG. 6 is a view illustrating a method of applying a deblocking filter.

[129] As described above, a deblocking filter may be applied to the reconstructed frame. The deblocking filter may be applied to the boundary of each CU or TU included in the current block based on the encoding/decoding order of the frame.

[130] Referring to FIG. 6, a method for applying a deblocking filter according to some embodiments of the present disclosure may include retrieving a target boundary to which deblocking filtering is to be applied (S610), determining a filtering intensity (S620), and applying a deblocking filter to the target boundary based on a certain filtration intensity (S630).

[131] Hereinafter, this document describes the determination of the filtering intensity applied to the target boundary (S620). According to some embodiments of the present disclosure, the filtering intensity may be determined according to the conditions of a transform block adjacent to the target boundary. In the following description, when the target boundary is a vertical boundary, based on the target boundary, the left block may be specified as a P block and the right block may be specified as a Q block. In addition, when the target boundary is a horizontal boundary, based on the target boundary, the upper block may be specified as a P block and the lower block may be specified as a Q block. In addition, in the following description, the first value, second value and third value of the filtering intensity, respectively, may be 0, 1 and 2, but the scope of the present disclosure is not limited to this definition.

[132] For example, when block-based residual domain differential pulse code modulation (BDPCM) is applied to a P-block sample and a Q-block sample included in one luminance CU, the filtering intensity for the target boundary may be determined to be a component of the former meaning. As another example, when a P-block sample and a Q-block sample included in one CU are internally predicted, the filtering intensity for the target boundary may be determined to constitute a third value. As another example, when the target boundary is a TU boundary, and combined inter and intra prediction (CIIP) is applied to a P-block sample and a Q-block sample included in one CU, the filtering intensity for the target boundary may be determined to be a third value . As another example, when the target boundary is a TU boundary, and at least one of the P-block sample and the Q-block sample included in one TU has a level of non-zero transform coefficients, the filtering intensity for the target boundary may be determined as a component of the second meaning. As another example, when the prediction mode of a sub-unit CU including a sample of P-blocks is different from the prediction mode of a sub-unit of a CU including a sample of Q-blocks, the filtering intensity for the target boundary may be determined as a component of the second value.

[133] As another example, when the current block is a luma block, and at least one of the following conditions is satisfied, the filter intensity for the target boundary may be determined to be a component of the second value. Meanwhile, when all of the following conditions are not satisfied, the filtering intensity for the target boundary can be determined as a component of the first value. For example, when both a subblock CU including a sample of P blocks and a subblock CU including a sample of Q blocks are encoded/decoded in the IBC mode, and the difference of the horizontal or vertical motion vector values of each subblock is equal to or greater than 4 units in 1/4 of the luminance signal sample, the filter intensity for the target boundary can be determined as a component of the second value. Alternatively, when a subunit CU including a sample of P blocks and a subunit CU including a sample of Q blocks refer to different reference frames or have different numbers of motion vectors, the filtering intensity for the target boundary may be determined as a component of the second value. When one motion vector is used to predict a sub-block of a CU including a sample of P-blocks and a sub-block of a CU including a sample of Q-blocks, or the difference between the horizontal values or vertical values of the motion vector of each sub-block is equal to or greater than 4 units at 1/4 of the luminance signal sample, the filter intensity for the target edge can be determined as a component of the second value. When two motion vectors and two different reference frames are used to predict a sub-block of a CU including a P-block sample, and two motion vectors and two identical reference frames are used to predict a sub-block of a CU including a Q-block sample blocks, and the difference between the horizontal values or vertical values of the motion vectors for an identical reference frame is equal to or greater than 4 units per 1/4 sample of the luminance signals, the filtering intensity for the target boundary may be determined as a component of the second value.

[134] When two motion vectors and identical two reference frames are used to predict a sub-block of a CU including a P-block sample, and two motions and identical two reference frames are used to predict a sub-block of a CU including the sample Q-blocks, the filter intensity for the target boundary can be determined to be a component of the second value if the following two conditions are satisfied. The first condition may mean a condition of a case in which the difference of the horizontal values or vertical values of the motion vector of list 0 of each sub-block is equal to or greater than 4 units in 1/4 of the luminance signal sample, or of a case in which the difference of the horizontal values or vertical values of the motion vector of list 1 of each subblock is equal to or greater than 4 units per 1/4 sample of luminance signals. The second condition may mean a condition of a case in which the difference of the horizontal values or vertical values of the motion vector of list 0 for predicting a sub-block CU including a sample of P-blocks and the motion vectors of list 1 for predicting a sub-block of a CU including a sample of Q-blocks is equal to or greater than 4 units in 1/4 of the luminance signal sample, or the case in which the difference of the horizontal values or vertical values of the list 1 motion vector for CU sub-block prediction including the P-block sample and the list 0 motion vectors for CU sub-block prediction , including the Q-block sample, is equal to or greater than 4 units per 1/4 of the luminance signal sample.

[135] The image encoding/decoding equipment may determine the filtering intensity for the target boundary based on the conditions described above. Meanwhile, when the filtering intensity is set to the first value, filtering may not be performed on the target boundary. A deblocking filter according to some embodiments of the present disclosure may be applied based on at least one of filter intensity and/or filter length.

[136] Hereinafter, this document describes a method for determining the filtering intensity according to the value of the reconstructed luminance signal sample.

[137] FIG. 7 is a view illustrating a method for determining filter intensity in a deblocking filter.

[138] In some embodiments of the present disclosure, a deblocking filter may be determined according to the variables β and tC. Here, the variables β and tC may constitute values determined by the quantization parameter qP_L. For example, the filtering intensity in the deblocking filter can be extracted by summing the offset determined according to the value of the reconstructed luminance signal sample with qP_L. For example, the reconstructed luminance signal level may be set to LL, and LL may be extracted according to Equation 1 below.

[139] equation 1

[140] LL=((p_0.0+p_0.3+q_0.0+q_0.3)>>2)/(1<<bitDepth)

[141] p_i, k and q_i, k of Equation 1 may mean the sample value of a P-block or a Q-block determined according to FIG. 7.

[142] Meanwhile, qP_L can be determined according to Equation 2 below.

[143] equation 2

[144] qP_L=((Q_p_Q+Q_p_P+1)>>1)+qpOffset

[145] Q_p_Q and Q_p_P of Equation 2 may represent quantization parameters of a CU including q_0.0 and p_0.0, and qpOffset may represent an offset transmitted in the SPS layer.

[146] In some embodiments of the present disclosure, a deblocking filter may be applied to an 8x8 area. The deblocking filter can be applied not only to the boundary of the CU and the boundary of the subblock or transform block constructing the CU. In this case, a sub-block may mean at least one of a sub-block or a transform block formed by a sub-block-based combining mode, an affine mode, an intra-sub-segment-based (ISP) mode, and/or a sub-block-based transform.

[147] A deblocking filter may be applied to a sub-block generated by ISP mode or sub-block-based transform when a non-zero coefficient is present in a boundary crossing a TU boundary. Meanwhile, a deblocking filter may be applied to a sub-block according to a sub-block-based fusion mode or an affine mode based on reference frames and the motion vector of neighboring sub-blocks.

[148] General view of mode based on internal subsegments (ISP)

[149] FIG. 8 is a view illustrating the internal sub-segment (ISP) mode of internal prediction technologies.

[150] In traditional intra prediction, the current block to be encoded/decoded (current block) is considered as one unit, and encoding/decoding is performed without partitioning. However, when using ISP, the current block can be divided in the horizontal or vertical direction, and internal predictive encoding/decoding can be performed. At this time, encoding/decoding may be performed in units of broken ISP sub-blocks to form a restored ISP sub-block, and the restored ISP sub-block may be used as a reference block of the next broken ISP sub-block.

[151] When the ISP mode is applied to the current block, intra prediction may be performed on each of the ISP sub-blocks obtained by dividing the current block in the horizontal or vertical direction. Thus, intra prediction, residual signal generation, and reconstructed signal generation are performed in units of ISP sub-blocks, and the reconstructed signal of the reconstructed sub-segment can be used as an intra-prediction reference sample of the next sub-segment.

[152] The image encoding equipment may determine the direction of the ISP partitioning using various methods (eg, an RDO-based method). The determined split direction may be explicitly signaled via the bitstream as information regarding the ISP split direction. The image decoding equipment may determine the ISP split direction of the current block based on information regarding the signaled split direction. When the ISP split direction is implicitly determined by an encoding parameter of the current block such as the size (width or height) of the current block, the image encoding equipment and the image decoding equipment can determine the ISP split direction of the current block using an identical method.

[153] Each of the ISP subblocks obtained by splitting the current block may need to include a minimum of 16 samples. For example, when the current block is a 4x4 block, it may be implicitly determined that the ISP is not applied. In addition, when the current block is a 4x8 block or an 8x4 block, as shown in FIG. 8(a), the current block to which the ISP is applied may be split into two ISP sub-blocks. In addition, when the current block is not a 4x4 block, a 4x8 block, or an 8x4 block, as shown in FIG. 8(b), the current block to which the ISP is applied may be divided into four ISP sub-blocks. In the examples shown in FIGS. 8(a) and 8(b), when the ISP partitioning direction is a horizontal direction, encoding and decoding may be performed in order from the upper ISP sub-block to the lower ISP sub-block. In addition, when the ISP partitioning direction is a vertical direction, encoding and decoding can be performed in order from the left ISP sub-block to the right ISP sub-block.

[154] For example, the size of an ISP sub-block or coefficient group according to the size of the current block may be determined based on Table 2 below.

[155] Tab. 2

[156]

Block size Coefficient Group Size 1 x N, N ≥ 16 1x16 N x 1, N ≥ 16 16×1 2 x N, N ≥ 8 2x8 N x 2, N ≥ 8 8×2 All other possible M x N cases 4x4

[157] Embodiment #1

[158] Hereinafter, this document describes a method for encoding/decoding an image using the above-described deblocking filter. According to an embodiment of the present disclosure, the filter length for the deblocking filter can be determined. Although, in the following description, the transform block, P-block and/or Q-block are described as a sub-block extracted based on the ISP mode, the scope of the present disclosure is not limited to this. Here, a transform block may mean at least one of a P-block or a Q-block adjacent to the target boundary. In addition, although the current block or transform block is described as being a luminance signal block in the following description, the scope of the present disclosure is not limited to this.

[159] According to an embodiment of the present disclosure, when the P-block width or the Q-block width is 4, the Q-block filter length applied to the target boundary, which is a vertical boundary, can be determined to be 1. In contrast, when the Q-block width -block is equal to or greater than 32, the length of the Q-block filter applied to the target boundary may be determined to be 7. Alternatively, the length of the Q-block filter applied to the target boundary may be determined to be 3.

[160] Meanwhile, when the P-block width or the Q-block width is 4, the length of the P-block filter applied to the target boundary, which is a vertical boundary, can be determined to be 1. In contrast, when the P-block width is equal to or greater than 32, the P-block filter length applied to the target boundary may be determined to be 7. Alternatively, the P-block filter length applied to the target boundary may be determined to be 3.

[161] Meanwhile, when the P-block height or the Q-block height is 4, the length of the Q-block filter applied to the target boundary, which is a horizontal boundary, can be determined to be 1. In contrast, when the Q-block height is equal to or greater than 32, the filter length applied to the target boundary may be determined to be 7. Alternatively, the Q-block filter length applied to the target boundary may be determined to be 3.

[162] Meanwhile, when the P-box height or Q-box height is 4, the length of the P-box filter applied to the target boundary, which is a horizontal boundary, can be determined to be 1. In contrast, when the P-box height is equal to or greater than 32, the filter length applied to the target boundary may be determined to be 7. Alternatively, the P-box filter length applied to the target boundary may be determined to be 3.

[163] According to an embodiment of the present disclosure, when the width or size of the transform block is equal to or less than a predetermined value, a problem may occur that filter parallelization cannot be performed. Hereinafter, this document will describe in detail the deblocking filter problem that may occur in the above-described embodiment.

[164] FIG. 9 to 11 are views illustrating a method for determining a filter length for a deblocking filter applied to a target boundary.

[165] FIG. 9-11 show states in which a 16×16 block is segmented into four 4×16 blocks, and the second and fourth 4×16 blocks, respectively, are segmented into two 2×16 sub-blocks and four 1×16 sub-blocks by ISP mode.

[166] Referring to FIG. 9, for the first filter boundary, the P-block filter length may be determined to be 1, and the Q-block filter boundary may be determined to be 3. Referring to FIG. 10, for the second filter boundary, the P-block filter length may be determined to be 3, and the Q-block filter length may be determined to be 1. Referring to FIG. 11, for the third filter boundary, the Q-block filter length may be determined to be 3, and for the fourth filter boundary, the P-block filter length may be determined to be 3.

[167] In FIG. 10 and 11, although the width of a sub-block is less than 4, since the sum of the filter length when the corresponding block is a P-block and the filter length when the corresponding block is a Q-block is greater than 4, the overlap between the filtering applied by both boundaries may arise. A problem may arise that parallel processing for filtering cannot be performed due to overlapping filter lengths. Later in this document, a method for resolving such a problem is described in detail.

[168] In particular, when the ISP mode is applied to the current block, the transform block generated as the result of the intra prediction may have a size smaller than the size of the conventionally specified maximum transform block. Thus, according to some embodiments of the present disclosure, parallel processing for filtering can be performed even on a block to which the ISP mode is applied.

[169] FIG. 12 is a view illustrating an image encoding/decoding method according to an embodiment of the present disclosure.

[170] Referring to FIG. 12, the image encoding/decoding method according to an embodiment of the present disclosure may include extracting a reconstructed block for the current block (S1210), extracting a target boundary for the reconstructed block (S1220), determining a filter length for a deblocking filter to be applied for the target boundaries (S1230), and/or applying a deblocking filter based on the determined filter length (S1240).

[171] In this case, the length of the filter may be determined based on at least one of the width or height of a transform block adjacent to the target boundary.

[172] Embodiment #2

[173] According to another embodiment of the present disclosure, when the width or height of the transform block satisfies a predetermined condition, the length of the filter applied to the target boundary can be determined taking this into account.

[174] For example, when the width or height of the transform block is smaller than the size of the minimum transform block, the length of the filter applied to the target boundary may be determined as a preset value. For example, when the width or height of the transform block is less than 4, the length of the filter applied to the target boundary may be determined to be 1.

[175] FIG. 13 and 14 are views illustrating a method for determining the length of a deblocking filter according to another embodiment of the present disclosure.

[176] FIG. 13 shows a method for determining the length of a deblocking filter that solves the problem according to the embodiment of FIG. 10. When the width or height of the transform block is equal to or less than 4, the length of the target edge filter may be determined to be 1. Referring to FIG. 13, since the width of a 2×16 sub-block is equal to or less than 4, both the Q-block filter length of the first target boundary and the P-block filter length of the second target boundary can be determined to be 1. Therefore, the overlap between the Q-block filter of the first target boundary and the P-block filter may not produce a second target boundary.

[177] FIG. 14 shows a method for determining the length of a deblocking filter that solves the problem according to the embodiment of FIG. 11. Referring to FIG. 13, since the width of the 1×16 ISP sub-block is equal to or less than 4, both the Q-block filter length of the first target boundary and the P-block filter length of the second target boundary can be determined to be 1. Therefore, the overlap between the Q-block filter of the third target boundary and the P-block filter may not produce a fourth target boundary.

[178] FIG. 15 and 16 are views illustrating a method for determining the length of a deblocking filter according to another embodiment of the present disclosure.

[179] Referring to FIG. 15, firstly, whether the extracted target boundary is a vertical boundary or a horizontal boundary can be determined (S1510). When the target boundary is a horizontal boundary (S1510-N), the horizontal boundary filter length extraction process can be performed (S1525). Next, a process for extracting horizontal edge filter lengths will be described using FIG. 16.

[180] For example, the image encoding/decoding equipment may first determine the length of the Q-block filter. When the target boundary is a vertical boundary (S1510-Y), it can be determined whether at least one of the P-block width or the Q-block width of the first value is equal to or less than or not (S1520). When at least one of the P-block width or the Q-block width is equal to or less than the first value (S1520-Y), the length of the Q-block filter for the target boundary may be determined as the first length (S1521). Meanwhile, when the P-block width and the Q-block width are greater than the first value (S1520-N), the Q-block width of the second value is equal to or greater than or not (S1530). When the Q-block width is equal to or greater than the second value (S1530-Y), the Q-block filter length for the target boundary may be determined as the second length (S1531). Meanwhile, when the Q block width is larger than the first value and smaller than the second value (S1530-N), the Q block filter length for the target boundary may be determined as the third length (S1532).

[181] Next, the image encoding/decoding equipment can determine the length of the P-block filter. After determining the length of the Q-block filter, it can be determined whether at least one of the P-block width or the Q-block width of the first value is equal to or less than or not (S1540). When at least one of the P-block width or the Q-block width is equal to or less than the first value (S1540-Y), the P-block filter length for the target boundary may be determined as the first length (S1541). Meanwhile, when the P-block width and the Q-block width are larger than the first value (S1540-N), it can be determined whether the P-block width is equal to or larger than the second value (S1550). When the P-block width is equal to or greater than the second value (S1550-Y), the P-block filter length for the target boundary may be determined as the second length (S1551). Meanwhile, when the P-block width is larger than the first value and smaller than the second value (S1550-N), the P-block filter length for the target boundary may be determined as the third length (S1552).

[182] For example, the first value and the second value may be 4 and 32, respectively. In addition, the first length, the second length and the third length, respectively, may have values of 1, 7 and 3, but this is only an example and is not limited to this. In addition, although the above example describes an embodiment in which the Q-block filter length is first retrieved and then the P-block filter length is retrieved, the embodiment in which the Q-block filter length is first retrieved and then the P-block filter length is retrieved blocks, and an embodiment in which the Q-block filter length and the P-block filter length are simultaneously retrieved may also be included within the scope of the present disclosure.

[183] Referring to FIG. 16, firstly, whether the extracted target boundary is a vertical boundary or a horizontal boundary can be determined (S1610). S1620-S1651 of FIG. 16 may be a detailed description of S1525 of FIG. 15. In addition, S1610 of FIG. 16 and S1510 of FIG. 15 may indicate an identical configuration.

[184] For example, the image encoding/decoding equipment may first determine the length of the Q-block filter. When the target boundary is a horizontal boundary (S1610-Y), at least one of the P-block height or the Q-block height is equal to or less than the first value (S1620). When at least one of the P-block width or the Q-block width is equal to or less than the first value (S1620-Y), the Q-block filter length for the target boundary may be determined as the first length (S1621). Meanwhile, when the height of the P block and the height of the Q block are greater than the first value (S1620-N), it can be determined whether the height of the Q block is equal to or greater than or not of the second value S1630). When the Q-block height is equal to or greater than the second value (S1630-Y), the Q-block filter length for the target boundary may be determined as the second length (S1631). Meanwhile, the Q-block height is greater than the first value and less than the second value (S1530-N), the Q-block filter length for the target boundary may be determined as the third length (S1532).

[185] Next, the image encoding/decoding equipment can determine the length of the P-block filter. After determining the length of the Q-block filter, it can be determined whether at least one of the P-block height or the Q-block height of the first value is equal to or less than or not (S1640). When at least one of the P-block height or the Q-block height is equal to or less than the first value (S1640-Y), the P-block filter length for the target boundary may be determined as the first length (S1641). Meanwhile, when the P-block height and the Q-block height are greater than the first value (S1640-N), the P-block height of the second value is equal to or greater than or not (S1650). When the P-block height is equal to or greater than the second length (S1650-Y), the P-block filter length for the target boundary may be determined as the second length (S1651). Meanwhile, when the P-block height is greater than the first value and less than the second value (S1650-N), the P-block filter length for the target boundary may be determined as the third length (S1652).

[186] For example, the first value and the second value may be 4 and 32, respectively. In addition, the first length, the second length and the third length, respectively, may have values of 1, 7 and 3, but this is only an example and is not limited to this. In addition, although the above example describes an embodiment in which the Q-block filter length is first retrieved and then the P-block filter length is retrieved, the embodiment in which the Q-block filter length is first retrieved and then the P-block filter length is retrieved blocks, and an embodiment in which the Q-block filter length and the P-block filter length are simultaneously retrieved may also be included within the scope of the present disclosure.

[187] According to the present embodiment, since a situation in which parallel processing of a deblocking filter is not possible due to reduction in the size of the transform block (for example, in the case of a sub-block generated by applying the ISP mode), or a situation in which overlap between filters occurs due to different target boundaries do not occur, the image encoding/decoding equipment can always perform parallel filtering processing, and the efficiency of image encoding/decoding can be increased.

[188] In this description, performing parallelization or parallel processing does not mean that the image encoding/decoding equipment always performs parallel processing. Even when parallel processing is supported, whether or not to perform parallel processing may be determined taking into account the hardware performance, service type, and quality of service of the image encoding/decoding equipment. For example, when the image encoding/decoding equipment is implemented using a multiprocessor system, parallel processing of the deblocking filter can be performed through an embodiment of the present disclosure. Meanwhile, when the image encoding/decoding equipment is implemented using a single processor, the deblocking filter can be sequentially applied to the target edge without applying the above-described embodiment.

[189] Although the exemplary methods of the present disclosure described above are presented as a sequence of operations for clarity of description, it is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order as necessary. To implement the method of the present disclosure, the described steps may further include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.

[190] In the present disclosure, image encoding equipment or image decoding equipment that performs a predetermined operation (step) may perform an operation (step) of confirming a condition or situation for executing the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding equipment or the image decoding equipment may perform the predetermined operation after determining whether or not the predetermined condition is satisfied.

[191] The various embodiments of the present disclosure are not intended to be a list of all possible combinations and are intended to describe characteristic aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more of them.

[192] Various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination of the above. If the present disclosure is implemented in hardware, the present disclosure may be implemented in application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays. (FPGA), general processors, controllers, microcontrollers, microprocessors, etc.

[193] In addition, the image decoding equipment and image encoding equipment to which embodiments of the present disclosure are applied may be included in a multimedia broadcast transceiver device, a mobile communication terminal, a home theater system video device, a digital cinema system video device, a surveillance camera, video chat device, real-time communication device such as video communication, mobile streaming device, storage medium, video recording camera, video-on-demand (VoD) service device, OTT video device ( video over networks), Internet streaming service device, 3D video device, video telephony system video device, medical video device, etc. and can be used to process video or data signals. For example, OTT video devices may include a gaming console, Blu-ray player, Internet-enabled television, home theater system, smartphone, tablet PC, digital video recorder (DVR), etc.

[194] FIG. 17 is a view showing a content streaming system to which an embodiment of the present disclosure is applicable.

[195] Referring to FIG. 17, the content streaming system to which the embodiment(s) of this document applies may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a media input device.

[196] The encoding server compresses content input from media input devices such as a smartphone, camera, video recorder, etc. into digital data to form a bitstream, and transmits the bitstream to the streaming server. As another example, when media input devices such as smartphones, cameras, video recorders, etc. directly generate a bitstream, the encoding server may be omitted.

[197] The bitstream may be generated by an image encoding method or image encoding equipment to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[198] The streaming server transmits multimedia data to the user device based on the user's request through the web server, and the web server serves as a medium for informing the user regarding the service. When a user requests a desired service from a web server, the web server may deliver it to the streaming server, and the streaming server may stream the media data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves to manage the command/response between devices in the content streaming system.

[199] The streaming server may receive content from a media storage and/or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may maintain the bit stream for a predetermined time.

[200] Examples of a user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable media player (PMP), a navigation device, a tablet PC, tablet PCs, ultrabooks, wearable devices (such as smart watches, smart glasses, head-mounted displays), digital TVs, desktop computers, digital signage system, etc.

[201] Each server in a content streaming system may operate as a distributed server, in which case data received from each server may be distributed.

[202] The scope of the disclosure includes software or computer-executable instructions (e.g., operating system, application, firmware, program, etc.) to enable operations according to the methods of various embodiments in hardware or computer, non-volatile computer-readable media having such software or instructions stored and executed in hardware or a computer.

Industrial applicability

[203] Embodiments of the present disclosure may be used to encode or decode an image.

Claims (43)

1. An image decoding method carried out by an image decoding equipment comprising a decoding module that decodes an image from a bitstream, the image decoding method comprising the steps of: extracting a target boundary of the deblocking filter in the reconstructed frame; determining the maximum length of the deblocking filter for the target boundary; And apply a deblocking filter to the target boundary based on the maximum length, wherein, based on the target boundary being a vertical boundary, the maximum length is determined based on the width of at least one of the first block adjacent to the left side of the target boundary or the second block adjacent to the right side of the target boundary, and wherein, based on the target boundary being a horizontal boundary, the maximum length is determined based on the height of at least one of a third block adjacent to the top side of the target boundary or a fourth block adjacent to the bottom side of the target boundary. 2. The image decoding method according to claim 1, wherein the first to fourth blocks are subblocks based on an internal sub-segment (ISP) mode, and wherein the maximum length is defined as equal to or less than the width of the first block or the second block, or the height of the third block or the fourth block. 3. The image decoding method of claim 1, wherein the maximum length comprises a first maximum length on a first block side and a second maximum length on a second block side. 4. The image decoding method of claim 3, wherein, based on the width of at least one of the first block and the second block being equal to or less than 4, each of the first maximum length and the second maximum length is determined to be 1. 5. The image decoding method according to claim 3, wherein based on the width of the first block equal to or greater than 32, the first maximum length is determined to be 7, wherein, based on the width of the second block being equal to or greater than 32, the second maximum length is determined to be 7. 6. The image decoding method of claim 3, wherein based on the width of the first block being less than 32, the first maximum length is determined to be 3, wherein, based on the width of the second block being less than 32, the second maximum length is determined to be 3. 7. The image decoding method of claim 1, wherein the maximum length comprises a third maximum length on a third block side and a fourth maximum length on a fourth block side. 8. The image decoding method of claim 7, wherein, based on the height of at least one of the third block or the fourth block being equal to or less than 4, each of the third maximum length and the fourth maximum length is determined to be 1. 9. The image decoding method of claim 7, wherein based on the third block height equal to or greater than 32, the third maximum length is determined to be 7, wherein, based on the height of the fourth block being equal to or greater than 32, the fourth maximum length is determined to be 7. 10. The image decoding method of claim 7, wherein based on the height of the third block being less than 32, the third maximum length is determined to be 3, however, based on the height of the fourth block being less than 32, the fourth maximum length is determined to be 3. 11. An image encoding method carried out by an image encoding equipment comprising an encoding module that encodes an image into a bitstream, the image encoding method comprising the steps of: extracting a target boundary of the deblocking filter in the reconstructed frame; determining the maximum length of the deblocking filter for the target boundary; And apply a deblocking filter to the target boundary based on the maximum length, wherein, based on the target boundary being a vertical boundary, the maximum length is determined based on the width of at least one of the first block adjacent to the left side of the target boundary or the second block adjacent to the right side of the target boundary, and wherein, based on the target boundary being a horizontal boundary, the maximum length is determined based on the height of at least one of a third block adjacent to the top side of the target boundary or a fourth block adjacent to the bottom side of the target boundary. 12. The image encoding method of claim 11, wherein the first to fourth blocks are subblocks based on an internal sub-segment (ISP) mode, and wherein the maximum length is defined as equal to or less than the width of the first block or the second block, or the height of the third block or the fourth block. 13. The image encoding method according to claim 11, wherein the maximum length is determined by comparing the width or height with a specified threshold value, wherein the number of target comparison blocks is determined to be 1 or 2, based on a predetermined threshold value. 14. A non-transitory computer-readable medium storing instructions that, when executed by a computer, direct the computer to execute an image encoding method, wherein the image encoding method comprises the steps of: extracting a target boundary of the deblocking filter in the reconstructed frame; determining the maximum length of the deblocking filter for the target boundary; And apply a deblocking filter to the target boundary based on the maximum length, wherein, based on the target boundary being a vertical boundary, the maximum length is determined based on the width of at least one of the first block adjacent to the left side of the target boundary or the second block adjacent to the right side of the target boundary, and wherein, based on the target boundary being a horizontal boundary, the maximum length is determined based on the height of at least one of a third block adjacent to the top side of the target boundary or a fourth block adjacent to the bottom side of the target boundary. 15. A method for transmitting a bit stream, containing: transmission of a bit stream formed by the stages in which: extracting a target boundary of the deblocking filter in the reconstructed frame; determining the maximum length of the deblocking filter for the target boundary; And apply a deblocking filter to the target boundary based on the maximum length, wherein, based on the target boundary being a vertical boundary, the maximum length is determined based on the width of at least one of the first block adjacent to the left side of the target boundary or the second block adjacent to the right side of the target boundary, and wherein, based on the target boundary being a horizontal boundary, the maximum length is determined based on the height of at least one of a third block adjacent to the top side of the target boundary or a fourth block adjacent to the bottom side of the target boundary.

Images