RU2788631C2 - Methods and systems for exponential partitioning - Google Patents

Abstract

FIELD: video encoding/decoding.

SUBSTANCE: invention relates to the field of technologies of compression and decompression of digital video, including decoding and encoding, in particular it is aimed at a method and systems for exponential partitioning of encoding units. A decoder is proposed, including a circuit made with the possibility of reception of a bit stream, determination of whether an exponential partitioning mode is turned on, partitioning of a block to the first and the second areas in accordance with a curve line, and restoration of pixel data of the block, using the curve line, wherein the first area and the second area have a non-rectangular shape.

EFFECT: increase in the quality of decompressed video.

8 cl, 19 dwg

Description

The technical field to which the invention belongs

The present invention relates generally to the field of digital video compression and decompression technologies, including decoding and encoding. In particular, the present invention is directed to a method and systems for exponentially splitting coding units.

State of the art

A video codec may include electronic circuitry or software that compresses or decompresses digital video. It can convert uncompressed video to compressed format and vice versa. In the context of video compression, a device that compresses the video (and/or performs some of its functions) can generally be called an encoder, and a device that decompresses the video (and/or performs some of its functions) can be called a decoder.

The compressed data format may follow a standard video compression specification. Compression can be lossy because the compressed video lacks some of the information that was present in the original video. The consequence of this may be that the decompressed video may be of lower quality than the original uncompressed video, as there is not enough information to accurately reconstruct the original video.

There can be complex relationships between video quality, amount of data used to represent the video (e.g. determined by bit rate), complexity of encoding and decoding algorithms, sensitivity to data loss and error, ease of editing, random access, end-to-end delay (e.g. latency) etc.

Disclosure of the essence of the invention

In one aspect of the invention, a decoder may include circuitry that may be configured to receive a bit stream. In an embodiment, the circuit may be further configured to determine if the exponential split mode is enabled and to split the block into a first region and a second region according to a curved line. In an embodiment, the circuit may also be configured to reconstruct the pixel data of the block using a curved line, where the first region and the second region may be non-rectangular.

The decoder may further include one or more of the following features, taken alone or in combination. In an embodiment, the exponential split mode may be signaled in the bitstream. In an embodiment, the curved line dividing the block into a first region and a second region may be characterized by a predetermined pattern. In another embodiment, the curved line dividing the block into a first region and a second region may be characterized by a predetermined coefficient value. Furthermore, in an embodiment, the exponential split mode is available for block sizes greater than or equal to 8x8 luma samples. In another embodiment, recovering the pixel data may include calculating a predictor for the first region using an associated motion vector contained in the bitstream. In another embodiment, the decoder may also include an entropy decoder processor that may be configured to receive a bitstream and decode the bitstream into quantized coefficients. The decoder may also include an inverse quantization and inverse transform processor, which may be configured to process the quantized coefficients, including performing an inverse discrete cosine transform. In addition, the decoder may include a deblocking filter, a frame buffer; and an intra-prediction processor. In an embodiment, the bitstream may include a parameter indicating whether the exponential split mode is enabled for the block. In another embodiment, the block may form part of a quadtree plus a binary decision tree. In addition, a block can be a non-leaf quadtree node plus a binary decision tree. In an embodiment, a block may be a coding tree unit or a coding unit. In another embodiment, the first region may be a coding unit or a prediction unit.

In a further aspect of the invention, the method may include receiving, by a decoder, a bitstream and determining, by said decoder, whether exponential split mode is enabled. The method may also include determining, by the decoder, a curved line dividing the block into a first region and a second region, and recovering, by the decoder, the pixel data of the block, and using the curved line.

The method may further include one or more of the following features, taken alone or in combination. In an embodiment, the exponential split mode may be signaled in the bitstream. In another embodiment, the curved line that divides the block into first and second regions may be characterized by a predetermined pattern. In an embodiment, the curved line dividing the block into a first region and a second region may be characterized by a predetermined coefficient value. In an embodiment, the exponential split mode may be available for block sizes greater than or equal to 8×8 luma samples. In another embodiment, recovering the pixel data may include calculating a predictor for the first region using an associated motion vector contained in the bitstream. In another embodiment, the decoder may include an entropy decoder processor that may be configured to receive a bitstream and decode the bitstream into quantized coefficients. The decoder may also include an inverse quantization and inverse transform processor, which may be configured to process the quantized coefficients, including performing an inverse discrete cosine transform. In addition, decoding may include a deblocking filter, a frame buffer, and an intra prediction processor. In an embodiment, the bitstream may include a parameter indicating whether the exponential split mode is enabled for the block. In another embodiment, the block may form part of a quadtree plus a binary decision tree. In addition, a block can be a non-leaf quadtree node plus a binary decision tree. In another embodiment, a block may be a coding tree unit or a coding unit. In an embodiment, the first region may be a coding unit or a prediction unit.

This document also describes non-volatile computer program products (i.e., physically embodied computer program products) that store instructions that, when executed by one or more data processors of one or more computing systems, cause at least one data processor to perform operations. Similarly, computer systems are also described, which may include one or more data processors and memory coupled to one or more data processors. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. In addition, the methods may be implemented by one or more data processors, either on a single computer system or distributed across two or more computer systems. Such computing systems may be connected and may exchange data and/or commands or other instructions and the like. via one or more connections, including a network connection (eg, the Internet, a wireless wide area network, a local area network, a wide area network, a wired network, etc.), a direct connection between one or more computing systems, etc.

Details of one or more embodiments of the subject matter described herein are set forth below in the detailed description of the invention with reference to the accompanying drawings. Other features and advantages of the subject matter described herein will be apparent from the description and drawings, as well as from the claims.

Brief description of the drawings

1 is a diagram illustrating an example of a block partition;

figure 2 is a diagram illustrating an example of a geometric partition;

3A is a diagram illustrating an example of exponential splitting, according to some aspects of the current subject matter, which improves compression efficiency;

3B is a series of graphs illustrating exemplary template exponential splits;

3C are exemplary curves associated with 4 predetermined coefficients that may define an exemplary exponential function;

3D is another exemplary block showing different start indices P1 and end indices P2 that break up a rectangular block;

4 is a system block diagram illustrating an example of a video encoder capable of exponential splitting;

FIG. 5A is a flowchart illustrating an exemplary exponentially split video encoding process according to some aspects of the current subject matter, which can reduce encoding complexity while improving compression efficiency;

FIG. 5B is a flowchart illustrating an exemplary exponentially split video encoding process using split parameters in accordance with some aspects of the current subject matter;

5C is a flowchart illustrating an exemplary exponentially split video coding process using an adaptive discrete cosine shape transform in accordance with some aspects of the current subject matter;

6 is a system block diagram illustrating an exemplary decoder capable of decoding a bitstream using exponential splitting;

7 is a flowchart illustrating an exemplary bitstream decoding process using exponential splitting;

Fig.8 - an example of splitting a frame based on a quadtree plus a binary tree;

Fig. 9 is an example of exponential splitting at the CU level of the quadtree plus binary tree shown in Fig. 8;

Fig. 10 is an image containing an apple that cannot be broken up efficiently by straight line segments;

11 is a diagram illustrating another exemplary block split according to exponential splitting.

Fig.12 is a diagram illustrating the inheritance of exponential splitting parameters by the current block from a neighboring block in space;

Fig.13 - examples of neighboring blocks in space for the current block; and

14 shows an exemplary current block with a time-adjacent block from which the current block inherits exponential splitting parameters.

Like reference numerals throughout the various drawings refer to like elements.

Implementation of the invention

Some implementations of the current subject matter relate to exponential splitting. With exponential splitting, a rectangular block can be split into non-rectangular regions using a curve versus a straight line segment. Using a curve to break blocks allows the break to more closely follow object boundaries, resulting in lower motion compensation prediction error, smaller residuals, and thus better compression efficiency. In some implementations, the curve may be characterized by an exponential function. A curve (eg, an exponential function) may be defined using predetermined coefficients and/or patterns that may be signaled in the bitstream for use by a decoder. In some implementations, exponential splitting may be available for luma samples of size 8x8 or larger. When splitting rectangular blocks using a curve, you can achieve greater compression efficiency for certain objects than technologies that are limited to splitting using a straight line segment, such as using geometric splitting.

In addition, some implementations of the current subject matter relate to predicting exponential splitting parameters using spatial and/or temporal reference blocks. In some implementations, for a given current block, an exponential split merge variable may be signaled to indicate that the current block may inherit all or some of these exponential split parameters from another block. This other block may be contiguous in space or time. By allowing blocks to inherit exponential splitting parameters from other blocks, the amount of signaling in the bitstream can be reduced, allowing greater compression efficiency.

In addition, some implementations of the current subject matter may include performing an adaptive discrete cosine shape transform (SADCT) on regions (eg, blocks) that have been decomposed into non-rectangular regions with a curve versus a straight line segment. If an exponential split is applied to a block, there is a possibility that the resulting regions (eg, splits) will result in one region having a low prediction error and another region having a high prediction error. Accordingly, the current subject matter may include performing a SADCT on a region having a low prediction error. By performing SADCT on a region having a low prediction error, compression efficiency can be improved. And, in some implementations, during decoding, an inverse SADCT may be performed on a single region split using exponential splitting. In some implementations, the inverse SADCT may be signaled in the bitstream as the choice of an additional transform to the full block discrete cosine transform (DCT) for a segment with low prediction error. In some implementations, the reverse SADCT may be performed based on exponential split parameters and without explicit signaling in the bitstream that requires the reverse SADCT to be performed.

Motion compensation may include an approach to predicting a video frame, or a portion thereof, given previous and/or future frames, given the movement of the camera and/or objects in the video. It can be used in encoding and decoding video data for video compression, such as encoding and decoding using the Moving Picture Experts Group (MPEG)-2 (also referred to as advanced video coding (AVC)) standard. Motion compensation allows you to describe the picture in terms of the transformation of the reference image into the current image. The reference picture may be a previous or future picture in time compared to the current picture. In the case where images can be accurately synthesized from previously transmitted and/or stored images, the compression efficiency can be improved.

Block division may refer to a video encoding method to find areas of similar motion. Some form of block partitioning can be found in video codec standards, including MPEG-2, H.264 (also called AVC or MPEG-4 Part 10), and H.265 (also called High Efficiency Video Coding (HEVC)). In exemplary block partitioning approaches, non-overlapping blocks of a video frame may be partitioned into rectangular sub-blocks to search for block partitions that contain pixels with similar motion. This approach can work well when all the pixels in the block partition have the same motion. The movement of pixels in a block can be determined relative to previously encoded frames.

Fig. 1 is a diagram illustrating an example of pixel blocking according to an embodiment. The original rectangular image or block 100, which may be directly a sub-block (eg, a node in an encoding tree), may be split into rectangular sub-blocks. For example, at step 110, block 100 is split into two rectangular sub-blocks 110a and 110b. Sub-blocks 110a and 110b may then be processed separately. As another example, at step 120 block 100 is divided into four rectangular sub-blocks 120a, 120b, 120c and 120d. The sub-blocks themselves may be further subdivided until it is determined that the pixels within the sub-blocks share the same motion, the minimum block size is reached, or other criteria are met. When pixels in a subblock have similar motion, the motion vector can describe the motion of all pixels in that area.

Some video coding approaches may include geometric partitioning, in which a rectangular block (eg, shown in FIG. 1) is further partitioned by a straight line segment into two regions, which may not be rectangular. For example, FIG. 2 is a diagram illustrating an example of a geometric partition. The exemplary rectangular block 200 (which may be M pixels wide and N pixels high and referred to as M×N pixels) may be split along the straight line segment P1P2 205 into two regions (region 0 and region 1). When the pixels in region 0 have the same motion, the motion vector can describe the motion of all pixels in that region. A motion vector may be used to compress region 0. Similarly, when pixels in region 1 have similar motion, the associated motion vector may describe the motion of the pixels in region 1. Such geometric partitioning may be signaled to a receiver (eg, decoder) by encoding positions P1 and P2 (or representations of positions P1 and P2) in the video bitstream.

When encoding video data using geometric partitioning, a straight line segment 205 (or, more specifically, points P1 and P2) can be defined. But a straight line segment may not be able to break the block in such a way as to reflect the boundaries of the object. As a result, splitting using straight line segments does not allow efficient block splitting (eg, such that any resulting remainder is small). This may be true in cases where the block may contain pixels (eg, luminance samples) representing an object or border having a curved (eg, non-straight) border. For example, let us now turn to Fig. 10, which shows an image containing an apple that cannot be effectively broken using straight line segments. The apple includes several rectangular boxes indicating areas of the image, however, if they are split using a straight line segment in accordance with the geometric split, the split cannot exactly follow the boundary of the object (for example, an apple).

Some implementations of the current subject matter include breaking up a rectangular block into non-rectangular regions with a curve versus a straight line segment. Using a curve for block splitting allows the splits to more closely follow object boundaries, resulting in lower prediction error, smaller residuals, and thus better compression efficiency. In some implementations, the curve may be characterized by an exponential function. The curve (eg, exponential function) may be represented using predetermined coefficients and/or indices, which may be signaled in the bitstream for use by the decoder. In some implementations, exponential splitting may be available for blocks greater than or equal to 8×8 luma samples. By splitting rectangular blocks with a curved line in the current subject matter, greater compression efficiency can be achieved for certain objects than technologies limited to straight line segmenting, such as geometric splitting.

FIG. 3A is a diagram illustrating an example of exponential splitting, according to some aspects of the current subject matter, which can improve compression efficiency. Rectangular block 300 includes pixels (eg, luminance samples). The rectangular block 300 may be, for example, 8×8 pixels (eg, luminance samples) or larger.

As shown in FIG. 3A, rectangular block 300 may be divided into two regions (eg, region 0 and region 1, labeled 310 and 315, respectively) by curved line 305. All luminance samples within region 310 may be considered to have the same or similar motion and can be represented by the same motion vector. Similarly, all luma samples in region 315 can be considered to have the same or similar motion and can be represented by the same motion vector. In some implementations, all brightness samples to the left of or above the curved segment 305 separating the rectangular block 300 may be considered to belong to region 0 (310). In some implementations, all brightness samples to the right or below the curved segment 305 separating the rectangular block 300 may be considered to belong to region 1 (315). In some implementations, all luma samples through which the curved line separating rectangular block 300 passes belong to region 0 (310). In some implementations, all luma samples that the curved line separating rectangular block 300 passes through can be considered to belong to region 1 (315). Other implementations are also possible.

In some implementations, by performing an exponential split, it is possible to reduce the number of possible splits that may occur (for example, compared to a geometric split), which can reduce the computational requirements for motion estimation in order to determine the appropriate split (for example, to determine the best linear segment for splitting the block). In some implementations, by performing exponential splitting, non-rectangular regions (eg, 310 and 315) can more closely follow object boundaries, thereby reducing prediction error, residual size, and improving compression efficiency compared to video encoding using geometric splitting.

Exponential splitting can be represented in a bitstream. In some implementations, the exponential split mode may be used and the appropriate parameters may be signaled in the bitstream. For example, exponential splitting may be represented in a bitstream by signaling predetermined exponential splitting patterns. FIG. 3B shows a series of graphs illustrating exemplary pattern splits according to an embodiment. These regular exponential partitions allow us to define exactly the set of given orientations. In some implementations, signaling can be done by including an index in one or more of these regular (eg, pattern) exponential splits, which are given. For example, FIG. 3C shows exemplary curves associated with 4 preset patterns (1,2,3,4) according to an embodiment. In some implementations, the number of pattern bends may vary.

As another example, exponential splits can be represented in the bitstream by signaling predetermined coefficients that indicate the degree of curvature, allowing for additional exponential functions to be provided.

In some implementations, the given pattern used in exponential split mode may specify a straight line segment. For example, in FIG. 3C, the segment indexed by a factor of 1 is a straight line, which can be considered a special case of exponential splitting, which can lead to a similar result to geometric splitting.

In some implementations, both orientation patterns, an example of which is shown in FIG. 3B, and predetermined patterns, an example of which is shown in FIG. 3C, can be used to effectively signal a large number of potential exponential splits.

In some implementations, a start and end index may be specified. For example, FIG. 3A shows a curved line starting at the lower left corner of a rectangular block 300 and ending at the upper right corner of a rectangular block 300 according to an embodiment. In some implementations, the start and end indices may be explicitly specified in the bitstream. For example, FIG. 3D shows another exemplary block showing different start indices P1 and end indices P2 that break rectangular block 300. Start indices P1 and end indices P2 may be signaled directly or may be indicated by an index into a set of predetermined values. In some embodiments, other options are possible.

In some implementations, exponential split parameters need not be included in the bitstream for every block undergoing exponential split (eg, for which exponential split mode is true). For example, for a given current block, exponential splitting parameters can be inherited from another block (sometimes called a parent block). The parent block may be contiguous in space and/or time. The parent block may be specified in the bitstream by an index for a given list and/or may be specified by at least creating a list of candidates whose index is signaled in the bitstream.

4 is a system block diagram illustrating an exemplary embodiment of a video encoder 400 capable of exponential splitting, such as with SADCT. An exemplary video encoder 400 receives input video 405, which may be initially segmented or split according to a processing scheme, such as a tree-structured macroblock partitioning scheme (eg, quadtree plus binary tree (QTBT)). An example of a macroblock partitioning scheme with a tree structure may include partitioning an image frame into larger block units called coding tree units (CTUs). In some implementations, each CTU may be further partitioned one or more times into a plurality of sub-blocks called coding units (CUs). The result of this partitioning may include a group of sub-blocks, which may be called Prediction Units (PUs). Alternatively, you can also use Transformation Units (TUs). Such a partitioning scheme may include performing exponential partitioning in accordance with some aspects of the current subject matter. For example, FIG. 8 shows an example of QTBT frame splitting, and FIG. 9 shows an example of exponential splitting at the QTBT CU level shown in FIG. 8. FIG.

An exemplary video encoder 400 includes an intra prediction processor 415, a motion estimation/compensation processor 420 (also referred to as an inter prediction processor) capable of supporting exponential splitting, a transform/quantization processor 425, an inverse quantization/inverse transform processor 430, an in-loop filter 435, a decoded picture buffer 440 and an entropy encoding processor 445 . In some implementations, motion estimation/compensation processor 420 may perform exponential splitting, including determining whether the current block can inherit exponential splitting parameters from another block, and which block should inherit exponential splitting parameters from which blocks. In some implementations, transform/quantization processor 425 may perform SADCT. Bitstream parameters that signal exponential splitting and inheritance modes may be input to entropy encoding processor 445 for inclusion in output bitstream 450.

During operation, for each frame block of the input video 405, it may be determined whether to process the block through intra image prediction or using motion estimation/compensation. The block may be provided to intra-prediction processor 410 or motion estimation/compensation processor 420. If the block is to be processed by intra prediction, intra prediction processor 410 may perform processing to derive a predictor. If the block is to be processed by motion estimation/compensation, motion estimation/compensation processor 420 may perform processing including using an exponential split to derive a predictor.

The remainder can be formed by subtracting the predictor from the input video. The remainder may be received by transform/quantization processor 425, which may perform transform processing (eg, SADCT) to generate coefficients that can be quantized. The quantized coefficients and any associated signaling information may be provided to entropy encoding processor 445 for entropy encoding and inclusion in output bitstream 450. Entropy encoding processor 445 may support encoding signaling information related to exponential splitting. In addition, the quantized coefficients may be provided to an inverse quantization/inverse transform processor 430, which may reproduce pixels that may be combined with a predictor and processed by an in-loop filter 435, the output of which is stored in a decoded image buffer 440 for use by the motion estimation/compensation processor 420 , which can support exponential splitting.

FIG. 5A is a flowchart illustrating an exemplary exponentially split video encoding process 500A, according to some aspects of the current subject matter, that reduces encoding complexity while improving compression efficiency. At step 510A, the video frame may be subjected to initial block segmentation, for example, using a macroblock partitioning scheme with a tree structure, which may include splitting the image frame into CTUs and CUs. At 520, a block for exponential splitting may be selected. The selection may include identifying, according to the metric rule, that the block should be processed according to the exponential split mode.

At 530A, an exponential split may be determined. A curved line (e.g. 305) can be defined that will split the pixels contained in the block according to their movement between frames into two non-rectangular regions (e.g. region 0 and region 1) such that the pixels (e.g. brightness samples ) in one of the regions (for example, in region 0) had a similar motion, and pixels in another region (for example, in region 1) had a similar motion. At 550A, a certain exponential split may be signaled in the bitstream. Signaling in the bitstream may include, for example, including an index in one or more predetermined patterns and/or coefficients.

FIG. 5B is a flowchart illustrating an exemplary exponentially split video encoding process 500B according to some aspects of the current subject matter that reduces encoding complexity while improving compression efficiency. At step 510B, the video frame may be subjected to initial block segmentation, for example, using a macroblock partitioning scheme with a tree structure, which may include splitting the image frame into CTUs and CUs. At 520B, a block for exponential splitting may be selected. The selection may include identifying, according to the metric rule, that the block should be processed according to the exponential split mode.

At 530B, an exponential split may be determined. A curved line (e.g., 305) can be defined that will split the pixels contained within a block, according to their movement between frames, into two non-rectangular regions (e.g., region 0 and region 1) such that the pixels (e.g., samples brightness) within one of the regions (for example, in region 0) had a similar motion, and pixels in another region (for example, in region 1) had a similar motion.

At block 540B, a split parameter representation may be determined, which may include determining whether the current block inherits the exponential split parameters from another block, and from which other block the current block will inherit.

At 550B, a certain exponential split may be signaled in the bitstream. Bitstream signaling may include, for example, including an index in a predetermined list of spatially and temporally neighboring blocks.

FIG. 5C is a flowchart illustrating an exemplary exponentially split and SADCT video encoding process 500C in accordance with some aspects of the current subject matter, which can reduce encoding complexity while improving compression efficiency. At step 510C, the video frame may be subjected to initial block segmentation, for example, using a macroblock partitioning scheme with a tree structure, which may include splitting the image frame into CTUs and CUs. At 520C, a block for exponential splitting may be selected. The selection may include identifying, according to the metric rule, that the block should be processed according to the exponential split mode.

At 530C, an exponential split may be determined. A curved line (e.g. 305) can be defined that will split the pixels contained in the block according to their movement between frames into two non-rectangular regions (e.g. region 0 and region 1) such that the pixels (e.g. brightness samples ) in one of the regions (for example, in region 0) had a similar motion, and pixels in another region (for example, in region 1) had a similar motion. At 540C, an appropriate transform for one or more of region 0 and region 1 may be determined. For example, it may be determined whether region 0 or region 1 has a low prediction error. In response to determining that region 0 has a low prediction error, region 0 may be encoded using SADCT. In response to determining that region 1 has a low prediction error, region 1 may be encoded using SADCT. In some implementations, an area may be considered to have a low prediction error when the prediction error is below a predetermined threshold.

At 550C, the determined exponential split and transform selection may be signaled in the bitstream. Bitstream signaling may include, for example, including an index in one or more predetermined patterns and/or coefficients. Bitstream signaling may include, for example, SADCT signaling as an additional transform choice for a full block DCT for a region (eg, region 0 or region 1) that has a low prediction error.

FIG. 6 is a system block diagram illustrating an exemplary decoder 600 capable of decoding bitstream 670 using exponential splitting and/or inverse SADCT. The decoder 600 includes an entropy decoder processor 610, an inverse quantization and inverse transform processor 620, a deblocking filter 630, a frame buffer 640, a motion compensation processor 650, and an intra prediction processor 660. In some implementations, bitstream 670 includes parameters that signal the exponential split mode. In some implementations, bitstream 670 includes parameters that signal the type of inverse transform applied (eg, inverse block DCT or inverse SADCT). Motion compensation processor 650 may recover pixel information using exponential splitting and/or inverse SADCT, as described herein.

In operation, bitstream 670 may be received by decoder 600 and provided to entropy decoder processor 610, which entropy decodes the bitstream into quantized coefficients. The quantized coefficients may be provided to an inverse quantization and inverse transform processor 620, which may perform inverse quantization and inverse transform using an inverse SADCT and according to signals in the bitstream. The inverse transform makes it possible to create a residual signal that can be added to the output signal of the motion compensation processor 650 or the intra prediction processor 660 according to the processing mode. The output of motion compensation processor 650 and intra prediction processor 660 may include a block prediction based on a previously decoded block. The sum of the prediction and the residual may be processed by the deblocking filter 630 and stored in the frame buffer 640 . For a given block (eg, CU or PU), when bitstream 670 signals that the split mode is exponential split, motion compensation processor 650 may construct a prediction based on the exponential split scheme described herein, including, for the current block, extracting from index bitstream into a given list of spatially and temporally adjacent blocks, and using exponential splitting parameters for the specified block to restore the current block.

7 is a flowchart illustrating an exemplary process 700 for decoding a bitstream using exponential splitting, which in some implementations may use an inverse SADCT. At 710, a block (eg, CTU, CU, PU) is received. The reception may include extracting and/or parsing the block and its associated signaling information from the bitstream. At block 720, it may be determined if the exponential split mode (eg, true) is enabled for the block. If the exponential split mode is not enabled (eg, false), the decoder may process the block using an alternative split mode, such as geometric split. If the exponential split mode is enabled (eg, true), at block 730, the decoder may extract and/or determine one or more parameters that characterize the exponential split and transform. For example, these parameters may include exponential coefficient indices, exponential coefficient values, orientation pattern indices, and/or curve start and end indices (eg, P1P2). Extraction parameters may include identifying and extracting parameters from a bitstream (eg, bitstream parsing). Said parameters may include transform parameters, for example, which may indicate whether the block is to be processed with an inverse SADCT. In addition, determining one or more parameters that characterize the exponential split may include determining that an exponential split merge has been signaled and determining a neighboring block from which the current block inherits the exponential split parameters using an index contained in the bitmap. stream. At 740, the block may be processed according to an exponential split (eg, to generate a prediction), including determining associated motion information for each region. In some implementations, at block 740, the block may be further processed using an inverse SADCT.

While several variations have been detailed above, other modifications or additions are possible. For example, in some implementations, exponential splitting may be applied to symmetrical blocks (8x8, 16x16, 32x32, 64x64, 128x128, etc.) as well as various asymmetric blocks (8x4, 16×8, etc.).

In some implementations, spatial and temporal exponential parameter prediction may be performed for a luma block size of 16x16 or greater, such as 64x64 and/or 128x128. In some implementations, a minimum block size of 16x16 may be set.

The split can be signaled in the bitstream based on rate-distortion decisions at the encoder. The encoding may be based on a combination of regular predetermined partitions (eg, patterns), space-time prediction of the partition, and additional offsets. Each exponentially split region can use motion-compensated prediction or intra-prediction. The boundary of the predicted areas can be smoothed before adding the remainder. For residual coding, the encoder may choose between a conventional rectangular DCT for the entire block and a shape-adaptive DCT for each region.

In some implementations, a quadtree plus binary decision tree (QTBT) may be implemented. In QTBT, at the unit level of the coding tree, the QTBT partitioning parameters are dynamically derived to adapt to local characteristics without passing any overhead. Subsequently, at the coding unit (CU) level, the decision tree of the joint classifier can eliminate unnecessary iterations and control the risk of false prediction. In some implementations, exponential splitting may be available as an additional splitting tool available on every QTBT leaf node. In some implementations, exponential splitting is available as an additional coding tool at the QTBT splitting CU level. For example, FIG. 8 shows an example of splitting a QTBT frame, and FIG. 9 shows an example of exponential splitting at the QTBT CU level shown in FIG. 8. FIG.

In some implementations, the decoder includes an exponential split processor that generates an exponential split for the current block and provides all split-related information to dependent processes. The exponential split processor can directly affect motion compensation, since it can be performed in segments in the case of an exponential block split. In addition, the partition processor may provide shape information to the intra-prediction processor and the transform coding processor.

In some implementations, additional syntax elements may be signaled at different levels of the bitstream hierarchy. To enable exponential splitting for the entire sequence, an enable flag may be encoded in a sequence parameter set (SPS). In addition, the CTU flag may be encoded at the coding tree unit (CTU) level to indicate whether any coding units (CU) use exponential splitting. The CU flag may be encoded to indicate whether the current coding unit uses exponential splitting. Parameters that precisely define the curved line in the block can be encoded. For each region, a flag may be decoded that specifies whether the current region is inter- or intra-predictable.

In some implementations, a minimum region size may be specified.

Referring now to FIG. 11, which is a graph illustrating another exemplary block 1100 split according to exponential splitting. Since exponential splitting is likely to be used for blocks (e.g., coding units) containing features with curved feature boundaries, it is likely that splitting will result in one region having a low prediction error and another region having a high error. forecasting. For example, as shown in FIG. 11, the block is curved according to exponential splitting. Assuming that the luma samples in the block represent the ball and the background, the two regions (S0 and S1) will include the luma samples corresponding to the background and the ball, respectively. As a result, the S0 region will have a high prediction error because the S0 region belongs to the background, while the S1 region will have a low prediction error because the S1 region belongs to the ball. Accordingly, some aspects of the current subject matter may include performing SADCT on a region having a low prediction error. By performing SADCT on an area having a low prediction error instead of a full block DCT, compression efficiency can be improved.

In some implementations, during decoding, the parameters for performing the inverse SADCT may be derived from the exponential split parameters. For example, the transform size can be determined from the exponential split pattern index.

In some implementations, SADCT may be implemented for block sizes of 64x64 and/or 128x128. In some implementations, SADCT may be signaled as an additional transform choice for DCT of full blocks for a segment with low prediction error.

Let us now turn to Fig.12, which shows a graph illustrating the inheritance of exponential split parameters by the current block 1205 from the spatially adjacent block 1210. The curve indicates the border 1215 of the object in the image. The current block 1205 and the spatially adjacent block 1210 indicate blocks of coding units or prediction units in a quadtree plus binary tree (QTBT). As shown in the drawing, the border 1215 of the object usually has a relatively uniform curvature. Both adjacent block 1210 and current block 1205 will be split using exponential splitting. Using some implementations of the current subject matter, instead of sending all exponential splitting parameters (e.g., indexes to form and/or orientation patterns, coefficients, start indices, end indices, and/or the like), exponential split merging can be signaled in the bitstream together with a neighboring block index 1210. During decoding of the current block, the current block may inherit some or all of the exponential split parameters from the specified neighboring block. FIG. 13 shows examples of adjacent blocks in space for the current block. Spatially adjacent blocks may include blocks (eg, coding units or prediction units) that are in the same location as A0 (bottom left), A1 (left), B0 (top right), B1 (top), and B2 (top left) (for example, overlap with them).

In addition, the neighboring block from which the current block inherits exponential splitting parameters may be adjacent in time. 14 shows an exemplary current block 1405 with a temporally adjacent block 1410 from which the current block 1405 inherits exponential split parameters. As shown in FIG. 14, reference picture 1415 includes an adjacent block 1410 that has a corresponding motion vector 1420 indicative of movement of adjacent block 1410 from reference picture 1415 to current picture 1425, which contains current block 1405. Using some implementations of the current subject matter , instead of sending all of the exponential split parameters (eg, indexes to generate and/or orientation patterns, coefficients, and/or the like), the exponential split merge may be signaled in the bitstream along with the index to an adjacent block 1410. During decoding of the current block 1405, the current block 1405 may inherit some or all of the exponential split parameters from the specified adjacent block 1410.

In some implementations, the parent block need not be a contiguous block, for example, it may be another block in the current frame that has been previously decoded.

In some implementations, the current block may inherit from another block only some exponential split parameters. For example, the current box may inherit from the first parent box the first parameter (eg, form template), and additional parameters (eg, start point, end point, orientation template, etc.) may be included in the bitstream.

The subject matter described herein provides many technical advantages. For example, some implementations of the current subject matter may provide for block splitting, which increases compression efficiency. In some implementations, effective visual effects can be achieved by implementing partitioning in such a way as to more closely follow the boundaries of the object. Similarly, in some implementations, blocking artifacts at the boundaries of an object can be reduced by implementing partitioning in a way that more closely follows the boundaries of the object. In some implementations, compression efficiency can be improved and complexity can be reduced by implementing SADCT as an additional transform choice for full block DCT and applying SADCT to exponentially split regions that have low prediction error.

One or more aspects or features of the subject matter described herein may be implemented as digital electronic circuits, integrated circuits, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), computer hardware, firmware, software and/or combinations thereof. These various aspects or features may include implementation in one or more computer programs that are executable and/or interpretable on a programmable system, including at least one programmable processor, which may be special purpose or general purpose, connected to receive data. and instructions from the data storage and transmission system and instructions to the storage system, at least one input device and at least one output device. The programmable system or computing system may include clients and servers. The client and server are usually remote from each other and usually communicate through a communication network. The relationship between the client and the server is due to the computer programs running on the respective computers and having a client-server relationship with each other.

These computer programs, which may also be referred to as programs, software, software applications, applications, components, or code, include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language , logic programming language and/or assembly language/machine language. As used herein, the term "computer-readable storage medium" refers to any computer software product, hardware device, and/or device, such as, for example, magnetic disks, optical disks, memory, and programmable logic devices (PLDs), used to provide machine instructions and /or data to a programmable processor, including a computer-readable storage medium that receives machine instructions in the form of a computer-readable signal. The term "machine readable signal" refers to any signal used to supply machine instructions and/or data to a programmable processor. The computer-readable storage medium may store such machine instructions in a non-temporal manner, such as in non-transitory solid-state memory, on a magnetic hard disk, or on any equivalent storage medium. The computer-readable storage medium may alternatively or additionally store such machine instructions in a temporary manner, such as in a processor cache or other random access memory cooperating with one or more physical processor cores.

To provide user interaction, one or more aspects or features of the subject matter described herein may be implemented on a computer having a display device such as, for example, a cathode ray tube (CRT) or a liquid crystal display (LCD). LCD) or light-emitting diode (LED, LED) monitor for displaying information to the user, and a keyboard and pointing device, such as a mouse or trackball, with which the user can enter data into the computer. Other types of devices may also be used to interact with the user. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touch screens or other touch devices such as single or multi-point resistive or capacitive trackpads, voice recognition hardware and software, optical scanners, optical pointers, digital image capture devices, and related interpretation software. etc.

In the above descriptions and in the claims, phrases such as "at least one of" or "one or more of" followed by a conjunctive list of elements or features may occur. The term "and/or" may also occur in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradicted by the context in which it is used, such phrase is intended to mean any of the listed elements or features alone or any of the listed elements or features in combination with any other listed elements or features. For example, each phrase is "at least one of A and B"; "one or more of A and B" and "A and/or B" means "A alone, B alone, or A and B together". A similar interpretation is also intended for lists of three or more items. For example, each phrase "at least one of A, B, and C;" "one or more of A, B and C" and "A, B and/or C" is intended to mean "A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together." In addition, the term "based" as used above and in the claims is intended to mean "based at least in part", so an unspecified feature or element is also acceptable.

The subject matter described herein may be embodied in systems, devices, methods, and/or articles, depending on the desired configuration. The implementations set forth in the previous description do not represent all implementations consistent with the subject matter described herein. Instead, they are only some examples consistent with aspects related to the described subject matter. Although some options are detailed above, other modifications or additions are possible. In particular, in addition to what is set forth herein, additional features and/or variations may be provided. For example, the implementations described above may target various combinations and subcombinations of the disclosed features and/or combinations and subcombinations of several additional features disclosed above. In addition, the logical sequences of operations depicted in the accompanying drawings and/or described herein do not necessarily require the specific order shown or sequential order to achieve the desired results. Other implementations may be within the scope of the following claims.

Claims (21)

1. A decoder containing a circuit configured to: receiving a bitstream including an encoded picture, the encoded picture including a coding tree unit containing a plurality of coding units, and signaling information including a first index for determining a start point and a second index for determining an end point of a non-rectangular non-rectangular boundary in the unit coding tree; determining, using the first index and the second index, a start point and an end point of a non-rectangular non-rectangular boundary in the coding tree unit; generating first predicted pixel values in a first region on a first side of the non-rectangular non-rectangular boundary; generating second predicted pixel values in a second region on a second side of the non-rectangular non-rectangular boundary; smoothing the first predicted pixel values and the second predicted pixel values on the non-rectangular non-rectangular boundary; and decoding the coding tree unit by adding residual pixel values to the smoothed first and second pixel values. 2. The decoder of claim 1, wherein the non-rectangular non-rectangular boundary is a curve. 3. The decoder according to claim. 1, in which the image is a curved object, and non-rectangular non-rectangular border represents the border of the object. 4. The decoder of claim 1, wherein the coding tree unit has a size of N×N, each coding unit has a size of N/2×N/2, where N is one of 128, 64, or 32. 5. The decoder of claim 1, wherein the start point is on a first side of the coding tree unit at a first offset distance from a first corner of the coding tree unit. 6. The decoder of claim 5, wherein the endpoint is located on a second side of the coding tree unit at a second offset distance from a second corner of the coding tree unit. 7. A decoder containing a circuit configured to: receiving a bitstream including an encoded image, wherein the encoded image includes a block of encoded pixels and signaling information including a first index used to determine the start point of the indirect non-rectangular partition and a second index used to determine the end point of the indirect non-rectangular partition in the block; determining, using the first index, a start point of an indirect non-rectangular split in the block, the start point being located on a first side of the block at a first offset distance from the first corner of the block; determining, using the second index, an end point of an indirect non-rectangular split in the block, the end point being located on the second side of the block at a second offset distance from the second corner of the block; generating first predicted pixel values in a first region on a first side of the indirect non-rectangular split; generating second predicted pixel values in a second region on a second side of the indirect non-rectangular split; smoothing the first predicted pixel values and the second predicted pixel values on the non-rectangular non-rectangular mesh; and adding residual pixel pixel values to the smoothed first and second predicted pixel values to decode the block. 8. The decoder of claim 7, wherein the encoded image includes a block of intra-predicted pixels, wherein the decoder includes an intra-prediction processor for decoding the block of intra-predicted pixels.

Images