RU2815738C2 - Determination of chromaticity coding mode using matrix-based intra-frame prediction - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to video encoding techniques. Result is achieved by the fact that the video processing method includes steps of performing a first determination that the video luminance video block is encoded using a matrix-based intra-frame prediction (MIP) mode, in which the prediction unit of the luminance video block is determined by performing, on the previously encoded video samples, a downsampling operation at the boundary, which is followed by a matrix-vector multiplication operation and which is selectively followed by an upsampling operation, is performed based on the first determination, a second determination with respect to the intra-frame chroma mode to be used for the chroma video block associated with the luma video block; and performing, based on the second determination, conversion between the chroma video block and the bitstream representation of the chroma video block.

EFFECT: high efficiency of video encoding.

17 cl, 41 dwg, 8 tbl

Description

Field of technology to which the invention relates

This patent document relates to video encoding technologies, devices and systems.

State of the art

Despite advances in video compression, digital video still accounts for the largest amount of bandwidth on the Internet and other digital communications networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video use is expected to continue to increase.

Disclosure of the invention

This document describes devices, systems and methods that relate to digital video encoding, and in particular to matrix-based intra-frame prediction methods for video encoding. The described methods can be applied to both existing video coding standards (eg, High Efficiency Video Coding (HEVC)) and future video coding standards (eg, Versatile Video Coding (VVC)) or codecs.

A first exemplary video processing method includes generating, for conversion between a current video block of video and an encoded representation of the current video block, a first most probable mode list (MPL) using a first rule-based procedure; and performing a conversion between the current video block and a coded representation of the current video block using the first MPM list, wherein the conversion of the current video block uses a matrix-based intra-prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on the previously encoded video samples, a down operation boundary sampling followed by a matrix-vector multiplication operation and selectively followed by an upsampling operation, wherein the rule specifies that the first procedure used to generate the first MPM list is the same as the second procedure used to generate the second MPM list to transform the others video video blocks that are encoded using a non-MIP intra-frame mode that is different from the MIP mode, and at least a portion of the first MPM list is generated based on at least a portion of the second MPM list.

A second exemplary video processing method includes generating, for conversion between a current video block of video and an encoded representation of the current video block, a rule-based most probable mode (MPM) list, wherein the rule is based on whether an adjacent video block of the current video block is encoded with using a matrix-based intra-prediction (MIP) mode in which a prediction block of an adjacent video block is determined by performing, on previously encoded video samples, an edge downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; and performing a conversion between the current video block and an encoded representation of the current video block using the MPM list, wherein the conversion applies a non-MIP mode to the current video block, wherein the non-MIP mode is different from the MIP mode.

A third exemplary video processing method includes decoding a current video block of video that is encoded in an encoded representation of the current video block using a matrix-based intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by performing on previously encoded samples video,edge downsampling operation followed by a matrix-vector,multiplication operation and selectively followed by an,upsampling operation; and updating a line buffer associated with the decoding without storing in the line buffer information indicating whether the current video block is encoded using the MIP mode.

A fourth exemplary video processing method includes performing a conversion between a current video block and a bitstream representation of the current video block, wherein the current video block is encoded using a matrix-based intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by executing, over previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, with a flag encoded in the bitstream representation using at most K contexts in the arithmetic encoding or process decoding, where the flag indicates whether the current video block is encoded using the MIP mode, with K greater than or equal to zero.

A fifth exemplary video processing method includes generating, for conversion between a current video block of video and a bitstream representation of the current video block, an intra-frame prediction mode for the current video block encoded using a matrix-based intra-frame prediction (MIP) mode in which the block the prediction of the current video block is determined by performing, on the previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; defining a rule for storing information indicating an intra-frame prediction mode based on whether the current video block is encoded using the MIP mode; and performing conversion in accordance with the rule, where the rule determines that the syntax element for the intra-frame prediction mode is stored in the bitstream representation for the current video block, and the rule determines that the mode index for the MIP mode for the current video block is not stored in the bitstream representation.

A sixth exemplary video processing method includes first determining that a video luminance video block is encoded using a matrix-based intra frame prediction (MIP) mode, in which the luminance video block prediction block is determined by performing, on previously encoded video samples, an operation downsampling at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; making, based on the first determination, a second determination regarding an intra-frame chrominance mode to be used for the chrominance video block associated with the luma video block; and performing, based on the second definition, a conversion between the chrominance video block and the bitstream representation of the chrominance video block.

A seventh exemplary video processing method includes performing a transformation between a current video block of video and an encoded representation of the current video block, the transformation being based on determining whether the current video block should be encoded using a matrix-based intra-frame prediction (MIP) mode in which the block The prediction of the current video block is determined by performing, on the previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation.

An eighth exemplary video processing method includes determining whether to encode a current video block of video into a bitstream representation of the current video block, in accordance with a rule, whether to use a matrix-based intra-frame prediction (MIP) mode and an encoding mode that is different from MIP mode for encoding the current video block, wherein the MIP mode includes determining a prediction block of the current video block by performing, on previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation ; and adding, based on the specified definition, an encoded representation of the current video block to the bitstream representation.

A ninth exemplary video processing method includes determining that the current video block of video is encoded into a bitstream representation using a matrix-based intra-frame prediction (MIP) mode and an encoding mode that is different from the MIP mode, wherein the MIP mode includes determining a prediction block of the current video block by performing, on the previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; and generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

A tenth exemplary video processing method includes determining the applicability of a contour filter to a reconstructed block of a current video block in a transformation between video and an encoded representation of the current video block, wherein the current video block is encoded using a matrix-based intra-frame prediction (MIP) mode in which a current video block prediction block is determined by performing, on the previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; and processing the current video block according to the specified definition.

An eleventh exemplary video processing method includes determining, for encoding a current video block of video into a bitstream representation of the current video block, the type of neighboring samples of the current video block to be used in accordance with a rule; and adding, based on said definition, an encoded representation of the current video block to a bitstream representation, wherein the current video block is encoded using a matrix-based intra-frame prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on the previously encoded video samples, an operation downsampling at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation.

A twelfth exemplary video processing method includes determining that a current video block of video is encoded in a bitstream representation using a matrix-based intra frame prediction (MIP) mode and using a neighbor sample type of the current video block in accordance with a rule, wherein the mode The MIP includes determining a prediction block of the current video block by performing, on previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; and generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

A thirteenth exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block, the conversion including generating, using a matrix-based intra-frame prediction (MIP) mode, a prediction block for the current video block by selecting and applying matrix multiplication using sample matrices and/or by selecting and adding offset using offset vectors for the current video block, wherein the samples are obtained by row and column averaging of previously encoded video samples, and the selection is based on the reframing information associated applying luminance chroma scaling (LMCS) transform to the reference image of the current video block.

A fourteenth exemplary video processing method includes determining that a current video block is to be encoded using a matrix-based intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by performing a downsampling operation on previously encoded video samples. at the boundary, followed by a matrix-vector multiplication operation and selectively followed by an upsampling operation; and performing, based on said definition, a conversion between the current video block and a bitstream representation of the current video block, the conversion being performed based on a rule of joint application of the MIP mode and the other encoding method.

A fifteenth exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block using a matrix-based intra-frame prediction (MIP) mode, wherein performing the conversion using the MIP mode includes generating a prediction block by application of matrix multiplication using matrices of samples obtained by averaging over rows and columns of previously encoded video samples, and the matrices depend on the bit depth of the sample.

A sixteenth exemplary video processing method includes generating an intermediate prediction for a current video block of video using a matrix-based intra-prediction (MIP) mode in which the prediction block of the current video block is determined by performing a downsampling operation on previously encoded video samples. at the boundary, followed by a matrix-vector multiplication operation and selectively followed by an upsampling operation; generating, based on the intermediate prediction signal, a final prediction signal; and performing, based on the final prediction signal, a conversion between the current video block and a bitstream representation of the current video block.

A seventeenth exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block using a matrix-based intra frame prediction (MIP) mode, wherein performing the conversion includes using an interpolation filter in an upsampling process to MIP mode, wherein in MIP mode matrix multiplication is applied to the first set of samples obtained as a result of averaging over rows and columns of previously encoded video samples, and an interpolation filter is applied to the second set of samples obtained as a result of matrix multiplication, and the interpolation filter excludes the bilinear filter interpolation.

An eighteenth exemplary video processing method includes performing a conversion between a current video block of a video and a bitstream representation of the current video block in accordance with a rule, where the rule determines the relationship between the applicability of a matrix-based intra-frame prediction (MIP) mode or a transform mode during conversion wherein the MIP mode includes determining a prediction block of the current video block by performing, on the previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, and the transform mode determines the use of the operation transforms to determine the prediction block for the current video block.

A nineteenth exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block using a matrix-based intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by performing on previously encoded samples video, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, wherein performing the conversion involves obtaining, in accordance with the rule, the boundary samples by applying a bit left shift or a bit right shift operation to the sum of at least one reference boundary sample, wherein the rule determines whether to apply a left shift operation or a right shift operation.

A twentieth exemplary video processing method includes performing a conversion between a current video block of video and a bitstream representation of the current video block using a matrix-based intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by performing on previously encoded samples video, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation, the prediction samples predSamples[ xHor + dX ][ yHor ] being determined according to the following formula in the upsampling operation: predSamples[ xHor + dX][yHor] = ((upHor - dX) * predSamples[xHor][yHor] + dX * predSamples[xHor + upHor][yHor] + offsetHor)/upHor, and predSamples[xVer][yVer + dY] = ((upVer - dY) * predSamples[xVer][yVer] + dY * predSamples[xVer][yVer + upVer] + offsetVer)/upVer, where offsetHor and offsetVer are integers, where upHor is the dependence of the width of the current video block on the given one a value based on the size of the current video block, where upVer is the dependence of the height of the current video block on a specified value based on the size of the current video block; where dX is equal to 1... upHor-1, where dY is equal to 1... upVer-1, and where xHor are positions based on upHor and yHor are positions based on upVer.

The twenty-first exemplary video processing method includes generating, for conversion between a current video block of video and a bitstream representation of the current video block, an intra-frame prediction mode for the current video block encoded using a matrix-based intra-frame prediction (MIP) mode in which a current video block prediction block is determined by performing, on the previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation; defining a rule for storing information indicating an intra-frame prediction mode based on whether the current video block is encoded using the MIP mode; and performing conversion in accordance with said rule, wherein the rule specifies that the bitstream representation excludes storing information indicating a MIP mode associated with the current video block.

In yet another characteristic aspect, the disclosed technology can be used to provide a method for processing video. This exemplary method includes determining that the current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode, generating, based on the determination, at least a portion of a most probable mode (MPM) list for the ALWIP mode based on at least a portion of the MPM list for the intra-frame mode without using ALWIP and performing, based on the MPM list for the ALWIP mode, a conversion between the current video block and a bitstream representation of the current video block.

In yet another characteristic aspect, the disclosed technology can be used to provide a method for processing video. This exemplary method includes the steps of determining that a luminance component of a current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode, inferring, based on the determination, an intra-frame chrominance mode, and performing, based on the intra-frame chrominance mode, chroma mode, conversion between the current video block and the bitstream representation of the current video block.

In yet another characteristic aspect, the disclosed technology can be used to provide a method for processing video. This exemplary method includes determining that a current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode and, based on the determination, performing a conversion between the current video block and a bitstream representation of the current video block.

In yet another characteristic aspect, the disclosed technology can be used to provide a method for processing video. This exemplary method includes determining that the current video block is encoded using an encoding mode other than an affine linear weighted intra-prediction (ALWIP) mode and, based on the determination, performing a conversion between the current video block and the bitstream representation current video block.

In yet another characteristic aspect, the above-described method is embodied in processor-executable code and stored on a computer-readable program medium.

In yet another characteristic aspect, a device is disclosed that is configured or operative to perform the method described above. The device may include a processor that is programmed to implement the method.

In yet another characteristic aspect, a video decoder apparatus may implement the method described herein.

The above and other aspects and features of the disclosed technology are described in more detail in the drawings, description and claims.

Brief description of drawings

Fig. 1 is an example of 33 intra-frame prediction directions.

Fig. 2 is an example of 67 intra-frame prediction modes.

Fig. 3 is an example of sample locations used to obtain linear model weights.

Fig. 4 is an example of four reference lines adjacent to the prediction block.

Fig. 5A and FIG. 5B – examples of subsections depending on block size.

Fig. 6 – ALWIP example for 4x4 blocks.

Fig. 7 – ALWIP example for 8x8 blocks.

Fig. 8 – ALWIP example for 8x4 blocks.

Fig. 9 – ALWIP example for 16×16 blocks.

Fig. 10 is an example of adjacent blocks used in compiling the MPM list.

Fig. 11 is a flowchart of an exemplary matrix intra-frame prediction method in accordance with the disclosed technology.

Fig. 12 is a flow diagram of another exemplary matrix intra-frame prediction method in accordance with the disclosed technology.

Fig. 13 is a block diagram of another exemplary matrix intra-frame prediction method in accordance with the disclosed technology.

Fig. 14 is a block diagram of another exemplary matrix intra-frame prediction method in accordance with the disclosed technology.

Fig. 15 is a block diagram of an example hardware platform for implementing the visual media encoding or visual media decoding methods described herein.

Fig. 16 is a block diagram showing an exemplary video processing system in which various methods disclosed herein may be implemented.

Fig. 17 is a block diagram that illustrates an example video coding system in which the methods of the present disclosure can be used.

Fig. 18 is a block diagram illustrating an example of a video encoder.

Fig. 19 is a block diagram illustrating an example of a video decoder.

Fig. 20-38 is an example flowchart of additional example methods for matrix intra-frame prediction in accordance with the disclosed technology.

Carrying out the invention

With the growing demand for higher resolution video, video encoding methods and technologies are becoming ubiquitous in modern technology. Typica LLY video codecs include electronic circuitry or software that compresses or decompresses digital video, and they are constantly being improved to improve encoding efficiency. A video codec converts uncompressed video to a compressed format or vice versa. There are complex relationships between video quality, the amount of data used to represent the video (which is determined by the bit rate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, and end-to-end latency. The compressed format typically follows a standard video compression specification, such as the High Efficiency Video Coding (HEVC) standard (also known as H.265 or MPEG-H Part 2), the to-be-developed Versatile Video Coding (VVC) standard, or other modern and /or future video encoding standards.

Embodiments of the disclosed technology can be applied to existing video coding standards (eg, HEVC, H.265) and future standards to improve performance. Section headings are used herein to enhance the readability of the description and are in no way intended to limit the discussion or embodiments (and/or implementations) to the relevant sections.

1. Brief overview of HEVC

1.1. Intra-frame prediction in HEVC/H.265

Intra-frame prediction involves generating samples for a given TB (transform block) using samples previously reconstructed in the color channel in question. The intra-prediction mode is signaled separately for the luma and chrominance channels, with the intra-prediction mode of the chrominance channel optionally depending on the intra-prediction mode of the luminance channel when using the DM_CHROMA mode. Although the intra-prediction mode is signaled at the PB (prediction block) level, the intra-prediction process is applied at the TB level according to the residual quadtree hierarchy for the CU, thereby allowing the encoding of one TB to have an impact on the encoding of the next TB in the CU and hence reducing the distance to readings used as reference values.

HEVC includes 35 intra-prediction modes - DC mode, planar mode and 33 directional or "angular" intra-prediction modes. The 33 angular intra prediction modes are shown in FIG. 1.

For PBs associated with chrominance color channels, the intra-frame prediction mode is defined as planar, DC, horizontal, vertical, "DM_CHROMA" mode, or sometimes diagonal "34" mode.

It should be noted that for 4:2:2 and 4:2:0 chroma formats, the chrominance PB may overlap two or four (respectively) luminance PBs; in this case, the luminance direction for DM_CHROMA is taken from the upper left corner of these luminance PBs.

The DM_CHROMA mode indicates that the luma PB color channel intra-frame prediction mode is applied to the chrominance PB channels. Since this is relatively common, the coding scheme for the most likely mode intra_chroma_pred_mode is biased in favor of selecting this mode.

2. Examples of intra-frame prediction in VVC

2.1. Intra-frame coding mode with 67 intra-frame prediction modes

To capture arbitrary edge directions present in natural video, the number of intra-frame directional modes is increased from 33, as used in HEVC, to 65. Additional directional modes are indicated by red dotted arrows in FIG. 2, both planar mode and DC mode remain the same. These denser directional intra-prediction modes apply to all block sizes and to both luma and chrominance intra-predictions.

2.2. Examples of cross-component linear model (CCLM) prediction

In some embodiments, and to reduce inter-component redundancy, JEM uses a inter-component linear model (CCLM) mode (also called LM), for which chrominance samples are predicted from reconstructed luminance samples of the same CU using a linear model as follows:

Where represents the predicted chroma samples in the CU and represents the reconstructed downsampled luminance samples of the same CU. The parameters α and β of the linear model are obtained from the relationship between the luminance values and the chrominance values from two samples, which are the luminance sample with the minimum sample value and the maximum sample within a set of adjacent downsampled luminance samples and their corresponding chroma samples. In fig. Figure 3 shows an example of the location of the left and top samples and the current block sample involved in CCLM mode.

The calculation of this parameter is performed as part of the decoding process, and not just as an encoder search operation. As a result, no syntax is used to pass the values of α and β to the decoder.

A total of 8 intra-frame modes are allowed for intra-chrominance encoding. These modes include five traditional intraframe modes and three intercomponent linear model modes (CCLM, LM_A, and LM_L). Chroma coding directly depends on the intra-frame prediction mode of the corresponding luminance block. Because I-slices allow a separate block structure for luma and chrominance components, one chrominance block can correspond to multiple luma blocks. Thus, the Chroma DM mode directly inherits the intra-frame prediction mode of the corresponding luma block, covering the center position of the current chrominance block.

2.3. Intra-frame prediction with multiple reference lines (MRL)

For multi-reference line (MRL) intra prediction, a large number of reference lines are used for intra-prediction. In fig. Figure 4 shows an example of 4 reference lines, where the samples of segments A and F are not extracted from the reconstructed neighboring samples, and are complemented by the nearest samples from segments B and E, respectively. For intra-frame prediction of an HEVC image, the nearest reference line (ie, reference line 0) is used. MRL uses 2 additional lines (reference line 1 and reference line 3). The selected reference line index (mrl_idx) is signaled and used to generate an intra-frame predictor. For a reference line, an idx index that is greater than 0 includes only the additional reference line modes in the MPM list and only the mpm index of the signal without the remaining mode.

2.4. Intra-frame subpartitions (ISP)

The Intra Subpartition (ISP) tool divides intraframe luminance prediction blocks vertically or horizontally into 2 or 4 subpartitions depending on the block size. For example, the minimum block size for an ISP is 4x8 (or 8x4). If the block size is greater than 4x8 (or 8x4), then the corresponding block is divided into 4 parts. In fig. Figure 5 shows examples of two possible options. All subsections satisfy the condition of having at least 16 samples.

For each subsection, the reconstructed samples are obtained by adding the residual signal to the prediction signal. In this paper, the residual signal is generated by processes such as entropy decoding, inverse quantization, and inverse transform. Thus, the reconstructed sample values of each subsection are available to generate a prediction for the next subsection, and each subsection is reprocessed. Additionally, the first subpartition processed will be the one that contains the top-left CU count and then continues down (horizontal partition) or to the right (vertical partition). As a result, the reference samples used to generate subsection prediction signals are located only to the left and above the lines. All subsections use the same intraframe mode.

2.5. Affine Linear Weighted Intra Prediction (ALWIP or Matrix Intra Prediction)

Affine Linear Weighted Intra Prediction (ALWIP, also known as Matrix Intra Prediction (MIP)) is proposed in JVET-N0217.

There are two tests performed in JVET-N0217. In Test 1, ALWIP is developed with a memory limit of 8 kilobytes and a maximum of 4 multiplications per sample. Test 2 is similar to Test 1, but further simplifies development in terms of memory requirements and model architecture.

○ A single set of matrices and displacement vectors for all block shapes.

○ Reducing the number of modes to 19 for all block shapes.

○ Reduced memory requirements to 5760 10-bit values, or 7.20 kilobytes.

○ Linear interpolation of the predicted samples is performed in one step in each direction, replacing the iterative interpolation as in the first test.

2.5.1. Test 1 JVET-N0217

To predict samples of a rectangular block with width W and height H, affine linear-weighted intra-prediction (ALWIP) takes as input one line H of reconstructed neighboring boundary samples to the left of the block and one line W of reconstructed neighboring boundary samples above the block. If reconstructed samples are not available, they are generated in the same way as in normal intra-frame prediction.

Generation of the prediction signal is based on the following three steps:

Four samples if W=H=4, and eight samples in all other cases, are extracted from the boundary samples by averaging.

Matrix-vector multiplication followed by offset addition is performed using the averaged samples as input. The result is a reduced prediction signal for the subsampled set of samples in the original block.

The prediction signal at the remaining positions is generated from the prediction signal in the subsampled set using linear interpolation, which is a one-step linear interpolation in each direction.

The matrix and displacement vectors required to generate the prediction signal are taken from three sets of matrices S ₀ , S ₁ , S ₂ . The set S ₀ consists of 18 matrices , each of which has 16 rows and 4 columns and 18 vectors offsets, each of size 16. The matrices and offset vectors from this set are used for 4x4 blocks. Set S ₁ consists of 10 matrices , each of which has 16 rows and 8 columns and 10 vectors offsets, each of size 16. The offset matrices and vectors of this set are used for blocks of size 4x8, 8x4, and 8x8. Finally, the set S ₂ consists of 6 matrices , each of which has 64 rows and 8 columns and 6 vectors offsets of size 64. The matrices and offset vectors from this set, or portions of these matrices and offset vectors, are used for all other block shapes.

The total number of multiplications required to compute a matrix-vector multiplication is always less than or equal to 4 × W × H. In other words, ALWIP modes require no more than four multiplications per sample.

2.5.2 Boundary averaging

At the first stage, the input boundaries And decrease to smaller limits And . Here are both borders And consist of 2 samples in the case of a 4x4 block, and in all other cases both boundaries consist of 4 samples.

In the case of a block of size 4×4 for 0≤ i < 2, it is determined

and is similarly defined .

Otherwise, if the width W of the block is given as W=4·2 ^k , for 0 ≤ I < 4, it is determined

and is similarly defined .

Two reduced borders And associated with the reduced boundary vector , which is therefore size four for blocks of the 4x4 shape and size eight for blocks of all other shapes. If mode is an ALWIP mode, this union is defined as follows:

Finally, a second version of the averaged boundary is required to interpolate the subsampled prediction signal over large blocks. Namely, if min(W,H)>8 and W≥H, write W , and for 0≤i<8 it is determined

If min(W,H)>8 and H>W, is defined in a similar way.

2.5.3. Generating a reduced prediction signal using matrix-vector multiplication

From the reduced input vector bdry _red, a reduced prediction signal pred _red is generated. The last signal is a subsampled block signal with width W _red and height H _red . Here W _red and H _red are defined as:

The reduced pred _red prediction signal is calculated by calculating the matrix-vector multiplication and adding an offset:

Here A is a matrix that has W _red ·H _red rows and 4 columns if W=H=4 and 8 columns in all other cases. b is a vector of size W _red ·H _red .

Matrix A and vector b are selected from one of the sets S ₀ , S ₁ , S ₂ as follows. The index idx=idx(W,H) is defined as follows:

In addition, we assume that m is defined as follows:

Then if or , And , we believe that And . In case And , let's say A is a matrix that is obtained by eliminating each row from , which in the case of W=4 corresponds to the odd x coordinate in the downsampled block, or, in the case of H=4, corresponds to the odd y coordinate in the downsampled block.

Finally, the reduced prediction signal is replaced by its transposition in the following cases:

○ W=H=4 and mode ≥ 18

○ max(W,H)=8 and mode ≥ 10

○ max(W,H) > 8 and mode ≥ 6

The number of multiplication operations required for the calculation is 4 in the case, since in this case there are 4 columns and 16 rows. In all other cases A has 8 columns and lines, and immediately checks what is required in these cases multiplication operations, that is, in these cases to calculate no more than 4 multiplication operations per count are also required.

2.5.4. Illustration of the entire ALWIP process

The entire process of averaging, matrix-vector multiplication, and linear interpolation is illustrated for various shapes in FIG. 6-9. It should be noted that the remaining forms are treated as in one of the cases depicted.

1. Given a 4x4 block, ALWIP has four average values along each boundary axis. The resulting four input samples enter into a matrix-vector multiplication. The matrices are taken from the set S ₀ . After adding the offset, there are 16 final prediction samples. Linear interpolation is not required to generate a prediction signal. Thus, a total of (4·16)/(4·4)=4 multiplications per count are performed.

2. Given an 8x4 block, ALWIP has four average values along each boundary axis. The resulting eight input samples go into matrix-vector multiplication. The matrices are taken from the set S ₁ . This gives 16 samples at odd positions of the prediction block. Thus, a total of (8·16)/(8·8)=2 multiplications per count are performed. After adding offset, these samples are interpolated vertically using the reduced upper bound. Horizontal interpolation is performed using the original left boundary.

3. Given an 8x4 block, ALWIP has four average values along the horizontal boundary axis and four original boundary values on the left boundary. The resulting eight input samples go into matrix-vector multiplication. The matrices are taken from the set S ₁ . This gives 16 samples at the odd horizontal and each vertical position of the prediction block. Thus, a total of (8·16)/(8·4)=4 multiplications per count are performed. After adding an offset, these samples are interpolated horizontally using the original left boundary.

4. Given a 16x16 block, ALWIP has four average values along each boundary axis. The resulting eight input samples go into matrix-vector multiplication. The matrices are taken from the set S ₂ . This gives 64 samples at odd positions of the prediction block. Thus, a total of (8·64)/(16·16)=2 multiplications per count are performed. After adding offset, these samples are interpolated vertically using the eight upper bound averages. Horizontal interpolation follows the use of the original left border. In this case, the interpolation process does not add any multiplications. Thus, only two multiplications per sample are required to calculate the ALWIP forecast.

For large forms the procedure is essentially the same, and it is easy to verify that the number of multiplications per count is less than four.

For W×8 ghb W>8 blocks, only horizontal interpolation is necessary, since the readings are given in odd horizontal and each vertical positions.

Finally, for W×4 blocks with W>8, let A_k be the matrix that is obtained by eliminating each row that corresponds to an odd entry on the horizontal axis of the downsampled block. Thus, the output size is 32, and again only horizontal interpolation remains to be done.

Transposed cases are handled accordingly.

2.5.5. One-step linear interpolation

For a block of size W×H with max(W,H)≥8, the prediction signal is obtained from the reduced prediction signal pred _red by W _red ×H _red using linear interpolation. Depending on the block shape, linear interpolation is performed in vertical, horizontal, or both directions. If linear interpolation is to be applied in both directions, it is applied first in the horizontal direction if W<H, and first applied in the vertical direction otherwise.

Without loss of generality, consider a block of size W×H with max(W,H)≥8 and W≥H. One-dimensional linear interpolation is then performed as follows. Without loss of generality, it is sufficient to describe linear interpolation in the vertical direction. First, the reduced prediction signal is expanded to the top using a boundary signal. Let us determine the coefficient U _ver =H/H _red of vertical upsampling, and write . We then define the extended reduced prediction signal using

A vertical linear interpolated prediction signal is then generated from this extended reduced prediction signal as follows:

for 0≤x<W _red , 0≤y<H _red and 0≤k<U _ver .

2.5.6. Signaling proposed intra-frame prediction modes

For each coding unit (CU) in intra-frame mode, a flag is sent in the bit stream indicating whether or not ALWIP mode should be applied to the corresponding prediction unit (PU). The last index signaling is harmonized with the MRL in the same way as in the JVET-M0043. If ALWIP mode is to be used, the ALWIP mode redmode index is signaled using the 3 MPMS MPM list.

Here, MPM inference is performed using the intra-frame mode of the top and left PUs as follows. There are three fixed tables , , which assign each conventional intra-frame prediction mode an ALWIP mode.

For each PU with width W and height H, an index is defined

,

which specifies from which of three sets the ALWIP parameters should be selected, as in section 2.5.3.

If the above unit prediction is available, belongs to the same CTU as the current PU, and is in intra-frame mode if , and if ALWIP applies to with ALWIP mode , we believe that

If the top PU is reachable, belongs to the same CTU as the current PU and is in intra-frame mode, and if the top PU is in legacy mode intra-frame prediction, we assume that

In all other cases we assume that

,

which means this mode is not available. In the same way, but without the restriction that the left PU must belong to the same CTU as the current PU, the mode .

Finally, three fixed lists are provided , by default, each containing three different ALWIP modes. From the default list and modes And three separate MPMs are created by replacing -1 with default values and eliminating repetition.

The left neighbor block and the top neighbor block, which are used in constructing the ALWIP MPM list, are A1 and B1, as shown in FIG. 10.

2.5.7. Obtaining an adaptive MPM list for traditional intraframe luma and chrominance prediction modes

The proposed ALWIP modes are harmonized with MPM-based coding of traditional intra-frame prediction modes as follows. The processes for obtaining the luma and chrominance MRM list for traditional intra-frame prediction modes use fixed tables , , which provide ALWIP mode mapping with a given PU in one of the traditional intra-frame prediction modes

To obtain a luma MPM list whenever a neighboring luma block that uses ALWIP mode is encountered , this block is processed as if it were using traditional intra-prediction mode . To obtain the MPM chrominance list whenever the current luma block uses the LWIP mode, the same mapping is used to convert the ALWIP mode to the traditional intra-frame prediction mode.

2.5.8. Corresponding modified working draft

In some embodiments, as described in this section, portions related to intra_lwip_flag, intra_lwip_mpm_flag, intra_lwip_mpm_idx, and intra_lwip_mpm_remainder have been added to the working draft based on embodiments of the disclosed technology.

In some embodiments, as described in this section, <begin> and <end> tags are used to indicate additions and modifications to the working design based on embodiments of the disclosed technology.

Syntax tables

Coding Unit Syntax

Semantics

<begin>intra_lwip_flag[x0][y0] equal to 1 indicates that the intra-prediction type for the luminance samples is affine linearly weighted intra-prediction. intra_lwip_flag[x0][y0] equal to 0 indicates that the intra prediction type for the luminance samples is not affine linear weighted intra prediction.

When intra_lwip_flag[x0][y0] is missing, it is assumed to be 0.

The syntax elements intra_lwip_mpm_flag[x0][y0], intra_lwip_mpm_idx[x0][y0], and intra_lwip_mpm_remainder [x0][y0] define the affine linear weighted intra-frame prediction mode for luminance samples. The array indices x0, y0 determine the location (x0, y0) of the top left luminance sample of the encoding block in question relative to the top left luminance sample of the image. When intra_lwip_mpm_flag[x0][y0] is equal to 1, then the affine linear weighted intra-prediction mode is obtained from adjacent intra-prediction coding blocks according to clause 8.4.X.

When intra_lwip_mpm_flag[x0][y0] is missing, it is assumed to be 1. <end>

intra_subpartitions_split_flag[x0][y0] indicates whether the intra-frame subpartition split type is horizontal or vertical. When intra_subpartitions_split_flag[x0][y0] is missing, it is obtained as follows:

- If intra_lwip_flag[x0][y0] is 1, intra_subpartitions_split_flag[x0][y0] is assumed to be 0.

- Otherwise the following applies:

- If cbHeight is greater than MaxTbSizeY, intra_subpartitions_split_flag[x0][y0] is assumed to be 0.

- Otherwise (cbWidth greater than MaxTbSizeY), intra_subpartitions_split_flag[x0][y0] is assumed to be 1.

Decoding process

8.4.1. General decoding process for coding units encoded in intra-frame prediction mode

The inputs to this process are:

- location (XCB, YCB) of brightness indicating the upper left reference of the current block coding in relation to the upper left reference of the brightness of the current image,

- variable cbWidth, which determines the width of the current encoding block in brightness samples,

- variable cbHeight, which determines the height of the current coding block in brightness samples,

is a treeType variable that determines whether a single or double tree is used, and if a double tree is used, it indicates whether the current tree corresponds to luma or chroma components.

The result of this process is a modified reconstructed image before in-loop filtering.

The process of obtaining the quantization parameters as defined in clause 8.7.1 is associated with the location (xCb, yCb) of the luminance, the width of the current coding block in luminance samples cbWidth and the height of the current coding block in luminance samples cbHeight and the treeType variable as input.

When treeType is SINGLE_TREE or treeType is DUAL_TREE_LUMA, the decoding process for luminance samples is defined as follows:

- If pcm_flag[xCb][yCb] is 1, the reconstructed image is modified as follows:

S _L [ xCb + i ][ yCb + j ] =

pcm_sample_luma[ ( cbHeight * j ) + i ] << ( BitDepth _Y − PcmBitDepth _Y ), (8-6)

with i = 0..cbWidth − 1, j = 0..cbHeight − 1

- Otherwise the following applies:

1. The intra-frame luminance prediction method is defined as follows:

- If intra_lwip_flag[xCb][yCb] is equal to 1, the acquisition process for affine linear-weighted intra-prediction mode is as specified in clause 8.4. X is related to the location (xCb, yCb) of the luminance, the width of the current coding block in luminance samples cbWidth, and the height of the current coding block in luminance samples cbHeight as input.

- Otherwise, the acquisition process for intra-frame luminance prediction mode, as specified in clause 8.4.2, is associated with the luminance address (XCB, YCB), the width of the current encoding block in luminance samples cbWidth, and the height of the current encoding block in luminance samples cbHeight as input data.

2. The general decoding process for intraframe blocks, as specified in clause 8.4.4.1, is associated with the brightness address (XCB, YCB), the treeType of the tree, the nTbW variable set to cbWidth, the nTbH variable set to cbHeight, the predModeIntra variable set to IntraPredModeY[xCb][yCb], and the variable cIdx set to 0 as input, and the result is the modified reconstructed image before intra-loop filtering.

...

<begin>

8.4.X. Acquisition process for affine linear-weighted intra-frame prediction mode

The inputs to this process are:

- a luminance location (XCB, YCB) defining the upper left reference of the current luminance encoding block with respect to the upper left luminance reference of the current image,

- variable cbWidth, which determines the width of the current encoding block in brightness samples,

- variable cbHeight, which determines the height of the current encoding block in brightness samples.

In this process, the affine linear weighted intra-frame prediction mode IntraPredModeY[xCb][yCb] is obtained.

IntraPredModeY[xCb][yCb] is obtained using the following ordered steps:

1. At adjacent locations (xNbA, yNbA) and (xNbB, yNbB) are set to (XCB - 1, YCB) and (XCB, YCB - 1), respectively.

2. If X is replaced by A or B, the candLwipModeX variables are obtained as follows:

- The process of obtaining availability for a block, as specified in clause 6.4.X [Edition (BB): Process of checking the availability of neighboring blocks tbd] is associated with the location (xCurr, yCurr) set to (xCb, yCb), and the neighboring location (xNbY, yNbY) set equal to (xNbX, yNbX) as input, and the output is assigned to availableX.

- The candLwipModeX mode of affine linear-weighted intraframe candidate prediction is obtained as follows:

- If one or more of the following conditions are true, candLwipModeX is set to -1.

- The availableX variable is set to false.

- CuPredMode [xNbX][yNbX] is not equal to MODE_INTRA, and mh_intra_flag[xNbX][yNbX] is not equal to 1.

- pcm_flag[xNbX][yNbX] is equal to 1.

- X is equal to B, and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY).

- Otherwise the following applies:

- The process of obtaining the size type for a block, as specified in clause 8.4.X.1, is related to the width of the current encoding block in luminance samples cbWidth and the height of the current encoding block in luminance samples cbHeight as input, and the output is assigned to the variable sizeId.

- If intra_lwip_flag[xNbX][yNbX] is equal to 1, then the process of obtaining the size type for a block, as specified in clause 8.4.X.1, is related to the width of the adjacent coding block in nbWidthX luminance samples and the height of the adjacent coding block in nbHeightX luminance samples as input, and the output is assigned to the sizeIdX variable.

- If sizeId is equal to sizeIdX, candLwipModeX is set to IntraPredModeY[xNbX][yNbX].

- Otherwise candLwipModeX is set to -1.

- Otherwise, candLwipModeX is obtained using IntraPredModeY[xNbX][yNbX] and sizeId as specified in Table 8-X1.

3. CandLwipModeList[x] with x = 0..2 is obtained as follows using lwipMpmCand[sizeId] as shown in Table 8-X2:

- If candLwipModeA and candLwipModeB are both -1, the following applies:

candLwipModeList[ 0 ] = lwipMpmCand[ sizeId ][ 0 ] (8-X1)

candLwipModeList[ 1 ] = lwipMpmCand[ sizeId ][ 1 ] (8-X2)

candLwipModeList[ 2 ] = lwipMpmCand[ sizeId ][ 2 ] (8-X3)

- Otherwise, the following is possible:

- If candLwipModeA is equal to candLwipModeB or candLwipModeA or candLwipModeB is equal to -1, the following applies:

candLwipModeList[0] = (candLwipModeA ! = -1) ? candLwipModeA: candLwipModeB (8-X4)

- If CandLwipModeList[0] is equal to lwipMpmCand[sizeId][0], the following applies:

candLwipModeList[ 1 ] = lwipMpmCand[ sizeId ][ 1 ] (8-X5)

candLwipModeList[ 2 ] = lwipMpmCand[ sizeId ][ 2 ] (8-X6)

- Otherwise the following applies:

candLwipModeList[1] = lwipMpmCand[sizeId][0] (8-X7)

candLwipModeList[2] = (candLwipModeList[0] ! = lwipMpmCand[sizeId][1]) ? lwipMpmCand[sizeId][1]: lwipMpmCand[sizeId][2] (8-X8)

- Otherwise the following applies:

candLwipModeList[0] = candLwipModeA (8-X9)

candLwipModeList[1] = candLwipModeB (8-X10)

- If candLwipModeA and candLwipModeB are not equal to lwipMpmCand[sizeId][0], the following applies:

candLwipModeList[2] = lwipMpmCand[sizeId][0] (8-X11)

- Otherwise the following applies:

- If candLwipModeA and candLwipModeB are not equal to lwipMpmCand[sizeId][1], the following applies:

candLwipModeList[2] = lwipMpmCand[sizeId][1] (8-X12)

- Otherwise the following applies:

candLwipModeList[2] = lwipMpmCand[sizeId][2] (8-X13)

4. IntraPredModeY[xCb][yCb] is obtained by applying the following procedure:

- If intra_lwip_mpm_flag[xCb][yCb] is 1, IntraPredModeY[xCb][yCb] is set to candLwipModeList[intra_lwip_mpm_idx[xCb][yCb]].

- Otherwise, IntraPredModeY[xCb][yCb] is obtained by performing the following ordered steps:

1. When candLwipModeList[i] is greater than candLwipModeList[j] for i = 0..1 and for each i, j = (i + 1)..2, both values are swapped as follows:

(candLwipModeList[i], candLwipModeList[j]) = Swap (candLwipModeList[i], candLwipModeList[j]) (8-X14)

2. IntraPredModeY[xCb][yCb] is obtained using the following ordered steps:

(1) IntraPredModeY[xCb][yCb] is set to intra_lwip_mpm_remaining[xCb][yCb].

(2) For i equal to 0 to 2 inclusive, when IntraPredModeY[xCb][yCb] is greater than or equal to candLwipModeList[i], the value of IntraPredModeY[xCb][yCb] is increased by one.

The variable IntraPredModeY[x][y] with x = xCb..xCb + cbWidth - 1 and y = yCb..yCb + cbHeight - 1 is set equal to IntraPredModeY[xCb][yCb].

8.4.X.1 Derivation process for prediction block size type

The inputs to this process are:

- variable cbWidth, which determines the width of the current encoding block in brightness samples,

- variable cbHeight, which determines the height of the current encoding block in brightness samples.

The result of this process is the sizeId variable.

The sizeId variable is obtained as follows:

- If both cbWidth and cbHeight are 4, sizeId is set to 0.

- Otherwise, if cbWidth and cbHeight are less than or equal to 8, sizeId is set to 1.

- Otherwise, sizeId is set to 2.

Table 8-X1. Specification of the mapping between intra-prediction modes and affine linear-weighted intra-prediction

IntraPredModeY[ xNbX ][ yNbX ] block size of type sizeId 0 1 2 0 17 0 5 1 17 0 1 2, 3 17 10 3 4, 5 9 10 3 6.7 9 10 3 8, 9 9 10 3 10, 11 9 10 0 12, 13 17 4 0 14, 15 17 6 0 16, 17 17 7 4 18, 19 17 7 4 20, 21 17 7 4 22, 23 17 5 5 24, 25 17 5 1 26, 27 5 0 1 28, 29 5 0 1 30, 31 5 3 1 32, 33 5 3 1 34, 35 34 12 6 36, 37 22 12 6 38, 39 22 12 6 40, 41 22 12 6 42, 43 22 14 6 44, 45 34 14 10 46, 47 34 14 10 48, 49 34 16 9 50, 51 34 16 9 52, 53 34 16 9 54, 55 34 15 9 56, 57 34 13 9 58, 59 26 1 8 60, 61 26 1 8 62, 63 26 1 8 64, 65 26 1 8 66 26 1 8

Table 8-X2. Specification of possible affine linear-weighted intra-frame prediction modes

candidate mode 0 1 2 lwipMpmCand[ 0 ] 17 34 5 lwipMpmCand[ 1 ] 0 7 16 lwipMpmCand[ 2 ] 1 4 6

<end>

8.4.2. Process of obtaining intra-frame luminance prediction mode

The inputs to this process are:

- location (XCB, YCB) of brightness indicating the upper left reference of the current luminance coding block in relation to the upper left reference of the brightness of the current image,

- variable cbWidth, which determines the width of the current encoding block in brightness samples,

- variable cbHeight, which determines the height of the current encoding block in brightness samples.

In this process, the intra-frame luminance prediction mode IntraPredModeY[xCb][yCb] is determined.

Table 8-1 shows the value for the intra-frame prediction mode IntraPredModeY[xCb][yCb] and its associated names.

Table 8-1. Intra-frame luminance prediction mode specification and associated names

Intra-frame prediction mode Associated title 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

Note: The intra-frame prediction modes INTRA_LT_CCLM, INTRA_L_CCLM and INTRA_T_CCLM are applicable to chrominance components only.

IntraPredModeY[xCb][yCb] is obtained using the following ordered steps:

1. Neighboring locations (xNbA, yNbA) and (xNbB, yNbB) are set to (XCB - 1, YCB + cbHeight - 1) and (XCB + cbWidth - 1, YCB - 1), respectively.

2. If X is replaced by A or B, the candIntraPredModeX variables are obtained as follows:

- The process of obtaining availability for a block, as specified in clause <begin> 6.4.X [Revision (BB): Process of checking the availability of neighboring blocks tbd] <end> is associated with the location (xCurr, yCurr) set to (xCb, yCb), and the neighboring location (xNbY, yNbY) set to (xNbX, yNbX) as input, and the output is assigned to availableX.

- The candidate intraframe luminance prediction mode candIntraPredModeX is obtained as follows:

- If one or more of the following conditions is true, candIntraPredModeX is set to INTRA_PLANAR.

- The availableX variable is set to false.

- CuPredMode [xNbX][yNbX] is not equal to MODE_INTRA, and ciip_flag[xNbX][yNbX] is not equal to 1.

- pcm_flag[xNbX][yNbX] is equal to 1.

- X is equal to B, and yCb - 1 is less than ((yCb >> CtbLog2SizeY) << CtbLog2SizeY).

- Otherwise CandIntraPredModeX is obtained as follows:

- If intra_lwip_flag[xCb][yCb] is 1, CandIntraPredModeX is obtained using the following ordered steps:

(1) The process of obtaining the size type for a block, as specified in clause 8.4.X.1, takes the width of the current coding block in luminance samples cbWidth and the height of the current coding block in luminance samples cbHeight as input, and the output is assigned to the variable sizeId .

(2) CandIntraPredModeX is obtained using IntraPredModeY[xNbX][yNbX] and sizeId as specified in Table 8-X3.

- Otherwise, CandIntraPredModeX is set to IntraPredModeY[xNbX][yNbX].

3. The ispDefaultMode1 and ispDefaultMode2 variables are defined as follows:

- If IntraSubPartitionsSplitType is equal to ISP_HOR_SPLIT, ispDefaultMode1 is set to INTRA_ANGULAR18, and ispDefaultMode2 is set to INTRA_ANGULAR5.

- Otherwise, ispDefaultMode1 is set to INTRA_ANGULAR50, and ispDefaultMode2 is set to INTRA_ANGULAR63.

...

Table 8-X3. Specification of the mapping between affine linear-weighted intra-prediction and intra-prediction modes

IntraPredModeY[ xNbX ][ yNbX ] block size of type sizeId 0 1 2 0 0 0 1 1 18 1 1 2 18 0 1 3 0 1 1 4 18 0 18 5 0 22 0 6 12 18 1 7 0 18 0 8 18 1 1 9 2 0 50 10 18 1 0 eleven 12 0 12 18 1 13 18 0 14 1 44 15 18 0 16 18 50 17 0 1 18 0 0 19 50 20 0 21 50 22 0 23 56 24 0 25 50 26 66 27 50 28 56 29 50 thirty 50 31 1 32 50 33 50 34 50

8.4.3 Acquisition process for intra-frame chroma prediction mode

The inputs to this process are:

- a luminance location (XCB, YCB) defining the upper left sample of the current chroma coding block relative to the upper left luminance sample of the current image,

- variable cbWidth, which determines the width of the current encoding block in brightness samples,

- variable cbHeight, which determines the height of the current encoding block in brightness samples.

This process produces the intra-frame chroma prediction mode IntraPredModeC [xCb][yCb].

The corresponding lumaIntraPredMode is obtained as follows:

- If intra_lwip_flag[xCb][yCb] is 1, lumaIntraPredMode is obtained using the following ordered steps:

(1) The process of obtaining the size type for a block, as specified in clause 8.4.X.1, takes the width of the current coding block in luminance samples cbWidth and the height of the current coding block in luminance samples cbHeight as input, and the output is assigned to the variable sizeId .

(2) The intra-frame luminance prediction mode is obtained using IntraPredModeY[xCb + cbWidth/2][yCb + cbHeight/2] and sizeId as specified in Table 8-X3, and the value of CandIntraPredModeX is assigned to the lumaIntraPredMode parameter.

- Otherwise lumaIntraPredMode is set to IntraPredModeY[xCb + cbWidth/2][yCb + cbHeight/2].

The intra-chroma prediction mode IntraPredModeC [xCB][YCB] is obtained using intra_chroma_pred_mode [xCB][YCB] and lumaIntraPredMode, as specified in Table 8-2 and Table 8-3.

...

xxx. Intra-frame sample prediction

<begin>

The input parameters in this process are:

- sample location (xTbCmp, yTbCmp) defining the upper left sample of the current transform block relative to the upper left sample of the current image,

- predModeIntra variable, defining the intra-frame prediction mode,

- variable nTbW, which determines the width of the conversion block,

- variable nTbH, which determines the height of the transformation block,

- variable nCbW, which determines the width of the coding block,

- variable nCbH, which determines the height of the coding block,

- variable cIdx, which determines the chromaticity component of the current block.

The output of this process is the predicted samples predSamples[x][y], with x = 0..nTbW - 1, y = 0..nTbH - 1.

The predicted samples predSamples[x][y] are obtained as follows:

- If intra_lwip_flag[xTbCmp][yTbCmp] is 1 and cIdx is 0, the affine linear-weighted intra-selective prediction process as specified in clause 8.4.4.2.X1 is associated with the location (xTbCmp, yTbCmp), intra-prediction mode predModeIntra, transform block width nTbW and height nTbH as input, and the output is predSamples.

- Otherwise, the general intra-frame sample prediction process as specified in clause 8.4.4.2.X1. associated with the location (xTbCmp, yTbCmp), intra prediction mode predModeIntra, transform block width nTbW and height nTbH, encoding block width nCbW and height nCbH and variable cIdx as input, and the output is predSamples.

8.4.4.2.X1 Affine linear-weighted selective intra-frame prediction

The input parameters in this process are:

- sample location (xTbCmp, yTbCmp) defining the upper left sample of the current transform block relative to the upper left sample of the current image,

- predModeIntra variable, defining the intra-frame prediction mode,

- variable nTbW, which determines the width of the conversion block,

- variable nTbH, which determines the height of the transformation block.

The output of this process is the Predicted samples predSamples[x][y], with x = 0..nTbW - 1, y = 0..nTbH - 1.

The process of obtaining the size type for a block, as specified in clause 8.4.X.1, is associated with the transform block width nTbW and transform block height nTbH as input, and the output is assigned to sizeId.

The variables numModes, BorderSize, predW, predH, and predC are obtained using sizeId as shown in Table 8-X4.

Table 8-X4. Specification of number of modes, boundary sample size and prediction sizes depending on sizeId

sizeId numModes boundarySize predW predH predC 0 35 2 4 4 4 1 19 4 4 4 4 2 eleven 4 Min( nTbW, 8 ) Min( nTbH, 8 ) 8

The isTransposed flag is obtained as follows:

isTransposed = ( predModeIntra > ( numModes / 2 ) ) ? 1 : 0 (8-X15)

The needUpsBdryHor and needUpsBdryVer flags are obtained as follows:

needUpsBdryHor = ( nTbW > predW ) ? TRUE : FALSE (8-X16)

needUpsBdryVer = ( nTbH > predH ) ? TRUE : FALSE (8-X17)

The upsBdryW and upsBdryH variables are obtained as follows:

upsBdryW = (nTbH> nTbW)? nTbW: predW (8-X18)

upsBdryH = (nTbH> nTbW)? predH: nTbH (8-X19)

The lwipW and lwipH variables are obtained as follows:

lwipW = (isTransposed == 1)? predH: predW (8-X20)

lwipH = (isTransposed = = 1)? predW: predH (8-X21)

To generate reference samples refT[x] at x = 0..nTbW - 1 and refL [y] at y = 0..nTbH - 1 the process for obtaining reference samples, as specified in clause 8.4.4.2.X2, is associated with location ( xTbCmp, yTbCmp) of the sample, the width nTbW of the transformation block, the height nTbH of the transformation block as input data, and the top and left reference samples refT[x] at x = 0..nTbW - 1 and refL [y] at y = 0.. nTbH - 1, respectively, as output data.

To generate boundary samples p[x] at x = 0..2 * boundarySize - 1, the following applies:

- The border reduction process, as specified in clause 8.4.4.2.X3, is associated for the upper reference samples with block size nTbW, reference samples refT, borderSize size, upsampling border flag needUpsBdryVer and upsampling border size upsBdryW as input and downsampling with boundary samples redT[x] at x = 0..boundarySize - 1 and upsampled boundary samples upsBdryT[x] at x = 0..upsBdryW - 1 as output.

- The boundary reduction process as specified in clause 8.4.4.2.X3 is associated for left reference samples with block size nTbH, reference samples refL, border size BorderSize, upsampling border flag needUpsBdryHor and upsampling border size upsBdryH as input and samples downsampled boundary redL[x] at x = 0..boundarySize - 1 and upsampled boundary samples upsBdryL[x] at x = 0..upsBdryH - 1 as output.

- The reduced top and left boundary samples redT and redL are assigned to the boundary sample array p as follows:

- If isTransposed is 1, p[x] is set to redL[x] when x = 0..boundarySize - 1, and p[x + BorderSize] is set to redT[x] when x = 0.. borderSize - 1.

- Otherwise, p[x] is set to redT[x] for x = 0..boundarySize - 1, and p[x + BorderSize] is set to redL[x] for x = 0..boundarySize - 1.

For the intraframe sample prediction process according to predModeIntra, the following ordered steps are applied:

1. Affine linearly weighted samples predLwip[x][y] for x = 0..lwipW - 1, y = 0..lwipH - 1 are obtained as follows:

- The modeId variable is obtained as follows:

modeId = predModeIntra − ( isTransposed = = 1) ? (numModes/2): 0 (8-X22)

- Weight matrix mWeight[x][y] at x = 0..2 * boundarySize - 1, y = 0..predC * predC - 1 is obtained using sizeId and modeId, as indicated in table 8-XX[TBD: add weight matrices].

- The bias vector vBias[y] at y = 0..predC * predC - 1 is obtained using sizeId and modeId as specified in Table 8-XX[TBD: add bias vectors].

- The sW variable is obtained using sizeId and modeId as specified in Table 8-X5.

- Affine linearly weighted samples predLwip[x][y] for x = 0..lwipW - 1, y = 0..lwipH - 1 are obtained as follows:

oW = 1 << ( sW – 1 ) (8-X23)

sB = BitDepth _Y – 1 (8-X24)

incW = ( predC > lwipW ) ? 2 : 1 (8-X25)

incH = ( predC > lwipH ) ? 2 : 1 (8-X26)

predLwip[ x ][ y ] = ( ( ) +

( vBias[ y * incH * predC + x * incW ] << sB ) + oW ) >> sW (8-X27)

2. Predicted samples predSamples[x][y] for x = 0..nTbW - 1, y = 0..nTbH - 1 are obtained as follows:

- When isTransposed is 1, predLwip[x][y], when x = 0..predW - 1, y = 0..predH - 1 is set to predLwip[y][x].

- If needUpsBdryVer is true or needUpsBdryHor is true, the prediction upsampling process as specified in clause 8.4.4.2.X4 is associated with the input block width predW, the input block height predH, the affine linear weighted samples predLwip, transform block width nTbW, transform block height nTbH, upsampling border width upsBdryW, upsampling border height upsBdryH, upsampling border top samples upsBdryT and upsampling border left samples upsBdryL as input data, and the output is an array of predSamples of predicted samples.

- Otherwise, predSamples[x][y] for x = 0..nTbW - 1, y = 0..nTbH - 1 is set equal to predLwip[x][y].

Table 8-X5. Specification of offsets sW weighting coefficients depending on sizeId and modeId

modeId sizeId 0 1 2 3 4 5 6 7 8 9 10 eleven 12 13 14 15 16 17 0 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 1 8 8 8 9 8 8 8 8 9 8 2 8 8 8 8 8 8

8.4.4.2.X2 Process of obtaining reference reading

The input parameters in this process are:

- location (xTbY, yTbY) of the sample defining the upper left brightness sample of the current transform block in relation to the upper left brightness sample of the current image,

- variable nTbW, which determines the width of the conversion block,

- variable nTbH, which determines the height of the transformation block.

The output parameters in this process are the top and left reference samples refT[x] at x = 0..nTbW - 1 and refL[y] at y = 0..nTbH - 1, respectively.

Adjacent samples refT[x] at x = 0..nTbW - 1 and refL[y] at y = 0..nTbH - 1 are samples created before the in-loop filtering process and are obtained as follows:

- The top and left adjacent locations of (xNbT, yNbT) brightness and (xNbL, yNbL) are specified as follows:

(xNbT, yNbT) = (xTbY + x, yTbY - 1) (8-X28)

(xNbL, yNbL) = (xTbY - 1, yTbY + y) (8-X29)

- The process of obtaining accessibility for a block, as specified in clause 6.4.X [Edition (BB): Process of checking the availability of neighboring blocks tbd] is associated with the current location (xCurr, yCurr) of the brightness set to (xTbY, yTbY), and the top neighboring location (xNbT, yNbT) brightness as input, and the output is assigned to availTop[x] at x = 0..nTbW - 1.

- The process of obtaining accessibility for a block, as specified in clause 6.4.X [Edition (BB): Process of checking the availability of neighboring blocks tbd] is associated with the current location (xCurr, yCurr) of the brightness set to (xTbY, yTbY), and the left neighboring location (xNbL, yNbL) brightness as input, and the output is assigned to availLeft[y] with y = 0..nTbH - 1.

- The upper reference readings refT[x] at x = 0..nTbW - 1 are obtained as follows:

- If all availTop[x] at x = 0..nTbW - 1 are true, the sample at location (xNbT, yNbT) is assigned to refT[x] at x = 0..nTbW - 1.

- Otherwise, if availTop[0] is false, all refT[x] at x = 0..nTbW - 1 are set to 1 << (BitDepth _Y - 1).

- Otherwise, the reference readings refT[x] at x = 0..nTbW - 1 are obtained using the following ordered steps:

1. The lastT variable is set to the x position of the first element in the sequence availTop[x] with x = 1..nTbW - 1, which is false.

2. For each x = 0..lastT - 1 sample at location (xNbT, yNbT) is assigned refT[x].

3. For each x = lastT..nTbW - 1 refT[x] is set equal to refT[lastT - 1].

- The left reference reading refL[y] at x = 0..nTbH - 1 is obtained as follows:

- If all availLeft[y] at y = 0..nTbH - 1 are true, the sample at location (xNbL, yNbL) is assigned to refL[y] at y = 0..nTbH - 1.

- Otherwise, if availLeft[0] is false, all refL[y] at y = 0..nTbH - 1 are set to 1 << (BitDepth _Y - 1).

- Otherwise, reference readings refL[y] at y = 0..nTbH - 1 are obtained using the following ordered steps:

1. The lastL variable is set to the y position of the first element in the sequence availLeft[y] with y = 1..nTbH - 1, which is false.

2. For each y = 0..lastL - 1 sample at location (xNbL, yNbL) is assigned refL[y].

3. For each y = lastL..nTbH - 1 refL[y] is set equal to refL[lastL - 1].

Boundary Reduction Process Specification

The input parameters in this process are:

- variable nTbX, which determines the size of the conversion block,

- reference readings refX[x] at x = 0..nTbX - 1,

- BorderSize variable, which determines the size of the downsampling border,

- needUpsBdryX flag indicating whether intermediate boundary samples are required for upsampling,

- upsBdrySize variable, which determines the size of the upsampling boundary.

The output of this process is downsampled boundary samples redX[x] at x = 0..boundarySize - 1 and upsampled boundary samples upsBdryX[x] at x = 0..upsBdrySize - 1.

The upsampled boundary samples upsBdryX[x] at x = 0.. upsBdrySize - 1 are obtained as follows:

- If needUpsBdryX is true and upsBdrySize is less than nTbX, the following applies:

uDwn = nTbX / upsBdrySize (8-X30)

upsBdryX[ x ] = ( + ( 1 << ( Log2( uDwn ) − 1 ) )) >> Log2( uDwn ) (8-X31)

- Otherwise (upsBdrySize equals nTbX) upsBdryX[x] is set equal to refX[x].

Reduced boundary samples RedX[x] at x = 0..boundarySize - 1 are obtained as follows:

- If BorderSize is less than upsBdrySize, the following applies:

bDwn = upsBdrySize / boundarySize (8-X32)

redX[ x ] = ( + ( 1 << ( Log2( bDwn ) − 1 ) )) >> Log2( bDwn ) (8-X33)

- Otherwise (BorderSize equals upsBdrySize) redX[x] is set equal to upsBdryX[x].

8.4.4.2.X4 Prediction upsampling process specification

The input parameters in this process are:

- variable predW, which determines the width of the input block,

- variable predH, which determines the height of the input block,

- affine linearly weighted samples predLwip[x][y], for x = 0..predW - 1, y = 0..predH - 1,

- variable nTbW, which determines the width of the conversion block,

- variable nTbH, which determines the height of the transformation block,

- upsBdryW variable, which determines the width of the upsampling boundary,

- upsBdryH variable, which determines the height of the upsampling boundary,

- upper boundary samples upsBdryT[x] with upsampling at x = 0..upsBdryW - 1,

- left boundary samples upsBdryL[x] with upsampling at x = 0..upsBdryH - 1.

The output of this process is the Predicted samples predSamples[x][y] at x = 0..nTbW - 1, y = 0..nTbH - 1.

Sparse Predicted samples predSamples[m][n] are obtained from predLwip[x][y] at x = 0..predW - 1, y = 0..predH - 1 as follows:

upHor = nTbW / predW (8-X34)

upVer = nTbH / predH (8-X35)

predSamples[ ( x + 1 ) * upHor − 1 ][ ( y + 1 ) * upVer − 1 ] = predLwip[ x ][ y ] (8-X36)

The upper limit samples upsBdryT[x] at x = 0..upsBdryW - 1 are assigned to predSamples[m][−1] as follows:

predSamples[ ( x + 1 ) * ( nTbW / upsBdryW ) − 1 ][ −1 ] = upsBdryT[ x ] (8-X37)

The left boundary samples upsBdryL[y] at y = 0..upsBdryH - 1 are assigned to predSamples[−1][n] as follows:

predSamples[ −1 ][ ( y + 1 ) * ( nTbH / upsBdryH ) − 1 ] = upsBdryL[ y ] (8-X38)

Predicted samples predSamples[x][y] for x = 0..nTbW - 1, y = 0..nTbH - 1 are obtained as follows:

- If nTbH is greater than nTbW, the following ordered steps apply:

1. When upHor is greater than 1, horizontal upsampling for all sparse positions (xHor, yHor) = (m * upHor - 1, n * upVer - 1) is applied with m = 0..predW - 1, n = 1..predH with dX = 1..upHor - 1 as follows:

predSamples[ xHor + dX ][ yHor ] = ( ( upHor − dX ) * predSamples[ xHor ][ yHor ] + dX * predSamples[ xHor + upHor ][ yHor ] ) / upHor (8-X39)

2. Vertical upsampling for all sparse positions (xVer, yVer) = (m, n * upVer - 1) with m = 0..nTbW - 1, n = 0..predH - 1 applied with dY = 1..upVer - 1 as follows:

predSamples[ xVer ][ yVer + dY ] = ( ( upVer − dY ) * predSamples[ xVer ][ yVer ] + dY * predSamples[ xVer ][ yVer + upVer ] ) / upVer (8-X40)

- Otherwise, the following steps apply:

1. When upVer is greater than 1, vertical upsampling is applied for all sparse positions (xVer, yVer) = (m * upHor - 1, n * upVer - 1) with m = 1..predW, n = 0..predH - 1 .with dY = 1..upVer - 1 as given in (8-X40).

2. Horizontal upsampling for all sparse positions (xHor, yHor) = (m * upHor - 1, n) with m = 0..predW - 1, n = 0..nTbH - 1 applied with dX = 1..upHor – 1, as indicated in (8-X39).

<end>

Table 9-9. Syntactic elements and associated binarizations

Syntax structure Syntactic element Binarization Process Input parameters coding_unit() cu_skip_flag[ ][ ] FL cMax = 1 pred_mode_ibc_flag FL cMax = 1 pred_mode_flag FL cMax = 1 <begin>intra_lwip_flag[ ][ ] FL cMax = 1 intra_lwip_mpm_flag[ ][ ] FL cMax = 1 intra_lwip_mpm_idx[ ][ ] TR cMax = 2, cRiceParam = 0 intra_lwip_mpm_remainder [ ][ ] FL cMax = (cbWidth = = 4 && cbHeight = = 4) ? 31 : ( (cbWidth <= 8 && cbHeight <= 8) ? 15 : 7) ...

Table 9-15. Purpose of ctxInc for syntax elements with context-encoded beans

Syntactic element binIdx 0 1 2 3 4 >= 5 ... completion na na na na na intra_lwip_flag[ ][ ] (Abs( Log2(cbWidth) – Log2(cbHeight) ) > 1) ?
3 : ( 0,1,2
(clause 9.5.4.2.2) na na na na na intra_lwip_mpm_flag[ ][ ] 0 na na na na na intra_lwip_mpm_idx[ ][ ] retraction retraction na na na na intra_lwip_mpm_remainder[ ][ ] retraction retraction retraction retraction retraction na

Table 9-16. ctxInc specification using left and top syntax elements

Syntactic element condL condA ctxSetIdx ... intra_lwip_flag[ x0 ][ y0 ] intra_lwip_flag[ xNbL ][ yNbL ] intra_lwip_flag[ xNbA ][ yNbA ] 0 ...

<end>

Brief description of ALWIP

To predict samples of a rectangular block with width W and height H, affine linear-weighted intra-prediction (ALWIP) takes one row of H reconstructed neighboring boundary samples to the left of the block and one row of W reconstructed neighboring boundary samples above the block as input. If reconstructed samples are not available, they are generated in a manner similar to traditional intra-frame prediction. ALWIP applies only to the intraframe block. For the intraframe chrominance block, traditional intraframe coding modes are used.

Generation of the prediction signal is based on the following three steps:

1. Four samples in the case of W=H=4 and eight samples in all other cases are extracted from the boundary samples by averaging.

2. Matrix-vector multiplication followed by offset addition is performed with the averaged samples as input. The result is a reduced prediction signal with respect to the subsampled set of samples in the original block.

3. The prediction signal at the remaining positions is generated from the prediction signal regarding the subsampled set using linear interpolation, which is a one-step linear interpolation in each direction.

If ALWIP mode is to be used, then the index ALWIP mode is signaled using an MPM list with 3 MPMS. Here, MPM acquisition is performed using the intra-frame modes of the top and left PUs as follows. There are three fixed tables , , which assign an ALWIP mode to each legacy mode intra-frame prediction.

For each PU with width W and height H, an index is defined

which specifies from which of the three sets the ALWIP parameters should be taken.

If the above unit prediction is available, belongs to the same CTU as the current PU, and is in intra-frame mode if and if ALWIP is applicable to with ALWIP mode , we believe that

If the above PU is available, belongs to the same CTU as the current PU, and is in intra-frame mode, and if the traditional intra-prediction mode is applied to the above PU, then we assume that

In all other cases we assume that

which means this mode is not available. In the same way, but without the restriction that the left PU must belong to the same CTU as the current PU, the mode .

Finally, three default fixed lists are provided , , each containing three different ALWIP modes. From the default list and modes And three separate MPMs are created by replacing -1 with default values and also eliminating duplicates.

To obtain a luma MPM list whenever a neighboring luma block that uses ALWIP mode is encountered , this block is processed as if it were using traditional intra-prediction mode .

3. Convert to VVC

3.1. Multiple Transform Selection (MTS)

In addition to DCT-II, which was used in HEVC, a multiple transform selection scheme (MTS) is used to encode the remainder of both inter- and intra-frame coded blocks. It uses a few selected transforms from DCT8/DST7. DST-VII and DCT-VIII are newly introduced transformation matrices.

3.2. Reduced Secondary Transform (RST) proposed in JVET-N0193

Reduced Secondary Transform (RST) applies a 16x16 and 16x64 non-separable transform to 4x4 and 8x8 blocks, respectively. The primary forward and inverse transformations are still performed in the same way as two passes of a one-dimensional horizontal/vertical transformation. Secondary forward and inverse transformations represent a separate processing step from these primary transformations. For the encoder, a primary forward transform is first performed, followed by a secondary forward transform and quantization and CABAC bit encoding. The decoder first performs CABAC bit decoding and inverse quantization, then a secondary inverse transform, followed by a primary inverse transform. RST applies only to intra-frame coded TUs in both intra- and inter-frame slices.

3.3. Unified MPM list for intra-frame encoding in JVET-N0185

A unified 6-MPM list is proposed for intra-frame blocks, regardless of whether multiple reference line (MRL) and intra-frame subsection (ISP) coding tools are used. The MPM list is compiled based on the intra-frame mode of the left and top adjacent block, as in VTM4.0. Assuming the Left mode is designated as Left and the top block mode is designated as Above, the unified MPM list is constructed as follows:

• When an adjacent block is unreachable, its intraframe mode is set to Planar mode by default.

• If both Left and Above modes are not angular:

a) MPM list → {Planar, DC, V, H, V-4, V+4}

• If one of the Left and Above modes is corner and the other is non-corner:

a) set Max mode as the larger mode on the left and top

b) MPM list → {Planar, Max, DC, Max -1, Max +1, Max -2}

• If both Left and Above modes are angular and different:

a) set Max mode as the larger mode on the left and top

b) If the difference between Left and Above modes is in the range from 2 to 62 inclusive

1) MPM list → {Planar, Left, Above, DC, Max -1, Max +1}

c) Otherwise

1) MPM list → {Planar, Left, Above, DC, Max -2, Max +2}

• If Left and Above are angular and the same:

a) MPM list → {Planar, Left, Left -1, Left +1, DC, Left -2}

In addition, the first codeword bin of the MPM index is a context-coded CABAC. A total of three contexts are used, corresponding to whether the current MRL intraframe is on, ISP is on, or a normal intraframe.

A2 and B2 represent the left neighbor block and the top neighbor block, which are used in constructing the unified MPM list, as shown in FIG. 10.

First, one MPM flag is encoded. If a block is encoded using one of the modes in the MPM list, the MPM index is additionally encoded. Otherwise, the indicator of the remaining modes (excluding MPM) is encoded.

4. Examples of shortcomings in existing implementations

The ALWIP design in JVET-N0217 has the following problems:

1) At the JVET conference in March 2019, unified 6-MPM listing for MRL mode, ISP mode and normal intra-frame mode was adopted. But affine linear-weighted prediction mode uses a different 3-MPM list compilation, which makes MPM list compilation more difficult. Complex MPM list design can compromise decoder throughput, especially for small blocks such as 4x4 samples.

2) ALWIP applies only to the luma component of the block. For the chrominance component of an ALWP-encoded block, the chrominance mode index is encoded and sent to the decoder, which may result in unnecessary signaling.

3) ALWIP interactions should be considered with other coding tools.

4) When calculating upsBdryX in upsBdryX[x] = (Log2( uDwn ) − 1))) >> Log2(uDwn) (8-X31), it is possible that Log2( uDwn ) − 1 is equal to -1, but the left shift by -1 is undefined.

5) When upsampling prediction samples, rounding is not applied.

6) During the deblocking process, ALWIP-encoded blocks are treated as normal intra-frame blocks.

5. Approximate methods of intra-frame matrix encoding

Embodiments of the technology disclosed herein overcome the shortcomings of existing implementations, thereby providing video encoding with higher encoding efficiency but lower computational complexity. The matrix intra-frame prediction methods for video encoding described herein provide improvements to both current and future video encoding standards, which are illustrated in the following examples described for various implementations. The examples of technology disclosed below explain general concepts and are not intended to be interpreted as limiting. A single example may combine various functions described in these examples unless explicitly stated otherwise.

In the following discussion, the intra prediction mode refers to the angular intra prediction mode (including DC, planar, CCLM and other possible intra prediction modes); wherein the intra-frame mode refers to the normal intra-frame mode, or MRL, or ISP, or ALWIP.

In the following discussion, "other intraframe modes" may refer to one or more intraframe modes other than ALWIP, such as normal intraframe mode, or MRL or ISP.

In the following discussion, SatShift(x, n) is defined as

Shift(x, n) is defined as Shift(x, n) = (x + offset0) >> n.

In one example, offset0 and/or offset1 are set to (1 << n) >> 1 or (1 << (n-1)). In another example, offset0 and/or offset1 are set to 0.

In another example, offset0=offset1= ((1<<n)>>1)-1 or ((1<<(n-1)))-1.

Clip3(min, max, x) is defined as

Listing MPMs for ALWIP

1. It is proposed that all or partial MPM list for ALWIP can be compiled in accordance with the full or partial MPM list construction procedure for non-ALWIP intra-frame mode (eg, normal intra-frame mode, MRL or ISP).

a) In one example, the MPM list size for ALWIP may be the same as the MPM list size for non-ALWIP intraframe mode.

(1) For example, the MPM list size is 6 for both ALWIP mode and non-ALWIP intraframe mode.

b) In one example, the MPM list for ALWIP may be derived from the MPM list for non-ALWIP intraframe mode.

(1) In one example, the MPM list may first be compiled for non-ALWIP intraframe mode. Some or all of it can then be converted to MPMs, which can be further added to the list of MPMs for ALWIP-encoded blocks.

1) Additionally, as an alternative, when adding the mapped MPM to the MPM list, a shortcut may be applied for ALWIP-encoded blocks.

2) Default modes can be added to the MPM list for ALWIP-encoded blocks.

a) In one example, default modes may be added before they are converted from the non-ALWIP intraframe mode MPM list.

b) Alternatively, default modes can be added after they have been converted from the intraframe non-ALWIP mode MPM list.

c) Alternatively, default modes may be added interleaved with those converted from the non-ALWIP intraframe mode MPM list.

d) In one example, the default modes could be fixed to be the same for all block types.

e) Alternatively, default modes can be determined according to encoded information such as neighbor block availability, neighbor block mode information, block size.

b) In one example, one intra-prediction mode in the MPM list for non-ALWIP intra-frame mode may be converted to its corresponding ALWIP intra-prediction mode when placed in the MPM list for ALWIP.

1) Alternatively, all intra-prediction modes in the non-ALWIP intra-mode MPM list may be converted to the corresponding ALWIP intra-prediction modes before being used to construct the ALWIP MPM list.

2) Alternatively, all candidate intra-prediction modes (which may include neighboring intra-prediction modes and default intra-prediction modes such as Planar and DC) may be converted to corresponding ALWIP intra-prediction modes before use to compile a list of MPMs for intra-frame no-ALWIP mode, if the list of MPMs for intra-frame mode without-ALWIP can be used to obtain a list of MPMs for ALWIP.

3) In one example, two converted ALWIP intra-frame prediction modes can be compared.

a) In one example, if they are the same, only one of them can be placed in the MPM list for ALWIP.

b) In one example, if they are the same, only one of them can be placed in the MPM list for non-ALWIP intraframe modes.

(3) In one example, K of the S intra-prediction modes in the MPM list for the non-ALWIP intra-frame mode may be selected as the MPM list for the ALWIP mode. For example, K is 3 and S is 6.

1) In one example, the first K intra-prediction modes in the MPM list for the non-ALWIP intra-frame mode may be selected as the MPM list for the ALWIP mode.

2. It is proposed that one or more adjacent blocks used to obtain the MPM list for ALWIP may also be used to obtain the MPM list for non-ALWIP intraframe mode (eg, normal intraframe mode, MRL or ISP).

a) In one example, the adjacent block to the left of the current block used to obtain the MPM list for ALWIP must be the same as the block used to obtain the MPM list for non-ALWIP intraframe modes.

(1) Assuming that the top left corner of the current block is (xCb, yCb), W and H are the width and height of the current block, then in one example the left adjacent block is used to obtain a list of MPMs for both ALWIP and intraframe modes no-ALWIP can cover positions (xCb-1, yCb). In an alternative example, the left adjacent block used to obtain the MPM list for both ALWIP mode and intra-ALWIP mode may cover the position (xCb-1, yCb+H-1).

(2) For example, A2 and B2 are the left neighbor block and the top neighbor block used in constructing the unified MPM list, as shown in FIG. 10.

b) In one example, the block adjacent to the current block that is used to obtain the MPM list for ALWIP must be the same as the block used to obtain the MPM list for non-ALWIP intraframe modes.

(1) Assuming the top left corner of the current block has position (xCb, yCb), W and H are the width and height of the current block, then in one example the top adjacent block is used to obtain the MPM list for both ALWIP and intraframe modes without-ALWIP, can cover positions (xCb, yCb-1). In an alternative example, the top adjacent block used to obtain the MPM list for both ALWIP mode and intraframe non-ALWIP mode may cover position (xCb + W-1, yCb-1).

(2) For example, A1 and B1 are the left neighbor block and the top neighbor block used in constructing the unified MPM list, as shown in FIG. 10.

3. It is proposed that the MPM list for ALWIP can be constructed differently according to the width and/or height of the current block.

a) In one example, different neighboring blocks may be accessed at different block sizes.

4. It is proposed that the MPM list for ALWIP and the MPM list for intraframe non-ALWIP modes can be constructed using the same procedure but with different parameters.

a) In one example, K of S intra-prediction modes in the MPM list generation procedure for non-ALWIP intra-frame modes may be obtained for the MPM list used in the ALWIP mode. For example, K is 3 and S is 6.

(1) In one example, the first K intra-prediction modes in the MPM list generation procedure may be obtained for the MPM list used in the ALWIP mode.

b) In one example, the first mode in the MPM list may be different.

(1) For example, the first mode in the MPM list for non-ALWIP intraframe mode may be Planar mode, but it may be X0 mode in the MPM list for ALWIP.

1) In one example, X0 may be an ALWIP intra-prediction mode converted from a Planar mode.

c) In one example, the padding modes in the MPM list may be different.

(1) For example, the first three padding modes in the MPM list for intraframe non-ALWIP mode may be Modes DC, Vertical and Horizontal, but they may be Mode X1, X2, X3 in the MPM list for ALWIP.

1) In one example, the modes X1, X2, X3 can be different for different sizeId.

(2) In one example, the number of filling modes may be different.

d) In one example, adjacent modes in the MPM list may be different.

(1) For example, normal intra-adjacent block prediction modes are used to construct the MPM list for non-ALWIP intra-frame modes, and they are converted to intra-frame prediction ALWIP modes to construct the MPM list for the ALWIP mode.

e) In one example, the biased modes in the MPM list may be different.

(1) For example, X+K0, where X is the normal intra-prediction mode, and K0 is an integer, can be placed in the MPM list for non-ALWIP intra-frame modes, and Y+K1, where Y is the ALWIP intra-prediction mode, and K1 is an integer that can be placed in the MPM list for ALWIP, where K0 may be different from K1.

1) In one example, K1 may depend on width and height.

5. It is proposed that an adjacent block is treated as unreachable if it is encoded with ALWIP when compiling the MPM list for the current block with non-ALWIP intraframe modes.

a) Alternatively, an adjacent block is treated as encoded with a given intra-prediction mode (eg, Planar mode) if it is encoded with ALWIP when compiling the MPM list for the current block with non-ALWIP intra-frame modes.

6. It is proposed that an adjacent block is treated as unavailable if it is encoded with non-ALWIP intraframe modes when constructing the MPM list for the current block in ALWIP mode.

a) Alternatively, an adjacent block is treated as encoded with the specified ALWIP intra-prediction mode X if it is encoded with non-ALWIP intra-frame mode when compiling the MPM list for the current block in ALWIP mode.

(1) In one example, X may depend on the dimensions of the block, such as width and/or height.

7. It is proposed to remove the ALWIP flag storage from the string buffer.

a) In one example, when the 2nd block to be accessed is located in a different LCU/CTU row/area than the current block, the conditional check of whether the 2nd block is ALWIP encoded is skipped.

b) In one example, when the 2nd block to be accessed is located in a different LCU/CTU row/area compared to the current block, the 2nd block is processed in the same way as non-ALWIP mode, e.g. like a normal intra-frame coded block.

8. When encoding the ALWIP flag, no more than K (K> = 0) contexts can be used.

a) In one example, K = 1.

9. It is proposed to store the converted intra-frame prediction mode of ALWIP coded blocks instead of directly storing the mode index associated with the ALWIP mode.

a) In one example, the decoded mode index associated with one coded ALWIP block is mapped to a normal intra-frame mode, for example, in accordance with , as described in section 2.5.7.

b) Additionally, as an alternative, storage of the ALWIP flag is removed entirely.

c) Additionally, as an alternative, the ALWIP mode persistence is removed completely.

d) Additionally, as an alternative, the status check of whether one adjacent/current block is encoded with the ALWIP flag may be omitted.

e) Additionally, as an alternative, mode mapping assigned to ALWIP-coded blocks and normal intra-frame predictions associated with one accessed block may be skipped.

ALWIP for different color components

10. It is proposed that the intended intra-frame chroma mode (eg DM mode) could always be used if the corresponding luminance block is encoded in ALWIP mode.

a) In one example, an intra-frame chroma mode is obtained as a DM mode without signaling if the corresponding luminance block is encoded in the ALWIP mode.

b) In one example, the corresponding luma block may be a block covering a corresponding chroma sample located at a given position (eg, the upper left corner of the current chroma block, the center of the current chroma block).

c) In one example, the DM mode may be obtained according to the intra-prediction mode of the corresponding luminance block, for example, by mapping the (ALWIP) mode to one of the normal intra-frame modes.

11. When the corresponding luminance block of chrominance blocks is encoded in ALWIP mode, multiple DM modes can be obtained.

12. It is proposed that a special mode be assigned to chrominance blocks if one corresponding luminance block is encoded in ALWIP mode.

a) In one example, a special mode is defined as a given normal intra-prediction mode regardless of the intra-prediction mode associated with ALWIP-coded blocks.

b) In one example, different intra-prediction methods may be assigned to this special mode.

13. It is intended that ALWIP may also be applied to chrominance components.

a) In one example, the matrix and/or offset vector may be different for different color components.

b) In one example, the matrix and/or bias vector may be specified jointly for Cb and Cr.

(1) In one example, the Cb and Cr components may be combined.

(2) In one example, the Cb and Cr components may alternate.

c) In one example, the chrominance component may share the same ALWIP intra-frame prediction mode as the corresponding luminance block.

(1) In one example, the same ALWIP intra-frame prediction mode is applied to the chrominance component if the corresponding luma block uses the ALWIP mode and the chroma block is encoded in the DM mode.

(2) In one example, the same ALWIP intra-frame prediction mode is applied to the chrominance component, and linear interpolation can then be skipped.

(3) In one example, the same ALWIP intra-frame prediction mode is applied to a subsampled matrix and/or offset vector chrominance component.

d) In one example, the number of ALWIP intra-frame prediction modes for different components may be different.

(1) For example, the number of ALWIP intra-frame prediction modes for chrominance components may be less than for luminance components for the same block width and height.

Applicability of ALWIP

14. It is proposed to signal the possibility of using ALWIP.

a) For example, signaling may be performed at the sequence level (e.g. in SPS), at the picture level (e.g. in PPS or picture header), at the slice level (e.g. in slice header), at the tile group level (e.g. in the tile group header), at the tile level, at the CTU row level, or at the CTU level.

b) For example, intra_lwip_flag cannot be signaled and set to 0 if ALWIP cannot be applied.

15. It is anticipated that the availability of ALWIP may depend on the width (W) and/or height (H) of the block.

c) For example, ALWIP may not apply if W>=T1 (or W>T1) and H>=T2 (or H>T2). For example, T1=T2=32;

(1) For example, ALWIP may not apply if W<=T1 (or W<T1) and H<=T2 (or H<T2). For example, T1=T2=32;

d) For example, ALWIP may not apply if W>=T1 (or W>T1) or H>=T2 (or H>T2). For example, T1=T2=32;

(1) For example, ALWIP may not apply if W<=T1 (or W<T1) or H<=T2 (or H<T2). For example, T1=T2=32 or T1=T2=8;

e) For example, ALWIP may not apply if W+H>=T (or W*H>T). For example, T = 256;

(1) For example, ALWIP may not apply if W+H<=T (or W+H<T). For example, T = 256;

f) For example, ALWIP may not apply if W*H>=T (or W*H>T). For example, T = 256;

(1) For example, ALWIP may not apply if W*H<=T (or W*H<T). For example, T = 256;

g) For example, intra_lwip_flag cannot be signaled and set to 0 if ALWIP cannot be tried.

Calculation problems in ALWIP

16. It is assumed that any shift operation involved in ALWIP can only perform a left or right shift by a number S, where S must be greater than or equal to 0.

a) In one example, the right shift operation may be different when S is 0 or greater than 0.

(1) In one example, upsBdryX[x] should be calculated as

uDwn>1, and upsBdryX[ x ]= , when uDwn is equal to 1.

b) In one example, upsBdryX[x] should be calculated as

upsBdryX[ x ]= ( + ( 1 << Log2( uDwn ) >>1 ) ) >> Log2( uDwn )

17. It is proposed that the results will be rounded towards or away from zero during the ALWIP upsampling process.

a) In one example

predSamples[ xHor + dX ][ yHor ] = ( ( upHor − dX ) * predSamples[ xHor ][ yHor ] + dX * predSamples[ xHor + upHor ][ yHor ] +offsetHor) / upHor (8-X39)

And

predSamples[ xVer ][ yVer + dY ] = ( ( upVer − dY ) * predSamples[ xVer ][ yVer ] + dY * predSamples[ xVer ][ yVer + upVer ]+offsetVer ) / upVer, (8-X40)

where offsetHor and offsetVer are integers. For example, offsetHor = upHor/2 and offsetVer=upVer/2.

Interoperability with other coding tools

18. It is proposed that ALWIP can be used for a CIIP encoded block.

a) In one example, the CIIP coded block may explicitly indicate whether the ALWIP intra prediction mode or a normal intra prediction mode, such as the Planar mode, is used to generate the intra prediction signal.

b) In one example, it can be implicitly inferred whether the ALWIP intra prediction mode or a normal intra prediction mode such as the Planar mode can be used to generate the intra prediction signal.

(1) In one example, the ALWIP intra-frame prediction mode may never be used in a CIIP-coded block.

1) Alternatively, normal intra-frame prediction can never be used in a CIIP-coded block.

(2) In one example, it can be inferred from the neighboring block information whether an ALWIP intra-prediction mode or a normal intra-prediction mode such as a Planar mode is used to generate an intra-prediction signal.

19. It is proposed that all or part of the procedure used to downsample adjacent luminance samples in CCLM mode can be used to downsample adjacent luminance samples in ALWIP mode.

a) Alternatively, all or part of the procedure used to downsample adjacent luminance samples in ALWIP mode can be used to downsample adjacent luminance samples in CCLM mode.

b) The neighbor downsampling procedure can be used with other parameters/arguments when used in the CCLM process and the ALWIP process.

c) In one example, a downsampling technique (such as selection of neighboring luminance locations, downsampling filters) in the CCLM process may be used in the ALWIP process.

d) The procedure used to downsample adjacent luminance samples at least includes selection of downsampling positions, downsampling filters, rounding and discarding operations.

20. It is assumed that a block encoded using the ALWIP mode cannot apply RST, or/and secondary transform, or/and rotation transform, or/and non-separable secondary transform (NSST).

a) In one example, whether or not such truncation can be applied may depend on block size information, eg, the same as the conditions described in (15).

b) Alternatively, ALWIP mode may be disabled when RST and/or secondary transform and/or rotation transform and/or NSST are applied.

c) Alternatively, a block encoded using the ALWIP mode may apply an RST and/or a secondary transform and/or a rotation transform and/or a non-separable secondary transform (NSST).

(1) In one example, the choice of transform matrix may depend on the ALWIP intra-frame prediction mode.

(2) In one example, the choice of transformation matrix may depend on the normal intra-prediction mode, which is converted from the ALWIP intra-prediction mode.

(3) In one example, the choice of transformation matrix may depend on the classification of the normal intra-prediction mode that is converted from the ALWIP intra-prediction mode.

21. It is assumed that a block encoded using the ALWIP mode cannot apply block-based DPCM (BDPCM) or residual RDPCM.

a) Alternatively, ALWIP mode may be disabled when using BDPCM or RDPCM.

22. It is proposed that a block encoded using the ALWIP mode may only use DCT-II as a transform.

a) One example always skips signaling the transformation matrix indices.

b) As an alternative, it is proposed that the mapping used for a block encoded using the ALWIP mode may be obtained implicitly instead of through explicit signaling. For example, the transformation may be selected according to the method proposed in JVET-M0303.

c) As an alternative, it is proposed that a block encoded using the ALWIP mode may use skip conversion only.

(1) Alternatively, when ALWIP is used, signaling to indicate the use of conversion skip is omitted.

d) In one example, ALWIP mode information (such as enable/disable, prediction mode index) may be conditionally signaled after the transformation matrix indications.

(1) In one example, ALWIP mode information indications may be signaled for a given transform matrix (such as skip transform or DCT-II).

(2) Additionally, as an alternative, indications of ALWIP mode information may be omitted for some given transformation matrices.

23. It is proposed that a block encoded using the ALWIP mode is considered to be normal intra-prediction encoded transformed from the ALWIP intra-prediction mode when the selected transformation is mode dependent.

24. ALWIP mode may not use skip conversion.

a) For example, in this case there is no need to additionally signal the use of skip conversion.

b) Alternatively, ALWIP mode may be disabled when skip conversion is applied.

(1) For example, there is no need to signal ALWIP mode information when skip translation is applied in this case.

25. In a filtering process, for example, which uses a deblocking filter, an adaptive sample offset (SAO), an adaptive loop filter (ALF), how to select filters and/or whether to filter samples can be determined by using ALWIP.

26. Unfiltered adjacent samples can be used in ALWIP mode.

a) Alternatively, filtered adjacent samples can be used in ALWIP mode.

b) In one example, filtered neighbor samples can be used for downsampling, and unfiltered neighbor samples can be used for upsampling.

c) In one example, unfiltered neighboring samples can be used for downsampling, and filtered neighboring samples can be used for upsampling.

d) In one example, filtered left neighbor samples can be used in upsampling, and unfiltered top neighbor samples can be used in upsampling.

e) In one example, unfiltered left neighbor samples can be used in upsampling, and filtered top neighbor samples can be used in upsampling.

f) In one example, the use of a filter or unfiltered neighboring samples may depend on the ALWIP mode.

(1) In one example, the ALWIP mode may be converted to a conventional intra-prediction mode, and whether filtered or unfiltered neighboring samples are used may depend on the converted conventional intra-prediction mode. For example, this solution is similar to traditional intra-frame prediction modes.

(2) Alternatively, whether filtered or unfiltered neighbor samples are used for the ALWIP mode can be signaled.

g) In one example, filtered samples may be generated in the same way as in traditional intra-prediction modes.

27. Which matrices and/or offset vectors are used may depend on the original shape restoration information (also known as LMCS, luma mapping to chrominance scaling).

a) In one example, different matrices and/or displacement vectors may be used when reshaping is turned on and off.

b) In one example, different matrices and/or displacement vectors may be used for different shape recovery parameters.

c) In one example, ALWIP may always run in the source area.

(1) For example, the adjacent sample is mapped to the original region (if original shape recovery is applied) before being used in ALWIP.

28. ALWIP mode can be disabled when restoring the original shape is used.

a) Alternatively, restoring to original shape can be disabled if ALWIP mode is enabled.

b) In one example, ALWIP may be disabled for HDR (high dynamic range) content when reshaping is applied.

29. The matrices used in ALWIP may depend on the bit depth of the sample.

30. It is assumed that the prediction generated by ALWIP can be considered as an intermediate signal that will be processed to obtain a prediction signal that will be used further.

a) In one example, a combination of position-dependent intra-frame prediction (PDPC) may be applied to the prediction generated by ALWIP to generate a prediction signal to be used subsequently.

b) In one example, the prediction of boundary samples produced by ALWIP can be filtered by neighboring samples to produce a prediction signal that will be used later.

31. It is proposed to use interpolation filters other than the bilinear interpolation filter in the ALWIP upsampling process.

a) In one example, 4-tap interpolation filters may be used in the ALWIP upsampling process.

(1) For example, 4-tap interpolation filters in VVC, used for motion compensation for chrominance components, can be used in the ALWIP upsampling process.

(2) For example, 4-tap interpolation filters in VVC used for angular intra-frame prediction can be used in the ALWIP upsampling process.

(3) For example, 8-tap interpolation filters in VVC used for motion compensation for the luminance component can be used in the ALWIP upsampling process.

The examples described above may be included in the context of the methods described below, for example, methods 1100-1400 and 2000-3800, which may be implemented in a video encoder and/or decoder.

In fig. 11 is a flowchart of an exemplary video processing method. Method 1100 includes, at step 1102, determining that the current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode.

Method 1100 includes, at step 1104, constructing, based on the determination of at least a portion of the MPM list for the ALWIP mode, based on at least a portion of the MPM list for the intra-ALWIP non-ALWIP mode.

Method 1100 includes, at step 1106, performing, based on the ALWIP mode MPM list, a conversion between the current video block and a bitstream representation of the current video block.

In some embodiments, the size of the ALWIP mode MPM list is identical to the size of the intraframe non-ALWIP mode MPM list. In the example, the ALWIP mode MPM list size is 6.

In some embodiments, method 1100 further comprises the step of inserting default modes into the MPM list for the ALWIP mode. In the example, the default modes are inserted before a portion of the MPM list for the ALWIP mode, which is based on the MPM list for the intraframe non-ALWIP mode. In another example, default modes are inserted after a portion of the MPM list for the ALWIP mode, which is based on the MPM list for the intraframe non-ALWIP mode. In yet another example, default modes are inserted by interleaving a portion of the MPM list for the ALWIP mode, which is based on the MPM list for the intraframe non-ALWIP mode.

In some embodiments, the construction of the MPM list for ALWIP mode and the MPM list for intraframe non-ALWIP mode is based on one or more neighboring blocks.

In some embodiments, the composition of the MPM list for ALWIP mode and the MPM list for intra-ALWIP mode is based on the height or width of the current video block.

In some embodiments, the MPM listing for the ALWIP mode is based on a first set of parameters that is different from the second set of parameters used to construct the MPM listing for the intra-ALWIP non-ALWIP mode.

In some embodiments, method 1100 further includes the step of determining that an adjacent block of the current video block was encoded using the ALWIP mode, and designating, when compiling an MPM list for the non-ALWIP intraframe mode, the adjacent block as unavailable.

In some embodiments, method 1100 further includes the step of determining that an adjacent block of the current video block was encoded using an intra-frame non-ALWIP mode, and designating, when compiling an MPM list for the ALWIP mode, the adjacent block as unavailable.

In some embodiments, the non-ALWIP intra-frame mode is based on a normal intra-frame mode, a multi-reference line (MRL) intra-frame prediction mode, or an intra-frame subpartition (ISP) tool.

In fig. 12 is a flowchart of an exemplary video processing method. Method 1200 includes, at step 1210, determining that the luminance component of the current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode.

Method 1200 includes, at 1220, inferring, based on the chroma determination, regarding an intra-frame mode.

Method 1200 includes, at step 1230, performing, based on the intra-frame chroma mode, a conversion between the current video block and a bitstream representation of the current video block.

In some embodiments, the luma component spans a given chrominance count of the chrominance component. In the example, the specified chrominance signal sample is the top left sample or center sample of the chrominance component.

In some embodiments, the intended intra-frame chroma mode is DM mode.

In some embodiments, the intended intra-frame chroma mode is ALWIP mode.

In some embodiments, the ALWIP mode is applied to one or more color components of the current video block.

In some embodiments, different ALWIP mode bias matrices or vectors are applied to different color components of the current video block. In one example, different matrices or displacement vectors are jointly defined for the Cb and Cr components. In another example, the Cb and Cr components are combined. In yet another example, the Cb and Cr components alternate.

In fig. 13 is a flowchart of an exemplary video processing method. Method 1300 includes, at step 1302, determining that the current video block is encoded using an affine linear weighted intra-prediction (ALWIP) mode.

Method 1300 includes, at step 1304, performing, based on the determination, a conversion between the current video block and a bitstream representation of the current video block.

In some embodiments, the determination is based on signaling in a sequence parameter set (SPS), picture parameter set (PPS), slice header, tile group header, tile header, coding tree unit (CTU) line, or CTU area.

In some embodiments, the determination is based on the height (H) or width (W) of the current video block. For example, W > T1 or H > T2. In another example, W ≥ T1 or H ≥ T2. In another example, W < T1 or H < T2. In another example, W ≤ T1 or H ≤ T2. In another example, T1 = 32 and T2 = 32.

In some embodiments, the determination is based on the height (H) or width (W) of the current video block. In one example, W+H ≤ T. In another example, W+H ≥ T. In yet another example, W×H ≤ T. In yet another example, W×H ≥ T. In yet another example, T = 256.

In fig. 14 is a flowchart of an exemplary video processing method. Method 1400 includes, at step 1402, determining that the current video block is encoded using an encoding mode that is other than an affine linear weighted intra-prediction (ALWIP) mode.

Method 1400 includes, at step 1404, performing, based on the determination, a conversion between the current video block and a bitstream representation of the current video block.

In some embodiments, the encoding mode is a combined intra-inter-prediction (CIIP) mode, and method 1400 further includes the step of selecting between an ALWIP mode and a normal intra-prediction mode. In one example, making a selection is based on explicit signaling in the bitstream representation of the current video block. In another example, the selection is made based on a given rule. In yet another example, the specified rule always selects the ALWIP mode when the current video block is encoded using the CIIP mode. In another example, the specified rule always selects the normal intra-prediction mode when the current video block is encoded using the CIIP mode.

In some embodiments, the encoding mode is a cross-component linear model (CCLM) prediction mode. In one example, the downsampling procedure for the ALWIP mode is based on the downsampling procedure for the CCLM prediction mode. In another example, the downsampling procedure for the ALWIP mode is based on a first set of parameters, and where the downsampling procedure for the CCLM prediction mode is based on a second set of parameters different from the first set of parameters. In yet another example, a downsampling procedure for an ALWIP mode or a CCLM prediction mode comprises at least one of: selecting downsampling positions, selecting downsampling filters, a rounding operation, or a discarding operation.

In some embodiments, method 1400 further includes the step of applying one or more of a reduced secondary transform (RST), a secondary transform, a rotation transform, or a non-separated secondary transform (NSST).

In some embodiments, method 1400 further includes the step of applying block differential pulse code modulation (DPCM) or residual DPCM.

6. Examples of implementation of the disclosed technology

In fig. 15 is a block diagram of a video processing apparatus 1500. Apparatus 1500 may be used to implement one or more of the methods described herein. The device 1500 may be embodied as a smartphone, tablet, computer, Internet of Things (IoT) receiver, etc. Apparatus 1500 may include one or more processors 1502, one or more memory devices 1504, and video processing hardware 1506. Processor(s) 1502 may be configured to implement one or more methods (including, but not limited to, methods 1100-1400 and 2000-3800) described herein. Memory(s) 1504 may be used to store data and code used to implement the methods and technologies described herein. Video processing hardware 1506 may be used to implement, in the form of hardware circuits, some of the technologies described herein.

In some embodiments, video encoding methods may be implemented using a device that is implemented on a hardware platform, as described with reference to FIG. 15.

In fig. 16 is a block diagram illustrating an example of a video processing system 1600 in which various technologies disclosed herein may be implemented. Various implementations may include some or all of the components of the system 1600. The system 1600 may include an input 1602 for receiving video content. Video content may be received in a raw or uncompressed format, such as 8- or 10-bit multi-pixel values, or may be presented in a compressed or encoded format. Input 1602 may represent a network interface, a peripheral bus interface, or a storage device interface. Examples of a network interface include wired interfaces such as Ethernet, passive optical network (PON), etc., and wireless interfaces such as Wi-Fi or cellular interfaces.

System 1600 may include an encoding component 1604 that may implement various encoding or encryption methods described herein. The encoding component 1604 allows the average video bit rate from the input 1602 to the output of the encoding component 1604 to be reduced to create an encoded representation of the video. Therefore, encoding technologies are sometimes called video compression technologies or video transcoding technologies. The output of encoding component 1604 may be either stored or transmitted via a connected link, as represented by component 1606. The stored or transmitted bitmap (or encoded) representation of the video received at input 1602 may be used by component 1608 to generate pixel values or display video that sent to display interface 1610. The process of producing user-viewable video from a bitstream representation is sometimes called video decompression. In addition, although certain video processing operations are referred to as "encoding" operations or tools, it should be understood that the encoding tools or operations are used in the encoder, and the corresponding decoding tools or operations that reverse the encoding results will be performed by the decoder.

Examples of the peripheral bus interface or display interface may include a universal serial bus (USB), a high-definition multimedia interface (HDMI), or a display port (Displayport), etc. Examples of storage device interfaces include SATA (Serial Advanced Technology), PCI, IDE interface, etc. The technologies described herein may be implemented in various electronic devices, such as mobile phones, laptops, smartphones, or other devices that are capable of digital data processing and/or video display.

Some embodiments of the disclosed technology include deciding or determining whether to enable a video processing tool or mode. In one example, when a video processing tool or mode is enabled, the encoder will use or implement the tool or mode when processing a video block, but may not necessarily change the resulting bitstream based on the use of the tool or mode. That is, the conversion from a video block to a video bitstream representation will use the video processing tool or mode when enabled, based on the decision or definition. In another example, when a video processing tool or mode is enabled, the decoder will process the bitstream knowing that the bitstream has been modified based on the video processing tool or mode. That is, the conversion from a video bitstream representation to a video block will be performed using the video processing tool or mode that was enabled based on the decision or determination.

Some embodiments of the disclosed technology include deciding or determining whether to disable a video processing tool or mode. In one example, when a video processing tool or mode is disabled, the encoder will not use the tool or mode when converting a video block to a video bitstream representation. In another example, when a video processing tool or mode is disabled, the decoder will process the bitstream knowing that the bitstream has not been modified using the video processing tool or mode that was disabled based on the decision or determination.

In fig. 17 is a block diagram that illustrates an example video encoding system 100 that may use the methods of the present disclosure. As shown in FIG. 17, the video encoding system 100 may include a source device 110 and a destination device 120. The source device 110, which may be referred to as a video encoder, produces encoded video data. A destination device 120, which may be referred to as a video decoder, may decode encoded video data produced by a source device 110. The source device 110 may include a video source 112, a video encoder 114, and an input/output (I/O) interface 116.

Video source 112 may include a source such as a video capture device, an interface for receiving video data from a video content provider, and/or a computer graphics system for generating video data, or a combination of such sources. Video data may contain one or more images. Video encoder 114 encodes video data from video source 112 to produce a bitstream. The bit stream may include a sequence of bits that form an encoded representation of video data. The bitstream may include encoded images and associated data. A coded image is a coded representation of an image. Associated data may include sequence parameter sets, image parameter sets, and other syntactic structures. I/O interface 116 may include a modulator/demodulator (modem) and/or a transmitter. The encoded video data may be transmitted directly to the recipient device 120 via I/O interface 116 via network 130a. The encoded video data may also be stored on storage medium/server 130b for access by recipient device 120.

The recipient device 120 may include an input/output interface 126, a video decoder 124, and a display device 122.

I/O interface 126 may include a receiver and/or a modem. I/O interface 126 may receive encoded video data from source device 110 or media/server 130b. Video decoder 124 may decode encoded video data. The display device 122 may display the decoded video data to the user. Display device 122 may be integrated with target device 120 or may be external to target device 120 that is configured to interface with the external display device.

Video encoder 114 and video decoder 124 may operate in accordance with a video compression standard, such as the High Efficiency Video Coding (HEVC) standard, the Versatile Video Coding (VVM) standard, and other existing and/or additional standards.

In fig. 18 is a block diagram illustrating an example of video encoder 200, which may be video encoder 114 in the system 100 illustrated in FIG. 17.

Video encoder 200 may be configured to perform any or all of the methods of the present disclosure. In the example shown in FIG. 18, video encoder 200 includes a variety of functional components. The methods described in the present disclosure may be shared among various components of video encoder 200. In some examples, the processor may be configured to perform any or all of the methods described in the present disclosure.

Functional components of the video encoder 200 may include a division block 201, a prediction block 202, which may include a mode selection block 203, a motion estimation block 204, a motion compensation block 205 and an intra-frame prediction block 206, a residual generation block 207, a transform block 208, a quantization unit 209, an inverse quantization unit 210, an inverse transform unit 211, a reconstruction unit 212, a buffer 213, and an entropy encoding unit 214.

In other examples, video encoder 200 may include more or fewer functional components, or different functional components. In an example, prediction block 202 may include an intra-block copy (IBC) block. The IBC block may perform prediction in an IBC mode in which at least one reference image is an image on which the current video block is located.

In addition, some components, such as motion estimation block 204 and motion compensation block 205, can be implemented with a high degree of integration, but in the example shown in FIG. 18, they are presented separately for ease of explanation.

The division unit 201 may divide the image into one or more video blocks. Video encoder 200 and video decoder 300 may support different video block sizes.

The mode selection unit 203 may select one of the encoding modes, intra-frame or inter-frame, for example, based on the error results, and provide the resulting intra-frame or inter-frame coded block to a residual generation unit 207 to generate the residual block data and to a recovery unit 212 to reconstruct the encoded block with for use as a reference image. In some example, the mode selector 203 may select a combination of an intra-frame and inter-frame prediction (CIIP) mode in which the prediction is based on an inter-frame prediction signal and an intra-frame prediction signal. The mode selector 203 may also select a motion vector resolution (eg, sub-pixel or integer pixel precision) for the block in the case of inter-frame prediction.

To perform inter-frame prediction for the current video block, motion estimator 204 may generate motion information for the current video block by comparing one or more reference frames from buffer 213 to the current video block. Motion compensation unit 205 may determine a predicted video block for the current video block based on motion information and decoded image samples from buffer 213 that are different from the image associated with the current video block.

The motion estimation unit 204 and the motion compensation unit 205 may perform different operations for the current video block, for example, depending on whether the current video block is in an I-slice, a P-slice, or a B-slice.

In some examples, motion estimator 204 may perform unidirectional prediction for the current video block, and motion estimator 204 may retrieve reference images from list 0 or list 1 for the reference video block for the current video block. The motion estimator 204 may then generate a reference index that indicates a reference image in list 0 or list 1 that contains the reference video block, and a motion vector that indicates a spatial offset between the current video block and the reference video block. The motion estimation unit 204 may output the reference index, the prediction direction indicator, and the motion vector as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current block based on the reference video block specified in the motion information of the current video block.

In other examples, motion estimator 204 may perform bidirectional prediction for the current video block, motion estimator 204 may look up reference images in list 0 for a reference video block for the current video block, and can look up reference images in list 1 for another reference video block for the current video block. Motion estimator 204 may then generate reference indices that indicate reference pictures in list 0 and list 1 containing reference video blocks and motion vectors that indicate spatial offsets between the reference video blocks and the current video block. The motion estimation unit 204 may output reference indices and motion vectors of the current video block as motion information of the current video block. The motion compensation unit 205 may generate a predicted video block of the current video block based on the reference video blocks specified in the motion information of the current video block.

In some examples, motion estimator 204 may output a complete set of motion information for the decoder decoding process.

In some examples, motion estimator 204 may not output the full set of motion information for the current video. Rather, motion estimator 204 may signal motion information of the current video block with reference to motion information of another video block. For example, motion estimator 204 may determine that the motion information of the current video block is sufficiently similar to the motion information of an adjacent video block.

In one example, motion estimator 204 may indicate, in a syntax structure associated with the current video block, a value that indicates to video decoder 300 that the current video block has the same motion information as another video block.

In another example, motion estimator 204 may identify, in the syntax structure associated with the current video block, another video block and a motion vector difference (MVD). The motion vector difference indicates the difference between the motion vector of the current video block and the motion vector of the specified video block. Video decoder 300 may use the motion vector of the specified video block and the difference of the motion vectors to determine the motion vector of the current video block.

As discussed above, video encoder 200 may predictively signal a motion vector. Two examples of prediction signaling techniques that may be implemented by video encoder 200 include enhanced motion vector prediction (AMVP) and merge mode signaling.

The intra-frame prediction unit 206 may perform intra-frame prediction for the current video block. When the intra-frame prediction unit 206 performs intra-frame prediction on the current video block, the intra-frame prediction unit 206 may generate prediction data for the current video block based on decoded samples of other video blocks in the same frame. The prediction data for the current video block may include the predicted video block and various syntax elements.

Residual generating unit 207 may generate residual data for the current video block by subtracting (eg, indicated by a minus sign) the predicted video block(s) of the current video block from the current video block. The current video block residual data may include residual video blocks that correspond to various sample components in the current video block.

In other examples, there may be no residual data for the current video block for the current video block, for example, in a skip mode, and the residual generating unit 207 may not perform the subtraction operation.

Transform processing unit 208 may generate one or more video blocks of transform coefficients for the current video block by applying one or more transforms to the remainder video block associated with the current video block.

After transform processing unit 208 produces a transform coefficient video block associated with the current video block, quantization unit 209 may quantize the transform coefficient video block associated with the current video block based on one or more quantization parameter (QP) values associated with the current video block.

The inverse quantization unit 210 and the inverse transform unit 211 may apply the inverse quantization and inverse transform to the transform coefficient video block, respectively, to recover the remainder video block from the transform coefficient video block. Reconstruction unit 212 may add the reconstructed residual video block to corresponding samples from one or more predicted video blocks generated by prediction unit 202 to obtain a reconstructed video block associated with the current block for storage in buffer 213.

After the reconstruction unit 212 restores the video block, a contour filtering operation may be performed to reduce video block artifacts in the video block.

Entropy encoding block 214 may receive data from other functional components of video encoder 200. When entropy encoding block 214 receives data, entropy encoding block 214 may perform one or more entropy encoding operations to generate entropy-encoded data and output a bitstream that includes entropy encoding -encoded data.

In fig. 19 is a block diagram illustrating an example of video decoder 300, which may be video decoder 114 in the system 100 illustrated in FIG. 17.

Video decoder 300 may be configured to perform any or all of the methods of the present disclosure. In the example shown in FIG. 19, video decoder 300 includes a variety of functional components. The methods described in the present disclosure may be shared among various components of the video decoder 300. In some examples, the processor may be configured to perform any or all of the methods described in the present disclosure.

In the example shown in FIG. 19, video decoder 300 includes an entropy decoding block 301, a motion compensation block 302, an intra-frame prediction block 303, an inverse quantization block 304, an inverse transform block 305, a reconstruction block 306, and a buffer 307. In some examples, the video decoder 300 may perform a decoding step, in generally the reverse of the encoding step described with respect to video encoder 200 (FIG. 18).

The entropy decoding unit 301 may extract the encoded bitstream. The coded bitstream may include entropy-coded video data (eg, coded blocks of video data). The entropy decoding unit 301 can decode the entropy-coded video data, and from the entropy-decoded video data, the motion compensation unit 302 can determine motion information including motion vectors, motion vector precision, reference picture list indices, and other motion information. The motion compensation unit 302 may, for example, determine such information by performing AMVP and merging mode.

Motion compensation block 302 may create motion compensation blocks, possibly performing interpolation based on interpolation filters. Identifiers for interpolation filters that must be used with sub-pixel precision can be included in syntax elements.

Motion compensation block 302 may use interpolation filters that are used by video encoder 20 during encoding of a video block to calculate interpolated values for sub-integer pixels of the reference block. The motion compensation unit 302 may determine the interpolation filters used by the video encoder 200 in accordance with the received syntax information and use the interpolation filters to create prediction blocks.

Motion compensation block 302 may use some syntax information to determine the block sizes used to encode the frame(s) and/or slice(s) of the encoded video sequence, partition information that describes how each image macroblock of the encoded video sequence is divided into partitions, modes , indicating how each section is encoded, one or more reference frames (and lists of reference frames) for each inter-frame coded block, and other information for decoding the coded video sequence.

Intra-frame prediction block 303 may use intra-frame prediction modes, such as those received in a bitstream, to form a prediction block from spatially contiguous blocks. The inverse quantization unit 303 performs inverse quantization, that is, the inverse operation of quantization, on the quantized video block coefficients provided in the bit stream and decoded by the entropy decoding unit 301. The inverse transform block 303 applies the inverse transform.

The recovery unit 306 may add the remainder blocks with the corresponding prediction blocks generated by the motion compensation unit 202 or the intra-frame prediction unit 303 to generate decoded blocks. If necessary, the deblocking filter can also be used to filter the decoded blocks to remove blocking artifacts. The decoded video blocks are then stored in a buffer 307, which provides reference blocks for subsequent motion compensation/intra-frame prediction and also produces decoded video for presentation on a display device screen.

In some embodiments, ALWIP mode or MIP mode is used to compute a prediction block of the current video block by performing, on previously encoded video samples, an edge downsampling operation (or averaging operation), followed by a matrix-vector multiplication operation, and followed by a selective (or if necessary) an upsampling operation (or a linear interpolation operation) follows. In some embodiments, ALWIP mode or MIP mode is used to compute a prediction block of the current video block by performing, on previously encoded video samples, an edge downsampling operation (or averaging operation) followed by a matrix-vector multiplication operation. In some embodiments, the ALWIP mode or MIP mode may also perform an upsampling operation (or a linear interpolation operation) after performing a matrix-vector multiplication operation.

In fig. 20 shows an example flowchart of an example method 2000 for matrix intra-frame prediction. Operation 2002 includes generating, for converting between the current video block of video and an encoded representation of the current video block, a first most probable mode list (MPL) using a first rule-based procedure. Operation 2004 includes performing a transformation between the current video block and a coded representation of the current video block using a first MPM list, where the transformation of the current video block uses a matrix intra-prediction (MIP) mode in which the prediction block of the current video block is determined by performing, on previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation, where the rule specifies that the first procedure used to generate the first MPM list is the same as the second procedure used to generate the second list MPM for converting other video video blocks that are encoded using an intra-frame mode without using MIP, which is different from the MIP mode, and where at least a portion of the first MPM list is compiled based on at least a portion of the second MPM list.

In some embodiments of method 2000, the size of the first MPM list for MIP mode is the same as the size of the second MPM list for intra-frame mode without using MIP. In some embodiments of method 2000, the size of the first MPM list for the MIP mode and the second MPM list for the non-MIP intra-frame mode is 6. In some embodiments of the method 2000, the size of the second MPM list for the non-MIP intra-frame mode is compiled before the first MPM list for the mode is compiled. MIP. In some embodiments of method 2000, part or all of the second non-MIP intra-frame mode MPM list is converted to MPMs that are added to part of the first MIP mode MPM list.

In some embodiments of method 2000, a subset of MPMs is added to a portion of the first MPM list for the MIP mode by abbreviating the MPM. In some embodiments, method 2000 further includes adding default intra-prediction modes to the first MPM list for the MIP mode. In some embodiments of method 2000, default intra-prediction modes are added to the first MIP mode MPM list before a portion of the first MIP mode MPM list that is based on a portion of the second MIP non-MIP mode MPM list. In some embodiments of method 2000, default intra-prediction modes are added to the first MIP mode MPM list after a portion of the first MIP mode MPM list that is based on a portion of the second non-MIP intra-mode MPM list. In some embodiments of method 2000, default intra-prediction modes are added to the first MPM list for the MIP mode interleaved with a portion of the first MPM list for the MIP mode that is based on a portion of the second MPM list for the non-MIP intra-mode. In some embodiments of method 2000, the default intra-frame prediction modes are the same for multiple types of video blocks. In some embodiments of method 2000, default intra-frame prediction modes are determined based on the encoded information of the current video block. In some embodiments of method 2000, the encoded information includes the availability of neighboring video blocks, mode information of neighboring video blocks, or the block size of the current video block.

In some embodiments of method 2000, one intra-frame prediction mode in a second MPM list for a non-MIP intra-frame mode is mapped to its corresponding MIP mode to produce a mapped MIP mode that is added to the first MPM list for the MIP mode. In some embodiments of method 2000, all intra-prediction modes in a second MPM list for a non-MIP intra-frame mode are mapped to its corresponding MIP mode to obtain a plurality of mapped MIP modes that are added to the first MPM list for the MIP mode. In some embodiments of method 2000, all intra-prediction modes are mapped to their corresponding MIP mode to obtain a plurality of mapped MIP modes that are used to construct a second list of MPMs for the non-MIP intra-frame mode. In some embodiments of method 2000, all intra-prediction modes include intra-prediction modes from adjacent video blocks of the current video block and default intra-prediction modes. In some embodiments of method 2000, default intra prediction modes include planar mode and direct current (DC) mode.

In some embodiments of method 2000, two intra-frame prediction modes in the second MPM list for the non-MIP intra-frame mode are mapped to their corresponding MIP modes to produce two mapped MIP modes, and one of the two mapped MIP modes is added to the first MPM list for the MIP mode in response. that these two converted MIP modes are the same. In some embodiments of method 2000, two intra-prediction modes in a second MPM list for the non-MIP intra-mode are mapped to their corresponding MIP modes to produce two mapped MIP modes, and one of the two mapped MIP modes is added to the second MPM list for the non-MIP intra-mode. MIP in response to the fact that the two converted MIP modes are the same. In some embodiments of method 2000, the second MPM list for the non-MIP intra-frame mode includes an S intra-frame prediction mode, and where K of the S intra-frame prediction mode is selected or selected for inclusion in the first MPM list for the MIP mode. In some embodiments of method 2000, K is 3 and S is 6. In some embodiments of method 2000, the first K intra-prediction modes in the second MPM list for the non-MIP intra-frame mode are selected for inclusion in the first MPM list for the MIP mode. In some embodiments of method 2000, a first MPM list for the MIP mode and a second MPM list for the non-MIP intra-frame mode are derived or compiled based on one or more adjacent video blocks of the current video block. In some embodiments of method 2000, the first MPM list for the MIP mode and the second MPM list for the intra-frame non-MIP mode are derived or compiled from the same adjacent video block located to the left of the current video block.

In some embodiments of method 2000, the same left adjacent video block is immediately to the left of the current video block and is aligned with the current video block. In some embodiments of method 2000, the same adjacent video block is located to the left and below the current video block. In some embodiments of method 2000, the first MPM list for the MIP mode and the second MPM list for the intra-frame mode without using MIP are derived or compiled based on the same neighboring video block, where the same neighboring video block is located to the left of the current video block in the lowest position , or the same adjacent video block is located above the current video block in the rightmost position. In some embodiments of method 2000, the first MPM list for the MIP mode and the second MPM list for the intra-frame mode without using MIP are derived or compiled based on the same adjacent video block located above the current video block.

In some embodiments of method 2000, the same adjacent video block located above is directly above the current video block and is aligned with the current video block. In some embodiments of method 2000, the same adjacent video block is located to the left and above the current video block. In some embodiments of method 2000, the first MPM list for the MIP mode and the second MPM list for the intra-frame mode without using MIP are derived or compiled based on the same neighboring video block, where the same neighboring video block is located above the current video block in the leftmost position, or the same adjacent video block is located to the left of the current video block in the uppermost position. In some embodiments of method 2000, the non-MIP intra-frame mode is based on an intra-frame prediction mode, a multi-reference line (MRL) intra-frame prediction mode, or an intra-subsection (ISP) tool. In some embodiments of method 2000, the first MPM list for the MIP mode is further based on the height or width of the current video block.

In some embodiments of method 2000, the first MPM list for the MIP mode is further based on the height or width of an adjacent video block of the current video block. In some embodiments of method 2000, the first MPM list for the MIP mode is constructed based on a first set of parameters that is different from the second set of parameters used to construct the second MPM list for the non-MIP intra-frame mode. In some embodiments of method 2000, the second MPM list for the non-MIP intra-mode includes S intra-prediction modes, and where K of the S intra-prediction modes are obtained such that they are included in the first MPM list for the MIP mode. In some embodiments of the 2000 method, K is 3 and S is 6.

In some embodiments of method 2000, the first K intra-prediction modes in the second MPM list for the non-MIP intra-frame mode are obtained such that they are included in the first MPM list for the MIP mode. In some embodiments of method 2000, the first mode specified in the first MPM list for the MIP mode is different from the first mode specified in the second MPM list for the non-MIP intra-frame mode. In some embodiments of method 2000, the first mode specified in the first MPM list for the MIP mode is the first mode, and the first mode specified in the second MPM list for the non-MIP intra-frame mode is a planar mode. In some embodiments of method 2000, the first mode in the first MPM list for the MIP mode is converted from a planar mode in the second MPM list to an intra-frame mode without using the MIP.

In some embodiments of method 2000, the first MIP list for the MIP mode includes a first set of padding modes that are different from the second set of padding modes included in the second MPM list for the non-MIP intraframe mode. In some embodiments of method 2000, the first set of fill modes includes a first mode, a second mode, and a third mode, and the second set of fill modes includes a direct current (DC) mode, a vertical mode, and a horizontal mode. In some embodiments of method 2000, a first mode, a second mode, and a third mode are included in the first set of fill modes based on the size of the current video block. In some embodiments of method 2000, the first MPM list for the MIP mode includes a first set of intra-neighbor video block prediction modes of the current video block, the second MPM list for the non-MIP intra-frame mode includes a second set of intra-neighbor video block prediction modes of the current video block, and the first set intra prediction modes is different from the second set of intra prediction modes.

In some embodiments of method 2000, the second set of intra-prediction modes includes intra-prediction modes that are mapped to MIP modes that are included in the first set of intra-prediction modes. In some embodiments of method 2000, the first MPM list for the MIP mode includes a first set of shifted intra-prediction modes, the second MPM list for the non-MIP intra-frame mode includes a second set of shifted intra-prediction modes and a first set of shifted prediction modes. The intra prediction modes are different from the second set of shifted intra prediction modes. In some embodiments of method 2000, the first set of shifted intra-prediction modes includes a MIP mode (Y) shifted by K1 according to the first formula Y+K1, the second set of shifted intra-prediction modes includes a non-MIP intra-mode (X) shifted on K0 according to the second formula X+K0, and K1 differs from K0. In some embodiments of method 2000, K1 depends on the width and height of the current video block.

In some embodiments, method 2000 further includes first determining that an adjacent video block of the current video block is encoded using a non-MIP intra-frame mode; and making a second determination, in response to the first determination that the adjacent video block is not available for compiling the first MPM list for the MIP mode.

In some embodiments, method 2000 further includes first determining that an adjacent video block of the current video block is encoded using a non-MIP intra-frame mode; and making, in response to the first determination, a second determination that the adjacent video block is encoded using a predetermined intra-MIP mode, wherein a first MPM list for the MIP mode is compiled using the predetermined intra-MIP mode. In some embodiments of method 2000, a given intra-frame MIP prediction mode depends on the width and/or height of the current video block.

In fig. 21 shows an example flowchart of an example method 2100 for matrix intra-frame prediction. Operation 2100 includes generating, for transforming between the current video block of video and an encoded representation of the current video block, a list of most likely modes (MPM) based on a rule, where the rule is based on the fact that a video block of an adjacent current video block is encoded using a matrix intra-prediction (MIP) mode ) in which a neighboring video block prediction block is determined by performing, on the previously encoded video samples, an edge downsampling operation, followed by a matrix-vector multiplication operation, and which is selectively followed by an upsampling operation. Operation 2104 includes performing a transformation between the current video block and an encoded representation of the current video block using the MPM list, where the transformation applies a non-MIP mode to the current video block, and where the non-MIP mode is different from the MIP mode.

In some embodiments of method 2100, the rule specifies that a neighboring MIP-encoded video block is treated as unavailable for MPM listing for the current non-MIP encoded video block. In some embodiments of method 2100, the rule specifies that a neighboring video block encoded in the MIP mode is determined to be encoded using a given intra-frame prediction mode. In some embodiments of method 2100, the rule specifies that a given intra prediction mode includes a planar mode.

In fig. 22 shows an example flowchart of an example method 2200 for matrix intra-frame prediction. Operation 2202 includes decoding the current video block of video, which is encoded into an encoded representation of the current video block using a matrix intra-prediction (MIP) mode, in which the prediction block of the current video block is determined by performing, on the previously encoded video samples, an edge downsampling operation, beyond which is followed by a matrix-vector multiplication operation, and which is selectively followed by an upsampling operation. Operation 2204 includes updating a line buffer associated with decoding without storing in the line buffer information indicating whether the current video block is encoded using the MIP mode.

In some embodiments, method 2200 further includes accessing a second video block of video, where the second video block is decoded before decoding the current video block, where the second video block is located in a different row of largest coding units (LCUs) or coding tree units (CTUs) or CTU area compared to with that of the current video block, and where the current video block is decoded without determining whether the second video block is encoded using the MIP mode. In some embodiments, method 2200 further includes accessing a second video block by determining that the second video block is encoded using a non-MIP intraframe mode without determining whether the second video block is encoded using a MIP mode where the second video block is located on a different line largest coding units (LCU) or coding tree units (CTU) or CTU area compared to that of the current video block; and decoding the current video block based on access to the second video block.

In fig. 23 shows an example flowchart of an example method 2300 for matrix intra-frame prediction. Operation 2302 includes performing a conversion between the current video block and a bitstream representation of the current video block, where the current video block is encoded using a matrix intra-prediction (MIP) mode in which a prediction block of the current video block is determined by performing a downsampling operation on the previously encoded video samples. at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, where the flag is encoded into the bitstream representation using at most K contexts in the arithmetic encoding process or in the decoding process, where the flag indicates whether the current one is encoded video block using MIP mode, and where K is greater than or equal to zero. In some embodiments of the method, 2300 K is 1. In some embodiments of the method, 2300 K is 4.

In fig. 24 shows an example flowchart of an example method 2400 for matrix intra-frame prediction. Operation 2402 includes generating, to transform between the current video video block and the bitstream representation of the current video block, an intra-frame mode prediction for the current video block encoded using a matrix intra-frame prediction (MIP) mode, in which a prediction block of the current video block is determined by performing on previously encoded ones. video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. Operation 2404 includes defining a rule for storing information indicating an intra-frame prediction mode based on whether the current video block is encoded using the MIP mode. Operation 2406 includes performing a conversion in accordance with a rule, where the rule specifies that the syntax element for the intra-frame prediction mode is stored in the bitstream representation for the current video block, and where the rule specifies that the mode index for the MIP mode for the current video block is not stored in the bitstream representation.

In some embodiments of method 2400, a mode index for the MIP mode is associated with an intra-frame prediction mode. In some embodiments of method 2400, the rule specifies that the bitstream representation excludes a flag that indicates that the current video block is encoded using the MIP mode. In some embodiments of method 2400, the rule specifies that the bitstream representation excludes storing information indicating the MIP mode associated with the current video block. In some embodiments, method 2400 further includes performing, after transforming, a second transform between a second video block of video and a bitstream representation of the second video block, where the second video block is an adjacent video block of the current video block, and where the second transform is performed without determining whether the second video block is encoded using MIP mode.

In fig. 25 shows an example flowchart of an example method 2500 for matrix intra-frame prediction. Operation 2502 includes making a first determination that the video luminance block is encoded using a matrix intra-prediction (MIP) mode, in which the luminance video block prediction block is determined by performing, on the previously encoded video samples, an edge downsampling operation followed by a matrix-vector multiplication operation, and which is selectively followed by an upsampling operation. Operation 2504 includes making, based on the first determination, a second determination regarding the intra-frame chrominance mode to be used for the chrominance video block associated with the luma video block. Operation 2506 includes performing, based on the second definition, a conversion between a chroma video block and a bitstream representation of the chroma video block.

In some embodiments of method 2500, making the second determination includes determining that the intra-frame chroma mode is a derived mode (DM) regardless of signaling. In some embodiments of method 2500, a luminance video block covers a given corresponding chrominance sample of the chrominance video block. In some embodiments of method 2500, the specified corresponding chroma sample is the top left sample or center sample of a chroma video block. In some embodiments of method 2500, the intraframe chroma mode includes a derived mode (DM). In some embodiments of the method 2500, DM is used to derive a first intra-prediction mode for a chroma video block from a second intra-prediction mode for a luminance video block. In some embodiments of method 2500, the MIP mode associated with a luma video block is mapped to a predetermined intra-frame mode.

In some embodiments of method 2500, a plurality of derived modes (DMs) are obtained in response to a first determination that a luma video block is encoded using a MIP mode. In some embodiments of method 2500, the intra-frame chroma mode includes a well-defined intra-frame prediction mode. In some embodiments of method 2500, the specified intra-frame mode or said intra-frame mode is a planar mode. In some embodiments of method 2500, the chrominance video block is encoded using the MIP mode. In some embodiments of method 2500, a chrominance video block is encoded using a first matrix or a first offset vector that is different from a second matrix or a second offset vector of the second chroma video block. In some embodiments of method 2500, a chrominance video block is a blue component, where a first matrix or first offset vector is defined for the blue component, where a second chrominance video block is a red component, where a second matrix or second offset vector is defined for a red component.

In some embodiments of method 2500, a blue component and a red component are combined. In some embodiments of method 2500, the blue and red color components alternate. In some embodiments of method 2500, the chrominance video block is encoded using a MIP mode that is the same as the MIP mode used for the luma video block. In some embodiments of method 2500, the chrominance video block is encoded using a derivative mode (DM). In some embodiments of method 2500, the upsampling operation (or linear interpolation method) is skipped after the chroma video block is encoded using the MIP mode. In some embodiments of method 2500, a chrominance video block is encoded using a MIP mode with a subsampled matrix and/or offset vector. In some embodiments of method 2500, the number of MIP modes for a luma video block and a chrominance video block is different. In some embodiments of method 2500, a chroma video block is associated with a first number of MIP modes, a luma video block is associated with a second number of MIP modes, and the first number of MIP modes is less than the second number of MIP modes. In some embodiments of method 2500, the chrominance video block and the luma video block are the same size.

In fig. 26 shows an example flowchart of an example method 2600 for matrix intra-frame prediction. Operation 2602 includes performing a transformation between the current video block of video and an encoded representation of the current video block, where the transformation is based on determining (or being determined) whether the current video block should be encoded using a matrix intra-frame prediction (MIP) mode in which the current video block's prediction block A video block is determined by performing, on the previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation.

In some embodiments of method 2600, a syntax element is included in the encoded representation in response to a determination that the current video block should be encoded in the MIP mode, and the syntax element specifies the use of the MIP mode to encode the current video block. In some embodiments of method 2600, the encoded representation includes a syntax element that indicates that the MIP mode is enabled or disabled for the current video block, and where the syntax element indicates that the MIP mode is enabled or disabled for encoding the current video block. In some embodiments of method 2600, a syntax element is not included in the encoded representation in response to a determination that the current video block should not be encoded in MIP mode, and the syntax element indicates whether MIP mode is used for the current video block.

In some embodiments of method 2600, a second syntax element indicating MIP mode enablement is further included in a sequence parameter set (SPS), picture parameter set (PPS), slice header, tile group header, tile header, coding tree units (CTU) string ) or CTU area. In some embodiments of method 2600, determining whether to use the MIP mode for conversion is based on the height (H) and/or width (W) of the current video block. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W ≥ T1 and H ≥ T2. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W ≤ T1 and H ≤ T2. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W ≥ T1 or H ≥ T2. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W ≤ T1 or H ≤ T2. In some embodiments of method 2600, T1 = 32 and T2 = 32. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W + H ≥ T. In some embodiments of method 2600 it is determined that the current video block is not encoded using the MIP mode in response to the fact that W + H ≤ T. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to the fact that W × H ≥ T. In some embodiments of method 2600, it is determined that the current video block is not encoded using the MIP mode in response to W × H ≤ T. In some embodiments of method 2600, T = 256.

In fig. 27A shows an exemplary flowchart of an exemplary video encoding method 2700A for matrix intra-frame prediction. Operation 2702A includes determining, for encoding the current video video block into a bitstream representation of the current video block in accordance with a rule, whether to use a matrix intra-prediction (MIP) mode and an encoding mode that is different from the MIP mode to encode the current video block, where the MIP mode includes determining a prediction block of the current video block by performing, on the previously encoded video samples, a boundary downsampling operation, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. Operation 2704A includes adding, based on the definition, a coded representation of the current video block to the bitstream representation.

In fig. 27B shows an exemplary flowchart of an exemplary video decoding method 2700B for matrix intra-frame prediction. Operation 2702B includes determining that the current video video block is encoded in a bitstream representation using a matrix intra-prediction (MIP) mode and an encoding mode that is different from the MIP mode, where the MIP mode includes determining a prediction block of the current video block by performing, over previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. Operation 2704B includes generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

In some embodiments of methods 2700A and/or 2700B, the encoding mode is a combined intra-interprediction (CIIP) mode, and where the method further comprises generating an intra-prediction signal for the current video block by selecting between a MIP mode and an intra-prediction mode. In some embodiments of methods 2700A and/or 2700B, making a selection is based on signaling in the bitstream representation of the current video block. In some embodiments of methods 2700A and/or 2700B, making a selection is based on a predetermined rule. In some embodiments of methods 2700A and/or 2700B, a predetermined rule selects the MIP mode in response to encoding the current video block using the CIIP mode. In some embodiments of methods 2700A and/or 2700B, a predetermined rule selects an intra-frame prediction mode in response to encoding the current video block using the CIIP mode. In some embodiments of method 2700A and/or 2700B, a predetermined rule selects a planar mode in response to encoding the current video block using the CIIP mode. In some embodiments of methods 2700A and/or 2700B, making a selection is based on information associated with neighboring video blocks of the current video block.

In some embodiments of methods 2700A and/or 2700B, the encoding mode is a cross-component linear model (CCLM) prediction mode. In some embodiments of methods 2700A and/or 2700B, a first downsampling procedure for downsampling an adjacent sample of the current video block in MIP mode uses at least a portion of a second downsampling procedure for the CCLM prediction mode in which adjacent luminance samples of the current video block are downsampled. In some embodiments of methods 2700A and/or 2700B, a first downsampling procedure for downsampling an adjacent sample of the current video block in the CCLM prediction mode uses at least a portion of a second downsampling procedure for the MIP prediction mode in which the adjacent luminance samples of the current video block are downsampled. In some embodiments of methods 2700A and/or 2700B, a first downsampling procedure for the MIP mode is based on a first set of parameters, and a first downsampling procedure for the CCLM prediction mode is based on a second set of parameters different from the first set of parameters.

In some embodiments of methods 2700A and/or 2700B, the first downsampling procedure for the MIP mode and the second downsampling procedure for the CCLM prediction mode include selecting neighboring luminance locations or selecting a downsampling filter. In some embodiments of methods 2700A and/or 2700B, adjacent luminance samples are downsampled using at least one of a selection of downsampling positions, a selection of downsampling filters, a rounding operation, or a discarding operation. In some embodiments of methods 2700A and/or 2700B, the rule specifies that block differential pulse coded modulation (BDPCM) or residual DPCM is not applied to the current video block encoded using the MIP mode. In some embodiments of methods 2700A and/or 2700B, the rule specifies that the MIP mode is prohibited from being applied to the current video block in response to the application of block differential pulse coded modulation (BDPCM) or residual DPCM.

In fig. 28 shows an example flowchart of an example method 2800 for matrix intra-frame prediction. Operation 2802 includes making a determination regarding the applicability of the edge filter to the reconstructed block of the current video block when converting between video and an encoded representation of the current video block, where the current video block is encoded using a matrix intra-frame prediction (MIP) mode, in which the prediction block of the current video block is determined by executing ,on previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. Operation 2804 includes processing the current video block as determined.

In some embodiments of method 2800, the loop filter includes a deblocking filter. In some embodiments of method 2800, the loop filter includes an adaptive sample offset (SAO). In some embodiments of method 2800, the loop filter includes an adaptive loop filter (ALF).

In fig. 29A shows an exemplary flowchart of an exemplary video encoding method 2900A for matrix intra-frame prediction. Operation 2902A includes determining, for encoding the current video block of video into a bitstream representation of the current video block, the type of neighboring samples of the current video block that will be used in accordance with the rule. Operation 2904A includes adding, based on the determination, an encoded representation of the current video block to the bitstream representation, where the current video block is encoded using a matrix intra-prediction (MIP) mode in which a prediction block of the current video block is determined by performing, on previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation.

In fig. 29B shows an exemplary flowchart of an exemplary video decoding method 2900B for matrix intra-frame prediction. Operation 2902B includes determining that the current video block of a video is encoded in a bitstream representation using a matrix intra-prediction (MIP) mode and using a neighbor sample type of the current video block in accordance with a rule, where the MIP mode includes determining a prediction block of the current video block by performing, on previously encoded video samples, an edge downsampling operation, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation. Operation 2904B includes generating a decoded representation of the current video block by parsing and decoding the bitstream representation.

In some embodiments of methods 2900A and/or 2900B, the rule specifies that the neighbor sample type is unfiltered neighbor samples. In some embodiments of methods 2900A and/or 2900B, the rule determines that the type of neighbor samples that include unfiltered neighbor samples is used for the upsampling process, the rule determines that the type of neighbor samples that include unfiltered neighbor samples is used for the downsampling process, and the rule specifies that the type of neighbor samples, which includes unfiltered neighbor samples, is used for the matrix-vector multiplication operation. In some embodiments of methods 2900A and/or 2900B, the rule specifies that the neighbor sample type is filtered neighbor samples. In some embodiments of methods 2900A and/or 2900B, the rule determines that the type of neighbor samples that include unfiltered neighbor samples is used for the upsampling method, and the rule determines that the type of neighbor samples that include filtered neighbor samples , is used for the downsampling method.

In some embodiments of methods 2900A and/or 2900B, the rule determines that the type of neighbor samples that include unfiltered neighbor samples is used for the downsampling method, and the rule determines that the type of neighbor samples that include filtered neighbor samples , is used for upsampling. In some embodiments of methods 2900A and/or 2900B, the rule determines that the type of neighbor samples that include unfiltered top neighbor samples is used for the upsampling method, and the rule determines that the type of neighbor samples that include filtered left adjacent samples, used for the upsampling method. In some embodiments of methods 2900A and/or 2900B, the rule determines that the type of neighbor samples that include the unfiltered left neighbor samples is used for the upsampling method, and the rule determines that the type of neighbor samples that include the filtered ones above adjacent samples, used for the upsampling method.

In some embodiments of methods 2900A and/or 2900B, the neighbor sample type includes unfiltered neighbor samples or filtered neighbor samples, and the rule determines whether unfiltered neighbor samples or filtered neighbor samples are used based on the MIP mode of the current video block.

In some embodiments, methods 2900A and/or 2900B further comprise converting the MIP mode to an intra-frame prediction mode, where the neighbor sample type includes unfiltered neighbor samples or filtered neighbor samples, and where a rule determines whether unfiltered neighbor samples or filtered neighbor samples will be used. samples based on intra-frame prediction mode. In some embodiments of methods 2900A and/or 2900B, the neighbor sample type includes unfiltered neighbor samples or filtered neighbor samples, and a rule determines whether unfiltered neighbor samples or filtered neighbor samples are used based on a syntax element or signaling. In some embodiments of methods 2900A and/or 2900B, filtered neighbor samples or filtered left neighbor samples or filtered top neighbor samples are generated using an intra-frame prediction mode.

In fig. 30 shows an example flowchart of an example method 3000 for matrix intra-frame prediction. Operation 3002 includes performing a transformation between the current video block of video and a bitstream representation of the current video block, where the transformation includes generating, using matrix intra-prediction (MIP) mode, a prediction block for the current video block by selecting and applying matrix multiplication using sample matrices and/or by selecting and adding offset using offset vectors for the current video block, where the samples are obtained by averaging across rows and columns of previously encoded video samples, and where the selection is based on the original shape recovery information associated with the luminance transform mapping using the scaling method chrominance (LMCS) with a reference image of the current video block. In some embodiments of method 3000, when the original shape changing method is enabled for a reference image, the original shape changing information is associated with matrices and/or offset vectors that are different from those associated with the original shape changing information when the original shape changing method is disabled for reference image. In some embodiments of method 3000, different matrices and/or offset vectors are used for different original shape change parameters of the original shape change information. In some embodiments of method 3000, performing the transformation includes performing intra-frame prediction on the current video block using a MIP mode in the source region. In some embodiments of method 3000, adjacent samples of the current video block are mapped to the original region in response to application of the original shape change information, and where the adjacent samples are mapped to the original region prior to use in the MIP mode.

In fig. 31 shows an example flowchart of an example method 3100 for matrix intra-frame prediction. Operation 3102 includes determining that the current video block should be encoded using a matrix intra-prediction (MIP) mode, in which the prediction block of the current video block is determined by performing, on the previously encoded video samples, an edge downsampling operation followed by an matrix-vector multiplication followed selectively by an upsampling operation. Operation 3104 includes performing, based on the determination, a transformation between the current video block and a bitstream representation of the current video block, where performing the transformation is based on a rule of joint application of the MIP mode and another encoding method.

In some embodiments of method 3100, the rule specifies that the MIP mode and the other encoding method are used exclusively mutually. In some embodiments of method 3100, the rule specifies that the other encoding method is a high dynamic range (HDR) method or a reshaping method.

In fig. 32 shows an example flowchart of an example method 3200 for matrix intra-frame prediction. Operation 3202 includes performing a conversion between the current video block of a video and a bitstream representation of the current video block using a matrix intra-prediction (MIP) mode, where performing a conversion using the MIP mode includes generating a prediction block by applying a matrix multiplication using the sample matrices obtained resulting from averaging over rows and columns of previously encoded video samples, and where the matrices depend on the bit depth of the sample.

In fig. 33 shows an example flowchart of an example method 3300 for matrix intra-frame prediction. Operation 3302 includes generating an intermediate prediction signal for the current video block using a matrix intra-prediction (MIP) mode in which the prediction block of the current video block is determined by performing, on the previously encoded video samples, an edge downsampling operation followed by a matrix operation. -vector multiplication followed selectively by an upsampling operation. Operation 3304 includes generating, based on the intermediate prediction signal, a final prediction signal. Operation 3306 includes performing, based on the final prediction signal, a conversion between the current video block and a bitstream representation of the current video block.

In some embodiments of method 3300, generation of the final prediction signal is accomplished by applying a position-dependent intra-frame prediction pattern (PDPC) to the intermediate prediction signal. In some embodiments of method 3300, generating the final prediction signal is accomplished by generating previously encoded video samples using the MIP mode and by filtering the previously encoded samples using adjacent samples of the current video block.

In fig. 34 shows an example flowchart of an example method 3400 for matrix intra-frame prediction. Operation 3402 includes performing a transformation between the current video block of video and a bitstream representation of the current video block using a matrix intra-prediction (MIP) mode, where performing the transformation includes using an interpolation filter in the upsampling process for the MIP mode, where, in the MIP mode, a matrix multiplication is applied to the first set of samples resulting from row and column averaging of the previously encoded video samples, and an interpolation filter is applied to a second set of samples resulting from the matrix multiplication, and where the interpolation filter excludes the bilinear interpolation filter.

In some embodiments of method 3400, the interpolation filter includes a 4-tap interpolation filter. In some embodiments of method 3400, motion compensation for the chroma components of the current video block is performed using a 4-tap interpolation filter. In some embodiments of method 3400, angular intra-frame prediction of the current video block is performed using a 4-tap interpolation filter. In some embodiments of method 3400, the interpolation filter includes an 8-tap interpolation filter, and motion compensation for the luminance component of the current video block is performed using the 8-tap interpolation filter.

In fig. 35 shows an example flowchart of an example method 3500 for matrix intra-frame prediction. Operation 3502 includes performing a conversion between the current video block of video and a bitstream representation of the current video block in accordance with a rule, where the rule specifies the relationship between the applicability of a matrix intra-prediction (MIP) mode or a transform mode at transform time, where the MIP mode includes determining prediction block of the current video block by performing, on the previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, and where the transform mode specifies the use of the transform operation to determine the prediction block for the current video block.

In some embodiments of method 3500, the transform mode includes a reduced secondary transform (RST), a secondary transform, a rotation transform, or a non-separable secondary transform (NSST). In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode should not be applied in response to applying the MIP mode to the current video block. In some embodiments of method 3500, the rule specifies whether to apply a transform operation using a transform mode based on the height (H) or width (W) of the current video block. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to W ≥ T1 and H ≥ T2. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to W ≤ T1 and H ≤ T2. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to W ≥ T1 or H ≥ T2. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to W ≤ T1 or H ≤ T2. In some embodiments of method 3500, T1 = 32, and T2 = 32. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to W + H ≥ T. In some embodiments of the method 3500, the rule indicates that a transform operation using the mode is not applied to the current video block in response to W + H ≤ T. In some embodiments of the transform method 3500, the rule indicates that a transform operation using the mode transform is not applied to the current video block in response to the fact that W × H ≥ T. In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is not applied to the current video block in response to the fact that W × H ≤ T. In some embodiments of the method, 3500 T = 256.

In some embodiments of method 3500, the rule specifies that a transform operation using a transform mode should be applied in response to the fact that the current video block is encoded using the MIP mode. In some embodiments of method 3500, the selection of the transform matrix or kernels for the transform operation is based on the current video block encoded using the MIP mode. In some embodiments, method 3500 further includes converting a MIP mode to an intra-frame prediction mode; and selecting a transformation matrix or kernel based on the transformed intra-prediction mode. In some embodiments, method 3500 further includes converting a MIP mode to an intra-frame prediction mode; and selecting a transformation matrix or kernel based on the classification of the transformed intra-prediction mode. In some embodiments of method 3500, the transformed intra prediction mode includes a planar mode.

In some embodiments of method 3500, the rule specifies that a transform operation using the transform mode is applied to the current video block when it is prohibited to apply the MIP mode to the current video block. In some embodiments of method 3500, the rule specifies that a discrete cosine transform type II (DCT-II) encoding method is applied to the current video block encoded using the MIP mode. In some embodiments of method 3500, the bitstream representation of the current video block eliminates signaling of transform matrix indices for the DCT-II transform coding method. In some embodiments of method 3500, performing the transform includes obtaining a transform matrix to be used in the DCT-II transform coding method. In some embodiments of method 3500, the bitstream representation includes MIP mode-related information that is signaled after specifying the transform matrix.

In some embodiments of method 3500, the bitstream representation includes indicating a MIP mode for the transform matrix. In some embodiments of method 3500, the rule specifies that the bitstream representation excludes specifying the MIP mode for given transform matrices. In some embodiments of method 3500, the rule specifies that a transform operation using a skip transform method is applied to the current video block in response to encoding the current video block using the MIP mode. In some embodiments of method 3500, the bitstream representation of the current video block avoids signaling a conversion skip method when the current video block is encoded using the MIP mode. In some embodiments of method 3500, the rule specifies that for a current video block encoded using the MIP mode, the MIP mode is converted to a specified intra-frame prediction mode when the conversion operation is performed by selecting a mode-dependent transform matrix or kernel. In some embodiments of method 3500, the rule indicates that a transform operation using a skip transform method is prohibited for the current video block that is encoded using the MIP mode. In some embodiments of method 3500, the bitstream representation excludes signaling that indicates use of the skip conversion technique. In some embodiments of method 3500, the rule indicates that a transform operation using a skip transform method is permitted for a current video block that is not encoded using the MIP mode. In some embodiments of method 3500, the bitstream presentation excludes signaling that indicates use of the MIP mode.

In fig. 36 shows an example flowchart of an example method 3600 for matrix intra-frame prediction. Operation 3602 includes performing a transformation between the current video block of a video and a bitstream representation of the current video block using a matrix intra-prediction (MIP) mode, in which the prediction block of the current video block is determined by performing, on previously encoded video samples, downsampling at an operation boundary, at followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, where performing the transformation involves obtaining, in accordance with the rule, the boundary samples by applying a left bit shift operation or a right bit shift operation to the sum of at least one reference boundary sample, and where the rule determines whether to apply a left shift operation or a right shift operation.

In some embodiments of method 3600, the rule specifies that a right shift operation is applied in response to the number of shifted bits being greater than or equal to zero. In some embodiments of method 3600, the rule specifies that a bit right shift operation is applied using a first method in response to a number of shifted bits greater than zero, and where the rule specifies that a bit right shift operation is applied using a second method in response to that the number of shifted bits is zero, where the first method is different from the second method.

In some embodiments of method 3600, the boundary samples upsBdryX[ x ] are calculated using one of the following equations: upsBdryX[ x ] = ( + (1 << (Log2( uDwn ) - 1))) >> Log2( uDwn ) if uDwn > 1, or upsBdryX[ x ] = if uDwn = 1, where uDwn is equal to the value of the current video block size function of the boundary size, refX[x] indicates the number of reference samples x, >> indicates the bit shift operation to the right, and << indicates the shift operation bits to the left. In some embodiments of method 3600, the boundary samples upsBdryX[ x ] are calculated using the following equation: upsBdryX[ x ] = ( + (1 << Log2( uDwn ) >> 1 )) >> Log2( uDwn ), where uDwn is equal to the value functions of the current video block on the boundary size, refX[x] indicates the number of reference samples x, >> indicates the right bit shift operation, and << indicates the left bit shift operation.

In some embodiments of method 3600, the size of the border is predetermined based on the size of the current video block. In some embodiments of method 3600, uDwn is calculated as uDwn = nTbs/BorderSize, where nTbs and BorderSize represent the current video block size and border size, respectively. In some embodiments of method 3600, the boundary samples upsBdryX[ x ] are calculated using one of the following equations: upsBdryX[ x ] = ( + (1 << (Log2( uDwn ) - 1))) >> Log2( uDwn ) if nTbs > BorderSize, or upsBdryX[ x ] = if nTbs = BorderSize.

In fig. 37 shows an example flowchart of an example method 3700 for matrix intra-frame prediction. Operation 3702 includes performing a transformation between the current video block of the video and a bitstream representation of the current video block using a matrix intra-prediction (MIP) mode, in which the prediction block of the current video block is determined by performing, on the previously encoded video samples, a downsampling operation at the boundary, beyond followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation, where the prediction samples predSamples[ xHor + dX ][ yHor ] are determined according to the following formula in the upsampling operation: predSamples[ xHor + dX ][ yHor ] = ( ( upHor − dX ) * predSamples[ xHor ][ yHor ] + dX * predSamples[ xHor + upHor ][ yHor ] + offsetHor) / upHor, and predSamples[ xVer ][ yVer + dY ] = ( ( upVer − dY ) * predSamples[ xVer ][ yVer ] + dY * predSamples[ xVer ][ yVer + upVer ]+ offsetVer ) / upVer, where offsetHor and offsetVer are integers, where upHor is the dependence of the width of the current video block on the specified value, based on the size of the current video block, where upVer is the dependence of the height of the current video block on a given value based on the size of the current video block; where dX is equal to 1... upHor-1, where dY is equal to 1... upVer-1, and where xHor are positions based on upHor and yHor are positions based on upVer. In some embodiments of method 3700, offsetHor = upHor/2 and offsetVer = upVer/2.

In fig. 38 shows an example flowchart of an example method 3800 for matrix intra-frame prediction. Operation 3802 includes generating, to transform between the current video video block and the bitstream representation of the current video block, an intra-frame prediction mode for the current video block encoded using a matrix intra-frame prediction (MIP) mode, in which the prediction block of the current video block is determined by performing on previously encoded video samples, a downsampling operation at the boundary, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. Operation 3804 includes defining a rule for storing information indicating an intra-frame prediction mode based on whether the current video block is encoded using the MIP mode. Operation 3806 includes performing a conversion in accordance with a rule, where the rule specifies that the bitstream representation excludes storing information indicating the MIP mode associated with the current video block.

This patent document may refer to a bitstream representation as an encoded representation and vice versa. From the foregoing, it will be understood that specific embodiments of the technology disclosed herein have been described herein for purposes of illustration, but various modifications may be made without departing from the scope of the invention. Accordingly, the technology disclosed herein is not limited except by the appended claims.

Implementations of the subject matter and functional operations described in this patent document may be implemented in the form of various systems, digital electronic circuits or computer software, firmware or hardware, including the structures disclosed herein, as well as their structural equivalents , or as combinations of one or more of them. The implementations of the subject matter described herein may be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on a tangible and non-transitory computer-readable storage medium, for execution or for control of operation from a processing device data. A computer-readable storage medium may be a computer-readable storage device, a computer-readable storage medium, a memory device, a composition of matter providing a propagated machine-readable signal, or a combination of one or more of these. The term "data processing unit" or "data processing device" includes all hardware devices, devices and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The device may include, in addition to hardware, code that creates an execution environment for the computer program in question, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

A computer program (also known as a program, software, application, script, or code) can be written in any form of a programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file on a file system. A program may be stored in a portion of a file that contains other programs or data (for example, one or more scripts stored in a document markup language), in a single file dedicated to the program in question, or in multiple coherent files (for example, files that store one or multiple modules, subprograms, or pieces of code). A computer program can be deployed for execution on a single computer or on multiple computers that are located in one location or distributed across multiple sites and interconnected by a communications network.

The processes and logical flows described herein may be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and producing output data. In addition, processes and logic flows may also be executed, and the device may also be implemented as a special purpose logic circuit, such as an FPGA (Field Programmable Gate Array) or ASIC (Application Application Integrated Circuit).

Processors suitable for executing a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor only accepts instructions and data to read memory or RAM or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. In general, a computer will also include, or be operatively coupled to receive data from or transmit data to, or both, one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks. However, the computer does not necessarily have to have such devices. Computer readable media suitable for storing computer program instructions and data include all types of non-volatile memory, data storage devices and storage devices, including by way of example semiconductor memory devices such as, for example, EPROMs, EEPROMs and flash memory devices. The processor and memory may be complemented by or incorporated into specialized logic circuitry.

It is intended that the description, together with the drawings, be considered as exemplary only, where the term "exemplary" means example. As used herein, the term “or” is intended to include “and/or” unless the context clearly indicates otherwise.

Although this patent document contains many specific details, they should not be construed as limiting the scope of the invention or anything that may be claimed, but rather they should be construed as describing features that may be specific to particular embodiments of particular inventions. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of one embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Moreover, although features have been described above as operating in certain combinations and even initially claimed as such, one or more features from a claimed combination may in some cases be excluded from the combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination.

Likewise, although operations are depicted in the drawings in a particular order, this should not be understood to require that such operations be performed in a particular order stated or in a sequential order, or that all illustrated operations be performed to achieve the desired results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood to require such separation in all embodiments.

Only a few implementations and examples have been described above, and other implementations, improvements and changes may be made based on what is described and illustrated in this patent document.

Claims (34)

1. A method for processing video, comprising the steps of: determining, for conversion between the video chrominance block and the video bitstream, whether to apply the first prediction mode to the chroma block; generating prediction samples for the chrominance block based on the specified definition; And performing a conversion between the chrominance block and the bitstream; wherein, in response to determining that the first prediction mode should be applied to a chroma block, prediction samples are generated by performing a marginal downsampling operation based on the size of the chroma block, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation. 2. The method of claim 1, wherein in response to determining that the first prediction mode should not be applied to the chroma block, in the case where the prediction mode of the second corresponding luminance block matches the first prediction mode, prediction samples are generated by performing a mode conversion operation intra-frame luminance prediction, followed by an operation of obtaining an intra-frame chrominance prediction mode, and followed by an operation of obtaining common intra-frame samples. 3. The method of claim 1 or 2, wherein determining that the first prediction mode should be applied to the chroma block is based on the fact that the prediction mode of the chroma block is the same as the prediction mode of the first corresponding luminance block. 4. The method of claim 3, wherein that the prediction mode of a chroma block is the same as the prediction mode of the first corresponding luminance block is determined based on an intra_chroma_pred_mode syntax element, wherein the intra_chroma_pred_mode syntax element is used to indicate a mapping relationship between the intra-chroma component prediction mode and the mode intra-frame prediction of the luminance component. 5. The method of claim 4, wherein the intra_chroma_pred_mode syntax element indicates that the DM mode is applied to the chroma block, wherein in the DM mode, the intra-chroma prediction mode is identical to the intra-luminance prediction mode. 6. The method according to any one of claims 3 to 5, wherein the first corresponding luminance block is a luminance block covering a luminance sample corresponding to the upper left sample of the chrominance block. 7. The method according to any one of claims 2 to 6, wherein in the intra-frame luminance prediction mode conversion operation, the converted luminance prediction mode is obtained based on the prediction mode of the second corresponding luminance block. 8. The method of claim 7, wherein the second corresponding luminance block is a luminance block covering a luminance sample corresponding to a central sample of the chrominance block. 9. The method as claimed in claim 7 or 8, wherein in response to the prediction mode of the second corresponding luminance block being the same as the first prediction mode, the converted luminance prediction mode is output as a normal intra-frame prediction mode. 10. The method of claim 9, wherein the normal intra-prediction mode applies distance-based weighted calculations to reference values in the vertical and horizontal directions to generate prediction values. 11. The method according to any one of claims 2 to 10, wherein in the operation of obtaining an intra-frame chrominance prediction mode, a second prediction mode for the chrominance block is derived based on the transformed luminance prediction mode, wherein the second prediction mode is different from the first prediction mode. 12. The method according to any one of claims 3 to 6, wherein in response to the fact that the size of the chroma block is the same as the size of the first corresponding luma block, the same matrix coefficients are used in the first prediction mode and in the prediction mode of the first corresponding luma block. 13. The method according to any one of claims 1 to 12, in which, at the conversion stage, the chrominance block is encoded into a bit stream. 14. The method according to any one of claims 1 to 12, in which, in the conversion step, the chrominance block from the bit stream is decoded. 15. A video data processing device comprising a processor and non-volatile memory in which instructions are stored, wherein the instructions, when executed by the processor, cause the processor to execute: determining, for conversion between a video chrominance block and a video bitstream, whether to apply a first prediction mode to the chrominance block; generating prediction samples for the chrominance block based on said definition; And performing a conversion between the chrominance block and the bitstream; wherein, in response to determining that a first prediction mode should be applied to a chroma block, prediction samples are generated by performing a marginal downsampling operation based on the size of the chroma block, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation. 16. A non-volatile computer-readable storage medium on which instructions are stored that cause the processor to execute: determining, for conversion between a video chrominance block and a video bitstream, whether to apply a first prediction mode to the chrominance block; generating prediction samples for the chrominance block based on said definition; And performing a conversion between the chrominance block and the bitstream; wherein, in response to determining that a first prediction mode should be applied to a chroma block, prediction samples are generated by performing a marginal downsampling operation based on the size of the chroma block, followed by a matrix-vector multiplication operation, and followed selectively by an upsampling operation. 17. A method for storing a video bitstream, comprising the steps of: determining whether to apply the first prediction mode to the chroma block; generating prediction samples for the chrominance block based on the specified definition; generating a bit stream based on the prediction samples; And storing the bit stream on a non-volatile computer readable recording medium; wherein, in response to determining that the first prediction mode should be applied to a chroma block, prediction samples are generated by performing a marginal downsampling operation based on the size of the chroma block, followed by a matrix-vector multiplication operation, and selectively followed by an upsampling operation.

Images