KR102747568B1 - Intercoding by size - Google Patents

Abstract

Techniques for implementing video processing techniques are described. In one exemplary implementation, a video processing method includes a step of determining how coding information of a current block is represented in the bitstream representation, based in part on whether a condition related to a size of the current block is satisfied, for conversion between a current block of video and a bitstream representation of the video. The method also includes a step of performing the conversion based on the determination.

Description

Intercoding by size

Cross-reference to related applications

This application is the national phase of International Application No. PCT/CN2020/078108, filed March 6, 2020, which claims priority and the benefit of International Application No. PCT/CN2019/077179, filed March 6, 2019. The entire disclosure of the aforementioned applications is incorporated herein by reference as part of the disclosure of the present application.

This patent document relates to image and video coding and decoding.

Digital video uses the most bandwidth on the Internet and other digital communication networks. As the number of connected user devices capable of receiving and displaying video increases, the demand for bandwidth for digital video use is expected to continue to increase.

This specification discloses various video processing techniques that can be used by video encoders and decoders during encoding and decoding operations.

In one exemplary embodiment, a video processing method is disclosed. The method comprises the step of determining that a first motion vector of a sub-block of a current block and a second motion vector, which is a representative motion vector for the current block, comply with a size constraint for a transformation between a current block of a video and a bitstream representation of the video using an affine coding tool. The method further comprises the step of performing the transformation based on the result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises a step of determining an affine model including six parameters for a transformation between a current block of video and a bitstream representation of the video. The affine model is inherited from affine coding information of a neighboring block of the current block. The method further comprises a step of performing a transformation based on the affine model.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining whether a bi-prediction coding technique is applicable to the block for conversion between a block of video and a bitstream representation of the video, based on a size of a block having a width W and a height H, where W and H are positive integers. The method also includes a step of performing a conversion according to a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining whether a coding tree partitioning process is applicable to the block for conversion between a block of video and a bitstream representation of the video, based on a size of a sub-block, which is a child coding unit of the block, according to the coding tree partitioning process. The sub-block has a width W and a height H, where W and H are positive integers. The method also includes a step of performing a conversion according to a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining whether an index of a bi-prediction with coding unit level weight (BCW) coding mode is derived based on a rule for a position of a current block for conversion between a current block of video and a bitstream representation of the video. In the BCW coding mode, a weight set including a plurality of weights is used to generate a bi-prediction value of the current block. The method also includes a step of performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes determining an intra-prediction mode of a current block independently of an intra-prediction mode of a neighboring block, for conversion between a current block of video coded using a combined inter and intra prediction (CIIP) coding technique and a bitstream representation of the video. The CIIP coding technique uses an intermedia inter prediction value and an intermedia intra prediction value to derive a final prediction value of the current block. The method also includes performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining an intra-prediction mode of a current block of video coded using an inter and intra integrated prediction (CIIP) coding technique and a bitstream representation of the video, based on a first intra-prediction mode of a first neighboring block and a second intra-prediction mode of a second neighboring block, for conversion between the current block of video and a bitstream representation of the video. The first neighboring block is coded using the intra-prediction coding technique, and the second neighboring block is coded using the CIIP coding technique. The first intra-prediction mode is given a different priority than the second intra-prediction mode. The CIIP coding technique uses an intermedia inter-prediction value and an intermedia intra-prediction value to derive a final prediction value of the current block. The method further comprises the step of performing the conversion based on the determination result.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining, based on a size of the current block, whether an inter and intra integrated prediction (CIIP) process is applicable to color components of the current block for conversion between a current block of video and a bitstream representation of the video. The CIIP coding technique uses intermedia inter prediction values and intermedia intra prediction values to derive a final prediction value of the current block. The method also includes a step of performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining, based on characteristics of a current block, whether an inter and intra integrated prediction (CIIP) coding technique should be applied to the current block for conversion between a current block of video and a bitstream representation of the video. The CIIP coding technique uses intermedia inter prediction values and intermedia intra prediction values to derive a final prediction value of the current block. The method also includes a step of performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises a step of determining whether a coding tool should be disabled for a current block for conversion between a current block of video and a bitstream representation of the video, based on whether the current block is coded with an Inter and Intra Integrated Prediction (CIIP) coding technique. The coding tool comprises at least one of: Bi-Directional Optical Flow (BDOF), Overlapped Block Motion Compensation (OBMC), or a decoder-side motion vector refinement process (DMVR). The method further comprises a step of performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a first precision P1 used for a motion vector for spatial motion prediction and a second precision P2 used for a motion vector for temporal motion prediction, for conversion between a block of video and a bitstream representation of the video. P1 and/or P2 are fractions, and neither of P1 nor P2 is signaled in the bitstream representation. The method further comprises the step of performing the conversion based on the result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining motion vectors (MVx, MVy) with precisions (Px, Py) for conversion between a block of video and a bitstream representation of the video. Px is associated with MVx, and Py is associated with MVy. MVx and MVy are stored as integers each having N bits, and MinX ≤ MVx ≤ MaxX, MinY ≤ MVy ≤ MaxY, and MinX, MaxX, MinY, and MaxY are real numbers. The method also includes a step of performing the conversion based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes a step of determining whether a shared merge list is applicable to the current block, based on a coding mode of the current block, for conversion between a current block of video and a bitstream representation of the video. The method also includes a step of performing the conversion based on the result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a second block of dimensions (W + N - 1) x (H + N - 1) for motion compensation during a transformation between a current block of video having a size of W x H and a bitstream representation of the video. The second block is determined based on a reference block of dimensions (W + N - 1 - PW) x (H + N - 1 - PH). N represents a filter size, and W, H, N, PW and PH are non-negative integers. Neither of PW and PH are equal to 0. The method further comprises the step of performing the transformation based on a result of the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a second block of dimensions (W + N - 1) x (H + N - 1) for motion compensation during the transformation between a current block of video having a size W x H and a bitstream representation of the video, wherein W and H are non-negative integers, and N is a non-negative integer and is based on a filter size. During the transformation, a refined motion vector is determined based on a multi-point search according to a motion vector refinement operation with respect to an original motion vector, and pixels long boundaries of a reference block are determined by repeating one or more non-boundary pixels. The method further comprises the step of performing the transformation based on the determined result.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a prediction value at a given location within the block based on a weighted sum of inter prediction values and intra prediction values at the location, for conversion between a block of video coded using an inter-intra integrated prediction (CIIP) coding technique and a bitstream representation of the video. The weighted sum is obtained based on adding an offset to an initial sum obtained based on the inter prediction values and the intra prediction values, and the offset is added prior to a right shift operation performed to determine the weighted sum. The method further comprises the step of performing the conversion based on a result of the determination.

In another exemplary aspect, a method of video processing is disclosed. The method includes a step of determining how coding information of a current block is represented in the bitstream representation, based in part on whether a condition associated with a size of the current block is satisfied for conversion between a current block of video and a bitstream representation of the video. The method also includes a step of performing the conversion based on this determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a set of modified motion vectors for transformation between a current video block of the video and a bitstream representation of the video. And the step of performing a transformation based on the set of modified motion vectors. The set of modified motion vectors due to a current block satisfying a condition is a modified version of the set of motion vectors associated with the current block.

In another exemplary aspect, a video processing method is disclosed. The method comprises the step of determining a unidirectional motion vector from a bidirectional motion vector if a block-wise condition is satisfied for transformation between a current block of video and a bitstream representation of the video. The unidirectional motion vector is then used as a merge candidate for transformation. The method further comprises the step of performing the transformation based on the determination.

In another exemplary aspect, a video processing method is disclosed. The method includes the steps of determining that motion candidates of a current block are limited to unidirectional prediction candidates based on a dimension of the current block for transforming between a current block of video and a bitstream representation of the video, and performing a transform based on the determination.

In another exemplary aspect, a video processing method is disclosed. The method comprises the steps of determining a size constraint between a representative motion vector of a current video block to be affine coded and a motion vector of a sub-block of the current video block, and performing a transformation between a bitstream representation of the current video block or sub-block and a pixel value using the size constraint.

In another exemplary aspect, another video processing method is disclosed. The method comprises the steps of determining one or more sub-blocks of a current video block for an affine coded current video block, each sub-block having a size of MxN pixels, where M and N are multiples of 2 or 4, conforming a motion vector of the sub-blocks to size constraints, and conditionally, based on a trigger, performing a transformation between a bitstream representation of the current video block and pixel values using the size constraints.

In another exemplary embodiment, another video processing method is disclosed. The method includes the steps of determining that a current video block satisfies a size condition, and performing a conversion between a bitstream representation of the current video block and pixel values based on the determination result, by excluding a bidirectional prediction encoding mode for the current video block.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of determining that a current video block satisfies a size condition, and performing a transformation between a bitstream representation of the current video block and pixel values based on the determination result, wherein an inter prediction mode is signaled in the bitstream representation according to the size condition.

In another exemplary embodiment, another video processing method is disclosed. The method includes the steps of determining that a current video block satisfies a size condition, and performing a transformation between a bitstream representation of the current video block and pixel values based on the determination result, wherein generating a merge candidate list during the transformation depends on the size condition.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of determining that a child coding unit of a current video block satisfies a size condition, and performing a conversion between a bitstream representation of the current video block and a pixel value based on the determination result, wherein a coding tree splitting process used to generate the child coding unit is dependent on the size condition.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of determining a weight index for a generalized bidirectional prediction (GBi) process for a current video block based on a position of the current video block, and performing a transformation between the current video block and its bitstream representation using the weight index to implement the GBi process.

In another exemplary aspect, another video processing method is disclosed. The method comprises the steps of determining that a current video block is coded as an intra-inter prediction (IIP) coded block and performing a conversion between the current video block and its bitstream representation using a simplification rule to determine an intra-prediction mode or a most probable mode (MPM) of the current video block.

In another exemplary aspect, another video processing method is disclosed. The method comprises the steps of determining that a current video block satisfies a simplification criterion, and performing a transformation between the current video block and a bitstream representation by disabling the use of an inter-intra prediction mode for the transformation or by disabling an additional coding tool used for the transformation.

In another exemplary aspect, another video processing method is disclosed. The method comprises the step of performing a conversion between a current video block and a bitstream representation of the current video block using a motion vector based encoding process, wherein (a) during the conversion process, a precision P1 is used to store a spatial motion prediction result and a precision P2 is used to store a temporal motion prediction result, and P1 and P2 are fractions, or (b) a precision Px is used to store an x-motion vector and a precision Py is used to store a y-motion vector, and Px and Py are fractions.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of interpolating a small sub-block of size W1xH1 within a larger sub-block of size W2xH2 of a current video block by fetching (W2 + N - 1 - PW) * (H2 + N - 1 - PH) blocks, pixel padding the fetched block, performing boundary pixel repeating on the pixel padded block and obtaining pixel values of the small sub-block, wherein W1, W2, H1, H2, and PW and PH are integers, and performing a conversion between the current video block and a bitstream representation of the current video block using the interpolated pixel values of the small sub-block.

In another exemplary aspect, another video processing method is disclosed. The method comprises: performing a motion compensation operation by fetching (W + N - 1 - PW) * (W + N - 1 - PH) reference pixels and padding the reference pixels outside the fetched reference pixels during a conversion between a current video block of dimension WxH and a bitstream representation of the current video block; and performing the conversion between the current video block and the bitstream representation of the current video block using a result of the motion compensation operation, wherein W, H, N, PW and PH are integers.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of determining, based on a size of a current video block, that bidirectional prediction or unidirectional prediction of the current video block is not allowed, and, based on a result of the determination, performing a conversion between a bitstream representation of the current video block and a pixel value by disabling the bidirectional prediction or unidirectional prediction mode.

In another exemplary aspect, another video processing method is disclosed. The method includes the steps of determining, based on a size of a current video block, that bidirectional prediction or unidirectional prediction of the current video block is not allowed, and, based on a result of the determination, performing a conversion between a bitstream representation of the current video block and a pixel value by disabling the bidirectional prediction or unidirectional prediction mode.

In another exemplary aspect, a video encoder device is disclosed. The video encoder includes a processor configured to implement the method described above.

In another exemplary aspect, a video encoder device is disclosed. The video encoder includes a processor configured to implement the method described above.

In another exemplary aspect, a computer-readable medium having code stored thereon is disclosed, the code implementing one of the methods described herein as processor-executable code.

These and other features are described throughout this specification.

Figure 1 shows an example of sub-block based prediction.
Figure 2a illustrates a four-parameter affine model.
Figure 2b illustrates the six-parameter affine model.
Figure 3 shows an example of an affine motion vector field for each sub-block.
Figure 4a shows an example of a candidate for AF_MERGE.
Figure 4b shows another example of a candidate for AF_MERGE.
Figure 5 illustrates candidate locations for affine merge mode.
Figure 6 illustrates an example of constrained sub-block motion vectors for a coding unit (CU) in affine mode.
Figure 7a illustrates an example of a 135 degree partition that divides one CU into two triangular prediction units.
Figure 7b illustrates an example of a 45 degree splitting pattern that splits one CU into two triangular prediction units.
Figure 8 shows examples of the locations of neighboring blocks.
Figure 9 illustrates an example of repeat boundary pixels of a reference block before interpolation.
Figure 10 illustrates an example of coding tree units (CTUs) and CTU (region) lines. The shaded CTUs (regions) are within one CUT (region) line, and the unshaded CTUs (regions) are within another CUT (region) line.
FIG. 11 is a block diagram of an example of a hardware platform for implementing a video decoder or video encoder device described herein.
Figure 12 is a flowchart of an exemplary video processing method.
Figure 13 shows an example of a motion vector difference MVD(0,1) mirrored between List 0 and List 1 in DMVR.
Figure 14 shows example MVs that can be checked in a single iteration.
Figure 15 shows the required reference sample and boundaries padded for calculation.
FIG. 16 is a block diagram of an exemplary video processing system in which the disclosed technique may be implemented.
Figure 17 is a flowchart representation of a method for video processing according to the present invention.
FIG. 18 is a flowchart representation of another method for video processing according to the present invention.
FIG. 19 is a flowchart representation of another method for video processing according to the present invention.
FIG. 20 is a flowchart representation of another method for video processing according to the present invention.
FIG. 21 is a flowchart representation of another method for video processing according to the present invention.
FIG. 22 is a flowchart representation of another method for video processing according to the present invention.
FIG. 23 is a flowchart representation of another method for video processing according to the present invention.
FIG. 24 is a flowchart representation of another method for video processing according to the present invention.
FIG. 25 is a flowchart representation of another method for video processing according to the present invention.
FIG. 26 is a flowchart representation of another method for video processing according to the present invention.
FIG. 27 is a flowchart representation of another method for video processing according to the present invention.
FIG. 28 is a flowchart representation of another method for video processing according to the present invention.
FIG. 29 is a flowchart representation of another method for video processing according to the present invention.
FIG. 30 is a flowchart representation of another method for video processing according to the present invention.
FIG. 31 is a flowchart representation of another method for video processing according to the present invention.
FIG. 32 is a flowchart representation of another method for video processing according to the present invention.
FIG. 33 is a flowchart representation of another method for video processing according to the present invention.
FIG. 34 is a flowchart representation of another method for video processing according to the present invention.
FIG. 35 is a flowchart representation of another method for video processing according to the present invention.
FIG. 36 is a flowchart representation of another method for video processing according to the present invention.

The section titles are used throughout this specification for ease of understanding and are not intended to limit the techniques and embodiments disclosed in each section to only that section.

1. Summary

This patent document relates to video/image coding technology. In particular, the technology relates to reducing the bandwidth and line buffer of various coding tools in the field of video/image coding. The technology can be applied to complete existing video coding standards such as HEVC or standard (Versatile Video Coding). In addition, the technology can be applied to future video/image coding standards or video/image codecs.

2. Background Technology

Video coding standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T produced H.261 and H.263, ISO/IEC produced MPEG-1 and MPEG-4 Visual, and these two organizations jointly produced the H.262/MPEG-2 Video and H.264/MPEG-4 Advanced Video Coding (AVC) and H.265/HEVC standards. Since H.262, video coding standards have been based on hybrid video coding architectures in which transform coding is utilized in addition to temporal prediction. To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, many new methods have been adopted by the JVET and added to the reference software called the Joint Exploration Model (JEM). In April 2018, the Joint Video Expert Team (JVET) was formed between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) to work on a VVC standard that aims to provide a 50% bitrate reduction compared to HEVC.

2.1 Inter prediction in HEVC/VVC

Interpolation filter

In HEVC, luma sub-samples are generated by an 8-tap interpolation filter and chroma sub-samples are generated by a 4-tap interpolation filter.

Filters can be separated in two dimensions: samples are filtered horizontally and then vertically.

2.2 Prediction techniques based on sub-blocks

Sub-block based prediction is first introduced into the video coding standard by HEVC Annex I (3D-HEVC). With sub-block based prediction, a block, such as a coding unit (CU) or a prediction unit (PU), is divided into multiple non-overlapping sub-blocks. Different motion information, such as a reference index or a motion vector (MV), can be assigned to different sub-blocks, and motion compensation (MC) is performed individually for each sub-block. Figure 1 demonstrates the concept of sub-block based prediction.

To explore future video coding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was jointly established by VCEG and MPEG in 2015. Since then, many new methods have been adopted by JVET and added to the reference software called the Joint Exploration Model (JEM).

In JEM, sub-block based predictions such as affine prediction, alternative temporal motion vector prediction (ATMVP), spatial-temporal motion vector prediction (STMVP), bi-directional optical flow (BIO), and Frame-Rate Up Conversion (FRUC) are adopted in several coding tools. Affine prediction has also been adopted in VVC.

2.3 Affine Prediction

In HEVC, only the translational motion model is applied for motion compensation prediction (MCP). However, there are many kinds of motions in the real world, such as zoom-in/out, rotation, viewpoint motion, and other irregular motions. In VVC, a simplified affine transform motion compensation prediction is applied. As shown in Fig. 2a and Fig. 2b, the affine motion field of a block is described by two (in a 4-parameter affine model) or three (in a 6-parameter affine model) control point motion vectors.

The motion vector field (MVF) of a block is described by the following mathematical equations, where the 4-parameter affine model (where the 4 parameters are defined by the variables a, b, e, and f ) is defined in Equation 1, and the 6-parameter affine model (where the 4 parameters are defined by the variables a, b, c, d, e, and f ) is defined in Equation 2, respectively:

Here, ( mv ^h ₀ , mv ^h ₀ ) is the motion vector of the top-left corner control point, ( mv ^h ₁ , mv ^h ₁ ) is the motion vector of the top-right corner control point, and ( mv ^h ₂ , mv ^h ₂ ) is the motion vector of the bottom-left corner control point, and together, three motion vectors are called control point motion vectors (CPMV), where (x, y) represent the coordinates of the representative point relative to the top-left sample within the current block. The CP motion vector can be signaled (as in affine AMVP mode), or derived on the fly (as in affine merge mode). w and h are the width and height of the current block. In practice, the division is implemented by a right-shift with rounding operation. In VTM, the representative point is defined as the center position of the sub-block, and for example, if the coordinates of the upper-left corner of the sub-block relative to the upper-left sample in the current block are (xs, ys), then the coordinates of the representative point are defined to be (xs+2, ys+2).

In a design without a saddle, Equations 1 and 2 are implemented as follows:

For the 4-parameter affine model shown in (1), it becomes:

For the 6-parameter affine shown in (2), it becomes:

Finally, the following holds,

Here, S represents the computational precision. For example, in VVC, S = 7. In VVC, the MV used in MC for the sub-block with the top-left sample at (xs, ys) is calculated by Equation 6) with the relationships x = xs + 2 and y = ys + 2.

To derive the motion vector of each 4×4 sub-block, the motion vector of the center sample of each sub-block, as illustrated in Fig. 3, is computed by Equation 1 or Equation 2 and rounded to a fractional accuracy of 1/16. Then, a motion compensation interpolation filter is applied to generate a prediction of each sub-block with the derived motion vector.

An affine model can be inherited from spatially neighboring affine-coded blocks, for example, the left, top, top right, bottom left and top left neighboring blocks as illustrated in Fig. 4a. For example, if the bottom left neighboring block A in Fig. 4a is coded in affine mode as indicated by A0 in Fig. 4b, the control point (CP) motion vectors ( mv ₀ ^N , mv _{1 N and mv 2 N ) of the top left corner, top right corner and bottom left corner of the neighboring CU/PU containing block A are fetched. Then, the top left corner/top right/bottom left motion vectors ( mv 0 C , mv 1} ^C and _mv ² C ₎ ^of the _current ^CU / _PU ⁽ used only for 6-parameter affine model) are computed based on mv ₀ ^N , mv ₁ ^N and mv ₂ ^N . In VTM-2.0, it should be noted that if the current block is affine coded, the sub-block (e.g., a 4×4 block in VTM) LT stores mv0 and RT stores mv1. If the current block is coded with a 6-parameter affine model, the LB stores mv2; otherwise (in the case of a 4-parameter affine model), the LB stores mv2'. The other sub-blocks store the MVs used for MC.

If CU is coded in affine merge mode, for example, in AF_MERGE mode, it takes the first block coded in affine mode from valid neighboring reconstructed blocks. And, the selection order for candidate blocks is from left, top, top right, bottom left to top left as shown in Fig. 4a.

The derived CP MVs ( mv ₀ ^C , mv ₁ ^C and mv ₂ ^C ) of the current block can be used as CP MVs in the affine merge mode. Alternatively, they can be used as MVPs for the affine inter mode in VVC. For the merge mode, if the current block is coded in the affine mode, it should be noted that after deriving the CP MVs of the current block, the current block can be further divided into multiple sub-blocks, and each block will derive its motion information based on the derived CP MVs of the current block.

2.4 Exemplary implementations in JVET

Unlike VTM, where only one affine spatial neighboring block can be used to induce affine motion for a block. In some embodiments, a separate list of affine candidates is constructed for the AF_MERGE mode.

1) Insert the inherited affine candidates into the candidate list.

An inherited affine candidate means that the candidate is derived from a reconstructed block of a valid neighbor coded in affine mode. As illustrated in Fig. 5, the scan order for these candidate blocks is A ₁ , B ₁ , B ₀ , A ₀ , and B ₂ . When a block (e.g., A ₁ ) is selected, the following two-step procedure is applied:

1.a First, derive two/three control points of the current block using three corner motion vectors of the CU covering the block.

1.b Based on the control points of the current block, derive sub-block movements for each sub-block within the current block.

2) Insert the configured affine candidate

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand , the constructed affine candidates are inserted into the candidate list.

The constructed affine candidates mean that these candidates are constructed by combining the neighboring motion information of each control point.

Motion information for control points is initially derived from the prescribed spatial and temporal neighborhoods, which are illustrated in Fig. 5. CPk (k=1, 2, 3, 4) represents the k-th control point. A ₀ , A ₁ , A ₂ , B ₀ , B ₁ , B _{2 ,} and B ₃ are spatial locations for predicting CPk (k=1, 2, 3); T is a temporal location for predicting CP4.

The coordinates of CP1, CP2, CP3, and CP4 are (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H are the width and height of the current block.

The movement information of each control point is obtained according to the following priority order:

2.a For CP1, the checking priority is B ₂ -> B ₃ -> A ₂ . B ₂ is used if it is available. Otherwise, if B ₂ is unavailable, B ₃ is used. If both B ₂ and B ₃ are unavailable, A ₂ is used. If all three candidates are unavailable, the motion information of CP1 cannot be obtained.

2.b For CP2, the check priority is B1->B0;

2.c For CP3, the check priority is A1->A0;

2.d For CP4, T is used.

Second, a combination of control points is used to construct a motion model.

The motion vectors of the three control points are required to compute the transformation parameters in the six-parameter affine model. The three control points can be selected from one of the following four combinations: ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}. For example, the CP1, CP2 and CP3 control points are used to construct the six-parameter affine motion model, denoted as Affine (CP1, CP2, CP3).

Motion vectors of two control points are required to compute the transformation parameters in the four-parameter affine model. The two control points can be selected from one of the following six combinations: ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}. For example, CP1 and CP2 control points are used to construct a four-parameter affine motion model, denoted as Affine (CP1, CP2).

The combinations of constructed affine candidates are inserted into the candidate list in the following order: {CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

3) Insert zero motion vector

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, zero motion vectors are inserted into the candidate list until the list is full.

2.5 List of Affine Merge Candidates

2.5.1 Affine Merge Mode

In the affine merge mode of VTM-2.0.1, only the first available affine neighbor can be used to derive the motion information of the affine merge mode. In some embodiments, the candidate list for the affine merge mode is constructed by searching for valid affine neighbors and combining the neighbor motion information of each control point.

The list of affine merge candidates is constructed in the following steps:

1) Insert the inherited affine candidate

An inherited affine candidate means that the candidate is derived from the affine motion model of its valid neighboring affine coded block. As illustrated in Fig. 5, the scan order for candidate positions in a common base is as follows: A1, B1, B0, A0, and B2.

After a candidate is derived, a full pruning process is performed to check whether the same candidate has already been inserted into the list. If there is a same candidate, the derived candidate is discarded.

2) Insert the configured affine candidate

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand (which is set to 5 in this contribution), the constructed affine candidates are inserted into the candidate list. The constructed affine candidates mean that these candidates are constructed by combining the neighboring motion information of each control point.

Motion information for control points is initially derived from the prescribed spatial and temporal neighbors. CPk(k=1, 2, 3, 4) represents the k-th control point. A0, A1, A2, B0, B1, B2, and B3 are spatial locations for predicting CPk(k=1, 2, 3); T is a temporal location for predicting CP4.

The coordinates of CP1, CP2, CP3, and CP4 are (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H are the width and height of the current block.

The movement information of each control point is obtained according to the following priority order:

For CP1, the check priority is B2->B3->A2. B2 is used if it is available. Otherwise, if B2 is unavailable, B3 is used. If both B2 and B3 are unavailable, A2 is used. If all three candidates are unavailable, the movement information of CP1 cannot be obtained.

For CP2, the inspection priority is B1->B0.

For CP3, the inspection priority is A1->A0.

For CP4, T is used.

Second, a combination of control points is used to construct affine merge candidates.

Motion information of three control points is required to construct a 6-parameter affine candidate. The three control points can be selected from one of the following four combinations: ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}. The combinations ({CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}) will be transformed into a 6-parameter motion model represented by the top-left, top-right and bottom-left control points.

Motion information of two control points is required to construct a 4-parameter affine candidate. The two control points can be selected from one of the following six combinations: ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}. The combinations ({CP1, CP4}, {CP2, CP3}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}) will be transformed into a 4-parameter motion model represented by the top-left and top-right control points.

The combinations of the constructed affine candidates are inserted into the candidate list in the following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}

For a reference list X of a combination (X is 0 or 1), the reference index with the highest usage ratio among the control points is selected as the reference index of the list X, and the motion vectors pointing to different reference pictures will be scaled.

After a candidate is derived, a full pruning process is performed to check whether the same candidate has already been inserted into the list. If there is a same candidate, the derived candidate is discarded.

3) Padding using zero motion vectors

If the number of candidates in the affine merge candidate list is less than 5, zero motion vectors with zero reference index are inserted into the candidate list until the list is full.

2.5.2 Example affine merge mode

In some embodiments, the affine merge mode can be simplified as follows:

1) The pruning process for the inherited affine candidates is simplified by comparing coding units covering neighboring positions instead of comparing affine candidates derived from VTM-2.0.1. At most two inherited affine candidates are inserted into the affine merge list. The pruning process for the constructed affine candidates is completely eliminated.

2) The MV scaling operation within the constructed affine candidate is removed. If the reference indices of the control points are different, the constructed motion model is discarded.

3) The number of configured affine candidates is reduced from 10 to 6.

4) In some embodiments, other merge candidates with sub-block predictions, such as ATMVP, are also added to the affine merge candidate list. In such a case, the affine merge candidate list is renamed to some other name, such as sub-block merge candidate list.

2.6 Example control point MV offset for affine merge mode

A new affine merge candidate is generated based on the CPMV offsets of the first affine merge candidate. If the first affine merge candidate merges a 4-parameter affine model, two CPMVs for each new affine merge candidate are derived by offsetting two CPMVs of the first affine merge candidate; otherwise (if the 6-parameter affine model is enabled), three CPMVs for each new affine merge candidate are derived by offsetting three CPMVs of the first affine merge candidate. In unidirectional prediction, the CPMV offsets are applied to the CPMVs of the first candidate. In bidirectional prediction with list 0 and list 1 in the same direction, the CPMV offsets are applied to the first candidate as follows:

In bidirectional prediction with lists 0 and 1 in opposite directions, CPMV offsets are applied to the first candidate as follows:

Different offset directions with different offset sizes can be used to generate new affine merge candidates. The following two implementations have been tested:

(1) Sixteen new affine merge candidates with eight different offset directions and two different offset sizes are generated as shown in the following offset set:

offset set = { (4, 0), (0, 4), (-4, 0), (0, -4), (-4, -4), (4, -4), (4, 4), (-4, 4), (8, 0), (0, 8), (-8, 0), (0, -8), (-8, -8), (8, -8), (8, 8), (-8, 8) }.

In this design, the affine merge list increases to 20. The total number of potential affine merge candidates is 31.

(2) Four new affine merge candidates with four different offset directions having one offset size are generated as shown in the following offset set:

Offset set = { (4, 0), (0, 4), (-4, 0), (0, -4) }.

The affine merge list is kept as 5 as in the case of VTM 2.0.1. Four temporally structured affine merge candidates are removed so that the number of potential affine merge candidates remains unchanged, for example, 15 in total. Assume that the coordinates of CPMV1, CPMV2, CPMV3 and CPMV4 are (0, 0), (W, 0), (H, 0) and (W, H). Note that CPMV4 is derived from temporal MV as illustrated in Fig. 6. The removed candidates are the following four temporally related affine merge candidates: {CP2, CP3, CP4}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}.

2.7 Bandwidth Issues in Affine Motion Compensation

Since the current block is split into 4×4 sub-blocks for the luma component and 2×2 sub-blocks for the two chroma components to perform motion compensation, the total bandwidth requirement is much higher than in the case without sub-block inter-prediction. To address the bandwidth issue, several approaches have been proposed.

2.7.1 Example 1

While 4x4 block is used as sub-block size for unidirectional affine coded CU, 8x4/4x8 block is used as sub-block size for bidirectional affine coded CU.

2.7.2 Example 2

For affine mode, the sub-block motion vectors of an affine CU are constrained to fall within a predefined motion vector field. Assume that the motion vector of the first (top-left) sub-block is (v _0x , v _0y ) and the motion vector of the second sub-block is (v _ix , v _iy ), where the values of v _ix and v _iy represent the following constraints:

If the motion vector of any sub-block exceeds a predefined motion vector field, the motion vector is clipped. An example of the concept of constrained sub-block motion vectors is provided in 6.

We assume that memory is fetched per CU instead of per sub-block, and that the worst-case memory bandwidth of an affine CU is chosen such that it does not exceed the memory bandwidth of a normal inter MC of an 8×8 bidirectional prediction block. Note that the values of H and V are adapted to the CU size and to unidirectional or bidirectional prediction.

2.7.3 Example 3

To reduce the memory bandwidth requirement in affine prediction, each 8x8 block within that block is considered a basic unit. The MVs of all four 4x4 sub-blocks within an 8x8 block are constrained such that the maximum difference between the integer parts of the four 4x4 sub-block MVs does not exceed 1 pixel. Therefore, the bandwidth becomes (8+7+1)*(8+7+1)/(8*8)=4 samples/pixel.

In some cases, after the MVs of all sub-blocks in the current block are computed as affine models, the MVs of the sub-blocks containing control points are first replaced by the corresponding control point MVs. This means that the MVs of the top-left, top-right and bottom-left sub-blocks are replaced by the top-left, top-right and bottom-left control point MVs, respectively. Then, for each 8x8 block in the current block, the MVs of all four 4x4 sub-blocks are clipped to ensure that the maximum difference between the integer parts of the four MVs does not exceed 1 pixel. Note that the sub-blocks containing control points (the top-left, top-right and bottom-left sub-blocks) participate in the MV clipping process using their corresponding control point MVs. During the clipping process, the MV of the top-right control point remains unchanged.

The clipping process applied to each 8x8 block is as follows:

1. The minimum and maximum values for the MV components, MVminx, MVminy, MVmaxx, and MVmaxy, are first determined for each 8x8 block as follows:

a) Take the minimum MV component among four 4x4 sub-block MVs.

MVminx = min(MVx0, MVx1, MVx2, MVx3)

MVminy = min(MVy0, MVy1, MVy2, MVy3)

b) Use the integer part of MVminx and MVminy as the minimum MV component.

MVminx = MVminx >> MV_precision << MV_precision

MVminy = MVminy >> MV_precision << MV_precision

c) The maximum MV component is computed as follows:

MVmaxx = MVminx + (2 << MV_precision) - 1

MVmaxy = MVminy + (2 << MV_precision) - 1

d) If the top-right control point is the current 8x8 block,

if (MV1x > MVmaxx)

MVminx = (MV1x >> MV_precision << MV_precision) - (1 << MV_precision)

MVmaxx = MVminx + (2 << MV_precision) - 1

if (MV1y > MVmaxy)

MVminy = (MV1y >> MV_precision << MV_precision) - (1 << MV_precision)

MVmaxy = MVminy + (2 << MV_precision) - 1

2. The MV components of each 4x4 block within these 8x8 blocks are clipped as follows:

MVxi = max(MVminx, min(MVmaxx, MVxi))

MVyi = max(MVminy, min(MVmaxy, MVyi))

Here, (MVxi, MVyi) is the MV of the ith sub-block within one 8x8 block, where i is 0, 1, 2, 3; (MV1x, MV1y) is the MV of the top-right control point; MV_precision is equal to 4, which corresponds to a motion vector fractional accuracy of 1/16. Since the difference between the integer parts of MVminx and MVmaxx (MVminy and MVmaxy) is 1 pixel, the maximum difference between the integer parts of four 4x4 sub-block MVs does not exceed 1 pixel.

Additionally, similar methods can be used to handle planar mode in some embodiments.

2.7.4 Example 4

In some embodiments, a limitation on affine mode for worst-case bandwidth reduction. To ensure that the worst-case bandwidth of an affine block is not worse than that of an INTER_4x8/INTER_8x4 block or even an INTER_9x9 block, the motion vector difference between affine control points is used to determine whether the sub-block size of the affine block is 4x4 or 8x8.

General affine limitations for worst case bandwidth reduction

The memory bandwidth reduction for affine mode is controlled by limiting the motion vector difference between affine control points (also called control point difference). In general, if the control point difference satisfies the constraints below, the affine motion is using 4x4 sub-blocks (i.e. 4x4 affine mode). Otherwise, it is using 8x8 sub-blocks (8x8 affine mode). The constraints for 6-parameter and 4-parameter models are given as follows.

To derive constraints for different block sizes ( w x h ), the motion vector differences of the control points are normalized as follows:

In the 4-parameter affine model, and It is set as follows:

therefore, and The norm is given as follows:

The constraint to ensure worst case bandwidth is to get INTER_4x8 or INTER_8x4:

Here, the left side of equation (17) represents the reduction or enlargement level of the sub-affine blocks, while (3.25) represents a pixel shift of 3.25.

The limitation to ensure worst case bandwidth is to obtain INTER_9x9

Here, (128 and 16 represent the normalization factor and motion vector precision, respectively).

2.8 Improved generalized bidirectional prediction

Some embodiments have improved the gain-complexity trade-off for GBi and have been adopted in BMS2.1. GBi is also called Bi-prediction with CU-level weight (BCW). BMS2.1 GBi applies unequal weights to predictors from L0 and L1 in bi-prediction mode. In inter-prediction mode, multiple weight pairs including the same weight pair (1/2, 1/2) are evaluated based on rate-distortion optimization (RDO), and the GBi index of the selected weight pair is signaled to the decoder. In merge mode, the GBi index is inherited from the neighboring CU. In BMS2.1 GBi, the predictor generation in bi-prediction mode is shown in Equation 19.

Here, P _GBi is the final predictor of GBi. w ₀ and w ₁ are the selected GBi weight pairs and are applied to the predictors in list 0 (L0) and list 1 (L1), respectively. RoundingOffset _GBi and shiftNum _GBi are used to normalize the final predictor in GBi . The supported w ₁ weight sets are {-1/4, 3/8, 1/2, 5/8, 5/4}, where five weights correspond to one identical weight pair and four unequal weight pairs. The blending gain, i.e. the sum of w ₁ and w ₀ is fixed to 1.0. Therefore, the corresponding w ₀ weight set is {5/4, 5/8, 1/2, 3/8, -1/4}. The selection of weight pairs is done at the CU level.

For non-low delay pictures, the weight set size is reduced from 5 to 3, where the w ₁ weight set is {3/8, 1/2, 5/8} and the w ₀ weight set is {5/8, 1/2, 3/8}. The weight set size reduction for non-low delay pictures applies to the BMS2.1 GBi and all GBi tests within this contribution.

In some embodiments, to further improve GBi performance, the following modifications are applied on top of the existing GBi design in BMS2.1.

2.8.1 GBi encoder bug fixes

In order to reduce the GBi encoding time, in the current encoder design, the encoder will store the unidirectional predictive motion vectors estimated from the GBi weights such as 4/8, and reuse them for the unidirectional predictive lookup of other GBi weights. This fast encoding method is applied to both the translational motion model and the affine motion model. In VTM2.0, the 6-parameter affine model is adopted together with the 4-parameter affine model. The BMS2.1 encoder does not distinguish between the 4-parameter affine model and the 6-parameter affine model when storing the unidirectional predictive affine MVs when the GBi weights are equal to 4/8. As a result, after encoding using the GBi weights 4/8, the 4-parameter affine MVs may be overwritten by the 6-parameter affine MVs. The saved 6-parameter affine MVs can be used for 4-parameter affine ME for other GBi weights, or the saved 4-parameter affine MVs can be used for 6-parameter affine ME. The proposed GBi encoder bug fix is to separate the saving of 4-parameter and 6-parameter affine MVs. If a GBi weight is equal to 4/8, the encoder saves those affine MVs based on their affine model type, and reuses the corresponding affine MVs based on their affine model type for other GBi weights.

CU size constraints for 2.8.2 GBi

In this method, GBi is disabled for small CUs. In inter prediction mode, if bidirectional prediction is used and the CU area is less than 128 luma samples, GBi is disabled without any signaling.

2.8.3 Merge mode with GBi

In case of merge mode, the GBi index is not signaled. Instead, it is inherited from the neighboring block into which it is merged. If a TMVP candidate is selected, GBi is turned off in these blocks.

2.8.4 Affine Prediction with GBi

If the current block is coded with affine prediction, GBi can be used. In the case of affine inter mode, the GBi index is signaled. In the case of affine merge mode, the GBi index is inherited from the neighboring block into which it is merged. If the constructed affine model is selected, GBi is turned off in these blocks.

2.9 Example Inter-Intra Prediction Mode (IIP)

In the inter-intra prediction mode, also called Inter and Intra Integrated Prediction (CIIP), multi-dimensional (multi-hypothesis) prediction combines one intra prediction and one merge indexed prediction. Such a block is treated as a special inter-coded block. In a merge CU, a flag is signaled for the merge mode to select one intra mode from the intra candidate list if the flag is true. For the luma component, the intra candidate list is derived from four intra prediction modes, including DC, planar, horizontal, and vertical modes, and the size of the intra candidate list can be 3 or 4 depending on the block shape. If the CU width is greater than twice the CU height, the horizontal mode is exclusively selected from the intra mode list, and if the CU height is greater than twice the CU width, the vertical mode is removed from the intra mode list. One intra prediction mode selected by the intra mode index and one merge indexed prediction selected by the merge index are combined using a weighted average. For chroma components, DM is always applied without additional signaling.

The weights for combining the predictions are described as follows. When DC or planar mode is selected, or CB width or height is less than 4, the same weights are applied. When horizontal/vertical mode is selected, for CBs with CB width and height greater than 4, a CB is first divided into four equal-area regions. Each set of weights, called (w_intra _i , w_inter _i ), will be applied to a corresponding region, where i is from 1 to 4, (w_intra ₁ , w_inter ₁ ) = ((6, 2), (w_intra ₂ , w_inter ₂ ) = (5, 3), (w_intra ₃ , w_inter ₃ ) = (3, 5), and (w_intra ₄ , w_inter ₄ ) = (2, 6). (w_intra ₁ , w_inter ₁ ) is for the region closest to the reference sample, and (w_intra ₄ , w_inter ₄ ) is for the region farthest from the reference sample. Then, the combined prediction can be computed by summing the two weighted predictions and shifting them right by 3 bits. Furthermore, the intra prediction mode for the intra hypothesis of the predictor is It can be stored for reference by neighboring CUs.

Assume that the intra and inter prediction values are PIntra and Pinter, and the weighting factors are w_intra and w_inter, respectively. The prediction value at position (x, y) is computed as (PIntra(x, y) * w_intra(x, y) + PInter(x, y) * w_inter(x, y)) >> N, where w_intra(x, y) + w_iner(x, y) = 2^N.

Signaling of intra prediction mode within an IIP-coded block

When the inter-intra mode is used, one of the four allowed intra-prediction modes, DC, planar, horizontal and vertical, is selected and signaled. Three Most Probable Modes (MPMs) are constructed from the left and above neighboring blocks. The intra-prediction mode of an intra-coded neighboring block or an IIP-coded neighboring block is treated as one MPM. If the intra-prediction mode is not one of the four allowed intra-prediction modes, it will be rounded up to vertical or horizontal mode depending on the angular difference. The neighboring blocks must be in the same CTU line as the current block.

Assume that the width and height of the current block are W and H. If W>2*H or H>2*W, only one of the three MPMs can be used in the inter-intra mode. Otherwise, all four valid intra-prediction modes can be used in the inter-intra mode.

It should be noted that the intra-prediction mode in inter-intra mode cannot be used to predict the intra-prediction mode in a normal intra-coded block.

Inter-intra prediction can only be used if W*H>=64.

2.10 Example triangle prediction mode

The concept of triangular prediction mode (TPM) is to introduce a new triangular partition for motion compensated prediction. As illustrated in Fig. 7a and Fig. 7b, it partitions one CU into two triangular prediction units in either the diagonal or the inverted diagonal direction. Each triangular prediction unit in the CU is inter-predicted using its own unidirectional prediction motion vector and a reference frame index derived from the unidirectional prediction candidate list. After predicting the triangular prediction unit, an adaptive weighting process is performed on the diagonal edges. Then, the transform and quantization processes are applied to the entire CU. Note that this mode is applied only to skip and merge modes.

2.10.1 List of one-way prediction candidates for TPM

The unidirectional prediction candidate list consists of five unidirectional prediction motion vector candidates. These are derived from seven neighboring blocks, which include five spatially neighboring blocks (1 to 5) and two temporally co-located blocks (6 to 7), as illustrated in Fig. 8. The motion vectors of the seven neighboring blocks are collected and added to the unidirectional prediction candidate list in the following order of the unidirectional prediction motion vectors: the L0 motion vector of the bidirectional prediction motion vector, the L1 motion vector of the bidirectional prediction motion vector, and the averaged motion vector of the L0 and L1 motion vectors of the bidirectional prediction motion vector. If the number of candidates is less than five, a zero motion vector is added to the list. The motion candidates added to this list are called TPM motion candidates.

More specifically, the following steps are involved:

1) Obtain motion candidates from A ₁ , B ₁ , B ₀ , A ₀ , B ₂ , Col and Col2 (corresponding to blocks 1 to 7 in Fig. 8) without any pruning operation.

2) Set the variable as follows: numCurrMergeCand = 0.

3) For each motion candidate derived from A ₁ , B ₁ , B ₀ , A ₀ , B ₂ , Col and Col2 , if numCurrMergeCand is less than 5 and the motion candidate is uni-predicted (from either list 0 or list 1), it is added to the merge list and numCurrMergeCand is increased by 1. This added motion candidate is named 'originally uni-predicted candidate'. Full pruning is applied.

4) For each motion candidate derived from A ₁ , B ₁ , B ₀ , A ₀ , B ₂ , Col and Col2 , if numCurrMergeCand is less than 5 and the motion candidate is a bidirectional prediction, the motion information from List 0 is added to the merge list (i.e., changed to a unidirectional prediction from List 0), and numCurrMergeCand is increased by 1. This added motion candidate is named 'Truncated List0-predicted candidate'. Full pruning is applied.

5) For each motion candidate derived from A ₁ , B ₁ , B ₀ , A ₀ , B ₂ , Col and Col2 , if numCurrMergeCand is less than 5 and the motion candidate is a bidirectional prediction, the motion information from List 1 is added to the merge list (i.e., changed to become a unidirectional prediction from List 1), and numCurrMergeCand is increased by 1. This added motion candidate is named 'Truncated List1-predicted candidate'. Full pruning is applied.

6) For each motion candidate derived from A ₁ , B ₁ , B ₀ , A ₀ , B ₂ , Col and Col2 , if numCurrMergeCand is less than 5 and the motion candidate is a bidirectional prediction,

- If the slice quantization parameter (QP) of the list 0 reference picture is smaller than the slice QP of the list 1 reference picture, the motion information of the list 1 is first scaled to the list 0 reference picture, and the average of two MVs (one is from the original list 0 and the other is a scaled MV from list 1), i.e., the averaged unidirectional prediction from the list 0 motion candidates, is added to the merge list, and numCurrMergeCand is increased by 1.

- Otherwise, the motion information in List 0 is first scaled to the List 1 reference picture, and the average of two MVs (one from the original List 1 and the other is the scaled MV from List 0), i.e., the averaged unidirectional prediction from the List 1 motion candidates, is added to the merge list, and numCurrMergeCand is increased by 1.

Full pruning is applied.

7) If numCurrMergeCand is less than 5, a zero motion vector candidate is added.

2.11 Decoder-side Motion Vector Refinement (DMVR) in VVC

For DMVR in VVC, MVD mirroring between List 0 and List 1 is assumed as illustrated in Fig. 13, and bilateral matching is performed to refine the MVs, for example, to find the best MVD among several MVD candidates. The MVs for the two reference picture lists are denoted as MVL0(L0X, L0Y), and MVL1(L1X, L1Y). The MVD denoted as (MvdX, MvdY) for List 0, which can minimize the cost function (e.g., SAD), is defined as the best MVD. For the SAD function, this is defined as the SAD between the reference block in list 0 derived by the motion vector (L0X+MvdX, L0Y+MvdY) in the list 0 reference picture and the reference block in list 1 derived by the motion vector (L1X-MvdX, L1Y-MvdY) in the list 1 reference picture.

The motion vector refinement process can be repeated twice. In each iteration, up to six MVDs (with integer-pel precision) can be checked in two steps as illustrated in Fig. 14. In the first step, MVDs (0, 0), (-1, 0), (1, 0), (0, -1), (0, 1) are checked. In the second step, one of MVDs (-1, -1), (-1, 1), (1, -1) or (1, 1) is selected and checked additionally. Assume that the function Sad(x, y) returns the SAD value of MVD (x, y). It is denoted as (MvdX, MvdY) and the MVD checked in the second step is determined as follows:

MvdX = -1;

MvdY = -1;

If (Sad(1, 0) < Sad(-1, 0))

MvdX = 1;

If (Sad(0, 1) < Sad(0, -1))

MvdY = 1;

In the first iteration, the starting point is the signaled MV, and in the second iteration, the starting point is the signaled MV plus the selected best MVD in the first iteration. DMVR is applied only when one reference picture is a preceding picture and the other reference picture is a succeeding picture, and the two reference pictures have the same picture order count distance from the current picture.

To further simplify the DMVR process, the following key features may be implemented in some embodiments:

1. Early termination when the SAD at position (0,0) between list0 and list1 is less than the critical value.

2. Early termination when the SAD between list0 and list1 is zero for some positions.

3. Block sizes for DMVR satisfy: W*H>=64 && H>=8, where W and H are the width and height of the block.

4. For DMVR with CU size larger than 16*16, split the CU into multiple 16x16 sub-blocks. Only when the width or height of the CU is larger than 16, it is split in the vertical or horizontal direction.

5. Reference block size (W+7)*(H+7) (for luma).

6. 25-point SAD-based integer-pel search (e.g. (+-) 2 refinement search range, single stage)

7. DMVR based on bilinear interpolation.

8. Sub-pixel refinement based on the "Parametric error surface equation". This procedure is performed only when, in the last MV refinement iteration, the minimum SAD cost is not equal to zero and the best MVD is (0, 0).

9. Luma/chroma MC with reference block padding (if required).

10. Refined MVs are used only for MC and TMVPs.

2.11.1 Purpose of DMVR

DMVR can be enabled if all of the following conditions are true:

- The DMVR enable flag in SPS (e.g. sps_dmvr_enabled_flag) is equal to 1.

- TPM flag, inter-affine flag, subblock merge flag (either ATMVP or affine merge), and MMVD flag are all equal to 0.

- The merge flag is equal to 1.

- The current block is bidirectionally predicted, and the Picture Order Count (POC) distance between the current picture and the reference picture in List 1 is equal to the POC distance between the reference picture in List 0 and the current picture.

- Current CU height is 8 or higher.

- The number of luma samples (CU width*height) is 64 or more.

2.11.2 Sub-pel refinement based on the "parametric error surface equation"

These methods are summarized as follows:

1. The parametric error surface fit is computed only if the center location is the best cost position for a given iteration.

2. The cost at the center location and the cost at positions (-1,0), (0,-1), (1,0) and (0,1) from the center are used to approximate the following 2-D parabolic error surface mathematical expression:

Here, ( corresponds to the location with the minimum cost, and C corresponds to the minimum cost value. By solving five mathematical equations with five unknowns, ( is calculated as follows:

( can be computed to any desired sub-pixel precision, by controlling the precision with which the division is performed (i.e. how many bits of the quotient are computed). For 1/16-pixel accuracy, only 4 bits within the absolute value of the quotient need be computed, which provides an implementation based on a fast shifted subtraction of two divisions required per CU.

3. Calculated ( is added to the integer distance refined MV to obtain sub-pixel accurate refined delta MV.

2.11.3 Reference samples required in DMVR

For a block of size W * H, assuming the maximum allowable MVD value is +/- offset (e.g. 2 in VVC) and the filter size is filterSize (e.g. 8 for luma and 4 for chroma in VVC), (W + 2 * offSet + filterSize - 1) * (H + 2 * offSet + filterSize - 1) reference samples are required. To reduce the memory bandwidth, only the center (W + filterSize - 1) * (H + filterSize - 1) reference samples are fetched and the other pixels are generated by repeating the boundaries of the fetched samples. An example for an 8 * 8 block is shown in Fig. 15, where 15 * 15 reference samples are fetched and the boundaries of the fetched samples are repeated to generate a 17 * 17 region.

During motion vector refinement, bilinear motion compensation is performed using these reference samples. Meanwhile, final motion compensation is also performed using these reference samples.

2.12 Calculating bandwidth for different block sizes

Based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the memory bandwidth for each block unit (one MxN luma block with two M/2 x N/2 chroma blocks for 4:2:0 color format) is written in Table 1 below.

Block size
MxN Required samples/pixels One way Two way 4*4 (11*11+2*5*5)/(4*4) 10.688 21.375 4*8 (11*15+2*5*7)/(4*8) 7.344 14.688 4*16 (11*23+2*5*11)/(4*16) 5.672 11.344 4*32 (11*39+2*5*19)/(4*32) 4.836 9.672 4*64 (11*71+2*5*35)/(4*64) 4.418 8.836 4*128 (11*135+2*5*67)/(4*128) 4.209 8.418 8*8 (15*15+2*7*7)/(8*8) 5.047 10.094 8*16 (15*23+2*7*11)/(8*16) 3.898 7.797 8*32 (15*39+2*7*19)/(8*32) 3.324 6.648 8*64 (15*71+2*7*35)/(8*64) 3.037 6.074 8*128 (15*135+2*7*67)/(8*128) 2.894 5.787

Table 1 Exemplary memory bandwidths

Similarly, based on the current 8-tap luma interpolation filter and 4-tap chroma interpolation filter, the memory bandwidth for each MxN luma block unit is written in Table 2 below.

Block size
MxN Required samples/pixels One way Two way 4*4 (11*11)/(4*4) 7.563 15.125 4*8 (11*15)/(4*8) 5.156 10.313 4*16 (11*23)/(4*16) 3.953 7.906 4*32 (11*39)/(4*32) 3.352 6.703 4*64 (11*71)/(4*64) 3.051 6.102 4*128 (11*135)/(4*128) 2.900 5.801 8*8 (15*15)/(8*8) 3.516 7.031 8*16 (15*23)/(8*16) 2.695 5.391 8*32 (15*39)/(8*32) 2.285 4.570 8*64 (15*71)/(8*64) 2.080 4.160 8*128 (15*135)/(8*128) 1.978 3.955

Table 2 Exemplary memory bandwidths

Therefore, regardless of color format, the bandwidth requirements for each block size, in descending order, are:

4*4 Bi > 4*8 Bi > 4*16 Bi > 4*4 Bi > 8*8 Bi > 4*32 Bi > 4*64Bi > 4*128 Bi >8*16 Bi >4*8 Uni >8 *32 Bi >… .

2.12 Motion vector accuracy issue in VTM-3.0

In VTM-3.0, the MV precision is stored as 1/16 luma pixel. When MV is signaled, the finest precision is 1/4 luma pixel.

3. Examples of problems solved by the disclosed embodiments

1. Bandwidth control methods related to affine prediction are not sufficiently clear and should have higher elasticity.

2. In HEVC design, the worst case for memory bandwidth requirement is 8x8 bidirectional prediction, and note that a single coding unit (CU) can also be split into asymmetric prediction modes, for example, a 16x16 can be split into two PUs of different sizes, such as 4x16 and 12x16. In VVC, due to the novel QTBT partition structure, a CU can be set to 4x16 and bidirectional prediction can be enabled. A bidirectionally predicted 4x16 CU requires higher memory bandwidth compared to a bidirectionally predicted 8x8 CU. It is not known how to handle block sizes requiring higher bandwidth (e.g., 4x16 or 16x4).

3. New coding tools like GBi cause more line buffer issues

4. Inter-intra mode requires more memory and logic to signal the intra-prediction mode used within an inter-coded block.

5. 1/16 luma pixel MV precision requires higher memory storage.

6. To interpolate four 4x4 blocks within one 8x8 block, (8 + 7 + 1) * (8 + 7 + 1) reference pixels need to be fetched, which requires about 14% more pixels compared to non-affine/non-planar mode 8x8 blocks.

7. The averaging operation in hybrid intra and inter prediction should align with other coding tools, such as weighted prediction, local illumination compensation, OBMC, and triangular prediction, where the offset is added before the transition is made.

4. Examples of embodiments

The techniques disclosed herein can reduce the bandwidth and line buffers required in affine prediction and other novel coding tools.

The following description should be considered as an example to illustrate the general concept and should not be construed in a narrow sense. Furthermore, the embodiments may be combined in any manner.

In the following description, the width and height of the affine-coded current CU are w and h, respectively. It is assumed that the number of interpolation filter taps (interpolation filter taps in motion compensation) is N (e.g., 8, 6, 4, or 2), and the current block size is WxH.

Bandwidth control for affine prediction

Example 1: Assuming that the motion vector of sub-block SB in an affine coded block is MV _SB (denoted as (MVx, MVy)), MV _SB can fall within a certain range relative to the representative motion vector MV'(MV'x, MV'y).

In some embodiments, MVx >=MV'x-DH0 and MVx<=MV'x+DH1 and MVy >=MV'y-DV0 and MVy<=MV'y+DV1, where MV' = (MV'x, MV'y). In some embodiments, DH0 may or may not be equal to DH1; DV0 may or may not be equal to DV1. In some embodiments, DH0 may or may not be equal to DV0; DH1 may or may not be equal to DV1. In some embodiments, DH0 may or may not be equal to DH1; DV0 may or may not be equal to DV1. In some embodiments, DH0, DH1, DV0 and DV1 can be signaled from encoder to decoder, e.g., in VPS/SPS/PPS/slice header/tile group header/tile/CTU/CU/PU. In some embodiments, DH0, DH1, DV0 and DV1 can be specified differently for different standard profiles/levels/tiers. In some embodiments, DH0, DH1, DV0 and DV1 can vary depending on the width and height of the current block. In some embodiments, DH0, DH1, DV0 and DV1 can vary depending on whether the current block is unidirectional or bidirectional. In some embodiments, DH0, DH1, DV0 and DV1 can vary depending on the location of the sub-block SB. In some embodiments, DH0, DH1, DV0 and DV1 can vary depending on how to obtain the MV'.

In some embodiments, MV' can be a CPMV, such as MV0, MV1 or MV2.

In some embodiments, MV' may be an MV used in MC for one of the corner sub-blocks, for example, MV0', MV1' or MV2' of FIG. 3.

In some embodiments, MV' is an affine model of the current block, which can be an MV derived for any location within or outside the current block. For example, it can be derived for the center location of the current block (e.g., x=w/2 and y=h/2).

In some embodiments, MV' can be an MV used in MC for any sub-block of the current block, for example, one of the central sub-blocks (C0, C1, C2 or C3 as shown in FIG. 3).

In some embodiments, if the MV _SB does not satisfy the constraints, the MV _SB should be clipped to a valid range. In some embodiments, the clipped MV _SB is stored in the MV buffer, which will be used to predict MVs of the next coded block. In some embodiments, the MV _SB before being clipped is stored in the MV buffer.

In some embodiments, if an MV _SB does not satisfy a constraint, the bit-stream is considered non-compliant (invalid). In one example, the standard may specify that an MV _SB must satisfy a constraint. This constraint must be satisfied by any constraint-compliant encoder, otherwise the encoder is considered non-compliant.

In some embodiments, MV _SB and MV' can be expressed in signaling MV precision (e.g., quarter-pixel precision). In some embodiments, MV _SB and MV' can be expressed in storage MV precision (e.g., 1/16 precision). In some embodiments, MV _SB and MV' can be rounded to a precision different from the signaling or storage precision (e.g., integer precision).

Example 2: For an affine coded block, each MxN (e.g., 8x4, 4x8 or 8x8) block within that block is considered as a basic unit. All 4x4 sub-block MVs within an MxN block are constrained such that the maximum difference between the integer parts of four 4x4 sub-block MVs does not exceed K pixels.

In some embodiments, whether and how these constraints are applied depends on whether the current block applies bidirectional or unidirectional prediction. For example, these constraints are applied only for bidirectional prediction, not unidirectional prediction. As another example, M, N and K are different for bidirectional and unidirectional prediction.

In some embodiments, M, N and K may vary depending on the width and height of the current block.

In some embodiments, whether to apply a constraint can be signaled from the encoder to the decoder, e.g., in the VPS/SPS/PPS/slice header/tile group header/tile/CTU/CU/PU. For example, an on/off flag is signaled to indicate whether to apply a constraint. As another example, M, N and K are signaled.

In some embodiments, M, N and K may be defined differently for different standard profiles/levels/tiers.

Example 3: The width and height of a sub-block can be computed differently for different affine coded blocks.

In some embodiments, the computation method is different for affine coded blocks with unidirectional prediction and bidirectional prediction. In one example, the sub-block size is fixed for blocks with unidirectional prediction (e.g., fixed to 4×4, 4×8 or 8×4). In another example, the sub-block size is computed for blocks with bidirectional prediction. In such cases, the sub-block size may be different for two different affine blocks with bidirectional prediction.

In some embodiments, for a bidirectionally predicted affine block, the width and/or height of a sub-block from reference list 0 and the width and/or height of a sub-block from reference list 1 can be different. In one example, assume that the width and height of a sub-block from reference list 0 are derived as Wsb0 and Hsb0, respectively; and that the width and height of a sub-block from reference list 1 are derived as Wsb1 and Hsb1, respectively. Then, the final width and height of the sub-block for both reference list 0 and reference list 1 are computed as Max(Wsb0, Wsb1) and Max(Hsb0, HSb1), respectively.

In some embodiments, the computed width and height of a sub-block applies only to the luma component. For the chroma component, this is always fixed, for example, to a 4x4 chroma sub-block, which corresponds to an 8x8 luma block with 4:2:0 color format.

In some embodiments, MVx-MV'x and MVy - MV'y are computed to determine the width and height of a sub-block. (MVx, MVy) and (MV'x, MV'y) are defined in Example 1.

In some embodiments, the MVs involved in these computations may be expressed in signaling MV precision (e.g., quarter-pixel precision). In one example, these MVs may be expressed in storage MV precision (e.g., 1/16 precision). As another example, these MVs may be rounded to a precision different from the signaling or storage precision (e.g., integer precision).

In some embodiments, the thresholds used in the calculations to determine the width and height of a sub-block may be signaled from the encoder to the decoder, e.g. in a VPS/SPS/PPS/slice header/tile group header/tile/CTU/CU/PU.

In some embodiments, the thresholds used in the calculations to determine the width and height of a sub-block may be different for different standard profiles/levels/tiers.

Example 4: To interpolate W1xH1 sub-blocks within a single W2xH2 sub-block/block, (W2 + N - 1 - PW) * (H2 + N - 1 - PH) blocks are first fetched, then the pixel padding method described in Example 6 (e.g., boundary pixel repetition method) is applied to generate a larger block, which is now used to interpolate W1xH1 sub-blocks. For example, W2=H2=8, W1=H1=4, and PW=PH=0.

In some embodiments, an integer part of the MV of any W1xH1 sub-block may be used to fetch the entire W2xH2 sub-block/block, and different boundary pixel repetition methods may be required accordingly. For example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 1 pixel, then an integer part of the MV of the top-left W1xH1 sub-block is used to fetch the entire W2xH2 sub-block/block. The right and bottom boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 1 pixel, then an integer part of the MV of the bottom-right W1xH1 sub-block is used to fetch the entire W2xH2 sub-block/block. The left and top boundaries of the reference block are repeated once.

In some embodiments, the MV of any W1xH1 sub-block may be modified first and then used to fetch the entire W2xH2 sub-block/blocks, and different boundary pixel repetition methods may be required accordingly. For example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 2 pixels, then the integer part of the MV of the top-left W1xH1 sub-block may be incremented by (1, 1) (where 1 means 1 integer pixel distance), and then used to fetch the entire W2xH2 sub-block/blocks. In such a case, the left, right, top and bottom boundaries of the reference block are repeated once. As another example, if the maximum difference between the integer parts of all W1xH1 sub-block MVs does not exceed 2 pixels, the integer part of the MV of the bottom-right W1xH1 sub-block can be incremented by (-1, -1) (where 1 means 1 integer pixel distance), and then used to fetch the entire W2xH2 sub-block/blocks. In this case, the left, right, top and bottom boundaries of the reference block are repeated once.

Bandwidth control for specific block sizes

Example 5: Bidirectional prediction is not allowed if w and h of the current block satisfy one or more of the following conditions:

A. w is equal to T1 and h is equal to T2, or h is equal to T1 and w is equal to T2. In one example, T1 = 4 and T2 = 16.

B. w is equal to T1 and h is not greater than T2, or h is equal to T1 and w is not greater than T2. In one example, T1 = 4 and T2 = 16.

C. w is not greater than T1 and h is not greater than T2, or h is not greater than T1 and w is not greater than T2. In one example, T1 = 8 and T2 = 8. In another example, T1==8, T2==4. In yet another example, T1==4 and T2==4.

In some embodiments, bidirectional prediction may be disabled for 4x8 blocks. In some embodiments, bidirectional prediction may be disabled for 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16 blocks. In some embodiments, bidirectional prediction may be disabled for 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x8, 8x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16, 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x8, 16x4 blocks. In some embodiments, bidirectional prediction may be disabled for 4x16, 8x4 blocks. In some embodiments, bidirectional prediction can be disabled for 4xN blocks, for example, where N <= 16. In some embodiments, bidirectional prediction can be disabled for Nx4 blocks, for example, where N <= 16. In some embodiments, bidirectional prediction can be disabled for 8xN blocks, for example, where N <= 16. In some embodiments, bidirectional prediction can be disabled for Nx8 blocks, for example, where N <= 16. In some embodiments, bidirectional prediction can be disabled for 4x8, 8x4, 4x16 blocks. In some embodiments, bidirectional prediction can be disabled for 4x8, 8x4, 16x4 blocks. In some embodiments, bidirectional prediction can be disabled for 8x4, 4x16, 16x4 blocks. In some embodiments, bidirectional prediction can be disabled for 4x8, 8x4, 4x16, and 16x4 blocks.

In some embodiments, the block size disclosed herein may refer to one color component, such as a luma component, and the determination of whether bidirectional prediction is disabled may apply to all color components. For example, if bidirectional prediction is disabled based on the block size of the luma component of a block, bidirectional prediction will also be disabled for the corresponding block of the other color components. In some embodiments, the block size disclosed herein may refer to one color component, such as a luma component, and the determination of whether bidirectional prediction is disabled may apply only to that color component.

In some embodiments, if bidirectional prediction is disabled for a block and the selected merge candidate is bidirectionally predicted, only one MV from reference list 0 or reference list 1 of the merge candidate is assigned to that block.

In some embodiments, if bidirectional prediction is disabled for a block, triangular prediction mode (TPM) is not allowed for that block.

In some embodiments, how to signal the prediction direction (unidirectional prediction from list 0/1, bidirectional prediction) can vary depending on the block dimension. In one example, an indication of unidirectional prediction from list 0/1 can be signaled if 1) block width * block height < 64 or 2) block width * block height = 64 but width is not equal to height. As another example, an indication of unidirectional prediction or bidirectional prediction from list 0/1 can be signaled if 1) block width * block height > 64 or 2) block width * block height = 64 and width is equal to height.

In some embodiments, both unidirectional prediction and bidirectional prediction may be disabled for 4x4 blocks. In some embodiments, it may be disabled for affine coded blocks. Alternatively, it may be disabled for non-affine coded blocks. In some embodiments, the indication of quad-tree splitting for 8x8 blocks, dual-tree splitting for 8x4 or 4x8 blocks, or triple-tree splitting for 4x16 or 16x4 blocks may be skipped. In some embodiments, the 4x4 blocks must be coded as intra-blocks. In some embodiments, the MV for the 4x4 blocks must have integer precision. For example, the IMV flag for the 4x4 blocks must be 1. As another example, the MV for the 4x4 blocks must be rounded to integer precision.

In some embodiments, bidirectional prediction is allowed. However, assuming the number of interpolation filter taps is N, instead of fetching (W + N - 1) * (H + N - 1) reference pixels, only (W + N - 1 - PW) * (W + N - 1 - PH) reference pixels are fetched. Meanwhile, pixels at the reference block boundaries (top, left, bottom and right boundaries) are repeated to generate a (W + N - 1) * (H + N - 1) block as illustrated in FIG. 9, which is used for the final interpolation. In some embodiments, PH is zero, and only the left and/or right boundaries are repeated. In some embodiments, PW is zero, and only the top and/or bottom boundaries are repeated. In some embodiments, both PW and PH are greater than zero, and the left and/or right boundaries are repeated first, followed by the top and/or bottom boundaries. In some embodiments, both PW and PH are greater than zero, and first the top and/or bottom boundaries are repeated, followed by the left and right boundaries. In some embodiments, the left boundary is repeated M1 times and the right boundary is repeated PW - M1 times. In some embodiments, the top boundary is repeated M2 times and the bottom boundary is repeated PH - M2 times. In some embodiments, this boundary pixel repetition method can be applied to some or all of the reference blocks. In some embodiments, PW and PH can be different for different color components, such as Y, Cb and Cr.

Figure 9 illustrates an example of repeat boundary pixels of a reference block before interpolation.

Example 6: In some embodiments, (W + N - 1 - PW) * (W + N - 1 - PH) reference pixels (instead of (W + N - 1) * (H + N - 1) reference pixels) can be fetched for motion compensation of a WxH block. Samples that are outside the range (W + N - 1 - PW) * (W + N - 1 - PH) but within the range (W + N - 1) * (H + N - 1) are padded to perform the interpolation process. In one padding method, pixels at the reference block boundaries (top, left, bottom and right boundaries) are repeated to generate a (W + N - 1) * (H + N - 1) block as illustrated in FIG. 11, which is used for the final interpolation.

In some embodiments, PH is zero and only the left and/or right boundaries are repeated.

In some embodiments, PW is zero and only the upper and/or lower boundaries are repeated.

In some embodiments, both PW and PH are greater than zero, and first the left and/or right boundaries are repeated, followed by the top and/or bottom boundaries.

In some embodiments, both PW and PH are greater than zero, and first the upper and/or lower boundaries are repeated, followed by the left and right boundaries.

In some embodiments, the left boundary is repeated M1 times and the right boundary is repeated PW - M1 times.

In some embodiments, the upper boundary is repeated M2 times and the lower boundary is repeated PH - M2 times.

In some embodiments, this boundary pixel repetition method may be applied to some or all of the reference blocks.

In some embodiments, PW and PH can be different for different color components, such as Y, Cb, and Cr.

In some embodiments, PW and PH may be different for different block sizes or shapes.

In some embodiments, PW and PH can be different for unidirectional and bidirectional prediction.

In some embodiments, padding may not be performed in affine mode.

In some embodiments, samples outside the range (W + N - 1 - PW) * (W + N - 1 - PH) but within the range (W + N - 1) * (H + N - 1) are set to a single value. In some embodiments, the single value is 1<<(BD-1), where BD is the bit-depth of the sample, e.g., 8 or 10. In some embodiments, the single value is signaled from the encoder to the decoder within a VPS/SPS/PPS/slice header/tile group header/tile/CTU row/CTU/CU/PU. In some embodiments, the single value is derived from samples within the range (W + N - 1 - PW) * (W + N - 1 - PH).

Example 7: Instead of fetching (W + filterSize - 1) * (H + filterSize - 1) reference samples in DMVR, (W + filterSize - 1 - PW) * (H + filterSize - 1 - PH) reference samples can be fetched, and all other required samples can be generated by iterating over the boundaries of the fetched reference samples, where PW >= 0 and PH >= 0.

In some embodiments, the method proposed in Example 6 can be used to pad non-fetched samples.

In some embodiments, padding may not be performed again in the final motion compensation of DMVR.

In some embodiments, whether or not to apply the above-described method may depend on the block dimension.

Example 8: The signaling method for inter_pred_idc can vary depending on whether w and h satisfy the conditions in Example 5. One example is shown in Table 3 below:

inter_pred_idc Name of inter_pred_idc !((w==T1 && h==T2) ||(h==T1 && w==T2)) (w==T1 && h==T2) ||(h==T1 && w==T2) 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Another example is shown in Table 4 below:

inter_pred_idc Name of inter_pred_idc !((w==T1 && h<=T2) ||(h==T1 && w<=T2)) (w==T1 && h<=T2) ||(h==T1 && w<=T2) 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Another example is shown in Table 5 below:

inter_pred_idc Name of inter_pred_idc !((w<=T1 && h<=T2) ||(h<=T1 && w<=T2)) (w<=T1 && h<=T2) ||(h<=T1 && w<=T2) 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Example 9 : The process of constructing the list of merge candidates may vary depending on whether w and h satisfy the conditions of Example 4. The following embodiments describe the case where w and h satisfy the conditions.

In some embodiments, if a merge candidate uses bidirectional prediction, only predictions from reference list 0 are retained, and the merge candidate is treated as a unidirectional prediction referencing reference list 0.

In some embodiments, if a merge candidate uses bidirectional prediction, only predictions from reference list 1 are retained, and the merge candidate is treated as a unidirectional prediction referencing reference list 1.

In some embodiments, if a merge candidate uses bidirectional prediction, that candidate is treated as unavailable, i.e., the merge candidate is removed from the merge list.

In some embodiments, a merge candidate list construction process for the triangle prediction mode is used instead.

Example 10: The coding tree splitting process may vary depending on whether the width and height of the child CU after the splitting satisfies the conditions of Example 5.

In some embodiments, a split is not allowed if the width and height of the child CU after the split satisfies the conditions of Example 5. In some embodiments, the signaling of a coding tree split may vary depending on whether a type of split is allowed. In one example, if a type of split is not allowed, the codeword representing that split is omitted.

Example 11 : Signaling of the skip flag and/or the Intra Block Copy (IBC) flag may vary depending on whether the width and/or height of the block satisfies certain conditions (e.g., the conditions mentioned in Example 5).

In some embodiments, the condition is that a luma block does not have more than X samples, for example X = 16;

In some embodiments, the condition is that the luma block has X samples, for example X = 16.

In some embodiments, the condition is that both the width and the height of the luma block are equal to X, for example X = 4;

In some embodiments, if one or some of the above conditions are true, inter mode and/or IBC mode are not allowed for such blocks.

In some embodiments, if inter mode is not allowed for a block, the skip flag may not be signaled for it. Or, further, the skip flag may be inferred to be false.

In some embodiments, if inter mode and IBC mode are not allowed for a block, the skip flag may not be signaled for this and may be implicitly driven to false (e.g., the block is driven to be coded in non-skip mode).

In some embodiments, if inter mode is not allowed for a block, but IBC mode is allowed for that block, the skip flag may still be signaled. In some embodiments, if the block is coded in skip mode, the IBC mode may not be signaled, and the IBC flag is implicitly driven to true (e.g., the block is driven to be coded in IBC mode).

Example 12: Signaling of the prediction mode may vary depending on whether the width and/or height of the block satisfies certain conditions (e.g., the conditions mentioned in Example 5).

In some embodiments, the condition is that a luma block does not have more than X samples, for example X = 16.

In some embodiments, the condition is that the luma block has X samples, for example X = 16.

In some embodiments, the condition is that both the width and the height of the luma block are equal to X, for example X = 4.

In some embodiments, if one or some of the above conditions are true, inter mode and/or IBC mode are not allowed for such blocks.

In some embodiments, signaling of indication of a particular mode may be skipped.

In some embodiments, if neither the Inter mode nor the IBC mode is allowed for a block, the signaling of the indication of the Inter and IBC modes is skipped, and the remaining allowed modes, such as whether it is an Intra mode or a Palette mode, can still be signaled.

In some embodiments, if neither inter mode nor IBC mode is allowed for a block, the prediction mode may not be signaled. Or, further, the prediction mode may be implicitly derived to be intra mode.

In some embodiments, if inter mode is not allowed for a block, the signaling of the inter mode indication is skipped, and the remaining allowed modes, such as whether it is intra mode or IBC mode, can still be signaled. Alternatively, the remaining allowed modes, such as whether it is intra mode or IBC mode or palette mode, can still be signaled.

In some embodiments, if inter mode is not allowed for a block, but IBC mode and intra mode are allowed for that block, an IBC flag may be signaled to indicate whether the block is coded in IBC mode. Alternatively, the prediction mode may not be signaled.

Example 13 : Signaling of the triangle mode may vary depending on whether the width and/or height of the block satisfies certain conditions (e.g., the conditions mentioned in Example 5).

In some embodiments, the condition is that the luma block size is one of several specific sizes. For example, the specific sizes may include 4x16 and/or 16x4.

In some embodiments, if the above condition is true, the triangle mode may not be allowed, and the flag indicating whether the current block is coded in triangle mode may not be signaled and may be caused to be false.

Example 14: Signaling of inter prediction direction may vary depending on whether the width and/or height of the block satisfies certain conditions (e.g., the conditions mentioned in Example 5).

In some embodiments, the condition is that the luma block size is one of several specific sizes. For example, the specific sizes may include 8x4 and/or 4x8 and/or 4x16 and/or 16x4.

In some embodiments, if the above condition is true, the block is only unidirectionally predicted, and the flag indicating whether the current block is bidirectionally predicted may not be signaled and may be driven to false.

Example 15: The signaling of SMVD (symmetric MVD) may vary depending on whether the width and/or height of the block satisfies certain conditions (e.g., the conditions mentioned in Example 5).

In some embodiments, the condition is that the luma block size is one of several specific sizes. In some embodiments, the condition is defined as the block size does not have more than 32 samples. In some embodiments, the condition is defined as whether the block size is 4x8 or 8x4. In some embodiments, the condition is defined as whether the block size is 4x4, 4x8 or 8x4. In some embodiments, the specific sizes can include 8x4 and/or 4x8 and/or 4x16 and/or 16x4.

In some embodiments, if certain conditions are true, the indication of use of SMVD (e.g., the SMVD flag) may not be signaled and may be caused to be false. For example, a block may be set to be unidirectionally predicted.

In some embodiments, if certain conditions are true, an indication of the use of SMVD (e.g., an SMVD flag) may still be signaled, but only List 0 or List 1 motion information may be utilized in the motion compensation process.

Example 16: Motion vectors (such as motion vectors derived from regular merge mode, ATMVP mode, MMVD merge mode, MMVD skip mode, etc.) or block vectors used for IBC may be changed depending on whether the width and/or height of the block satisfies certain conditions.

In some embodiments, the condition is that the luma block size is one of several specific sizes. For example, the specific sizes may include 8x4 and/or 4x8 and/or 4x16 and/or 16x4.

In some embodiments, if the above condition is true, the motion vector of the corresponding block or block vector can be converted into a uni-directional motion vector (e.g., inherited from a neighboring block with some offset) if the derived motion information is bi-directional prediction. This process is called a conversion process, and the final uni-directional motion vector is named a 'converted uni-directional' motion vector. In some embodiments, the motion information of the reference picture list X (e.g., X is 0 or 1) can be maintained, and this motion information and the motion information of the list Y (Y is 1 - X) can be discarded. In some embodiments, the motion information of the reference picture list X (e.g., X is 0 or 1) and the motion information of the list Y (Y is 1 - X) can be jointly utilized to derive a new motion candidate point into the list X. In one example, the motion vector of the new motion candidate can be an averaged motion vector of the two reference picture lists. As another example, the motion information of list Y can be first scaled to match list X. Then, the motion vector of the new motion candidate can be the averaged motion vector of the two reference picture lists. In some embodiments, the motion vector in the prediction direction X can be not used (e.g., the motion vector in the prediction direction X is changed to (0, 0) and the reference index in the prediction direction X is changed to -1), and the prediction direction can be changed to 1 - X, where X = 0 or 1. In some embodiments, the transformed unidirectional motion vectors can be used to update the HMVP lookup table. In some embodiments, the derived bidirectional motion information, e.g., the bidirectional MVs before being converted to unidirectional MVs, can be used to update the HMVP lookup table. In some embodiments, the transformed unidirectional motion vectors can be stored and used for motion prediction of subsequent coded blocks, TMVP, deblocking, etc. In some embodiments, the derived bidirectional motion information, e.g., the bidirectional MVs before being converted to unidirectional MVs, can be stored and used for motion prediction of subsequent coded blocks, TMVP, deblocking, etc. In some embodiments, the converted unidirectional motion vectors can be used for motion refinement. In some embodiments, the derived bidirectional motion information can be used for motion refinement and/or sample refinement, e.g., using an optical flow method. In some embodiments, the prediction blocks generated according to the derived bidirectional motion information can be first refined, and then only one prediction block can be utilized to derive the final prediction and/or reconstruction block of a block.

In some embodiments, if certain conditions are true, (bidirectionally predicted) motion vectors may be converted to unidirectional motion vectors before being used as base merging candidates in MMVD.

In some embodiments, if certain conditions are true (e.g., if the dimension of the block satisfies the conditions specified in Example 5 above), (bidirectionally predicted) motion vectors may be converted to unidirectional motion vectors before being inserted into the merge list.

In some embodiments, the transformed unidirectional motion vectors may come only from reference list 0. In some embodiments, if the current slice/tile group/picture is bidirectionally predicted, the transformed unidirectional motion vectors may come from reference list 0 or list 1. In some embodiments, if the current slice/tile group/picture is bidirectionally predicted, the transformed unidirectional motion vectors from reference list 0 and list 1 may be interleaved within the merge list and/or the MMVD base merge candidate list.

In some embodiments, how the motion information is converted into unidirectional motion vectors may vary depending on the reference pictures. In some embodiments, if all the reference pictures (e.g., tiles/tile groups) of one video data unit are past pictures in display order, the List 1 motion information may be utilized. In some embodiments, if at least one of the reference pictures (e.g., tiles/tile groups) of one video data unit is a past picture and at least one is a future picture in display order, the List 0 motion information may be utilized. In some embodiments, how the motion information is converted into unidirectional motion vectors may vary depending on the low-latency check flag.

In some embodiments, the transform process may be called immediately before the motion compensation process. In some embodiments, the transform process may be called immediately after the motion candidate list (e.g., merge list) construction process. In some embodiments, the transform process may be called before calling the MVD adding process in the MMVD process. That is, the MVD adding process follows the design of unidirectional prediction instead of bidirectional prediction. In some embodiments, the transform process may be called before calling the sample refinement process in the PROF process. That is, the MVD adding process follows the design of unidirectional prediction instead of bidirectional prediction. In some embodiments, the transform process may be called before calling the BIO (also known as BDOF) process. That is, in some cases, BIO may be disabled because it has been converted to unidirectional prediction. In some embodiments, the transform process may be called before calling the DMVR process. That is, in some cases, DMVR may be disabled because it has been converted to unidirectional prediction.

Example 17: In some embodiments, how the list of motion candidates is generated may vary depending on the block dimension, for example as described above in Example 5.

In some embodiments, for a certain block dimension, all motion candidates derived from spatial blocks and/or temporal blocks and/or HMVPs and/or other kinds of motion candidates may be restricted to be unidirectionally predicted.

In some embodiments, for a certain block dimension, a motion candidate derived from a spatial block and/or a temporal block and/or an HMVP and/or another kind of motion candidate, if bidirectionally predicted, may first be unidirectionally predicted before being added to the candidate list.

Example 18: Whether shared merge lists are allowed may depend on the encoding mode.

In some embodiments, a shared merge list may not be allowed for blocks coded in regular merge mode, but may be allowed for blocks coded in IBC mode.

In some embodiments, if a block split from a parent shared node is coded in regular merge mode, updates to the HMVP table may be disabled after encoding/decoding that block.

Example 19: In the above disclosed examples, the block size/width/height for the luma block can also be changed to the block size/width/height for the chroma blocks such as Cb, Cr, or G/B/R.

Reduced line buffer for GBi mode

Example 20: Whether the GBi weighted index can be inherited or predicted from a neighboring block (including CABAC context selection) depends on the location of the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not within the same coding tree unit (CTU, also known as the largest coding unit (LCU)) as the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not within the same CTU line or CTU row as the current block.

In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not within the same MxN region as the current block. For example, M=N=64. In such a case, a single tile/slice/picture is split into multiple non-overlapping MxN regions.

In some embodiments, the GBi weighted index cannot be inherited or predicted from a neighboring block that is not within the same M×N region line or M×N region row as the current block. For example, M=N=64. CTU line/row and region line/row are illustrated in Fig. 10.

In some embodiments, if the top-left corner (or any other location) of the current block is (x, y), and the top-left corner (or any other location) of the neighboring block is (x', y'), then this cannot be inherited or predicted from the neighboring block if one of the following conditions is satisfied:

(1) x / M != x' / M. For example, M = 128 or 64.

(2) y / N != y' / N. For example, N = 128 or 64.

(3) ((x / M != x' / M) && (y / N != y' / N)). For example, M = N=128 or M= N = 64.

(4) ((x / M != x' / M) || (y / N != y' / N)). For example, M = N=128 or M= N = 64.

(5) x >> M != x' >> M . For example, M = 7 or 6.

(6) y >> N != y' >> N. For example, N = 7 or 6.

(7) ((x >> M != x' >>M) && (y >> N != y' >>N)). For example, M = N=7 or M= N = 6.

(8) ((x >> M != x' >>M) || (y >> N != y' >>N)). For example, M = N=7 or M= N = 6.

In some embodiments, a flag is signaled in the PPS or slice header or tile group header or tile to indicate whether GBi can be applied within a picture/slice/tile group/tile. In some embodiments, whether GBi is used and how GBi is used (e.g., how many candidate weights and values of the weights are present) can be derived for a picture/slice/tile group/tile. In some embodiments, this derivation can depend on information such as QP, temporal layer, POC distance, etc.

Figure 10 illustrates an example of CTU (region) lines. The shaded CTUs (regions) are within one CUT (region) line, and the unshaded CTUs (regions) are within another CUT (region) line.

Simplification for inter-intra prediction (IIP)

Example 21: Coding of intra-prediction modes within an IIP-coded block is coded independently of the intra-prediction modes of neighboring IIP-coded blocks.

In some embodiments, only the intra-prediction mode of an intra-coded block may be used in coding the intra-prediction mode for an IIP-coded block, e.g., during the MPM list construction process.

In some embodiments, intra-prediction modes within an IIP-coded block are coded without mode prediction from any neighboring blocks.

Example 22 : When both are used to code the intra-prediction mode of a new IIP-coded block, the intra-prediction mode of the IIP-coded block may have a lower priority than the intra-prediction mode of the intra-coded block.

In some embodiments, when deriving MPMs for an IIP-coded block, intra-prediction modes of both the IIP-coded block and the intra-coded neighboring blocks are utilized. However, intra-prediction modes from the intra-coded neighboring blocks may be inserted into the MPM list before intra-prediction modes from the IIP-coded neighboring blocks.

In some embodiments, intra prediction modes from intra-coded neighboring blocks may be inserted into the MPM list after intra prediction modes from IIP-coded neighboring blocks.

Example 23 : The intra-prediction mode within an IIP-coded block can also be used to predict the intra-prediction mode of an intra-coded block.

In some embodiments, the intra-prediction mode within an IIP-coded block may be used to derive MPMs for a normal intra-coded block. In some embodiments, when used to derive MPMs for a normal intra-coded block, the intra-prediction mode within an IIP-coded block may have a lower priority than the intra-prediction mode within an intra-coded block.

In some embodiments, an intra-prediction mode within an IIP-coded block may also be used to predict an intra-coded block or an intra-prediction mode of an IIP-coded block only if one or more of the following conditions are satisfied:

1. The two blocks must be within the same CTU line.

2. The two blocks must be within the same CTU.

3. Two blocks will be within the same M×N region, where, for example, M=N=64.

4. The two blocks will be within a line of M×N area, where, for example, M=N=64.

Example 24: In some embodiments, the MPM construction process for an IIP-coded block should be identical to the MPM construction process for a regular intra-coded block.

In some embodiments, six MPMs are used for inter-coded blocks with inter-intra prediction.

In some embodiments, only some of the MPMs are used for IIP-coded blocks. In some embodiments, the first one is always used. Or, further, there is no need to signal the MPM flag and the MPM index. In some embodiments, the first four MPMs can be utilized. Or, further, there is no need to signal the MPM flag, but the MPM index needs to be signaled.

In some embodiments, each block may select one of the MPM lists based on the intra prediction modes contained in the MPM list, e.g., the mode with the minimum index compared to a given mode (e.g., a planar mode).

In some embodiments, each block may select a subset of modes from the MPM list and signal a mode index within the subset.

In some embodiments, the context used to code the intra MPM mode is reused to code the intra modes within the IIP-coded blocks. In some embodiments, different contexts used to code the intra MPM mode are employed to code the intra modes within the IIP-coded blocks.

Example 25: In some embodiments, for angular intra prediction modes excluding horizontal and vertical directions, the same weights are utilized for intra-prediction blocks and inter-prediction blocks generated for an IIP-coded block.

Example 26: In some embodiments, for certain locations, zero weights may be applied in the IIP coding process.

In some embodiments, zero weights may be applied to intra prediction blocks used in the IIP coding process.

In some embodiments, zero weighting may be applied to inter prediction blocks used in the IIP coding process.

Example 27: In some embodiments, the intra-prediction mode of an IIP-coded block can be selected only as one of the MPMs, regardless of the size of the current block.

In some embodiments, the MPM flag is not signaled and is inferred to be 1, regardless of the size of the current block.

Example 28 : For IIP-coded blocks, luma-predict-chroma mode (LM) is used instead of derived mode (DM) mode to perform intra-prediction on chroma components.

In some embodiments, both DM and LM may be allowed.

In some embodiments, multiple intra prediction modes may be allowed for chroma components.

In some embodiments, whether multiple modes are allowed for chroma components may depend on the color format. In one example, for a 4:4:4 color format, the allowed chroma intra prediction modes may be the same as those for the luma component.

Example 29: Inter-intra prediction may not be allowed in one or more of the following specific cases:

A. w == T1 || h == T1, and for example T1 = 4.

B. w > T1 || h > T1, and for example T1 = 64.

C. (w == T1 && h == T2) || (w == T2 && h == T1), for example, T1 = 4 and T2 = 16.

Example 30 : Inter-intra prediction may not be allowed for blocks with bidirectional prediction.

In some embodiments, if a selected merge candidate for a certain IIP-coded block uses bidirectional prediction, it will be converted into a unidirectional prediction merge candidate. In some embodiments, only predictions from reference list 0 are retained, and the merge candidate is treated as a unidirectional prediction referring to reference list 0. In some embodiments, only predictions from reference list 1 are retained, and the merge candidate is treated as a unidirectional prediction referring to reference list 1.

In some embodiments, an additional constraint is added that the selected merge candidate must be a unidirectional predictive merge candidate. Alternatively, the signaled merge index for the IIP-coded block indicates the index of a unidirectional predictive merge candidate (i.e., bidirectional predictive merge candidates are not counted).

In some embodiments, the merge candidate list construction process used in the triangle prediction mode may be utilized to derive the motion candidate list for the IIP-coded block.

Example 31: When inter-intra prediction is applied, some coding tools may not allow it.

In some embodiments, bi-directional optical flow (BIO) is not applied for bidirectional prediction.

In some embodiments, overlapped block motion compensation (OBMC) is not applied.

In some embodiments, the decoder-side motion vector derivation/refinement process is not allowed.

Example 32: The intra prediction process used in inter-intra prediction may be different from that used in normal intra-coded blocks.

In some embodiments, neighboring samples may be filtered in different ways. In some embodiments, neighboring samples are not filtered prior to performing intra-prediction used in inter-intra prediction.

In some embodiments, the position-dependent intra prediction sample filtering process is not performed on the intra prediction used in inter-intra prediction. In some embodiments, multiline intra-prediction is not allowed in inter-intra prediction. In some embodiments, wide-angle intra-prediction is not allowed in inter-intra prediction.

Example 33: Assume that the intra and inter prediction values in hybrid intra and inter prediction are PIntra and Pinter, and the weighting factors are w_intra and w_inter, respectively. The predicted value at position (x, y) is computed as (PIntra(x, y) * w_intra(x, y) + PInter(x, y) * w_inter(x, y) + offset(x, y)) >> N, where w_intra(x, y) + w_iner(x, y) = 2^N, and offset(x, y) = 2^(N - 1). In one example, N = 3.

Example 34: In some embodiments, MPM flags signaled within a normal intra-coded block and an IIP-coded block must share the same arithmetic coding context.

Example 35: In some embodiments, MPM is not required to code intra-prediction modes within an IIP-coded block (assuming block width and height are w and h).

In some embodiments, the four modes {planar, DC, vertical, horizontal} are binarized as 00, 01, 10 and 11 (there can be any arbitrary mapping rule, such as 00-planar, 01-DC, 10-vertical, 11-horizontal).

In some embodiments, the four modes {plane, DC, vertical, horizontal} are binarized as 0, 10, 110 and 111 (there can be any mapping rule such as 0-plane, 10-DC, 110-vertical, 111-horizontal).

In some embodiments, the four modes {plane, DC, vertical, horizontal} are binarized as 1, 01, 001 and 000 (there can be any arbitrary mapping rule, such as 1-plane, 01-DC, 001-vertical, 000-horizontal).

In some embodiments, where W>N*H (where N is an integer, e.g. 2) can be used, only three modes {planar, DC, vertical} can be used. The three modes are binarized as 1, 01, 11 (there can be any mapping rule, e.g. 1-planar, 01-DC, 11-vertical).

In some embodiments, where W>N*H (where N is an integer such as 2) can be used, only three modes {planar, DC, vertical} can be used. The three modes are binarized as 0, 10, 00 (there can be any arbitrary mapping rule, such as 0-planar, 10-DC, 00-vertical).

In some embodiments, where H>N*W (where N is an integer such as 2) can be used, only three modes {plane, DC, horizontal} can be used. The three modes are binarized as 1, 01, 11 (there can be any arbitrary mapping rule, such as 1-plane, 01-DC, 11-horizontal).

In some embodiments, only three modes {plane, DC, horizontal} can be used, where H>N*W (where N is an integer such as 2). The three modes are binarized as 0, 10, 00 (there can be any arbitrary mapping rule, such as 0-plane, 10-DC, 00-horizontal).

Example 36: In some embodiments, only DC and Planar modes are used within an IIP-coded block. In some embodiments, a flag is signaled to indicate whether DC or Planar is used.

Example 37: In some embodiments, IIP is performed differently for different color components.

In some embodiments, inter-intra-prediction is not performed on chroma components (e.g., Cb and Cr).

In some embodiments, the intra-prediction mode for the chroma component is different from that for the luma component within the IIP-coded block. In some embodiments, the DC mode is always used for the chroma. In some embodiments, the planar mode is always used for the chroma. In some embodiments, the LM mode is always used for the chroma.

In some embodiments, how IIP is performed for different color components may vary depending on the color format (e.g., 4:2:0 or 4:4:4).

In some embodiments, how IIP is performed for different color components may vary depending on the block size. For example, if the width or height of the current block is 4 or less, inter-intra-prediction is not performed on chroma components (e.g., Cb and Cr).

MV precision issues

In the following description, the precision used for the stored MVs for spatial motion prediction is denoted as P1, and the precision used for the stored MVs for temporal motion prediction is denoted as P2.

Example 38: P1 and P2 may be the same, or they may be different.

In some embodiments, P1 is a 1/16 luma pixel and P2 is a 1/4 luma pixel. In some embodiments, P1 is a 1/16 luma pixel and P2 is a 1/8 luma pixel. In some embodiments, P1 is a 1/8 luma pixel and P2 is a 1/4 luma pixel. In some embodiments, P1 is a 1/8 luma pixel and P2 is a 1/8 luma pixel. In some embodiments, P2 is a 1/16 luma pixel and P1 is a 1/4 luma pixel. In some embodiments, P2 is a 1/16 luma pixel and P1 is a 1/8 luma pixel. In some embodiments, P2 is a 1/8 luma pixel and P1 is a 1/4 luma pixel.

Example 39: P1 and P2 may not be fixed. In some embodiments, P1/P2 may be different for different standard profiles/levels/tiers. In some embodiments, P1/P2 may be different for pictures within different temporal layers. In some embodiments, P1/P2 may be different for pictures of different widths/heights. In some embodiments, P1/P2 may be signaled from encoder to decoder within VPS/SPS/PPS/Slice Header/Tile Group Header/Tile/CTU/CU.

Example 40: For MV (MVx, MVy), the precision for MVx and MVy can be different, and are denoted as Px and Py.

In some embodiments, Px/Py may be different for different standard profiles/levels/tiers. In some embodiments, Px/Py may be different for pictures within different temporal layers. In some embodiments, Px may be different for pictures of different widths. In some embodiments, Py may be different for pictures of different heights. In some embodiments, Px/Py may be signaled from encoder to decoder within VPS/SPS/PPS/Slice Header/Tile Group Header/Tile/CTU/CU.

Example 41 : Before storing MV (MVx, MVy) for temporal motion prediction, it must be transformed to exact precision.

In some embodiments, if P1>=P2, then MVx=Shift(MVx, P1-P2), MVy=Shift(MVy, P1-P2). In some embodiments, if P1>=P2, then MVx=SignShift(MVx, P1-P2), MVy=SignShift(MVy, P1-P2). In some embodiments, if P1<P2, MVx= MVx <<(P2-P1), MVy= MVy<<(P2-P1).

Example 42 : Assume that the precision of MV(MVx, MVy) is Px and Py, and that MVx or MVy is stored by an N-bit integer. The range of MV(MVx, MVy) is MinX <=MVx<= MaxX, and MinY <=MVy<= MaxY.

In some embodiments, MinX may be equal to MinY and may not be equal to MinY. In some embodiments, MaxX may be equal to MaxY and may not be equal to MaxY. In some embodiments, {MinX, MaxX} may vary over Px. In some embodiments, {MinY, MaxY} may vary over Py. In some embodiments, MinX, MaxX, MinY, MaxY} may vary over N. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for MVs stored for spatial motion prediction and MVs stored for temporal motion prediction. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for different standard profiles/levels/tiers. In some embodiments, {MinX, MaxX, MinY, MaxY} may be different for pictures in different temporal layers. In some embodiments, {MinX, MaxX, MinY, MaxY} can be different for pictures of different width/height. In some embodiments, {MinX, MaxX, MinY, MaxY} can be signaled from encoder to decoder in VPS/SPS/PPS/slice header/tile group header/tile/CTU/CU. In some embodiments, {MinX, MaxX} can be different for pictures of different width. In some embodiments, {MinY, MaxY} can be different for pictures of different height. In some embodiments, MVx is clipped to [MinX, MaxX] before being stored for spatial motion prediction. In some embodiments, MVx is clipped to [MinX, MaxX] before being stored for temporal motion prediction. In some embodiments, MVy is clipped to [MinY, MaxY] before being stored for spatial motion prediction. In some embodiments, MVy is clipped to [MinY, MaxY] before being stored for temporal motion prediction.

Reduce line buffer for affine merge mode

Example 43: The affine model (derived CPMVs or affine parameters) inherited by an affine merge candidate from a neighboring block is always a 6-parameter affine model.

In some embodiments, if a neighboring block is coded with a 4-parameter affine model, the affine model is still inherited as a 6-parameter affine model.

In some embodiments, whether a 4-parameter affine model from a neighboring block is inherited as a 6-parameter affine model or a 4-parameter affine model may depend on the location of the current block. In some embodiments, a 4-parameter affine model from a neighboring block is inherited as a 6-parameter affine model if the neighboring block is not within the same Coding Tree Unit (CTU) (also known as Largest Coding Unit (LCU)) as the current block. In some embodiments, a 4-parameter affine model from a neighboring block is inherited as a 6-parameter affine model if the neighboring block is not within the same CTU line or CTU row as the current block. In some embodiments, a 4-parameter affine model from a neighboring block is inherited as a 6-parameter affine model if the neighboring block is not within the same M×N region as the current block. For example, M=N=64. In this case, a tile/slice/picture is partitioned into multiple non-overlapping MxN regions. In some embodiments, a 4-parameter affine model from a neighboring block is inherited as a 6-parameter affine model if the neighboring block is not in the same MxN region line or MxN region row as the current block. For example, M=N=64. The CTU line/row and region line/row are illustrated in Fig. 10.

In some embodiments, assuming that the top-left corner (or any other location) of the current block is (x, y) and the top-left corner (or any other location) of the neighboring block is (x', y'), a 4-parameter affine model from the neighboring block is inherited as a 6-parameter affine model if the neighboring block satisfies one or more of the following conditions:

(a) x / M != x' / M. For example, M = 128 or 64.

(b) y / N != y' / N. For example, N = 128 or 64.

(c) ((x / M != x' / M) && (y / N != y' / N)). For example, M = N=128 or M= N = 64.

(d) ((x / M != x' / M) || (y / N != y' / N)). For example, M = N=128 or M= N = 64.

(e) x >> M != x' >> M . For example, M = 7 or 6.

(f) y >> N != y' >> N . For example, N = 7 or 6.

(g) ((x >> M != x' >>M) && (y >> N != y' >>N)). For example, either M = N=7 or M= N = 6.

(h) ((x >> M != x' >>M) || (y >> N != y' >>N)). For example, M = N=7 or M= N = 6.

5. Embodiment

The following description shows how the disclosed technology can be implemented within the syntax structure of the current VVC standard. New additions are shown in bold , and deletions are shown in italics .

5.1 Embodiment #1 (Disable 4x4 inter prediction, and disable bidirectional prediction for 4x8, 8x4, 4x16 and 16x4 blocks)

7.3.6.6 Coding Unit Syntax

coding_unit(x0, y0, cbWidth, cbHeight, treeType) { technician if( tile_group_type != I | | sps_ibc_enabled_flag ) { if( treeType != DUAL_TREE_CHROMA && !( cbWidth == 4 && cbHeight == 4 && !sps_ibc_enabled_flag ) ) cu_skip_flag [ x0 ][ y0 ] ae(v) if( cu_skip_flag[ x0 ][ y0 ] = = 0 && tile_group_type != I && !( cbWidth == 4 && cbHeight == 4 ) ) pred_mode_flag ae(v) if( ( ( tile_group_type = = I && cu_skip_flag[ x0 ][ y0 ] = =0 ) | |
( tile_group_type != I && CuPredMode[ x0 ][ y0 ] != MODE_INTRA ) ) &&
sps_ibc_enabled_flag && !( cbWidth == 4 && cbHeight == 4 && cu_skip_flag[ x0 ][ y0 ] = = 1 ) ) pred_mode_ibc_flag ae(v) } if(CuPredMode[x0][y0] = = MODE_INTRA) { if( sps_pcm_enabled_flag &&
cbWidth >= MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&
cbHeight >= MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY ) pcm_flag [ x0 ][ y0 ] ae(v) if( pcm_flag[ x0 ][ y0 ] ) { while( !byte_aligned( ) ) pcm_alignment_zero_bit f(1) pcm_sample(cbWidth, cbHeight, treeType) } else { if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) { if( ( y0 % CtbSizeY ) > 0 ) intra_luma_ref_idx[x0][y0] ae(v) if (intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
( cbWidth <= MaxTbSizeY | | cbHeight <= MaxTbSizeY ) &&
( cbWidth * cbHeight > MinTbSizeY * MinTbSizeY )) intra_subpartitions_mode_flag[x0][y0] ae(v) if( intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&
cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) intra_subpartitions_split_flag[x0][y0] ae(v) if( intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 ) intra_luma_mpm_flag[ x0 ][ y0 ] ae(v) if( intra_luma_mpm_flag[x0][y0]) intra_luma_mpm_idx[x0][y0] ae(v) else intra_luma_mpm_remainder[x0][y0] ae(v) } if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA ) intra_chroma_pred_mode[x0][y0] ae(v) } } else if( treeType != DUAL_TREE_CHROMA ) { /* MODE_INTER or MODE_IBC */ if(cu_skip_flag[x0][y0] = = 0) merge_flag[ x0 ][ y0 ] ae(v) if(merge_flag[x0][y0]) { merge_data(x0, y0, cbWidth, cbHeight) } else if (CuPredMode[x0][y0] = = MODE_IBC) { mvd_coding( x0, y0, 0, 0 ) mvp_l0_flag[ x0 ][ y0 ] ae(v) if( sps_amvr_enabled_flag &&
( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0 ][ y0 ][ 1 ] != 0 ) ) { amvr_precision_flag[x0][y0] ae(v) } } else { if( tile_group_type = = B ) inter_pred_idc[ x0 ][ y0 ] ae(v) if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) { inter_affine_flag[x0][y0] ae(v) if( sps_affine_type_flag && inter_affine_flag[ x0 ][ y0 ] ) cu_affine_type_flag[x0][y0] ae(v) } … }

7.4.7.6 Coding Unit Semantics

If pred_mode_flag is equal to 0, it specifies that the current coding unit is coded in inter prediction mode. If pred_mode_flag is equal to 1, it specifies that the current coding unit is coded in intra prediction mode. The variable CuPredMode[ x ][ y ] is derived as follows for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1:

- If pred_mode_flag is equal to 0, CuPredMode[ x ][ y ] is set to be equal to MODE_INTER.

- Otherwise (if pred_mode_flag is equal to 1), CuPredMode[ x ][ y ] is set to be equal to MODE_INTRA.

If pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I tile group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 , and 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, it specifies that the current coding unit is coded in IBC prediction mode. If pred_mode_ibc_flag is equal to 0, it specifies that the current coding unit is not coded in IBC prediction mode.

If pred_mode_ibc_flag is not present, it is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I tile group or when decoding a coding unit coded in skip mode and cbWidth is equal to 4 and cbHeight is equal to 4 , or 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, the variables CuPredMode[ x ][ y ] are set equal to MODE_IBC for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1.

inter_pred_idc[ x0 ][ y0 ] specifies whether list0, list1, or bidirectional prediction is used for the current coding unit according to Table 7-9. The array indices x0, y0 specify the position ( x0, y0 ) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

inter_pred_idc Name of inter_pred_idc (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 | | ( cbWidth + cbHeight ) = = 20 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Table 7-9 - Name associations for inter prediction modes

If inter_pred_idc[ x0 ][ y0 ] does not exist, it is inferred to be equal to PRED_L0 .

8.5.2.1 General Information

The inputs to this process are:

- Luma position of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture ( xCb, yCb ),

- Variable cbWidth, which specifies the width of the current coding block within the luma samples.

- Variable cbHeight, which specifies the height of the current coding block within the luma samples.

The outputs of this process are:

- Luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ] with 1/16 fractional-sample accuracy,

- Reference indices refIdxL0 and refIdxL1,

- Prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ],

- Bidirectional prediction weight index gbiIdx.

Let the variable LX be the RefPicList[ X ] of the current picture, where X is either 0 or 1.

To derive variables mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], refIdxL0 and refIdxL1, and predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], the following applies:

- If merge_flag[ xCb ][ yCb ] is equal to 1, then the derivation process for luma motion vector for merge mode as specified in Section 8.5.2.2 is called with inputs luma position ( xCb , yCb ), variables cbWidth and cbHeight , and outputs luma motion vector mvL0[ 0 ][ 0 ], mvL1[ 0 ][ 0 ], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

- Otherwise, the following applies:

- For each X that is substituted with 0 or 1 in variables predFlagLX[ 0 ][0 ], mvLX[ 0 ][0 ] and refIdxLX, in PRED_LX, and in the syntax elements ref_idx_lX and MvdLX, the following sequence of steps is applied:

1. The variables refIdxLX and predFlagLX[ 0 ][0 ] are derived as follows:

- If inter_pred_idc[ xCb ][ yCb ] is equal to PRED_LX or PRED_BI,

refIdxLX = ref_idx_lX[xCb][yCb] (8-266)

predFlagLX[ 0 ][0 ] = 1 (8-267)

- Otherwise, the variables refIdxLX and predFlagLX[ 0 ][ 0 ] are defined by:

refIdxLX = -1 (8-268)

predFlagLX[ 0 ][0 ] = 0 (8-269)

2. The variable mvdLX is derived as follows:

mvdLX[ 0 ] = MvdLX[ xCb ][ yCb ][ 0 ] (8-270)

mvdLX[ 1 ] = MvdLX[ xCb ][ yCb ][ 1 ] (8-271)

3. If predFlagLX[ 0 ][ 0 ] is equal to 1, the induction process for luma motion vector prediction in Section 8.5.2.8 is called with luma coding block position ( xCb, yCb ), coding block width cbWidth, coding block height cbHeight and variable refIdxLX as input, and the output is mvpLX.

4. When predFlagLX[ 0 ][ 0 ] is equal to 1, the luma motion vector mvLX[ 0 ][ 0 ] is derived as follows:

uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-272)

mvLX[ 0 ][ 0 ][ 0 ] = ( uLX[ 0 ] >= 2 ¹⁷ ) ? ( uLX[ 0 ] - 2 ¹⁸ ): uLX[ 0 ] (8-273)

uLX[ 1 ] = ( mvpLX[ 1 ] + mvdLX[ 1 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-274)

mvLX[ 0 ][ 0 ][ 1 ] = ( uLX[ 1 ] >= 2 ¹⁷ ) ? ( uLX[ 1 ] - 2 ¹⁸ ): uLX[ 1 ] (8-275)

NOTE 1 - As specified above, the resulting values of mvLX[ 0 ][ 0 ] and mvLX[ 0 ][ 0 ][ 1 ] will always be in the range -2 ¹⁷ to 2 ¹⁷ - 1, inclusive.

- The bidirectional prediction weight index gbiIdx is set to be equal to gbi_idx[ xCb ][ yCb ].

If all of the following conditions are true, refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0:

- predFlagL0[ 0 ][ 0 ] will be equal to 1.

- predFlagL1[ 0 ][ 0 ] will be equal to 1.

- ( cbWidth + cbHeight = = 8 ) | | ( cbWidth + cbHeight = = 12 ) | | ( cbWidth + cbHeight = = 20 )

= cbWidth is equal to 4; cbHeight is equal to 4.

The update process for the motion vector predictor list based on the history as specified in Section 8.5.2.16 is called with luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

9.5.3.8 Binarization process for inter_pred_idc

Inputs to this process are a request for the binarization process for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

The output of this process is a binarization of the syntax elements.

The binarization for the syntax element inter_pred_idc is specified in Table 9-9.

The value of inter_pred_idc Name of inter_pred_idc Bin string (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 | | ( cbWidth + cbHeight ) = = 20 0 PRED_L0 00 0 1 PRED_L1 01 1 2 PRED_BI 1 -

Table 9-9 - Binarization for inter_pred_idc

9.5.4.2.1 General Information

inter_pred_idc[ x0 ][ y0 ] ( cbWidth + cbHeight ) != 8 && ( cbWidth + cbHeight ) != 12 && ( cbWidth + cbHeight ) != 20 ? 7 - ( ( 1 +
Log2( cbWidth ) + Log2( cbHeight ) ) >> 1 )
: 4 4 na na na na

Table 9-10 - Assignment of ctxInc to a syntax element with context-coded bins

5.2 Embodiment #2 (Disable 4x4 inter prediction)

7.3.6.6 Coding Unit Syntax

coding_unit(x0, y0, cbWidth, cbHeight, treeType) { technician if( tile_group_type != I | | sps_ibc_enabled_flag ) { if( treeType != DUAL_TREE_CHROMA && !( cbWidth == 4 && cbHeight == 4 && !sps_ibc_enabled_flag ) ) cu_skip_flag[ x0 ][ y0 ] ae(v) if( cu_skip_flag[ x0 ][ y0 ] = = 0 && tile_group_type != I && !( cbWidth == 4 && cbHeight == 4 ) ) pred_mode_flag ae(v) if( ( ( tile_group_type = = I && cu_skip_flag[ x0 ][ y0 ] = =0 ) | |
( tile_group_type != I && CuPredMode[ x0 ][ y0 ] != MODE_INTRA ) ) &&
sps_ibc_enabled_flag && !( cbWidth == 4 && cbHeight == 4 && cu_skip_flag[ x0 ][ y0 ] = = 1 ) ) pred_mode_ibc_flag ae(v) } if(CuPredMode[x0][y0] = = MODE_INTRA) { if( sps_pcm_enabled_flag &&
cbWidth >= MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&
cbHeight >= MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY ) pcm_flag[ x0 ][ y0 ] ae(v) if( pcm_flag[ x0 ][ y0 ] ) { while( !byte_aligned( ) ) pcm_alignment_zero_bit f(1) pcm_sample(cbWidth, cbHeight, treeType) } else { if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) { if( ( y0 % CtbSizeY ) > 0 ) intra_luma_ref_idx[x0][y0] ae(v) if (intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
( cbWidth <= MaxTbSizeY | | cbHeight <= MaxTbSizeY ) &&
( cbWidth * cbHeight > MinTbSizeY * MinTbSizeY )) intra_subpartitions_mode_flag[x0][y0] ae(v) if( intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&
cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) intra_subpartitions_split_flag[x0][y0] ae(v) if( intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 ) intra_luma_mpm_flag[ x0 ][ y0 ] ae(v) if( intra_luma_mpm_flag[x0][y0]) intra_luma_mpm_idx[x0][y0] ae(v) else intra_luma_mpm_remainder[x0][y0] ae(v) } if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA ) intra_chroma_pred_mode[x0][y0] ae(v) } } else if( treeType != DUAL_TREE_CHROMA ) { /* MODE_INTER or MODE_IBC */ if(cu_skip_flag[x0][y0] = = 0) merge_flag[ x0 ][ y0 ] ae(v) if(merge_flag[x0][y0]) { merge_data(x0, y0, cbWidth, cbHeight) } else if (CuPredMode[x0][y0] = = MODE_IBC) { mvd_coding( x0, y0, 0, 0 ) mvp_l0_flag[ x0 ][ y0 ] ae(v) if( sps_amvr_enabled_flag &&
( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0 ][ y0 ][ 1 ] != 0 ) ) { amvr_precision_flag[x0][y0] ae(v) } } else { if( tile_group_type = = B ) inter_pred_idc[ x0 ][ y0 ] ae(v) if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) { inter_affine_flag[x0][y0] ae(v) if( sps_affine_type_flag && inter_affine_flag[ x0 ][ y0 ] ) cu_affine_type_flag[x0][y0] ae(v) } … }

7.4.7.6 Coding Unit Semantics

If pred_mode_flag is equal to 0, it specifies that the current coding unit is coded in inter prediction mode. If pred_mode_flag is equal to 1, it specifies that the current coding unit is coded in intra prediction mode. The variable CuPredMode[ x ][ y ] is derived as follows for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1:

- If pred_mode_flag is equal to 0, CuPredMode[ x ][ y ] is set to be equal to MODE_INTER.

- Otherwise (pred_mode_flag is equal to 1), CuPredMode[ x ][ y ] is set to be equal to MODE_INTRA.

If pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I tile group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 , and 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, it specifies that the current coding unit is coded in IBC prediction mode. If pred_mode_ibc_flag is equal to 0, it specifies that the current coding unit is not coded in IBC prediction mode.

If pred_mode_ibc_flag is not present, it is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I tile group or when decoding a coding unit coded in skip mode and cbWidth is equal to 4 and cbHeight is equal to 4 , or 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, the variables CuPredMode[ x ][ y ] are set equal to MODE_IBC for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1.

5.3 Embodiment #3 (Disable bidirectional prediction for 4x8, 8x4, 4x16 and 16x4 blocks)

7.4.7.6 Coding Unit Semantics

inter_pred_idc[ x0 ][ y0 ] specifies whether list0, list1, or bidirectional prediction is used for the current coding unit according to Table 7-9. The array indices x0, y0 specify the position ( x0, y0 ) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

inter_pred_idc Name of inter_pred_idc (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 | | ( cbWidth + cbHeight ) = = 20 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Table 7-9 - Name associations for inter prediction modes

If inter_pred_idc[ x0 ][ y0 ] does not exist, it is inferred to be equal to PRED_L0 .

8.5.2.1 General Information

The inputs to this process are:

- Luma position of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture ( xCb, yCb ),

- Variable cbWidth, which specifies the width of the current coding block within the luma samples.

- Variable cbHeight, which specifies the height of the current coding block within the luma samples.

The outputs of this process are:

- Luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ] with 1/16 fractional-sample accuracy,

- Reference indices refIdxL0 and refIdxL1,

- Prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ],

- Bidirectional prediction weight index gbiIdx.

Let the variable LX be the RefPicList[ X ] of the current picture, where X is either 0 or 1.

To derive variables mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], refIdxL0 and refIdxL1, and predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], the following applies:

- If merge_flag[ xCb ][ yCb ] is equal to 1, then the derivation process for luma motion vector for merge mode as specified in Section 8.5.2.2 is called with inputs luma position ( xCb , yCb ), variables cbWidth and cbHeight , and outputs luma motion vector mvL0[ 0 ][ 0 ], mvL1[ 0 ][ 0 ], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

- Otherwise, the following applies:

- For each X that is substituted with 0 or 1 in variables predFlagLX[ 0 ][ 0 ], mvLX[ 0 ][ 0 ] and refIdxLX, in PRED_LX, and in syntax elements ref_idx_lX and MvdLX, the following sequence of steps is applied:

5. The variables refIdxLX and predFlagLX[ 0 ][0 ] are derived as follows:

- If inter_pred_idc[ xCb ][ yCb ] is equal to PRED_LX or PRED_BI,

refIdxLX = ref_idx_lX[xCb][yCb] (8-266)

predFlagLX[ 0 ][0 ] = 1 (8-267)

- Otherwise, the variables refIdxLX and predFlagLX[ 0 ][ 0 ] are defined by:

refIdxLX = -1 (8-268)

predFlagLX[ 0 ][0 ] = 0 (8-269)

6. The variable mvdLX is derived as follows:

mvdLX[ 0 ] = MvdLX[ xCb ][ yCb ][ 0 ] (8-270)

mvdLX[ 1 ] = MvdLX[ xCb ][ yCb ][ 1 ] (8-271)

7. If predFlagLX[ 0 ][ 0 ] is equal to 1, the induction process for luma motion vector prediction in Section 8.5.2.8 is called with luma coding block position ( xCb, yCb ), coding block width cbWidth, coding block height cbHeight and variable refIdxLX as input, and the output is mvpLX.

8. When predFlagLX[ 0 ][ 0 ] is equal to 1, the luma motion vector mvLX[ 0 ][ 0 ] is derived as follows:

uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-272)

mvLX[ 0 ][ 0 ][ 0 ] = ( uLX[ 0 ] >= 2 ¹⁷ ) ? ( uLX[ 0 ] - 2 ¹⁸ ): uLX[ 0 ] (8-273)

uLX[ 1 ] = ( mvpLX[ 1 ] + mvdLX[ 1 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-274)

mvLX[ 0 ][ 0 ][ 1 ] = ( uLX[ 1 ] >= 2 ¹⁷ ) ? ( uLX[ 1 ] - 2 ¹⁸ ): uLX[ 1 ] (8-275)

NOTE 1 - As specified above, the resulting values of mvLX[ 0 ][ 0 ] and mvLX[ 0 ][ 0 ][ 1 ] will always be in the range -2 ¹⁷ to 2 ¹⁷ - 1, inclusive.

- The bidirectional prediction weight index gbiIdx is set to be equal to gbi_idx[ xCb ][ yCb ].

If all of the following conditions are true, refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0:

- predFlagL0[ 0 ][ 0 ] will be equal to 1.

- predFlagL1[ 0 ][ 0 ] will be equal to 1.

- ( cbWidth + cbHeight = = 8 ) | | ( cbWidth + cbHeight = = 12 ) | | ( cbWidth + cbHeight = = 20 ) will be satisfied.

- cbWidth will be equal to 4; cbHeight will be equal to 4.

The update process for the motion vector predictor list based on the history as specified in Section 8.5.2.16 is called with luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

9.5.3.8 Binarization process for inter_pred_idc

Inputs to this process are a request for the binarization process for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

The output of this process is a binarization of the syntax elements.

The binarization for the syntax element inter_pred_idc is specified in Table 9-9.

The value of inter_pred_idc Name of inter_pred_idc Bin string (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 | | ( cbWidth + cbHeight ) = = 20 0 PRED_L0 00 0 1 PRED_L1 01 1 2 PRED_BI 1 -

Table 9-9 - Binarization for inter_pred_idc

9.5.4.2.1 General Information

inter_pred_idc[ x0 ][ y0 ] ( cbWidth + cbHeight ) != 8 && ( cbWidth + cbHeight ) != 12 && ( cbWidth + cbHeight ) != 20 ? 7 - ( ( 1 +
Log2( cbWidth ) + Log2( cbHeight ) ) >> 1 )
: 4 4 na na na na

Table 9-10 - Assignment of ctxInc to a syntax element with context-coded bins

Embodiment #4 (Disable 4x4 inter prediction, disable bidirectional prediction for 4x8, 8x4 blocks)

7.3.6.6 Coding Unit Syntax

coding_unit(x0, y0, cbWidth, cbHeight, treeType) { technician if( tile_group_type != I | | sps_ibc_enabled_flag ) { if( treeType != DUAL_TREE_CHROMA && !( cbWidth == 4 && cbHeight == 4 && !sps_ibc_enabled_flag ) ) cu_skip_flag[ x0 ][ y0 ] ae(v) if( cu_skip_flag[ x0 ][ y0 ] = = 0 && tile_group_type != I && !( cbWidth == 4 && cbHeight == 4 ) ) pred_mode_flag ae(v) if( ( ( tile_group_type = = I && cu_skip_flag[ x0 ][ y0 ] = =0 ) | |
( tile_group_type != I && CuPredMode[ x0 ][ y0 ] != MODE_INTRA ) ) &&
sps_ibc_enabled_flag && !( cbWidth == 4 && cbHeight == 4 && cu_skip_flag[ x0 ][ y0 ] = = 1 ) ) pred_mode_ibc_flag ae(v) } if(CuPredMode[x0][y0] = = MODE_INTRA) { if( sps_pcm_enabled_flag &&
cbWidth >= MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&
cbHeight >= MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY ) pcm_flag[ x0 ][ y0 ] ae(v) if( pcm_flag[ x0 ][ y0 ] ) { while( !byte_aligned( ) ) pcm_alignment_zero_bit f(1) pcm_sample(cbWidth, cbHeight, treeType) } else { if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) { if( ( y0 % CtbSizeY ) > 0 ) intra_luma_ref_idx[x0][y0] ae(v) if (intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
( cbWidth <= MaxTbSizeY | | cbHeight <= MaxTbSizeY ) &&
( cbWidth * cbHeight > MinTbSizeY * MinTbSizeY )) intra_subpartitions_mode_flag[x0][y0] ae(v) if( intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&
cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) intra_subpartitions_split_flag[x0][y0] ae(v) if( intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 ) intra_luma_mpm_flag[ x0 ][ y0 ] ae(v) if( intra_luma_mpm_flag[x0][y0]) intra_luma_mpm_idx[x0][y0] ae(v) else intra_luma_mpm_remainder[x0][y0] ae(v) } if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA ) intra_chroma_pred_mode[x0][y0] ae(v) } } else if( treeType != DUAL_TREE_CHROMA ) { /* MODE_INTER or MODE_IBC */ if(cu_skip_flag[x0][y0] = = 0) merge_flag[ x0 ][ y0 ] ae(v) if(merge_flag[x0][y0]) { merge_data(x0, y0, cbWidth, cbHeight) } else if (CuPredMode[x0][y0] = = MODE_IBC) { mvd_coding( x0, y0, 0, 0 ) mvp_l0_flag[ x0 ][ y0 ] ae(v) if( sps_amvr_enabled_flag &&
( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0 ][ y0 ][ 1 ] != 0 ) ) { amvr_precision_flag[x0][y0] ae(v) } } else { if( tile_group_type = = B ) inter_pred_idc[ x0 ][ y0 ] ae(v) if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) { inter_affine_flag[x0][y0] ae(v) if( sps_affine_type_flag && inter_affine_flag[ x0 ][ y0 ] ) cu_affine_type_flag[x0][y0] ae(v) } … }

7.4.7.6 Coding Unit Semantics

If pred_mode_flag is equal to 0, it specifies that the current coding unit is coded in inter prediction mode. If pred_mode_flag is equal to 1, it specifies that the current coding unit is coded in intra prediction mode. The variable CuPredMode[ x ][ y ] is derived as follows for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1:

- If pred_mode_flag is equal to 0, CuPredMode[ x ][ y ] is set to be equal to MODE_INTER.

- Otherwise (if pred_mode_flag is equal to 1), CuPredMode[ x ][ y ] is set to be equal to MODE_INTRA.

If pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I tile group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 , and 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, it specifies that the current coding unit is coded in IBC prediction mode. If pred_mode_ibc_flag is equal to 0, it specifies that the current coding unit is not coded in IBC prediction mode.

If pred_mode_ibc_flag is not present, it is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I tile group or when decoding a coding unit coded in skip mode and cbWidth is equal to 4 and cbHeight is equal to 4 , or 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, the variables CuPredMode[ x ][ y ] are set equal to MODE_IBC for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1.

inter_pred_idc[ x0 ][ y0 ] specifies whether list0, list1, or bidirectional prediction is used for the current coding unit according to Table 7-9. The array indices x0, y0 specify the position ( x0, y0 ) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

inter_pred_idc Name of inter_pred_idc (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Table 7-9 - Name associations for inter prediction modes

If inter_pred_idc[ x0 ][ y0 ] does not exist, it is inferred to be equal to PRED_L0 .

8.5.2.1 General Information

The inputs to this process are:

- Luma position of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture ( xCb, yCb ),

- Variable cbWidth, which specifies the width of the current coding block within the luma samples.

- Variable cbHeight, which specifies the height of the current coding block within the luma samples.

The outputs of this process are:

- Luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ] with 1/16 fractional-sample accuracy,

- Reference indices refIdxL0 and refIdxL1,

- Prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ],

- Bidirectional prediction weight index gbiIdx.

Let the variable LX be the RefPicList[ X ] of the current picture, where X is either 0 or 1.

To derive variables mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], refIdxL0 and refIdxL1, and predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], the following applies:

- If merge_flag[ xCb ][ yCb ] is equal to 1, then the derivation process for luma motion vector for merge mode as specified in Section 8.5.2.2 is called with inputs luma position ( xCb , yCb ), variables cbWidth and cbHeight , and outputs luma motion vector mvL0[ 0 ][ 0 ], mvL1[ 0 ][ 0 ], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

- Otherwise, the following applies:

- For each X that is substituted with 0 or 1 in variables predFlagLX[ 0 ][ 0 ], mvLX[ 0 ][ 0 ] and refIdxLX, in PRED_LX, and in syntax elements ref_idx_lX and MvdLX, the following sequence of steps is applied:

1. The variables refIdxLX and predFlagLX[ 0 ][0 ] are derived as follows:

- If inter_pred_idc[ xCb ][ yCb ] is equal to PRED_LX or PRED_BI,

refIdxLX = ref_idx_lX[xCb][yCb] (8-266)

predFlagLX[ 0 ][0 ] = 1 (8-267)

- Otherwise, the variables refIdxLX and predFlagLX[ 0 ][ 0 ] are defined by:

refIdxLX = -1 (8-268)

predFlagLX[ 0 ][0 ] = 0 (8-269)

2. The variable mvdLX is derived as follows:

mvdLX[ 0 ] = MvdLX[ xCb ][ yCb ][ 0 ] (8-270)

mvdLX[ 1 ] = MvdLX[ xCb ][ yCb ][ 1 ] (8-271)

3. If predFlagLX[ 0 ][ 0 ] is equal to 1, the induction process for luma motion vector prediction in Section 8.5.2.8 is called with luma coding block position ( xCb, yCb ), coding block width cbWidth, coding block height cbHeight and variable refIdxLX as input, and the output is mvpLX.

4. When predFlagLX[ 0 ][ 0 ] is equal to 1, the luma motion vector mvLX[ 0 ][ 0 ] is derived as follows:

uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-272)

mvLX[ 0 ][ 0 ][ 0 ] = ( uLX[ 0 ] >= 2 ¹⁷ ) ? ( uLX[ 0 ] - 2 ¹⁸ ): uLX[ 0 ] (8-273)

uLX[ 1 ] = ( mvpLX[ 1 ] + mvdLX[ 1 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-274)

mvLX[ 0 ][ 0 ][ 1 ] = ( uLX[ 1 ] >= 2 ¹⁷ ) ? ( uLX[ 1 ] - 2 ¹⁸ ): uLX[ 1 ] (8-275)

NOTE 1 - As specified above, the resulting values of mvLX[ 0 ][ 0 ] and mvLX[ 0 ][ 0 ][ 1 ] will always be in the range -2 ¹⁷ to 2 ¹⁷ - 1, inclusive.

- The bidirectional prediction weight index gbiIdx is set to be equal to gbi_idx[ xCb ][ yCb ].

If all of the following conditions are true, refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0:

- predFlagL0[ 0 ][ 0 ] is equal to 1.

- predFlagL1[ 0 ][ 0 ] is equal to 1.

- ( cbWidth + cbHeight = = 8 ) | | ( cbWidth + cbHeight = = 12 )

- cbWidth is equal to 4.

- cbHeight is equal to 4.

The update process for the motion vector predictor list based on the history as specified in Section 8.5.2.16 is called with luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

9.5.3.8 Binarization process for inter_pred_idc

Inputs to this process are a request for the binarization process for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

The output of this process is a binarization of the syntax elements.

The binarization for the syntax element inter_pred_idc is specified in Table 9-9.

The value of inter_pred_idc Name of inter_pred_idc Bin string (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 0 PRED_L0 00 0 1 PRED_L1 01 1 2 PRED_BI 1 -

Table 9-9 - Binarization for inter_pred_idc

9.5.4.2.1 General Information

inter_pred_idc[ x0 ][ y0 ] ( cbWidth + cbHeight ) != 8 && ( cbWidth + cbHeight ) != 12 ? 7 - ( ( 1 +
Log2( cbWidth ) + Log2( cbHeight ) ) >> 1 )
: 4 4 na na na na

Table 9-10 - Assignment of ctxInc to a syntax element with context-coded bins

5.5 Embodiment #5 (Disable 4x4 inter prediction, disable bidirectional prediction for 4x8, 8x4 blocks, and disable shared merge list for regular merge mode)

7.3.6.6 Coding Unit Syntax

coding_unit(x0, y0, cbWidth, cbHeight, treeType) { technician if( tile_group_type != I | | sps_ibc_enabled_flag ) { if( treeType != DUAL_TREE_CHROMA && !( cbWidth == 4 && cbHeight == 4 && !sps_ibc_enabled_flag ) ) cu_skip_flag[ x0 ][ y0 ] ae(v) if( cu_skip_flag[ x0 ][ y0 ] = = 0 && tile_group_type != I && !( cbWidth == 4 && cbHeight == 4 ) ) pred_mode_flag ae(v) if( ( ( tile_group_type = = I && cu_skip_flag[ x0 ][ y0 ] = =0 ) | |
( tile_group_type != I && CuPredMode[ x0 ][ y0 ] != MODE_INTRA ) ) &&
sps_ibc_enabled_flag && !( cbWidth == 4 && cbHeight == 4 && cu_skip_flag[ x0 ][ y0 ] = = 1 ) ) pred_mode_ibc_flag ae(v) } if(CuPredMode[x0][y0] = = MODE_INTRA) { if( sps_pcm_enabled_flag &&
cbWidth >= MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&
cbHeight >= MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY ) pcm_flag[ x0 ][ y0 ] ae(v) if( pcm_flag[ x0 ][ y0 ] ) { while( !byte_aligned( ) ) pcm_alignment_zero_bit f(1) pcm_sample(cbWidth, cbHeight, treeType) } else { if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_LUMA ) { if( ( y0 % CtbSizeY ) > 0 ) intra_luma_ref_idx[x0][y0] ae(v) if (intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
( cbWidth <= MaxTbSizeY | | cbHeight <= MaxTbSizeY ) &&
( cbWidth * cbHeight > MinTbSizeY * MinTbSizeY )) intra_subpartitions_mode_flag[x0][y0] ae(v) if( intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&
cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) intra_subpartitions_split_flag[x0][y0] ae(v) if( intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&
intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 ) intra_luma_mpm_flag[ x0 ][ y0 ] ae(v) if( intra_luma_mpm_flag[x0][y0]) intra_luma_mpm_idx[x0][y0] ae(v) else intra_luma_mpm_remainder[x0][y0] ae(v) } if( treeType = = SINGLE_TREE | | treeType = = DUAL_TREE_CHROMA ) intra_chroma_pred_mode[x0][y0] ae(v) } } else if( treeType != DUAL_TREE_CHROMA ) { /* MODE_INTER or MODE_IBC */ if(cu_skip_flag[x0][y0] = = 0) merge_flag[ x0 ][ y0 ] ae(v) if(merge_flag[x0][y0]) { merge_data(x0, y0, cbWidth, cbHeight) } else if (CuPredMode[x0][y0] = = MODE_IBC) { mvd_coding( x0, y0, 0, 0 ) mvp_l0_flag[ x0 ][ y0 ] ae(v) if( sps_amvr_enabled_flag &&
( MvdL0[ x0 ][ y0 ][ 0 ] != 0 | | MvdL0[ x0 ][ y0 ][ 1 ] != 0 ) ) { amvr_precision_flag[x0][y0] ae(v) } } else { if( tile_group_type = = B ) inter_pred_idc[ x0 ][ y0 ] ae(v) if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ) { inter_affine_flag[x0][y0] ae(v) if( sps_affine_type_flag && inter_affine_flag[ x0 ][ y0 ] ) cu_affine_type_flag[x0][y0] ae(v) } … }

7.4.7.6 Coding Unit Semantics

If pred_mode_flag is equal to 0, it specifies that the current coding unit is coded in inter prediction mode. If pred_mode_flag is equal to 1, it specifies that the current coding unit is coded in intra prediction mode. The variable CuPredMode[ x ][ y ] is derived as follows for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1:

- If pred_mode_flag is equal to 0, CuPredMode[ x ][ y ] is set to be equal to MODE_INTER.

- Otherwise (pred_mode_flag is equal to 1), CuPredMode[ x ][ y ] is set to be equal to MODE_INTRA.

If pred_mode_flag is not present, it is inferred to be equal to 1 when decoding an I tile group or a coding unit with cbWidth equal to 4 and cbHeight equal to 4 , and 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, it specifies that the current coding unit is coded in IBC prediction mode. If pred_mode_ibc_flag is equal to 0, it specifies that the current coding unit is not coded in IBC prediction mode.

If pred_mode_ibc_flag is not present, it is inferred to be equal to the value of sps_ibc_enabled_flag when decoding an I tile group or when decoding a coding unit coded in skip mode and cbWidth is equal to 4 and cbHeight is equal to 4 , or 0 when decoding a P or B tile group, respectively.

If pred_mode_ibc_flag is equal to 1, the variables CuPredMode[ x ][ y ] are set equal to MODE_IBC for x = x0..x0 + cbWidth - 1 and y = y0..y0 + cbHeight - 1.

inter_pred_idc[ x0 ][ y0 ] specifies whether list0, list1, or bidirectional prediction is used for the current coding unit according to Table 7-9. The array indices x0, y0 specify the position ( x0, y0 ) of the top-left luma sample of the considered coding block relative to the top-left luma sample of the picture.

inter_pred_idc Name of inter_pred_idc (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 0 PRED_L0 PRED_L0 1 PRED_L1 PRED_L1 2 PRED_BI n.a.

Table 7-9 - Name associations for inter prediction modes

If inter_pred_idc[ x0 ][ y0 ] does not exist, it is inferred to be equal to PRED_L0 .

8.5.2.1 General Information

The inputs to this process are:

- Luma position of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture ( xCb, yCb ),

- Variable cbWidth, which specifies the width of the current coding block within the luma samples.

- Variable cbHeight, which specifies the height of the current coding block within the luma samples.

The outputs of this process are:

- Luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ] with 1/16 fractional-sample accuracy,

- Reference indices refIdxL0 and refIdxL1,

- Prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ],

- Bidirectional prediction weight index gbiIdx.

Let the variable LX be the RefPicList[ X ] of the current picture, where X is either 0 or 1.

To derive variables mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], refIdxL0 and refIdxL1, and predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], the following applies:

- If merge_flag[ xCb ][ yCb ] is equal to 1, then the derivation process for luma motion vector for merge mode as specified in Section 8.5.2.2 is called with inputs luma position ( xCb , yCb ), variables cbWidth and cbHeight , and outputs luma motion vector mvL0[ 0 ][ 0 ], mvL1[ 0 ][ 0 ], reference indices refIdxL0, refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

- Otherwise, the following applies:

- For each X that is substituted with 0 or 1 in variables predFlagLX[ 0 ][ 0 ], mvLX[ 0 ][ 0 ] and refIdxLX, in PRED_LX, and in syntax elements ref_idx_lX and MvdLX, the following sequence of steps is applied:

5. The variables refIdxLX and predFlagLX[ 0 ][0 ] are derived as follows:

- If inter_pred_idc[ xCb ][ yCb ] is equal to PRED_LX or PRED_BI,

refIdxLX = ref_idx_lX[xCb][yCb] (8-266)

predFlagLX[ 0 ][0 ] = 1 (8-267)

- Otherwise, the variables refIdxLX and predFlagLX[ 0 ][ 0 ] are defined by:

refIdxLX = -1 (8-268)

predFlagLX[ 0 ][0 ] = 0 (8-269)

6. The variable mvdLX is derived as follows:

mvdLX[ 0 ] = MvdLX[ xCb ][ yCb ][ 0 ] (8-270)

mvdLX[ 1 ] = MvdLX[ xCb ][ yCb ][ 1 ] (8-271)

7. If predFlagLX[ 0 ][ 0 ] is equal to 1, the induction process for luma motion vector prediction in Section 8.5.2.8 is called with luma coding block position ( xCb, yCb ), coding block width cbWidth, coding block height cbHeight and variable refIdxLX as input, and the output is mvpLX.

4. When predFlagLX[ 0 ][ 0 ] is equal to 1, the luma motion vector mvLX[ 0 ][ 0 ] is derived as follows:

uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-272)

m vLX[ 0 ][ 0 ][ 0 ] = ( uLX[ 0 ] >= 2 ¹⁷ ) ? ( uLX[ 0 ] - 2 ¹⁸ ): uLX[ 0 ] (8-273)

uLX[ 1 ] = ( mvpLX[ 1 ] + mvdLX[ 1 ] + 2 ¹⁸ ) % 2 ¹⁸ (8-274)

mvLX[ 0 ][ 0 ][ 1 ] = ( uLX[ 1 ] >= 2 ¹⁷ ) ? ( uLX[ 1 ] - 2 ¹⁸ ): uLX[ 1 ] (8-275)

NOTE 1 - As specified above, the resulting values of mvLX[ 0 ][ 0 ] and mvLX[ 0 ][ 0 ][ 1 ] will always be in the range -2 ¹⁷ to 2 ¹⁷ - 1, inclusive.

- The bidirectional prediction weight index gbiIdx is set to be equal to gbi_idx[ xCb ][ yCb ].

If all of the following conditions are true, refIdxL1 is set equal to -1, predFlagL1 is set equal to 0, and gbiIdx is set equal to 0:

- predFlagL0[ 0 ][ 0 ] will be equal to 1.

- predFlagL1[ 0 ][ 0 ] will be equal to 1.

- ( cbWidth + cbHeight = = 8 ) | | ( cbWidth + cbHeight = = 12 )

- cbWidth will be equal to 4.

- cbHeight will be equal to 4.

The update process for the motion vector predictor list based on the history as specified in Section 8.5.2.16 is called with luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ], reference indices refIdxL0 and refIdxL1, prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ], and bidirectional prediction weight index gbiIdx.

8.5.2.2 Derivation process for luma motion vectors for merge mode

This process is only called if merge_flag[ xCb ][ yPb ] is equal to 1, where ( xCb, yCb ) specifies the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture.

The inputs to this process are:

- Luma position of the top-left sample of the current luma coding block relative to the top-left luma sample of the current picture ( xCb, yCb ),

- Variable cbWidth, which specifies the width of the current coding block within the luma samples.

- Variable cbHeight, which specifies the height of the current coding block within the luma samples.

The outputs of this process are:

- Luma motion vectors mvL0[ 0 ][ 0 ] and mvL1[ 0 ][ 0 ] with 1/16 fractional-sample accuracy,

- Reference indices refIdxL0 and refIdxL1,

- Prediction list utilization flags predFlagL0[ 0 ][ 0 ] and predFlagL1[ 0 ][ 0 ],

- Bidirectional prediction weight index gbiIdx.

The bidirectional prediction weight index gbiIdx is set to be equal to 0.

The variables xSmr, ySmr, smrWidth, smrHeight, and smrNumHmvpCand are derived as follows:

xSmr = IsInSmr[xCb][yCb] ? SmrX[xCb][yCb]: xCb (8-276)

ySmr = IsInSmr[xCb][yCb] ? SmrY[xCb][yCb]: yCb (8-277)

smrWidth = IsInSmr[ xCb ][ yCb ] ? SmrW[xCb][yCb]: cbWidth (8-278)

smrHeight = IsInSmr[ xCb ][ yCb ] ? SmrH[xCb][yCb]: cbHeight (8-279)

smrNumHmvpCand = IsInSmr[ xCb ][ yCb ] ? NumHmvpSmrCand: NumHmvpCand (8-280)

8.5.2.6 Inductive process for history-based merge candidates

The inputs to this process are:

- mergeCandList, the list of merge candidates

- Variable isInSmr, which determines whether the current coding unit is within a shared merge candidate region.

- numCurrMergeCand, the number of available merge candidates in the list.

The outputs of this process are:

- modified merge candidate list mergeCandList,

- numCurrMergeCand The modified number of merge candidates in the list.

Both variables isPrunedA ₁ and isPrunedB ₁ are set to equal FALSE.

The array smrHmvpCandList and the variable smrNumHmvpCand are derived as follows:

smrHmvpCandList = isInSmr ? HmvpSmrCandList : HmvpCandList (8-353)

smrNumHmvpCand = isInSmr ? NumHmvpSmrCand: NumHmvpCand (8-354)

For each candidate in smrHmvpCandList[ hMvpIdx ] whose index is hMvpIdx = 1..smrNumHmvpCand, the following sequence of steps is repeated until numCurrMergeCand equals ( MaxNumMergeCand - 1):

1. The variable sameMotion is derived as follows:

- If all of the following conditions are true for any merge candidate N (N is either A ₁ or B ₁ ), then both sameMotion and isPrunedN are set to TRUE:

- hMvpIdx is less than or equal to 2.

- Candidate smrHmvpCandList[smrNumHmvpCand - hMvpIdx] is equal to merge candidate N.

- isPrunedN is equal to FALSE.

- Otherwise, sameMotion is set to FALSE.

2. If sameMotion is FALSE, the candidate smrHmvpCandList[smrNumHmvpCand - hMvpIdx] is added to the merge candidate list as follows:

mergeCandList[ numCurrMergeCand++ ] = smrHmvpCandList[ smrNumHmvpCand - hMvpIdx ] (8-355)

9.5.3.8 Binarization process for inter_pred_idc

Inputs to this process are a request for the binarization process for the syntax element inter_pred_idc, the current luma coding block width cbWidth and the current luma coding block height cbHeight.

The output of this process is a binarization of the syntax elements.

The binarization for the syntax element inter_pred_idc is specified in Table 99.

The value of inter_pred_idc Name of inter_pred_idc Bin string (　cbWidth　+　cbHeight　)　　!=　　8 ( cbWidth + cbHeight ) = = 8 | | ( cbWidth + cbHeight ) = = 12 0 PRED_L0 00 0 1 PRED_L1 01 1 2 PRED_BI 1 -

Table 9-9 - Binarization for inter_pred_idc

9.5.4.2.1 General Information

inter_pred_idc[ x0 ][ y0 ] ( cbWidth + cbHeight ) != 8 && ( cbWidth + cbHeight ) != 12 ? 7 - ( ( 1 +
Log2( cbWidth ) + Log2( cbHeight ) ) >> 1 )
: 4 4 na na na na

Table 9-10 - Assignment of ctxInc to a syntax element with context-coded bins

FIG. 11 is a block diagram of a video processing device (1100). The device (1100) may be used to implement one or more of the methods described herein. The device (1100) may be implemented in a smart phone, a tablet, a computer, an Internet of Things (IoT) receiver, etc. The device (1100) may include one or more processors (1102), one or more memories (1104), and video processing hardware (1106). The processor(s) (1102) may be configured to implement one or more of the methods described herein. The memory (memories) (1104) may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware (1106) may be used within hardware circuitry to implement some of the techniques described herein.

FIG. 12 is a flowchart of one exemplary method (1200) of video processing. The method (1200) includes the steps of determining (1202) size constraints between a representative motion vector of a current video block to be affine coded and a motion vector of a sub-block of the current video block and performing a conversion between a bitstream representation of the current video block or sub-block and its pixel values using the size constraints (1204).

In this specification, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of a video to a corresponding bitstream representation, or vice versa. The bitstream representation of a current video block may correspond to bits that are co-located or distributed elsewhere in the bitstream, for example, as specified by a syntax. For example, a macroblock may be encoded using bits in terms of a transformed and coded error residual value, and also in a header and other fields in the bitstream.

It will be appreciated that the disclosed techniques are useful for implementing embodiments in which the implementation complexity of video processing is reduced because the memory requirements or line buffer size requirements are reduced. Some of the techniques disclosed herein can be explained using the description based on the sections that follow.

1. As a video processing method,

A step of determining a size constraint between a representative motion vector of a current video block being affine coded and a motion vector of a sub-block of the current video block; and

A video processing method, comprising the step of performing a conversion between a bitstream representation and pixel values of a current video block or sub-block using the above size constraints.

2. In Section 1,

The steps for performing the above conversion are:

A video processing method comprising generating a bitstream representation from the pixel values.

3. In Section 1,

The steps for performing the above conversion are:

A video processing method comprising generating the pixel values from the bitstream representation.

4. In any one of Sections 1 to 3,

The above size restrictions are,

Contains constraint values of motion vectors (MVx, MVy) of sub-blocks according to MVx>=MV'x-DH0 and MVx<=MV'x+DH1 and MVy>=MV'y-DV0 and MVy<=MV'y+DV1,

MV' = (MV'x, MV'y),

MV' represents the representative motion vector,

DH0, DH1, DV0 and DV1 are video processing methods that represent positive numbers.

5. In Section 4,

The above size restrictions are:

i. DH0 will be equal to DH1 or DV0 will be equal to DV1

ii. DH0 is equal to DV0 or DH1 is equal to DV1.

iii. DH0 and DH1 are different from each other or DV0 and DV1 are different from each other.

iv. DH0, DH1, DV0 and DV1 shall be signaled in the bitstream representation at the video parameter set level or the sequence parameter set level or the picture parameter set level or the slice header level or the tile group header level or the tile level or the coding tree unit level or the coding unit level or the prediction unit level.

v. DH0, DH1, DV0 and DV1 will be functions of the mode of video processing.

vi. DH0, DH1, DV0 and DV1 will vary depending on the width and height of the current video block.

vii. DH0, DH1, DV0 and DV1 will vary depending on whether the current video block is coded using unidirectional prediction or bidirectional prediction.

viii. DH0, DH1, DV0 and DV1 will vary depending on the location of the sub-block.

A video processing method comprising at least one of:

6. In any one of Sections 1 to 5,

A video processing method, wherein the above representative motion vector corresponds to a control point motion vector of the current video block.

7. In any one of Sections 1 to 5,

A video processing method, wherein the above representative motion vector corresponds to a motion vector for a corner sub-block of the current video block.

8. In any one of Sections 1 to 7,

The precision used for the motion vector of the above sub-block and the representative motion vector is,

A video processing method corresponding to the motion vector signaling precision within the above bitstream representation.

9. In any one of Sections 1 to 7,

The precision used for the motion vector of the above sub-block and the representative motion vector is,

A video processing method corresponding to a storage precision for storing the above motion vector.

10. As a video processing method,

A step of determining one or more sub-blocks of a current video block for an affine-coded current video block, each sub-block having a size of MxN pixels, where M and N are multiples of 2 or 4;

A step of conforming the motion vector of the above sub-block to the size constraints; and

A video processing method, comprising the step of conditionally performing a transformation between a bitstream representation of the current video block and pixel values using the size constraints based on a trigger.

11. In Section 10,

The steps for performing the above conversion are:

A video processing method comprising generating a bitstream representation from the pixel values.

12. In Section 10,

The steps for performing the above conversion are:

A video processing method comprising generating the pixel values from the bitstream representation.

13. In any one of Sections 10 to 12,

The above size restrictions are,

The maximum difference between the integer parts of the sub-block motion vectors of the current video block is limited to K pixels or less,

A video processing method where K is an integer.

14. In any one of Sections 10 to 13,

The above method,

A video processing method, which applies only when the current video block is coded using bidirectional prediction.

15. In any one of Sections 10 to 13,

The above method,

A video processing method, which applies only when the current video block is coded using unidirectional prediction.

16. In any one of Sections 10 to 13,

A video processing method, wherein the values of M, N or K are a function of the unidirectional prediction or bidirectional prediction mode of the current video block.

17. In any one of Sections 10 to 13,

A video processing method wherein the values of M, N or K are a function of the height or width of the current video block.

18. In any one of Sections 10 to 17,

The above trigger is,

A video processing method, wherein the bitstream representation is included at the video parameter set level or the sequence parameter set level or the picture parameter set level or the slice header level or the tile group header level or the tile level or the coding tree unit level or the coding unit level or the prediction unit level.

19. In Section 18,

A video processing method wherein the above trigger signals the values of M, N or K.

20. In any one of Sections 10 to 19,

One or more sub-blocks of the current video block,

A video processing method, wherein the method is calculated based on the type of affine coding used for the current video block.

21. In Section 20,

A video processing method, wherein two different methods are used to compute sub-blocks of unidirectional prediction and bidirectional prediction affine prediction modes.

22. In Section 21,

If the current video block above is a bidirectionally predicted affine block,

A video processing method wherein sub-blocks from different reference lists have different widths or heights.

23. In any one of the clauses 20 to 22,

A video processing method, wherein said one or more sub-blocks correspond to a luma component.

24. In any one of the clauses 10 to 23,

The width and height of one of the above one or more sub-blocks,

A video processing method, wherein a motion difference vector is determined by using a motion vector value of the current video block and a motion vector value of one of the one or more sub-blocks.

25. In any one of the clauses 20 to 23,

The above calculation is,

A video processing method based on pixel precision signaled within the bitstream representation.

26. As a video processing method,

a step of determining that the current video block satisfies the size condition; and

A video processing method, comprising: performing a conversion between a bitstream representation of the current video block and pixel values based on a decision result, by excluding a bidirectional prediction encoding mode for the current video block.

27. As a video processing method,

a step of determining that the current video block satisfies the size condition; and

Based on the decision result, a step of performing a conversion between a bitstream representation of the current video block and a pixel value is included,

A video processing method, wherein an inter prediction mode is signaled within the bitstream representation according to the size conditions.

28. As a video processing method,

a step of determining that the current video block satisfies the size condition; and

Based on the decision result, a step of performing a conversion between a bitstream representation of the current video block and a pixel value is included,

A video processing method wherein generating a list of merge candidates during said transformation depends on said size condition.

29. As a video processing method,

A step of determining that a child coding unit of the current video block satisfies a size condition; and

Based on the decision result, a step of performing a conversion between a bitstream representation of the current video block and a pixel value is included,

A video processing method wherein a coding tree splitting process used to generate the child coding units depends on the size condition.

30. In any one of the clauses 26 to 29,

The above size conditions are as follows:

(a) w is equal to T1 and h is equal to T2, or h is equal to T1 and w is equal to T2;

(b) w is equal to T1 and h is less than or equal to T2, or h is equal to T1 and w is less than or equal to T2;

(c) w is less than or equal to T1 and h is less than or equal to T2, or h is less than or equal to T1 and w is less than or equal to T2.

is one of them,

A video processing method where w is the width and h is the height.

31. In Section 30,

A video processing method wherein T1 = 8 and T2 = 8, or T1 = 8 and T2 = 4, or T1 = 4 and T2 = 4, or T1 = 4 and T2 = 16.

32. In any one of the clauses 26 to 29,

The above transformation is,

A video processing method comprising generating a bitstream representation from pixel values of the current video block or generating pixel values of the current video block from the bitstream representation.

33. As a video processing method,

A step of determining a weight index for a generalized bidirectional prediction (GBi) process for a current video block based on a position of the current video block; and

A video processing method, comprising the step of performing a transformation between the current video block and a bitstream representation of the current video block using the weight index to implement the GBi process.

34. In Section 33,

The above transformation is,

A video processing method comprising generating a bitstream representation from pixel values of the current video block or generating pixel values of the current video block from the bitstream representation.

35. In section 33 or 34,

The above decision-making steps are:

For the current video block at the first position, inheriting or predicting other weight indices of neighboring blocks, and

A video processing method, comprising calculating a GBI for the current video block at the second position without inheriting from the neighboring block.

36. In Section 35,

The second position above is,

A video processing method, comprising the current video block located within a coding tree unit different from the neighboring block.

37. In Section 35,

The second position above is,

A video processing method, wherein the current video block corresponds to a neighboring block and is within a different coding tree unit line or a different coding tree unit row.

38. As a video processing method,

A step of determining that the current video block is coded as an intra-inter prediction (IIP) coded block; and

A video processing method, comprising the step of performing a conversion between the current video block and a bitstream representation of the current video block using a simplification rule for determining an intra-prediction mode or a most probable mode (MPM) of the current video block.

39. In Section 38,

The above transformation is,

A video processing method comprising generating a bitstream representation from pixel values of the current video block or generating pixel values of the current video block from the bitstream representation.

40. In section 38 or 39,

The above simplification rule is,

A video processing method, comprising: determining an intra-prediction coding mode of a current video block to be intra-inter-prediction (IIP) coded such that the intra-prediction coding mode of a current video block is independent of other intra-prediction coding modes of neighboring video blocks.

41. In section 38 or 39,

The above intra-prediction coding mode is,

A video processing method, wherein the bitstream representation is expressed using coding independent of the coding of neighboring blocks.

42. In any one of the clauses 38 to 40,

The above simplification rule is,

A video processing method, wherein a priority is given to a selection so as to prefer a coding mode of an intra-coded block over a coding mode of an intra-predictive coded block.

43. In Section 38,

The above simplification rule is,

A video processing method, comprising: determining the MPM by inserting intra prediction modes from intra-coded neighboring blocks prior to inserting intra prediction modes from IIP-coded neighboring blocks.

44. In verse 38,

The above simplification rule is,

A video processing method, which prescribes that the same composition process as used for other normal intra-coded blocks be used for determining said MPM.

45. A video processing method,

a step of determining that the current video block satisfies the simplification criterion; and

A video processing method comprising the step of performing a conversion between said current video block and its bitstream representation by disabling the use of an inter-intra prediction mode for said conversion or by disabling an additional coding tool used for said conversion.

46. In Section 45,

The above transformation is,

A video processing method comprising generating a bitstream representation from pixel values of the current video block or generating pixel values of the current video block from the bitstream representation.

47. In section 45 or 46,

The above simplification criteria are:

Including that the width or height of the current video block above is equal to T1,

T1 is an integer, video processing method.

48. In section 45 or 46,

The above simplification criteria are:

Including that the width or height of the current video block is greater than T1,

T1 is an integer, video processing method.

49. In section 45 or 46,

The above simplification criteria are:

A video processing method, wherein the width of the current video block is equal to T1, and the height of the current video block is equal to T2.

48. In section 45 or 46,

The above simplification criteria are:

A video processing method, wherein the current video block uses a bidirectional prediction mode.

49. In section 45 or 46,

A video processing method wherein the additional coding tool comprises bi-directional optical flow (BIO) coding.

50. In section 45 or 46,

The above additional coding tool is a video processing method including a nested block motion compensation mode.

51. A video processing method,

Comprising a step of performing a conversion between a current video block and a bitstream representation of said current video block using a motion vector based encoding process,

(a) During the conversion process, precision P1 is used to store the spatial motion prediction result, precision P2 is used to store the temporal motion prediction result, and P1 and P2 are fractions, or

(b) A video processing method, wherein a precision Px is used to store an x-motion vector, a precision Py is used to store a y-motion vector, and Px and Py are fractions.

52. In Section 51,

Video processing method where P1, P2, Px and Py are different numbers.

53. In Section 52,

P1 is 1/16 luma pixel, P2 is 1/4 luma pixel, or

P1 is 1/16 luma pixel, P2 is 1/8 luma pixel, or

P1 is 1/8 luma pixel, P2 is 1/4 luma pixel, or

P1 is 1/8 luma pixel, P2 is 1/8 luma pixel, or

P2 is 1/16 luma pixel, P1 is 1/4 luma pixel, or

P2 is 1/16 luma pixel, P1 is 1/8 luma pixel, or

A video processing method where P2 is 1/8 luma pixel and P1 is 1/4 luma pixel.

54. In section 51 or 52,

P1 and P2 are,

A video processing method for different pictures in different temporal layers contained within the bitstream representation.

55. In section 51 or 52,

The computed motion vector is,

A video processing method wherein the video is processed through a precision correction process before being stored as a temporal motion prediction.

56. In section 51 or 52,

The above storage is,

Includes storing the x-motion vector and the y-motion vector as N-bit integers,

The range of values of the x-motion vectors is [MinX, MaxX], and the range of values of the y-motion vectors is [MinY, MaxY].

The above ranges are as follows:

a. MinX is the same as MinY,

b. MaxX is equal to MaxY,

c. {MinX, MaxX} depends on Px;

d. {MinY, MaxY} depends on Py;

e. {MinX, MaxX, MinY, MaxY} depends on N;

f. {MinX, MaxX, MinY, MaxY} are different for MVs stored for spatial motion prediction and for other MVs stored for temporal motion prediction;

g. {MinX, MaxX, MinY, MaxY} are different for pictures in different temporal layers;

h. {MinX, MaxX, MinY, MaxY} are different for pictures with different widths or heights;

i. {MinX, MaxX} are different for pictures with different widths;

j. {MinY, MaxY} are different for pictures with different heights;

k. MVx is clipped to [MinX, MaxX] before being stored for spatial motion prediction;

l. MVx is clipped to [MinX, MaxX] before being stored for temporal motion prediction;

m. MVy is clipped to [MinY, MaxY] before being stored for spatial motion prediction;

n. MVy is clipped to [MinY, MaxY] before being stored for temporal motion prediction.

A video processing method, which satisfies one or more of the following:

59. A video processing method,

A step of interpolating a small sub-block of size W1xH1 within a large sub-block of size W2xH2 of the current video block by fetching (W2 + N - 1 - PW) * (H2 + N - 1 - PH) blocks;

Step of pixel padding the fetched block;

A step of performing boundary pixel repeating on a pixel-padded block and obtaining pixel values of said small sub-block - W1, W2, H1, H2, and PW and PH are integers -; and

A video processing method, comprising the step of performing a conversion between said current video block and a bitstream representation of said current video block using interpolated pixel values of said smaller sub-blocks.

60. In Section 59,

The above transformation is,

A video processing method, comprising generating the current video block from the bitstream representation, or generating the bitstream representation from the current sub-block.

61. In section 59 or 60,

A video processing method where W2=H2=8, W1=H1=4, and PW=PH=0.

62. A video processing method,

During a conversion between a current video block of dimension WxH and a bitstream representation of the current video block, performing the motion compensation operation by fetching (W + N - 1 - PW) * (W + N - 1 - PH) reference pixels and padding the reference pixels outside the fetched reference pixels; and

Comprising a step of performing a conversion between said current video block and a bitstream representation of said current video block using the result of said motion compensation operation,

W, H, N, PW and PH are integers, video processing method.

63. In Section 62,

The above transformation is,

A video processing method, comprising generating the current video block from the bitstream representation, or generating the bitstream representation from the current sub-block.

64. In section 62 or 63,

The above padding is,

A video processing method comprising repeating a left or right boundary of a fetched pixel.

65. In section 62 or 63,

The above padding is,

A video processing method comprising repeating an upper or lower boundary of a fetched pixel.

66. In section 62 or 63,

The above padding is,

A video processing method comprising setting pixel values to constants.

67. In Section 38,

The above rules are,

A video processing method, wherein the same arithmetic coding context as used for other intra-coded blocks is used during said transformation.

68. In Section 38,

The transformation of the current video block above is:

A video processing method, excluding using MPM for the current video block.

69. In Section 38,

The above simplification rule is,

A video processing method, which specifies that only DC and planar modes be used for the bitstream representation of the current video block, which is an IIP coded block.

70. In Section 38,

The above simplification rule is,

A video processing method, which specifies different intra-prediction modes for luma and chroma components.

71. In verse 44,

A subset of the above MPMs,

A video processing method used for the current video block being IIP coded.

72. In Section 38,

The above simplification rule is,

A video processing method, wherein the above MPM is selected based on an intra prediction mode included in the MPM list.

73. In Section 38,

The above simplification rule is,

A video processing method, wherein a subset of MPMs is to be selected from a list of MPMs and a mode index associated with said subset is signaled.

74. In verse 38,

The context used to code intra MPM modes is

A video processing method, which is used to code intra mode for the current video block being IIP coded.

75. In verse 44,

The same weights are used for the intra-prediction blocks and inter-prediction blocks generated for the current video block.

A video processing method, wherein the above current video block is an IIP coded block.

76. In verse 44,

A video processing method, wherein in an IIP coding process for the current video block, zero weights are used for positions.

77. In Section 77,

A video processing method, wherein the above zero weights are applied to intra prediction blocks used in the IIP coding process.

78. In verse 77,

A video processing method, wherein the above zero weights are applied to inter prediction blocks used in the IIP coding process.

79. A video processing method,

A step of determining that bidirectional prediction or unidirectional prediction of the current video block is not allowed based on the size of the video block; and

A video processing method, comprising: performing a transformation between a bitstream representation of the current video block and pixel values based on a decision result by disabling a bidirectional prediction mode or a unidirectional prediction mode. For example, disallowed modes are not used to encode or decode the current video block. The transformation operation may represent video coding or compression, or video decoding or decompression.

80. In verse 79,

The current video block above is 4x8,

The above decision-making steps are:

A method of video processing, comprising determining that bidirectional prediction is not allowed. Other examples are provided in Example 5.

81. In verse 79,

The current video block above is 4x8 or 8x4,

The above decision-making steps are:

A method of video processing, comprising determining that bidirectional prediction is not allowed.

82. In verse 79,

The current video block above is 4xN, where N is an integer less than or equal to 16,

A video processing method, wherein the step of determining comprises determining that bidirectional prediction is not allowed.

83. In any one of Sections 26 to 29 and Sections 79 to 82,

The size of the current block above is,

A video processing method corresponding to the size of a color component or a luma component of the current block.

84. In Section 83,

Disabling bidirectional or unidirectional prediction,

A video processing method applied to all three components of the current video block.

85. In Section 83,

Disabling bidirectional or unidirectional prediction,

A video processing method, wherein the size is applied only to the color component used as the size of the current block.

86. In any one of the clauses 79 to 85,

The above transformation is,

A video processing method, wherein the method comprises disabling bidirectional prediction, further using a merge candidate that is bidirectionally predicted, and then assigning to the current video block only one motion vector from only one reference list.

87. In verse 79,

The current video block above is 4x4,

The above decision-making steps are:

A method of video processing, comprising determining that both bidirectional prediction and unidirectional prediction are not allowed.

88. In Section 87,

A video processing method, wherein the current video block is coded as an intra-block.

89. In Section 87,

The current video block above is,

A video processing method limited to using integer pixel motion vectors.

Additional examples and embodiments for Sections 78 to 89 are described in Example 5.

90. A method for processing video,

A step of determining a video coding condition for the current video block based on the size of the current video block; and

A video processing method, comprising the step of performing a conversion between the current video block and a bitstream representation of the current video block based on the video coding conditions.

91. In Section 90,

The above video coding conditions are:

A video processing method, comprising selectively signaling a skip flag or an intra block coding flag within the bitstream representation.

92. In section 90 or 91,

The above video coding conditions are:

A video processing method, comprising: optionally signaling a prediction mode for said current video block.

93. In any one of Sections 90 to 92,

The above video coding conditions are:

A video processing method, comprising optionally signaling triangle mode coding of said current video block.

94. In any one of Articles 90 to 93,

The above video coding conditions are:

A video processing method, comprising: optionally signaling an inter prediction direction for said current video block.

95. In any one of Sections 90 to 94,

The above video coding conditions are:

A video processing method, which provides for selectively changing a motion vector or a block vector used for intra block duplication of the current video block.

96. In any one of Articles 90 to 95,

The above video coding conditions are:

A video processing method, which depends on the height in pixels of the current video block.

97. In any one of Articles 90 to 96,

The above video coding conditions are:

A video processing method, which depends on the width in pixels of the current video block.

98. In any one of Articles 90 to 95,

A video processing method, wherein the above video coding condition depends on whether the current video block is square in shape.

Additional examples of Sections 90 through 98 are provided in Items 11 through 16 listed in Section 4 of this specification.

99. A video encoder device comprising a processor configured to perform a method as set forth in any one of clauses 1 to 98.

100. A video decoder device comprising a processor configured to perform a method as set forth in any one of clauses 1 to 98.

101. A computer-readable medium having code stored thereon,

A computer-readable medium, wherein the code, when executed by a processor, causes the processor to implement a method described in any one of clauses 1 to 98.

FIG. 16 is a block diagram illustrating one exemplary video processing system (1600) in which various techniques disclosed herein may be implemented. Various implementations may include some or all of the components of the system (1600). The system (1600) may include an input (1602) for receiving video content. The video content may be received in a raw or uncompressed format, for example as 8 or 10 bit multi-component pixel values, or may be in a compressed or encoded format. The input (1602) may represent a network interface, a peripheral bus interface, or a storage interface. Examples of network interfaces include wired interfaces such as Ethernet, a passive optical network (PON), and the like, and wireless interfaces such as Wi-Fi or a cellular interface.

The system (1600) may include a coding component (1604) that may implement various coding or encoding methods described herein. The coding component (1604) may reduce the average bitrate of the video from an input (1602) to an output of the coding component (1604) to generate a coded representation of the video. Therefore, the coding technique is sometimes referred to as a video compression or video transcoding technique. The output of the coding component (1604) may be stored or transmitted via a connected communication unit, as represented by component (1606). The stored or communicated bitstream (or coded) representation of the video received at the input (1602) may be used by component (1608) to generate pixel values or displayable video that is transmitted to a display interface (1610). The process of generating viewable video from the bitstream representation is sometimes referred to as video decompression. Furthermore, while some video processing operations are referred to as "coding" operations or tools, it will be understood that the coding tool or operation is used in an encoder and a corresponding decoding tool, or that the operation of inverting the result of the coding will be performed by a decoder.

Examples of peripheral bus interfaces or display interfaces may include universal serial bus (USB), high definition multimedia interface (HDMI), or DisplayPort. Examples of storage interfaces include serial advanced technology attachment (SATA), PCI, IDE interfaces, etc. The techniques described herein may be implemented in various electronic devices, such as mobile phones, laptops, smart phones, or other devices capable of performing digital data processing and/or video display.

FIG. 17 is a flowchart representation of a method (1700) for video processing according to the present invention. The method (1700) includes, at operation 1702, determining that a first motion vector of a sub-block of a current block and a second motion vector, which is a representative motion vector for the current block, comply with a size constraint for transforming between a current block of video and a bitstream representation of the video using an affine coding tool. The method (1700) further includes, at operation 1704, performing the transform based on the determined result.

In some embodiments, the first motion vector of a sub-block is represented by (MVx, MVy) and the second motion vector is represented by (MV'x, MV'y). The size constraints indicate that MVx>=MV'x-DH0, MVx<=MV'x+DH1, MVy>=MV'y-DV0, and MVy<=MV'y+DV1, where DH0, DH1, DV0 and DV1 are positive. In some embodiments, DH0 = DH1. In some embodiments, DH0 ≠ DH1. In some embodiments, DV0 = DV1. In some embodiments, DV0 ≠ DV0. In some embodiments, DH0 = DV0. In some embodiments, DH0 ≠ DV0. In some embodiments, DH1 = DV1. In some embodiments, DH1 ≠ DV1.

In some embodiments, at least one of DH0, DH1, DV0, or DV1 is signaled in the bitstream representation at the video parameter set level, the sequence parameter set level, the picture parameter set level, the slice header, the tile group header, the tile level, the coding tree unit level, the coding unit level, or the prediction unit level. In some embodiments, DH0, DH1, DV0, and DV1 are different for different profiles, levels, or tiers of the transform. In some embodiments, DH0, DH1, DV0, and DV1 are based on the width or height of the current block. In some embodiments, DH0, DH1, DV0, and DV1 are based on the prediction mode of the current block, wherein the prediction mode is a unidirectional prediction mode or a bidirectional prediction mode. In some embodiments, DH0, DH1, DV0, and DV1 are based on the location of a sub-block within the current block.

In some embodiments, the second motion vector comprises a motion vector of a control point of the current block. In some embodiments, the second motion vector comprises a motion vector for a second sub-block of the current block. In some embodiments, the second sub-block comprises a center sub-block of the current block. In some embodiments, the second sub-block comprises a corner sub-block of the current block. In some embodiments, the second motion vector comprises a motion vector derived for a location inside or outside the current block, where the location is coded using the same affine model as the current block. In some embodiments, the location comprises a center location of the current block.

In some embodiments, the first motion vector is adjusted to satisfy the size constraint. In some embodiments, if the first motion vector does not satisfy the size constraint for the second motion vector, the bitstream is invalid. In some embodiments, the first motion vector and the second motion vector are represented according to a motion vector signaling precision in the bitstream representation. In some embodiments, the first motion vector and the second motion vector are represented according to a storage precision for storing the motion vector. In some embodiments, the first motion vector and the second motion vector are represented according to a precision different from the motion vector signaling precision or the storage precision for storing the motion vector.

FIG. 18 is a flowchart representation of a method (1800) for video processing according to the present invention. The method (1800) includes, at operation 1802, determining an affine model including six parameters for transformation between a current block of video and a bitstream representation of the video. The affine model is inherited from affine coding information of a neighboring block of the current block. The method (1800) includes, at operation 1804, performing the transformation based on the affine model.

In some embodiments, the neighboring block is coded using a second affine model having six parameters, and the affine model is identical to the second affine model. In some embodiments, the neighboring block is coded using a third affine model having four parameters. In some embodiments, the affine model is determined based on the position of the current block. In some embodiments, the affine model is determined according to the third affine model if the neighboring block is not in the same coding tree unit (CTU) as the current block. In some embodiments, the affine model is determined according to the third affine model if the neighboring block is not in the same CTU line or the same CTU row as the current block.

In some embodiments, a tile, slice, or picture is divided into multiple non-overlapping regions. In some embodiments, the affine model is determined according to a third affine model if the neighboring block is not in the same region as the current block. In some embodiments, the affine model is determined according to the third affine model if the neighboring block is not in the same region line or region row as the current block. In some embodiments, each region has a size of 64x64. In some embodiments, the top-left corner of the current block is represented by (x, y) and the top-left corner of the neighboring block is represented by (x', y'), where the affine model is determined according to the third affine model if conditions for x, y, x' and y' are satisfied. In some embodiments, the condition indicates that x / M ≠ x' / M, where M is a positive integer. In some embodiments, M is 128 or 64. In some embodiments, the condition indicates that y / N ≠ y' / N, where N is a positive integer. In some embodiments, N is 128 or 64. In some embodiments, the condition indicates that x / M ≠ x' / M and y / N ≠ y' / N, where M and N are positive integers. In some embodiments, M = N = 128 or M = N = 64. In some embodiments, the condition indicates that x >> M ≠ x' >> M, where M is a positive integer. In some embodiments, M is 6 or 7. In some embodiments, the condition indicates that y >> N ≠ y' >> N, where N is a positive integer. In some embodiments, N is 6 or 7. In some embodiments, the condition states that x >> M ≠ x' >> M and y >> N ≠ y' >> N, where M and N are positive integers. In some embodiments, M = N = 6 or M = N = 7.

FIG. 19 is a flowchart representation of a method (1900) for video processing according to the present invention. The method (1900) includes, at operation 1902, determining whether a bi-prediction coding technique is applicable to the block for conversion between a block of video and a bitstream representation of the video, based on a size of a block having a width W and a height H, where W and H are positive integers. The method (1900) includes, at operation 1904, performing the conversion according to the determination result.

In some embodiments, the bidirectional predictive coding scheme is not applicable when W = T1 and H = T2, and T1 and T2 are positive integers. In some embodiments, the bidirectional predictive coding scheme is not applicable when W = T2 and H = T1, and T1 and T2 are positive integers. In some embodiments, the bidirectional predictive coding scheme is not applicable when W = T1 and H ≤ T2, and T1 and T2 are positive integers. In some embodiments, the bidirectional predictive coding scheme is not applicable when W ≤ T2 and H = T1, and T1 and T2 are positive integers. In some embodiments, T1 = 4 and T2 = 16. In some embodiments, the bidirectional predictive coding scheme is not applicable when W ≤ T1 and H ≤ T2, and T1 and T2 are positive integers. In some embodiments, T1 = T2 = 8. In some embodiments, T1 = 8 and T2 = 4. In some embodiments, T1 = T2 = 4. In some embodiments, T1 = 4 and T2 = 8.

In some embodiments, when the bidirectional predictive coding technique is applicable, an indicator indicating information about the bidirectional predictive coding technique is signaled in the bitstream. In some embodiments, when the bidirectional predictive coding technique is not applicable to the block, an indicator indicating information about the bidirectional predictive coding technique for the block is excluded from the bitstream. In some embodiments, the bidirectional predictive coding technique is not applicable when the size of the block is one of 4x8 or 8x4. In some embodiments, the bidirectional predictive coding technique is not applicable when the size of the block is 4xN or Nx4, where N is a positive integer and N ≤ 16. In some embodiments, the size of the block corresponds to a first color component of the block, and whether the bidirectional predictive coding technique is applicable is determined with respect to the first color component and the remaining color components of the block. In some embodiments, the size of the block corresponds to a first color component of the block, and whether the bidirectional predictive coding technique is applicable is determined only for the first color component. In some embodiments, the first color component includes a luma component.

In some embodiments, the method further comprises the step of assigning a single motion vector from the first reference list or the second reference list if the selected merge candidate is determined to be coded using the bidirectional predictive coding technique, if the bidirectional predictive coding technique is not applicable to the current block. In some embodiments, the method further comprises the step of determining that a triangular prediction mode is not applicable to the block, if the bidirectional predictive coding technique is not applicable to the current block. In some embodiments, whether the bidirectional predictive coding technique is applicable is associated with a prediction direction, and the prediction direction is further associated with a uni-prediction coding technique, and the prediction direction is signaled in the bitstream based on a size of the block. In some embodiments, the uni-prediction coding technique is signaled in the bitstream if (1) WxH < 64 or (2) WxH = 64, and W is not equal to H. In some embodiments, information about a unidirectional predictive coding scheme or a bidirectional coding scheme is signaled in the bitstream if (1) W x H > 64 or (2) W x H = 64, where W is equal to H.

In some embodiments, when the size of the block is 4x4, the restriction indicates that neither the bidirectional coding technique nor the unidirectional technique is applicable to the block. In some embodiments, the restriction is applicable when the block is affine coded. In some embodiments, the restriction is applicable when the block is not affine coded. In some embodiments, the restriction is applicable when the block is intra-coded. In some embodiments, the restriction is not applicable when the motion vector of the block has integer precision.

In some embodiments, signaling that the block is generated based on a split of a parent block is skipped in the bitstream, wherein the parent block has a size of (1) 8x8 for a quad-tree split, (2) 8x4 or 4x8 for a binary tree split, or (3) 4x16 or 16x4 for a triple tree split. In some embodiments, an indicator that the motion vector has integer precision is set to 1 in the bitstream. In some embodiments, the motion vector of the block is rounded to integer precision.

In some embodiments, the bidirectional predictive coding technique is applicable to the block. The reference block has a size of (W + N - 1 - PW) x (H + N - 1 - PH), and boundary pixels of the reference block are repeated to generate a second block having a size of (W + N - 1) x (H + N - 1) for interpolation operation, where N represents an interpolation filter tap, and N, PW and PH are integers. In some embodiments, PH = 0, and at least the pixels at the left boundary or the right boundary are repeated to generate the second block. In some embodiments, PW = 0, and at least the pixels at the top boundary or the bottom boundary are repeated to generate the second block. In some embodiments, PW > 0 and PH > 0, and the second block is generated by repeating at least the pixels at the left boundary or the right boundary and then repeating at least the pixels at the top boundary or the bottom boundary. In some embodiments, PW > 0 and PH > 0, and the second block is generated by repeating pixels of at least an upper boundary or a lower boundary and then repeating pixels of at least a left boundary or a right boundary. In some embodiments, pixels of the left boundary are repeated M1 times and pixels of the right boundary are repeated (PW - M1) times. In some embodiments, pixels of the upper boundary are repeated M2 times and pixels of the lower boundary are repeated (PH - M2) times. In some embodiments, for the transformation, how boundary pixels of the reference block are repeated is applied to some or all of the reference block. In some embodiments, PW and PH are different for different components of the block.

In some embodiments, the merge candidate list construction process is performed based on the size of the block. In some embodiments, if (1) the merge candidate is coded using a bidirectional predictive coding technique and (2) bidirectional prediction is not applicable to the block depending on the size of the block, the merge candidate is considered a unidirectional prediction candidate pointing to a first reference list in a unidirectional predictive coding technique. In some embodiments, the first reference list includes reference list 0 or reference list 1 of the unidirectional predictive coding technique. In some embodiments, if (1) the merge candidate is coded using a bidirectional predictive coding technique and (2) bidirectional prediction is not applicable to the block depending on the size of the block, the merge candidate is considered unavailable. In some embodiments, during the merge candidate list construction process, unavailable merge candidates are removed from the merge candidate list. In some embodiments, if bidirectional prediction is not applicable to the block depending on the size of the block, a merge candidate list construction process for a triangular prediction mode is invoked.

FIG. 20 is a flowchart representation of a method (2000) for video processing according to the present invention. The method (2000) includes, at operation 2002, determining whether a coding tree partitioning process is applicable to the block for conversion between a block of video and a bitstream representation of the video, based on a size of a sub-block, which is a child coding unit of the block, according to the coding tree partitioning process. The sub-block has a width W and a height H, where W and H are positive integers. The method (2000) further includes, at operation 2004, performing the conversion according to a result of the determination.

In some embodiments, the coding tree splitting process is not applicable when W = T1 and H = T2, and T1 and T2 are positive integers. In some embodiments, the coding tree splitting process is not applicable when W = T2 and H = T1, and T1 and T2 are positive integers. In some embodiments, the coding tree splitting process is not applicable when W = T1 and H ≤ T2, and T1 and T2 are positive integers. In some embodiments, the coding tree splitting process is not applicable when W ≤ T2 and H = T1, and T1 and T2 are positive integers. In some embodiments, T1 = 4 and T2 = 16. In some embodiments, the coding tree splitting process is not applicable when W ≤ T1 and H ≤ T2, and T1 and T2 are positive integers. In some embodiments, T1 = T2 = 8. In some embodiments, T1 = 8 and T2 = 4. In some embodiments, T1 = T2 = 4. In some embodiments, T1 = 4. In some embodiments, T2 = 4. In some embodiments, if the coding tree splitting process is not applicable to the current block, signaling of the coding tree splitting process is omitted from the bitstream.

FIG. 21 is a flowchart representation of a method (2100) for video processing according to the present invention. The method (2100) includes, at operation 2102, determining whether an index of a bi-prediction with coding unit level weight (BCW) coding mode is derived based on a rule for a position of the current block for conversion between a current block of video and a bitstream representation of the video. In the BCW coding mode, a weight set including a plurality of weights is used to generate a bi-prediction value of the current block. The method (2100) further includes, at operation 2104, performing the conversion based on the determination result.

In some embodiments, the bidirectional prediction value of the current block is generated as a non-averaging weighted sum of two motion vectors, provided that at least one weight in the weight set is applied. In some embodiments, the rule provides that the index is not derived according to the neighboring block if the current block and the neighboring block are located in different coding tree units or largest coding units. In some embodiments, the rule provides that the index is not derived according to the neighboring block if the current block and the neighboring block are located in different lines or rows within one coding tree unit. In some embodiments, the rule provides that the index is not derived according to the neighboring block if the current block and the neighboring block are located in different non-overlapping regions of one tile, slice, or picture of the video. In some embodiments, the rule provides that the index is not derived according to the neighboring block if the current block and the neighboring block are located in different rows of different non-overlapping regions of one tile, slice, or picture of the video. In some embodiments, each region has a size of 64x64.

In some embodiments, the top corner of the current block is represented by (x, y), and the top corner of the neighboring block is represented by (x', y'). The rule provides that the indices are not derived along the neighboring blocks if (x, y) and (x', y') satisfy some condition. In some embodiments, the condition states that x / M ≠ x' / M, where M is a positive integer. In some embodiments, M is 128 or 64. In some embodiments, the condition states that y / N ≠ y' / N, where N is a positive integer. In some embodiments, N is 128 or 64. In some embodiments, the condition states that (x / M ≠ x' / M) and (y / N ≠ y' / N), where M and N are positive integers. In some embodiments, M = N = 128 or M = N = 64. In some embodiments, the condition indicates that x >> M ≠ x' >> M, where M is a positive integer. In some embodiments, M is 6 or 7. In some embodiments, the condition indicates that y >> N ≠ y' >> N, where N is a positive integer. In some embodiments, N is 6 or 7. In some embodiments, the condition indicates that (x >> M ≠ x' >> M) and (y >> N ≠ y' >> N), where M and N are positive integers. In some embodiments, M = N = 6 or M = N = 7.

In some embodiments, whether a BCW coding mode is applicable to a picture, a slice, a group of tiles, or a tile is signaled in a picture parameter set, a slice header, a tile group header, or a tile, respectively, in the bitstream. In some embodiments, whether a BCW coding mode is applicable to a picture, a slice, a group of tiles, or a tile is derived based on information associated with the picture, slice, group of tiles, or tile. In some embodiments, such information includes at least a quantization parameter (QP), a temporal layer, or a Picture Order Count distance.

FIG. 22 is a flowchart representation of a method (2200) for video processing according to the present invention. The method (2200) includes, at operation 2202, determining an intra-prediction mode of a current block independently of intra-prediction modes of neighboring blocks for conversion between a current block of video coded using a combined inter and intra prediction (CIIP) coding technique and a bitstream representation of the video. The CIIP coding technique uses intermedia inter-prediction values and intermedia intra-prediction values to derive a final prediction value of the current block. The method (2200) further includes, at operation 2204, performing the conversion based on the determination result.

In some embodiments, the intra-prediction mode of the current block is determined without reference to the intra-prediction prediction mode of any neighboring block. In some embodiments, the neighboring block is coded using a CIIP coding technique. In some embodiments, the intra-prediction of the current block is determined based on the intra-prediction mode of a second neighboring block that is coded using an intra-prediction coding technique. In some embodiments, whether to determine the intra-prediction mode of the current block according to the second intra-prediction mode is based on whether a certain condition is satisfied, which condition specifies a relationship between the first block, the current block, and the second neighboring block, the second block. In some embodiments, this determination is part of a most probable mode (MPM) construction process of the current block to derive a list of MPM modes.

FIG. 23 is a flowchart representation of a method (2300) for video processing according to the present invention. The method (2300) includes, at operation 2302, determining an intra-prediction mode of a current block according to a first intra-prediction mode of a first neighboring block and a second intra-prediction mode of a second neighboring block for conversion between a current block of video coded using an Inter and Intra Integrated Prediction (CIIP) coding technique and a bitstream representation of the video. The first neighboring block is coded using the intra-prediction coding technique and the second neighboring block is coded using the CIIP coding technique. The first intra-prediction mode is given a different priority than the second intra-prediction mode. The CIIP coding technique uses intermedia inter-prediction values and intermedia intra-prediction values to derive a final prediction value of the current block. The method (2300) also further includes, in operation 2304, a step of performing the transformation based on the decision result.

In some embodiments, this determination is part of a most likely mode (MPM) construction process of the current block to derive a list of MPM modes. In some embodiments, the first intra-prediction mode is positioned before the second intra-prediction mode in the list of MPM candidates. In some embodiments, the first intra-prediction mode is positioned after the second intra-prediction mode in the list of MPM candidates. In some embodiments, the coding of the intra-prediction mode bypasses the most likely mode (MPM) construction process of the current block. In some embodiments, the method further comprises determining an intra-prediction mode of a subsequent block based on the intra-prediction mode of the current block, wherein the subsequent block is coded using an intra-prediction coding technique and the current block is coded using a CIIP coding technique. In some embodiments, this determination is part of a most likely mode (MPM) construction process of the subsequent block. In some embodiments, in the MPM configuration process of the subsequent block, the intra-prediction mode of the current block is given a lower priority than the intra-prediction mode of another neighboring block that is coded using the intra-prediction coding technique. In some embodiments, whether to determine the intra-prediction mode of the subsequent block based on the intra-prediction mode of the current block is based on whether a certain condition is satisfied, which condition specifies a relationship between the first block, the subsequent block, and the second block, the current block. In some embodiments, the condition is as follows:

(1) The first block and the second block shall be located within the same line of the coding tree unit (CTU);

(2) Block 1 and Block 2 will be located within the same CTU;

(3) The first block and the second block shall be within the same area, or

(4) The first block and the second block shall be within the same line of the above area.

At least one of the regions is included. In some embodiments, the width of the region is equal to the height of the region. In some embodiments, the region has a size of 64x64.

In some embodiments, only a subset of the list of most probable modes (MPMs) for the common intra coding scheme is used for the current block. In some embodiments, the subset comprises a single MPM mode within the list of MPM modes for the common intra coding scheme. In some embodiments, the single MPM mode is the first MPM mode within the list. In some embodiments, an index indicating the single MPM mode is omitted in the bitstream. In some embodiments, the subset comprises the first four MPM modes within the list of MPM modes. In some embodiments, an index indicating an MPM mode within the subset is signaled in the bitstream. In some embodiments, a coding context for coding an intra-coded block is reused for coding the current block. In some embodiments, a first MPM flag for an intra-coded block and a second MPM flag for the current block share a same coding context within the bitstream. In some embodiments, the intra-prediction mode of the current block is selected from a list of MPM modes, regardless of the size of the current block. In some embodiments, the processed MPM configuration is enabled by default, and the flag indicating the MPM configuration process is omitted in the bitstream. In some embodiments, the MPM list construction process is not required for the current block.

In some embodiments, the luma-prediction-chroma mode is used to process the chroma component of the current block. In some embodiments, the derived mode is used to process the chroma component of the current block. In some embodiments, multiple intra-prediction modes are used to process the chroma component of the current block. In some embodiments, the multiple intra-prediction modes are used based on the color format of the chroma component. In some embodiments, when the color format is 4:4:4, the multiple intra-prediction modes are the same as the intra-prediction mode for the luma component of the current block. In some embodiments, each of the four intra-prediction modes is coded using one or more bits, and the four intra-prediction modes include a planar mode, a DC mode, a vertical mode, and a horizontal mode. In some embodiments, the four intra-prediction modes are coded using 00, 01, 10, and 11. In some embodiments, the four intra-prediction modes are coded using 0, 10, 110, and 111. In some embodiments, the four intra-prediction modes are coded using 1, 01, 001, and 000. In some embodiments, only a subset of the four intra-prediction modes is available when the width W and the height H of the current block satisfy certain conditions. In some embodiments, this subset includes the planar mode, the DC mode, and the vertical mode when W > N × H, where N is an integer. In some embodiments, the planar mode, the DC mode, and the vertical mode are coded using 1, 01, and 11. In some embodiments, the planar mode, the DC mode, and the vertical mode are coded using 0, 10, and 00. In some embodiments,

These subsets include planar mode, DC mode, and horizontal mode, where H > N × W, where N is an integer. In some embodiments, planar mode, DC mode, and horizontal mode are coded using 1, 01, and 11. In some embodiments, planar mode, DC mode, and horizontal mode are coded using 0, 10, and 00. In some embodiments, N = 2. In some embodiments, only DC mode and planar mode are used for the current block. In some embodiments, an indicator indicating DC mode or planar mode is signaled in the bitstream.

FIG. 24 is a flowchart representation of a method (2400) for video processing according to the present invention. The method (2400) includes, at operation 2402, determining whether an inter-intra integrated prediction (CIIP) process is applicable to the color components of the current block for conversion between a current block of video and a bitstream representation of the video, based on the size of the current block. The CIIP coding technique uses intermedia inter prediction values and intermedia intra prediction values to derive a final prediction value of the current block. The method (2400) further includes, at operation 2404, performing the conversion based on the determination result.

In some embodiments, the chroma component includes a chroma component, and the CIIP process is not performed on the chroma component if the width of the current block is less than 4. In some embodiments, the chroma component includes a chroma component, and the CIIP process is not performed on the chroma component if the height of the current block is less than 4. In some embodiments, the intra-prediction mode for the chroma component of the current block is different from the intra-prediction mode for the luma component of the current block. In some embodiments, the chroma component is in: a DC mode, a planar mode, or a luma-prediction-chroma mode.

One of the following is used. In some embodiments, the intra-prediction mode for a chroma component is determined based on a color format of the chroma component. In some embodiments, the color format includes 4:2:0 or 4:4:4.

FIG. 25 is a flowchart representation of a method (2500) for video processing according to the present invention. The method (2500) includes, at operation 2502, determining whether an inter and intra integrated prediction (CIIP) coding technique should be applied to the current block for conversion between a current block of video and a bitstream representation of the video, based on characteristics of the current block. The CIIP coding technique uses intermedia inter prediction values and intermedia intra prediction values to derive a final prediction value of the current block. The method (2500) further includes, at operation 2504, performing the conversion based on the determination result.

In some embodiments, the characteristic comprises a size of a current block having a width W and a height H, where W and H are integers, and the inter-intra prediction coding scheme is disabled for the current block if the size of the block satisfies a condition. In some embodiments, the condition indicates that W is equal to T1, where T1 is an integer. In some embodiments, the condition indicates that H is equal to T1, where T1 is an integer. In some embodiments, T1 = 4. In some embodiments, T1 = 2. In some embodiments, the condition indicates that W is greater than T1 or that H is greater than T1, where T1 is an integer. In some embodiments, T1 = 64 or 128. In some embodiments, the condition indicates that W is equal to T1 and H is equal to T2, where T1 and T2 are integers. In some embodiments, this condition indicates that W is equal to T2 and H is equal to T1, where T1 and T2 are integers. In some embodiments, T1 = 4 and T2 = 16.

In some embodiments, the characteristic current block includes a coding scheme applied thereto, and the CIIP coding scheme is disabled for the current block if the coding scheme satisfies a certain condition. In some embodiments, the condition indicates that the coding scheme is a bidirectional predictive coding scheme. In some embodiments, the bidirectional predictive coded merge candidate is converted into a unidirectional predictive coded merge candidate, so that the inter-intra predictive coding scheme can be applied to the current block. In some embodiments, the converted merge candidate is associated with a reference list 0 of the unidirectional predictive coding scheme. In some embodiments, the converted merge candidate is associated with a reference list 1 of the unidirectional predictive coding scheme. In some embodiments, only the unidirectional predictive coded merge candidates of the block are selected for the conversion. In some embodiments, the bidirectional predictive coded merge candidates are discarded to determine a merge index indicating the merge candidate in the bitstream representation. In some embodiments, the inter-intra predictive coding scheme is applied to the current block according to the determination result. In some embodiments, the merge candidate list construction process for the triangle prediction mode is used to derive the motion candidate list of the current block.

FIG. 26 is a flowchart representation of a method (2600) for video processing according to the present invention. The method (2600) includes, at operation 2602, determining whether a coding tool should be disabled for the current block for conversion between a current block of video and a bitstream representation of the video based on whether the current block is coded with Inter and Intra Integrated Prediction (CIIP) coding technique. The CIIP coding technique uses intermedia inter prediction values and intermedia intra prediction values to derive a final prediction value of the current block. The coding tool includes at least one of: Bi-Directional Optical Flow (BDOF), Overlapped Block Motion Compensation (OBMC), or a decoder-side motion vector refinement process (DMVR). The method (2600) further includes, at operation 2604, performing the conversion based on the determination result.

In some embodiments, the intra-prediction process for the current block is different from the intra-prediction process for the second block, which is coded using an intra-prediction coding technique. In some embodiments, in the intra-prediction process for the current block, filtering of neighboring samples is skipped. In some embodiments, in the intra-prediction process for the current block, a position-dependent intra-prediction sample filtering process is disabled. In some embodiments, in the intra-prediction process for the current block, a multi-line intra-prediction process is disabled. In some embodiments, in the intra-prediction process for the current block, a wide-angle intra-prediction process is disabled.

FIG. 27 is a flowchart representation of a method (2700) for video processing according to the present invention. The method (2700) includes, at operation 2702, determining a first precision P1 used for a motion vector for spatial motion prediction and a second precision P2 used for a motion vector for temporal motion prediction, for conversion between a block of video and a bitstream representation of the video. P1 and/or P2 are fractions. P1 and P2 are different from each other. The method (2700) further includes, at operation 2704, performing the conversion based on the determination result.

In some embodiments, the first precision is 1/16 luma pixel and the second precision is 1/4 luma pixel. In some embodiments, the first precision is 1/16 luma pixel and the second precision is 1/8 luma pixel. In some embodiments, the first precision is 1/8 luma pixel and the second precision is 1/4 luma pixel. In some embodiments, the first precision is 1/16 luma pixel and the second precision is 1/4 luma pixel. In some embodiments, the first precision is 1/16 luma pixel and the second precision is 1/8 luma pixel. In some embodiments, the first precision is 1/8 luma pixel and the second precision is 1/4 luma pixel. In some embodiments, at least one of the first or second precisions is less than 1/16 luma pixel.

In some embodiments, at least one of the first precision or the second precision can vary. In some embodiments, the first precision or the second precision can vary depending on a profile, level, or tier of the video. In some embodiments, the first precision or the second precision can vary depending on a temporal layer within a picture of the video. In some embodiments, the first precision or the second precision can vary depending on a size of a picture of the video.

In some embodiments, at least one of the first precision or the second precision is signaled within a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a tile group header, a tile, a coding tree unit, or a coding unit within the bitstream representation. In some embodiments, a motion vector is represented by (MVx, MVy) and a precision for the motion vector is represented by (Px, Py), where Px is associated with MVx and Py is associated with MVy. In some embodiments, Px and Py can vary depending on a profile, level, or tier of the video. In some embodiments, Px and Py can vary depending on a temporal layer of a picture within the video. In some embodiments, Px and Py can vary depending on a width of a picture within the video. In some embodiments, Px and Py are signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a tile group header, a tile, a coding tree unit, or a coding unit within the bitstream representation. In some embodiments, a decoded motion vector is represented as (MVx, MVy), and the motion vector is adjusted according to a second precision before the motion vector is stored as a temporal motion prediction motion vector. In some embodiments, the temporal motion prediction motion vector is adjusted to be (Shift(MVx, P1-P2), Shift(MVy, P1-P2)), where P1 and P2 are integers, P1 ≥ P2, and Shift represents a right shift operation for unsigned numbers. In some embodiments, the temporal motion prediction motion vector is adjusted to be (SignShift(MVx, P1-P2), SignShift(MVy, P1-P2)), where P1 and P2 are integers, and P1 ≥ P2, and SignShift denotes a right shift operation on a signed number. In some embodiments, the temporal motion prediction motion vector is adjusted to be (MVx << (P1-P2)), MVy << (P1-P2)), where P1 and P2 are integers, P1 ≥ P2, and << denotes a left shift operation on a signed number or an unsigned number.

FIG. 28 is a flowchart representation of a method (2800) for video processing according to the present invention. The method (2800) includes, at operation 2802, determining motion vectors (MVx, MVy) with precisions (Px, Py) for conversion between a block of video and a bitstream representation of the video. Px is associated with MVx, and Py is associated with MVy. MVx and MVy are represented using N bits, MinX ≤ MVx ≤ MaxX, MinY ≤ MVy ≤ MaxY, and MinX, MaxX, MinY, and MaxY are real numbers. The method (2800) further includes, at operation 2804, performing the conversion based on the determination result.

In some embodiments, MinX = MinY. In some embodiments, MinX ≠ MinY. In some embodiments, MaxX = MaxY. In some embodiments, MaxX ≠ MaxY.

In some embodiments, at least one of MinX or MaxX is based on Px. In some embodiments, the motion vector has a precision denoted as (Px, Py) and at least one of MinY or MaxY is based on Py. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY is based on N. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY for a spatial motion prediction motion vector is different from a corresponding MinX, MaxX, MinY, or MaxY for a temporal motion prediction motion vector. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY can vary depending on a profile, level, or tier of the video. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY can vary depending on a temporal layer of a picture in the video. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY can vary depending on a size of a picture in the video. In some embodiments, at least one of MinX, MaxX, MinY, or MaxY is signaled within a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a tile group header, a tile, a coding tree unit, or a coding unit within the bitstream representation. In some embodiments, MVx is clipped to [MinX, MaxX] before being used for spatial motion prediction or temporal motion prediction. In some embodiments, MVy is clipped to [MinY, MaxY] before being used for spatial motion prediction or temporal motion prediction.

FIG. 29 is a flowchart representation of a method (2900) for video processing according to the present invention. The method (2900) includes, at operation 2902, determining whether a shared merge list is applicable to the current block, depending on a coding mode of the current block, for conversion between a current block of video and a bitstream representation of the video. The method (2900) also includes, at operation 2904, performing the conversion based on the determination result.

In some embodiments, the shared merge list is not applicable when the current block is coded using the regular merge mode. In some embodiments, the shared merge list is applicable when the current block is coded using the Intra Block Copy (IBC) mode. In some embodiments, the method further comprises: prior to performing the transform, maintaining a table of motion candidates based on past transforms and bitstream representations of the video; and after performing the transform, disabling updating of the table of motion candidates if the current block is a child of a parent block to which the shared merge list can be applied, and the current block is coded using the regular merge mode.

FIG. 30 is a flowchart representation of a method (3000) for video processing according to the present invention. The method (3000) includes, at operation 3002, determining a second block of dimensions (W + N - 1) x (H + N - 1) for motion compensation during a transformation between a current block of video having a size of W x H and a bitstream representation of the video. The second block is determined based on a reference block of dimensions (W + N - 1 - PW) x (H + N - 1 - PH). N represents a filter size, and W, H, N, PW and PH are non-negative integers. Neither of PW nor PH are equal to 0. The method (3000) further includes, at operation 3004, performing the transformation based on the determined result.

In some embodiments, pixels in a second block located outside the reference block are determined by repeating one or more boundaries of the reference block. In some embodiments, PH = 0, and at least a left boundary or a right boundary of the reference block is repeated to generate the second block. In some embodiments, PW = 0, and at least a top boundary or a bottom boundary of the reference block is repeated to generate the second block. In some embodiments, PW > 0 and PH > 0, and the second block is generated by repeating at least a left boundary or a right boundary of the reference block, and then repeating at least a top boundary or a bottom boundary of the reference block. In some embodiments, PW > 0 and PH > 0, and the second block is generated by repeating at least a top boundary or a bottom boundary of the reference block, and then repeating at least a left boundary or a right boundary of the reference block.

In some embodiments, the left boundary of the reference block is repeated M1 times and the right boundary of the reference block is repeated (PW - M1) times, where M1 is a positive integer. In some embodiments, the upper boundary of the reference block is repeated M2 times and the lower boundary of the reference block is repeated (PH - M2) times, where M2 is a positive integer. In some embodiments, at least one of the PW or the PH is different for different color components of the current block, where the color components include at least a luma component or one or more chroma components. In some embodiments, at least one of the PW or the PH can vary depending on a size or shape of the current block. In some embodiments, at least one of the PW or the PH can vary depending on a coding characteristic of the current block, where the coding characteristic includes unidirectional predictive coding or bidirectional predictive coding.

In some embodiments, pixels in the second block outside the reference block are set to a value. In some embodiments, the value is 1 << (BD - 1), where BD is the bit-depth of pixel samples in the reference block. In some embodiments, BD is 8 or 10. In some embodiments, the one value is derived based on pixel samples of the reference block. In some embodiments, the one value is signaled in a video parameter set, a sequence parameter set, a picture parameter set, a slice header, a tile group header, a tile, a coding tree unit row, a coding tree unit, a coding unit, or a prediction unit. In some embodiments, padding of pixels in the second block located outside the reference block is disabled when the current block is affine coded.

FIG. 31 is a flowchart representation of a method (3100) for video processing according to the present invention. The method (3100) comprises, at operation 3102, determining a second block of dimensions (W + N - 1) x (H + N - 1) for motion compensation during the transformation between a current block of video having a size of W x H and a bitstream representation of the video. W and H are non-negative integers, where N is a non-negative integer and is based on a filter size. During the transformation, a refined motion vector is determined based on a multi-point search according to a motion vector refinement operation with respect to an original motion vector, wherein pixels long boundaries of the reference block are determined by repeating one or more non-boundary pixels. The method (3100) further comprises, at operation 3104, performing the transformation based on the determined result.

In some embodiments, processing the current block comprises filtering the current block in a motion vector refinement operation. In some embodiments, whether the reference block is applicable to processing of the current block is determined based on a dimension of the current block. In some embodiments, interpolating the current block comprises: interpolating a plurality of sub-blocks of the current block based on the second block. Each sub-block has a size of W1xH1, where W1 and H1 are non-negative integers. In some embodiments, W1=H1=4, W=H=8, and PW=PH=0. In some embodiments, the second block is determined entirely based on an integer portion of a motion vector of at least one of the plurality of sub-blocks. In some embodiments, if the maximum difference between the integer parts of the motion vectors of all the plurality of sub-blocks is less than or equal to 1 pixel, the reference block is determined based on the integer part of the motion vector of the upper-left sub-block of the current block, and each of the right boundary and the bottom boundary of the reference block is repeated once to obtain a second block. In some embodiments, if the maximum difference between the integer parts of the motion vectors of all the plurality of sub-blocks is less than or equal to 1 pixel, the reference block is determined based on the integer part of the motion vector of the lower-right sub-block of the current block, and each of the left boundary and the top boundary of the reference block is repeated once to obtain a second block. In some embodiments, the second block is determined entirely based on the changed motion vector of one of the plurality of sub-blocks.

In some embodiments, if the maximum difference between the integer parts of the motion vectors of all of the plurality of sub-blocks is less than or equal to two pixels, the motion vector of the upper-left sub-block of the current block is modified by adding one integer pixel distance to each component to obtain a modified motion vector. A reference block is determined based on the modified motion vector, and each of the left boundary, the right boundary, the upper boundary, and the lower boundary of the reference block is iterated once to obtain a second block.

In some embodiments, if the maximum difference between the integer parts of the motion vectors of all of the plurality of sub-blocks is less than or equal to two pixels, the motion vector of the lower-right sub-block of the current block is modified by subtracting one integer pixel distance from each component to obtain a modified motion vector. A reference block is determined based on the modified motion vector, and each of the left boundary, the right boundary, the upper boundary, and the lower boundary of the reference block is repeated once to obtain a second block.

FIG. 32 is a flowchart representation of a method (3200) for video processing according to the present invention. The method (3200) includes, at operation 3202, determining a prediction value at a given location within the block based on a weighted sum of inter prediction values and intra prediction values at the location for conversion between a block of video coded using an inter and intra integrated prediction (CIIP) coding technique and a bitstream representation of the video. The weighted sum is obtained based on adding an offset to an initial sum obtained based on the inter prediction values and the intra prediction values, and the offset is added prior to a weighted operation performed to determine the weighted sum. The method (3200) further includes, at operation 3204, performing the conversion based on a result of the determination.

In some embodiments, a location within the block is represented by (x, y), an inter prediction value at location (x, y) is represented by Pinter(x, y), an intra prediction value at location (x, y) is represented by Pintra(x, y), an inter-prediction weight at location (x, y) is represented by w_inter(x, y), and an intra-prediction weight at location (x, y) is represented by w_intra(x, y). The prediction value at location (x, y) is determined such that (Pintra(x, y) × w_intra(x, y) + Pinter(x, y) × w_inter(x, y) + offset(x, y)) >> N, where w_intra(x, y) + w_inter(x, y) = 2^N and offset(x, y) = 2^(N-1), where N is a positive integer. In some embodiments, N = 2.

In some embodiments, the weighted sum is determined using the same weights for the inter-prediction value and the intra-prediction value at that location. In some embodiments, a zero weight is used depending on the location within the block to determine the weighted sum. In some embodiments, a zero weight is applied to the inter-prediction value. In some embodiments, a zero weight is applied to the intra-prediction value.

FIG. 33 is a flowchart representation of a method (3300) for video processing according to the present invention. The method (3300) includes, at operation 3302, determining how coding information of a current block is represented in the bitstream representation, based in part on whether a condition associated with a size of the current block is satisfied, for conversion between a current block of video and a bitstream representation of the video. The method (3300) also includes, at operation 3304, performing a conversion based on this determination.

In some embodiments, the coding information is signaled in the bitstream representation if the condition is satisfied. In some embodiments, the coding information is omitted from the bitstream representation if the condition is satisfied. In some embodiments, if the condition is satisfied, at least one of an IBC (intra block copy) coding tool or an inter prediction coding tool is disabled for the current block. In some embodiments, the coding information is omitted from the bitstream representation if the IBC coding tool and the inter prediction coding tool are disabled. In some embodiments, the coding information is signaled in the bitstream representation if the IBC coding tool is enabled and the inter prediction tool is disabled.

In some embodiments, the coding information includes an operation mode associated with the IBC tool and/or the inter prediction coding tool, wherein the operation mode includes at least a normal mode or a skip mode. In some embodiments, the coding information includes an indicator indicating use of a prediction mode associated with the coding tool. In some embodiments, the coding tool is not used when the coding information is omitted from the bitstream representation. In some embodiments, the coding tool is used when the coding information is omitted from the bitstream representation. In some embodiments, the coding tool includes an IBC (intra block copy) coding tool or an inter prediction coding tool.

In some embodiments, when an indicator is omitted from the bitstream, one or more indicators indicating one or more other coding tools are signaled in the bitstream representation. In some embodiments, the one or more other coding tools include at least an intra coding tool or a palette coding tool. In some embodiments, the indicator distinguishes between an inter mode and an intra mode, wherein the one or more indicators include a first indicator indicating an IBC mode that is signaled in the bitstream when the inter prediction coding tool is disabled and the IBC coding tool is enabled, and a second indicator indicating a palette mode.

In some embodiments, the coding information includes a third indicator for a skip mode, and the skip mode is an inter-skip mode or an IBC skip mode. In some embodiments, the third indicator for the skip mode is signaled when the inter prediction coding tool is disabled and the IBC coding tool is enabled. In some embodiments, the prediction mode is determined to be the IBC mode when the inter prediction coding tool is disabled and the IBC coding tool is enabled, despite the indicator indicating that use of the IBC mode is omitted in the bitstream representation, and the skip mode is applicable to the current block.

In some embodiments, the coding information includes a triangle mode. In some embodiments, the coding information includes an inter prediction direction. In some embodiments, the coding information is omitted from the bitstream representation if the current block is coded using a unidirectional predictive coding tool. In some embodiments, the coding information includes an indicator indicating the use of a symmetric motion vector difference (SMVD) method. In some embodiments, if the condition is satisfied, the current block is set to be unidirectionally predicted despite the indicator indicating the use of the SMVD method in the bitstream representation. In some embodiments, if the indicator indicating the use of the SMVD method is omitted from the bitstream representation, only the list 0 or list 1 associated with the unidirectional predictive coding tool is used in the motion compensation process.

In some embodiments, the condition indicates that the width of the current block is T1 and the height of the current block is T2, where T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T2 and the height of the current block is T1, where T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T1 and the height of the current block is less than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is T2 and the height of the current block is less than or equal to T1, where T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is less than or equal to T1 and the height of the current block is less than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the condition indicates that the width of the current block is less than or equal to T1 and the height of the current block is less than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the width of the current block is greater than or equal to T1 and the height of the current block is greater than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the condition specifies that the width of the current block is greater than or equal to T1 and the height of the current block is greater than or equal to T2, where T1 and T2 are positive integers. In some embodiments, T1 is 4 and T2 is 16. In some embodiments, T1 and T2 are 8. In some embodiments, T1 is 8 and T2 is 4. In some embodiments, T1 and T2 are 4. In some embodiments, T1 and T2 are 32. In some embodiments, T1 and T2 are 64. In some embodiments, T1 and T2 are 128.

In some embodiments, the condition indicates that the current block has a size of 4x8, 8x4, 4x16, or 16x4. In some embodiments, the condition indicates that the block has a size of 4x8, 8x4. In some embodiments, the condition indicates that the size of the current block is 4xN or Nx4, where N is a positive integer and N ≤ 16. In some embodiments, the condition indicates that the size of the current block is 8xN or Nx8, where N is a positive integer and N ≤ 16. In some embodiments, the condition indicates that the color component of the current block is less than or equal to N samples. In some embodiments, N is 16. In some embodiments, N is 32.

In some embodiments, the condition indicates that the chroma component of the current block has a width equal to its height. In some embodiments, the condition indicates that the chroma component of the current block has a size of 4x8, 8x4, 4x4, 4x16 or 16x4. In some embodiments, the chroma component includes a luma component or one or more chroma components. In some embodiments, if the condition indicates that the current block has a width of 4 and a height of 4, the inter prediction tool is disabled. In some embodiments, if the condition indicates that the current block has a width of 4 and a height of 4, the IBC prediction tool is disabled.

In some embodiments, the list 0 associated with the single predictive coding tool is used in the motion compensation process if (1) the current block has a width of 4 and a height of 8, or (2) the current block has a width of 8 and a height of 4. The double predictive coding tool is disabled in the motion compensation process.

FIG. 34 is a flowchart representation of a method (3400) for video processing according to the present invention. The method (3400) includes, at operation 3402, determining a modified motion vector set for transformation between a current block of video and a bitstream representation of the video. The method (3400) includes, at operation 3404, performing a transformation based on the modified motion vector set. Due to the current block satisfying the condition, the modified motion vector set is a modified version of the motion vector set associated with the current block.

In some embodiments, the set of motion vectors is derived using one of regular merge coding techniques, sub-block temporal motion vector prediction (sbTMVP), or Merge with motion vector difference (MMVD). In some embodiments, the set of motion vectors includes block vectors used in an intra block copy (IBC) coding technique. In some embodiments, the condition specifies that the size of the luma component of the current block is equal to a predefined size. In some embodiments, the predefined size includes at least one of 8x4 or 4x8.

In some embodiments, the step of changing the set of motion vectors comprises the step of converting the motion vectors into unidirectional motion vectors when the motion information of the current block is determined in a bidirectional manner, including a first motion vector referencing a first reference picture in a first reference picture list for a first prediction direction and a second motion vector referencing a second reference picture in a second reference picture list for a second prediction direction. In some embodiments, the motion information of the current block is derived based on motion information of a neighboring block. In some embodiments, the method comprises the step of discarding information of one of the first prediction direction or the second prediction direction. In some embodiments, the step of discarding information comprises the step of changing a motion vector of a prediction direction whose motion is discarded to (0, 0). In some embodiments, the step of discarding information comprises the step of changing a reference index of the prediction direction whose motion is discarded to -1. In some embodiments, the step of discarding information comprises the step of changing the discarded prediction direction to another prediction direction.

In some embodiments, the method further comprises a step of deriving a new motion candidate based on information about the first prediction direction and information about the second prediction direction. In some embodiments, the step of deriving the new motion candidate comprises a step of determining a motion vector of the new motion candidate based on an average of a first motion vector in the first prediction direction and a second motion vector in the second prediction direction. In some embodiments, the step of deriving the new motion candidate comprises a step of determining a scaled motion vector by scaling the first motion vector in the first prediction direction according to the information about the second prediction direction; and a step of determining a motion vector of the new motion candidate based on an average of the scaled motion vector and the second motion vector in the second prediction direction.

In some embodiments, the first reference picture list is list 0 and the second reference picture list is list 1. In some embodiments, the first reference picture list is list 1 and the second reference picture list is list 0. In some embodiments, the method comprises updating a table of motion candidates determined based on past transformations using the transformed motion vector. In some embodiments, the table comprises a History Motion Vector Prediction (HMVP) table.

In some embodiments, the method comprises a step of updating a table of motion candidates determined based on past transformations using information about the first prediction direction and the second prediction direction prior to the step of modifying the set of motion vectors. In some embodiments, the transformed motion vector is used for a motion compensation operation for the current block. In some embodiments, the transformed motion vector is used for predicting a motion vector of a next (subsequent) block. In some embodiments, the transformed motion vector is used in a deblocking process for the current block.

In some embodiments, the information about the first prediction direction and the second prediction direction is used for a motion compensation operation for the current block prior to the step of changing the motion vector set. In some embodiments, the information is used for predicting a motion vector of a subsequent block prior to the step of changing the motion vector set. In some embodiments, the information about the first prediction direction and the second prediction direction is used for a deblocking process for the current block prior to the step of changing the motion vector set.

In some embodiments, the transformed motion vectors are used for a motion refinement operation for the current block. In some embodiments, prior to the step of modifying the set of motion vectors, information about the first prediction direction and the second prediction direction is used for the refinement operation for the current block. In some embodiments, the refinement operation includes at least an optical flow coding operation including a Bi-Directional Optical Flow (BDOF) operation or a Prediction Refined with Optical Flow (PROF) operation.

In some embodiments, the method comprises the steps of generating intermediate prediction blocks based on information about the first prediction direction and information about the second prediction direction, improving the intermediate prediction blocks, and determining a final prediction block based on one of the intermediate prediction blocks.

FIG. 35 is a flowchart representation of a method (3500) for video processing according to the present invention. The method (3500) includes, at operation 3502, determining a unidirectional motion vector from a bidirectional motion vector if a block-level condition is satisfied for transformation between a current block of video and a bitstream representation of the video. The unidirectional motion vector is then used as a merge candidate for transformation. The method (3500) includes, at operation 3504, performing the transformation based on the determination.

In some embodiments, the transformed unidirectional motion vector is used as a base merge candidate in an MMVD (Merge with Motion Vector Difference) coding process. In some embodiments, the transformed unidirectional motion vector is then inserted into a merge list. In some embodiments, the unidirectional motion vector is transformed based on one prediction direction of the bidirectional motion vector. In some embodiments, the unidirectional motion vector is transformed based on only one prediction direction, and only one prediction direction is associated with reference picture list 0. In some embodiments, if the video data unit in which the current block is located is bi-predicted, only one prediction direction for the first candidate in the candidate list is associated with reference picture list 0, and only one prediction direction for the second candidate in the candidate list is associated with reference picture list 1. The candidate list includes a merge candidate list or an MMVD base candidate list.

In some embodiments, the video data unit includes a current slice, a group of tiles, or a picture of the video. In some embodiments, the unidirectional motion vector is determined based on reference picture list 1 if all reference pictures of the video data unit are past pictures in display order. In some embodiments, the unidirectional motion vector is determined based on reference picture list 0 if at least a first reference picture of the reference pictures of the video data unit is a past picture and at least a second reference picture of the reference pictures of the video data unit is a future picture in display order. In some embodiments, the unidirectional motion vectors determined based on different reference picture lists are interleaved in the merge list.

In some embodiments, the unidirectional motion vector is determined based on a low delay check indicator. In some embodiments, the unidirectional motion vector is determined prior to a motion compensation process during the transform. In some embodiments, the unidirectional motion vector is determined prior to a step of adding a motion vector difference in a MMVD (Merge With Motion Vector Difference) coding process. In some embodiments, the unidirectional motion vector is determined prior to a sample refinement process. In some embodiments, the unidirectional motion vector is determined prior to a BDOF (Bi-Directional Optical Flow) process. In some embodiments, the BDOF process is subsequently disabled based on the determination to use the unidirectional motion vector.

In some embodiments, the unidirectional motion vector is determined prior to the PROF (Prediction Refinement with Optical Flow) process. In some embodiments, the unidirectional motion vector is determined prior to the DMVR (Decoder-side Motion Vector Refinement) process. In some embodiments, the DMVR process is subsequently disabled based on the unidirectional motion vector.

In some embodiments, the block-dimensional condition indicates that the width of the current block is T1 and the height of the current block is T2, where T1 and T2 are positive integers. In some embodiments, the block-dimensional condition indicates that the width of the current block is T2 and the height of the current block is T1, where T1 and T2 are positive integers. In some embodiments, the block-dimensional condition indicates that the width of the current block is T1 and the height of the current block is less than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the block-dimensional condition indicates that the width of the current block is T2 and the height of the current block is less than or equal to T1, where T1 and T2 are positive integers.

FIG. 36 is a flowchart representation of a method (3600) for video processing according to the present invention. The method (3600) includes, at operation 3602, determining that motion candidates of a current block are limited to unidirectional prediction candidates based on dimensions of the current block for transformation between a current block of video and a bitstream representation of the video. The method (3600) includes, at operation 3604, performing a transformation based on the determination.

In some embodiments, if the dimension of the current block satisfies the condition, all motion candidates of the current block are restricted to bidirectional prediction candidates. In some embodiments, the bidirectional motion candidates are converted to unidirectional prediction candidates if the dimension of the current block satisfies the condition. In some embodiments, the dimension of the current block indicates that the width of the current block is T1 and the height of the current block is T2, where T1 and T2 are positive integers. In some embodiments, the condition on the block dimension indicates that the width of the current block is T2 and the height of the current block is T1, where T1 and T2 are positive integers. In some embodiments, the condition on the block dimension indicates that the width of the current block is T1 and the height of the current block is less than or equal to T2, where T1 and T2 are positive integers. In some embodiments, the dimension of the current block indicates that the width of the current block is T2 and the height of the current block is less than or equal to T1, where T1 and T2 are positive integers. In some embodiments, T1 is 4 and T2 is 8. In some embodiments, T1 is 4 and T2 is 4. In some embodiments, T1 is 8 and T2 is also 8.

In some embodiments, performing the transformation in the method described above comprises generating a bitstream representation based on a current block of the video. In some embodiments, performing the transformation in the method described above comprises generating a current block of the video from the bitstream representation.

In this document, the term “video processing” may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion from a pixel representation of the video to a corresponding bitstream representation or vice versa. The bitstream representation of the current video block may correspond to co-located bits in the bitstream, or may be spread over different locations, as defined in the syntax, for example. For example, a macroblock may be encoded in terms of transformed and coded error residual values, and may be encoded using bits in a header or other fields in the bitstream. Additionally, during the conversion, the decoder may parse the bitstream with the knowledge that some fields may or may not be present based on the decisions made in the solution described above. Similarly, the encoder may determine whether certain syntax fields are included or not, and may generate the bitstream representation accordingly by including or excluding the syntax fields from the bitstream representation.

The disclosed solutions, examples, embodiments, modules and functional operations and other solutions, examples, embodiments, modules and functional operations described herein may be implemented as digital electronic circuits, or as computer software, firmware, or hardware including the structures disclosed herein and their structural equivalents, or as a combination of one or more of these. The disclosed embodiments and other embodiments may be implemented as one or more computer program products, for example, as one or more modules of computer program instructions encoded on a computer-readable medium for execution by or controlling the operation of a data processing apparatus. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials for affecting a machine-readable propagated signal, or a combination of one or more of these. The term "data processing apparatus" encompasses any apparatus, device, or machine for processing data, including, for example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, such devices may include code that creates an execution environment for the computer program of interest, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. A propagated signal is an artificially generated signal, for example, an electrical, optical, or electromagnetic signal generated by a machine that encodes information to be transmitted to a suitable receiver device.

A computer program (also known as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, such as a stand-alone program, or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored within a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), within a single file dedicated to the program in question, or within a number of coordinated files (e.g., a file that stores one or more modules, sub-programs, or portions of code). A computer program may be implemented to be executed on a single computer, or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

The processes and logic flows described herein can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. Additionally, the processes and logic flows can be performed by, and devices can also be implemented as, special purpose logic circuitry, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

Processors suitable for executing a computer program include, for example, both general-purpose and application-specific microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. Essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Typically, the computer will further include, or be operatively coupled to, one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or to receive data from or transfer data to or from such devices. However, the computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include, for example, semiconductor memory devices, such as EPROMs, EEPROMs, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and all forms of non-volatile memory, media, and memory devices, including CD-ROMs and DVD disks. The processor and memory may be supplemented by, or incorporated within, special purpose logic circuitry.

While this patent specification contains many features, they should not be construed as limiting any subject matter or the scope of what may be claimed, but rather as descriptions of features that may be unique to particular embodiments of a particular technology. Some features that are described in this patent specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented separately in multiple embodiments or in any suitable subcombination. Furthermore, even if features may be described above and initially claimed as acting in particular combinations, one or more features of the claimed combinations may in some cases be excised from the combination, and the claimed combination may be intended to be a subcombination or a variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be construed as requiring that these operations be performed in the particular order illustrated or in sequential order, or that all of the illustrated operations be performed, to achieve desired results. Moreover, the separation of various system components within the embodiments described in this patent specification should not be construed as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, improvements, and variations may be made based on what is described and illustrated in this patent specification.

Claims (54)

In a video processing method,
For conversion between a current block of a video and a bitstream of the video, a step of determining whether multiple flags of the current block exist in the bitstream based on whether a same dimension condition related to the size of the current block is satisfied; and
A step of performing the conversion based on the above decision; Including,
Whether a skip flag indicating whether skip mode is used for the current block and a flag indicating whether inter prediction mode is used for the current block exist in the bitstream is based on the same dimension condition indicating that the width of the current block is equal to 4 and the height of the current block is equal to 4 respectively.
In the case where the sum of the width and height of the current block is 12, the context increasement used for inter prediction direction flag coding is equal to a fixed integer that is not a function of the width or height of the current block,
A video processing method, wherein, when the size of the current block is 8x4 or 4x8, a weight index is set to 0 for the current block, and the weight index is used to find a weight from a set of weights set to generate a weighted sum from prediction samples of list 0 and list 1.
In paragraph 1,
A video processing method, wherein a merge data syntax structure is not included in the bitstream when the skip flag value is 1. In paragraph 1,
A video processing method, wherein the plurality of flags includes an intra block copy flag indicating whether an intra block copy mode is used for the current block, wherein whether the intra block copy flag exists in the bitstream is based on the same dimension condition.
In the third paragraph,
A video processing method, wherein, when the above same dimension condition is satisfied and the skip flag value is 1, the intra block copy flag does not exist in the bitstream.
In paragraph 4,
A video processing method, wherein the intra block copy flag is a pred_mode_ibc_flag corresponding to a coding unit level flag.
In paragraph 1,
A video processing method, wherein the skip flag is not present in the bitstream, corresponding to a video sequence level flag value that satisfies the above same dimension condition and indicates that intra block copy mode is not allowed for the entire video sequence.
In paragraph 6,
The above video sequence level flag is sps_ibc_enabled_flag, a video processing method. In paragraph 1,
A video processing method, wherein the plurality of flags includes a flag related to whether triangle mode is used for the current block.
In paragraph 1,
A video processing method wherein the above inter prediction direction flag is used to indicate whether list 0, list 1, or bidirectional prediction is used for the current block.
In Article 9,
A video processing method, wherein the bidirectional prediction is not used for the current block if the size of the current block is 8x4 or 4x8.
In paragraph 1,
A video processing method, wherein said transform comprises encoding said current block into said bitstream.
In paragraph 1,
A video processing method, wherein the transform comprises decoding the current block from the bitstream.
In a video data processing device including a processor and a non-transitory memory including instructions, the instructions, when executed by the processor, cause the processor to:
For conversion between a current block of a video and a bitstream of the video, it is determined whether multiple flags of the current block exist in the bitstream based on whether a same dimension condition related to the size of the current block is satisfied, and
The above conversion is performed based on the above decision,
Whether a skip flag indicating whether skip mode is used for the current block and a flag indicating whether inter prediction mode is used for the current block exist in the bitstream is based on the same dimension condition indicating that the width of the current block is equal to 4 and the height of the current block is equal to 4 respectively.
In the case where the sum of the width and height of the current block is 12, the context increasement used for inter prediction direction flag coding is equal to a fixed integer that is not a function of the width or height of the current block,
A video data processing device, wherein, when the size of the current block is 8x4 or 4x8, a weight index is set to 0 for the current block, and the weight index is used to find a weight from a set of weights for generating a weighted sum from prediction samples of list 0 and list 1.
In a non-transitory computer-readable storage medium storing instructions, the instructions cause a processor to:
For conversion between a current block of a video and a bitstream of the video, it is determined whether multiple flags of the current block exist in the bitstream based on whether a same dimension condition related to the size of the current block is satisfied, and
The above conversion is performed based on the above decision,
Whether a skip flag indicating whether skip mode is used for the current block and a flag indicating whether inter prediction mode is used for the current block exist in the bitstream is based on the same dimension condition indicating that the width of the current block is equal to 4 and the height of the current block is equal to 4 respectively.
In the case where the sum of the width and height of the current block is 12, the context increasement used for inter prediction direction flag coding is equal to a fixed integer that is not a function of the width or height of the current block,
A non-transitory computer-readable storage medium, wherein a weight index is set to 0 for the current block when the size of the current block is 8x4 or 4x8, and the weight index is used to find a weight from a set of weights for generating a weighted sum from prediction samples of list 0 and list 1.
A non-transitory computer-readable recording medium storing a bitstream of a video generated by a method performed by a video processing device, the method comprising:
A step of determining whether multiple flags of the current block exist in the bitstream based on whether a same dimension condition related to the size of the current block is satisfied, for conversion between a current block of a video and a bitstream of the video;
a step of generating the bitstream based on the above decision; and
Comprising the step of storing the above bitstream in a non-transitory computer-readable recording medium,
Whether a skip flag indicating whether skip mode is used for the current block and a flag indicating whether inter prediction mode is used for the current block exist in the bitstream is based on the same dimension condition indicating that the width of the current block is equal to 4 and the height of the current block is equal to 4 respectively.
In the case where the sum of the width and height of the current block is 12, the context increasement used for inter prediction direction flag coding is equal to a fixed integer that is not a function of the width or height of the current block,
A non-transitory computer-readable recording medium, wherein, when the size of the current block is 8x4 or 4x8, a weight index is set to 0 for the current block, and the weight index is used to find a weight from a set of weights for generating a weighted sum from prediction samples of list 0 and list 1. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete