TWI829769B - Motion vector accessing range for affine - Google Patents

Abstract

A method of video processing includes accessing motion vectors(MV) stored in at least one of basic unit blocks adjacent to a current block; and performing a video processing between the current block and a bitstream representation of the current block based on the accessed MVs.

Description

Affine motion vector access range

This patent document relates to video encoding technology, devices and systems. [Cross-reference to related applications] This application timely claims International Patent Application No. PCT/CN2018/107629 filed on September 26, 2018 and International Patent Application No. Priority and rights of PCT/CN2018/107869. The entire disclosures of International Patent Application No. PCT/CN2018/107629 and International Patent Application No. PCT/CN2018/107869 are hereby incorporated by reference as part of the disclosure of this application.

Motion compensation (MC) is a technique in video processing that predicts frames in a video by taking into account the motion of the camera and/or objects in the video, given previous and/or future frames. Motion compensation can be used in the encoding of video data for video compression.

This document discloses methods, systems, and apparatus involving the use of affine motion compensation in video encoding and decoding.

In an exemplary aspect, a method of video processing is disclosed. The method includes: accessing a motion vector (MV) stored in at least one basic unit block adjacent to a current block; and performing a bit stream of the current block and the current block based on the accessed MV. Video processing between representations.

In one exemplary aspect, a video processing device is disclosed, including a processor configured to implement the methods described herein.

In yet another representative aspect, various techniques described herein may be implemented as a computer program product stored on non-transitory computer-readable media. A computer program product contains program code for performing the methods described herein.

In yet another representative aspect, a video decoder device can implement a method as described herein.

The details of one or more implementations are set forth in the accompanying attachments, drawings, and the following description. Other features will become apparent from the description and drawings and from the claims.

This document provides several techniques that can be implemented as digital video encoders and decoders. Section headings are used in this document to facilitate understanding and do not limit the scope of the techniques and embodiments disclosed in each section to that section only.

In this document, the term "video processing" may refer to video encoding, video decoding, video compression, or video decompression. For example, a video compression algorithm may be applied during the conversion of a pixel representation of a video to a corresponding bitstream representation, or vice versa.

1 Overview

The present invention relates to video/image coding technology. Specifically, it relates to affine prediction in video/image coding. It can be applied to existing video coding standards, such as HEVC, or to a yet-to-be-finalized standard (Multifunction Video Coding). It can also be applied to future video/image encoding standards or video/image codecs.

2. Introduction

Sub-block-based prediction was first introduced into the video coding standard by HEVC Annex I (3D-HEVC). With sub-block based prediction, a block such as a coding unit (CU) or a prediction unit (PU) is divided into several non-overlapping sub-blocks. Different sub-blocks may be assigned different motion information, such as reference indexes or motion vectors (MVs), and motion compensation (MC) is performed individually for each sub-block. Figure 1 illustrates the concept of sub-block based prediction.

To explore future video encoding technologies beyond HEVC, the Joint Video Exploration Team (JVET) was co-founded by VCEG and MPEG in 2015. Since then, JVET has adopted many new methods and incorporated them into reference software called the Joint Exploration Model (JEM).

In JEM, sub-block based prediction is employed in several coding tools such as affine prediction, alternative temporal motion vector prediction (ATMVP), space-time motion vector prediction (STMVP), bidirectional optical flow (BIO) and frame rate up conversion (FRUC). Affine prediction is also adopted into VVC.

2.1 Affine prediction

In HEVC, only the translational motion model is applied for motion compensated prediction (MCP). In the real world, there are many kinds of motion, such as zooming in/out, rotation, perspective motion and other irregular motions. In VVC, a simplified affine transform motion compensated prediction is applied. As shown in Figures 2A-2B, the affine motion field of a block is described by two (in the 4-parameter affine model) or three (in the 6-parameter affine model) control point motion vectors.

Figures 2A-2B show simplified affine motion models (a) 4-parameter affine model; (b) 6-parameter affine model.

The motion vector field (MVF) of a block is described by the following equation with a 4-parameter affine model:

(1)

and a 6-parameter affine model:

(2)

where ( mv ^h ₀ , mv ^v ₀ ) is the motion vector of the upper left control point, and ( mv ^h ₁ , mv ^v ₁ ) is the motion vector of the upper right control point, and ( mv ^h ₂ , mv ^v ₂ ) is the motion vector of the lower left control point. The motion vector of the corner control point. Point (x, y) represents the coordinates of a representative point relative to the upper left sample within the current block. CP motion vectors can be signaled (as in affine AMVP mode) or derived on-the-fly (as in affine merge mode). w and h are the width and height of the current block. In practice, division is implemented by right shift with rounding. In VTM, the representative point is defined as the center position of the sub-block. For example, when the coordinates of the upper-left corner of the sub-block relative to the upper-left sample within the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2).

In a division-free design, (1) and (2) are implemented as:

(3)

For the 4-parameter affine model shown in (1):

(4)

For the 6-parameter affine model shown in (2):

(5)

Finally,

(6)

(7)

Where S represents the calculation accuracy, for example, in VVC, S=7. In VVC, the MV used in the MC of the sub-block at (xs, ys) for the upper left sample is calculated by (6), where x=xs+2 and y=ys+2.

To derive the motion vector of each 4 × 4 sub-block, the motion vector of the center sample of each sub-block is calculated according to equation (1) or (2), as shown in Figure 3, and rounded to 1/16 fractional accuracy. Then, a motion compensated interpolation filter is applied to generate predictions for each sub-block with the derived motion vectors.

The affine model may be inherited from spatially adjacent affine coding blocks, such as the left, upper, upper right, lower left and upper left neighbor blocks, as shown in Figure 4A. For example, if the adjacent left block A in Figure 4A is encoded in affine mode, as designated by A0 in Figure 4B, then the control points of the upper left corner, upper right corner and lower left corner of the adjacent CU/PU containing block A are obtained ( CP) motion vectors mv ₀ ^N , mv ₁ ^N and mv ₂ ^N . And calculate the motion vector mv ₀ ^C , mv ₁ ^C and mv ₂ ^C of the upper left corner/upper right/lower left corner on the current CU/PU based on mv ₀ ^N , mv ₁ ^N and mv ₂ ^N (which is only used for the 6-parameter affine model ). It should be noted that in VTM-2.0, if the current block is affine coded, then the sub-block (such as a 4×4 block in VTM) LT stores mv0 and RT stores mv1. If the current block is encoded with a 6-parameter affine model, then LB stores mv2; otherwise (in the case of a 4-parameter affine model), then LB stores mv2'. Other sub-blocks store MVs for MC.

It should be noted that when a CU is encoded with affine merge mode, i.e. in AF_MERGE mode, it obtains the first block encoded with affine mode from the valid neighboring reconstructed blocks. And the selection order of candidate blocks is from left, top, top right, bottom left to top left, as shown in Figure 4A.

The derived CP MVs mv ₀ ^C , mv ₁ ^C and mv ₂ ^C of the current block can be used as CP MVs in the affine merge mode. Or they can be used in VVC as MVP for affine inter mode. It should be noted that for merge mode, if the current block is encoded with affine mode, after deriving the CP MV of the current block, the current block can be further divided into multiple sub-blocks, and each block will be based on the derived CP MV of the current block derive its motion information.

2.2 JVET-K0186

Unlike VTM where only one affine spatial neighbor block can be used to derive the affine motion of a block, in JVET-K0186 it is proposed to construct a separate list of affine candidates for the AF_MERGE mode. Follow the steps below.

1) Insert inherited affine candidates into the candidate list

Figure 5 shows an example of candidate locations for the affine merge pattern.

Inherited affine candidates are candidates derived from valid neighboring reconstruction blocks encoded with affine patterns.

As shown in Figure 5, the scanning order of candidate blocks is A ₁ , B ₁ , B ₀ , A ₀ and B ₂ . When selecting a block (e.g., A1), a two-step process is applied:

a) First, use the three corner motion vectors of the CU covering the block to derive the two/three control points of the current block.

b) Based on the control points of the current block, to derive the sub-block motion of each sub-block within the current block.

2) Insert constructed affine candidates

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand , the constructed affine candidate is inserted into the candidate list.

The constructed affine candidate refers to the candidate constructed by combining the neighboring motion information of each control point.

First, the motion information of the control point is derived from the specified spatial neighborhood and temporal neighborhood, as shown in Figure 5. CPk (k=1, 2, 3, 4) represents the kth control point. A ₀ , A ₁ , A ₂ , B ₀ , B ₁ , B ₂ and B ₃ are the predicted spatial domain positions of CPk (k=1, 2, 3); T is the predicted time domain position of CP4.

The coordinates of CP1, CP2, CP3 and CP4 are (0,0), (W,0), (H,0) and (W,H) respectively, where W and H are the width and height of the current block.

Motion information for each control point is obtained according to the following priority order:

- For CP1, the check priority is B ₂ ->B ₃ ->A ₂ . B ₂ is used if B ₂ is available. Otherwise, if _B2 is not available, _B3 is used. If neither B ₂ nor B ₃ is available, then A ₂ is used. If all three candidates are not available, motion information for CP1 cannot be obtained.

- For CP2, the check priority is B1->B0;

- For CP3, the check priority is A1->A0;

- For CP4, use T.

Secondly, the combination of control points is used to construct the motion model.

The motion vectors of three control points are needed to calculate the transformation parameters in the 6-parameter affine model. Three control points can be selected from one of the following four combinations: ({CP1, CP2, CP4}, {CP1, CP2, CP3}, {CP2, CP3, CP4}, {CP1, CP3, CP4}). For example, use CP1, CP2 and CP3 control points to construct a 6-parameter affine motion model, referred to as affine (CP1, CP2, CP3).

The motion vectors of the two control points are needed to calculate the transformation parameters in the 4-parameter affine model. Two control points can be selected from one of the following six combinations ({CP1, CP4}, {CP2, CP3}, {CP1, CP2}, {CP2, CP4}, {CP1, CP3}, {CP3, CP4}) . For example, use CP1 and CP2 control points to construct a 4-parameter affine motion model, referred to as affine (CP1, CP2).

The constructed combinations of affine candidates are inserted into the candidate list in the following order:

{CP1, CP2, CP3}, {CP1, CP2, CP4}, {CP1, CP3, CP4}, {CP2, CP3, CP4}, {CP1, CP2}, {CP1, CP3}, {CP2, CP3}, {CP1, CP4}, {CP2, CP4}, {CP3, CP4}.

3) Insert zero motion vector

If the number of candidates in the affine merge candidate list is less than MaxNumAffineCand, zero motion vectors are inserted into the candidate list until the list is full.

2.3 ATMVP (Advanced Temporal Motion Vector Prediction)

In the 10th JVET meeting, Advanced Temporal Motion Vector Prediction (ATMVP) was included in the Benchmark Set (BMS)-1.0 reference software, which derives a Multiple motions for sub-blocks of a coding unit (CU). Although it improves the efficiency of temporal motion vector prediction, the following complexity issues are identified for existing ATMVP designs:

If multiple reference pictures are used, the co-located pictures of different ATMVP CUs may be different. This means that multiple reference images of the playing field need to be retrieved.

The motion information of each ATMVP CU is always derived based on 4×4 units, resulting in multiple calls of motion derivation and motion compensation for each 4×4 sub-block within an ATMVP CU.

Some further simplifications of ATMVP were proposed and have been adopted in VTM2.0.

2.3.1 Simplified co-located block derivation using a fixed co-located image

In this approach, a simplified design is proposed to use the same co-located picture as in HEVC, which is signaled at the slice header, as the co-located picture for ATMVP derivation. At the block level, if the reference picture of a neighboring block is different from the co-located picture, the HEVC temporal MV scaling method is used to scale the MV of the block, and the scaled MV is used in ATMVP.

Refer to the motion vector used to retrieve the motion field in the co-located picture R _col as MV _col . To minimize the impact due to MV scaling, the MVs in the spatial candidate list used to derive MV _col are selected in the following way: if the reference picture of the candidate MV is a co-located picture, then this MV is selected and used as MV _col , without using any scaling. Otherwise, the MV with the reference picture closest to the co-located picture is selected to derive MV _col using scaling.

2.3.2 Adaptive ATMVP sub-block size

In this approach, we propose to support slice-level patching of sub-block sizes for ATMVP motion derivation. Specifically, a preset sub-block size for ATMVP motion derivation is signaled at the sequence level. Additionally, a flag is signaled at the slice level to indicate whether the preset sub-block size is used for the current slice. If this flag is false, the corresponding ATMVP sub-block size is further signaled in the slice header of the slice.

2.4 STMVP (Space-time Motion Vector Prediction)

STMVP was proposed and adopted in JEM, but has not yet been adopted in VVC. In STMVP, the motion vectors of sub-CUs are deduced recursively in raster scan order. Figure 6 illustrates this concept. Let us consider an 8×8 CU containing four 4×4 sub-CUs A, B, C and D. Neighboring 4×4 blocks in the current frame are labeled a, b, c and d.

The derivation of the motion of sub-CU A starts with the identification of its two spatial neighbors. The first neighborhood is the N×N block above sub-CU A (block c). If block c is not available or is intra-coded, other N×N blocks above sub-CU A are checked (from left to right, starting with block c). The second neighborhood is the block to the left of sub-CU A (block b). If block b is not available or is intra-coded, check other blocks to the left of sub-CU A (from top to bottom, starting with block b). The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of the given list. Next, the temporal motion vector predictor (TMVP) of sub-block A is derived by following the same process as the TMVP derivation specified in HEVC. Retrieve the motion information of the co-located block at position D and scale accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is designated as the motion vector of the current sub-CU.

Figure 6 shows an example of one CU with four sub-blocks (A-D) and its neighboring blocks (a-d).

2.5 Example affine inheritance method

To reduce the storage requirements of affine model inheritance, some embodiments may implement a scheme in which the affine model is inherited by accessing only the MVs of one column to the left of the current block and the MVs of one column above the current block.

Affine model inheritance can be performed by deriving the CP MV of the current block from the lower left MV and lower right MV of the affine-coded neighboring blocks, as shown in Figure 7.

a. In an example, mv ₀ ^C = ( mv ₀ ^Ch , mv ₀ ^Cv ) and mv ₁ ^C = ( mv ₁ ^Ch , mv ₁ ^Cv ) are given by mv ₀ ^N = ( mv ₀ ^Nh , mv ₀ ^Nv ) and mv ₁ ^N = ( mv ₁ ^Nh , mv ₁ ^Nv ) is derived as follows:

(8)

where w and w' are the width of the current block and the width of the neighboring block respectively. (x ₀ , y ₀ ) are the coordinates of the upper left corner of the current block, and (x' ₀ , y' ₀ ) are the coordinates of the lower left corner of the adjacent block.

Alternatively, furthermore, the division operations during the calculation of a and b may be replaced by shifts with or without addition operations.

b. For example, affine model inheritance is performed by deriving the CP MV of the current block from the lower left MV and lower right MV of the affine-encoded neighboring blocks from “B” and “C” in Figure 4A.

i. Alternatively, affine model inheritance is performed by deriving the CP MV of the current block from the lower left MV and lower right MV of the affine-encoded neighboring blocks from “B”, “C” and “E” in Figure 4A.

c. For example, only if the neighboring block is in the M×N area above (or upper right, or upper left) the M×N area containing the current block, it is derived by the lower left MV and lower right MV of the neighboring block encoded by the affine The CP MV of the current block is used for affine model inheritance.

d. For example, y ₀ = y' ₀ .

i. Alternatively, y ₀ = 1+y ₀ '.

e. For example, if the current block inherits the affine model by deriving the CP MV of the current block from the lower left MV and lower right MV of the affine-coded neighboring blocks, then the current block is considered to use the 4-parameter affine model.

Figure 7 shows an example of affine inheritance by derivation from two base CPs of neighboring blocks.

Affine model inheritance can be performed by deriving the CP MV of the current block from the upper right MV and lower right MV of the affine-coded neighboring blocks, as shown in Figure 8.

a. For example, mv ₀ ^C = ( mv ₀ ^Ch , mv ₀ ^Cv ) and mv ₁ ^C = ( mv ₁ ^Ch , mv ₁ ^Cv ) can be given by mv ₀ ^N = ( mv ₀ ^Nh , mv ₀ ^Nv ) and mv ₁ ^N =( mv ₁ ^Nh , mv ₁ ^Nv ) is derived as follows:

(9)

where h' is the height of the neighboring block. w is the width of the current block. (x ₀ , y ₀ ) are the coordinates of the upper left corner of the current block, and (x' ₀ , y' ₀ ) are the coordinates of the upper right corner of the adjacent block.

Alternatively, furthermore, the division operations during the calculation of a and b may be replaced by shifts with or without addition operations.

b. For example, affine model inheritance is performed by deriving the CP MV of the current block from the upper right MV and lower right MV of the neighboring blocks from the affine encoding of "A" and "D" in Figure 4A.

i. Alternatively, affine model inheritance is performed by deriving the CP MV of the current block from the upper right MV and lower right MV of the affine-encoded neighboring blocks from “A”, “D” and “E” in Figure 4A.

c. For example, only if the neighboring block is in the M×N area to the left (or upper left, or lower left) of the M×N area containing the current block, it is derived by the upper right MV and lower right MV of the neighboring block encoded by the affine The CP MV of the current block performs affine model inheritance.

d. For example, x ₀ = x' ₀ .

i. Alternatively, x ₀ = 1+ x' ₀ .

e. For example, if the current block inherits the affine model by deriving the CP MV of the current block from the upper right MV and lower right MV of the affine-encoded neighboring blocks, then the current block is considered to use the 4-parameter affine model.

If the current block does not use a 6-parameter affine model, more CP MVs can be derived and stored for use in motion vector prediction and/or filtering processes.

a. In one example, the stored lower left MV can be used for motion prediction, including affine model inheritance for later encoded PU/CUs.

b. In one example, the stored lower left MV can be used in motion prediction of subsequent pictures.

c. In one example, the stored lower left MV can be used in the deblocking filtering process.

d. If the affine-coded block does not use a 6-parameter affine model, the CP MV of the lower left corner is derived for the affine-coded block and stored in the lower left MV unit, which is 4×4 in VVC.

i. The CP MV in the lower left corner (denoted as mv ₂ = ( mv ₂ ^h , mv ₂ ^v )) is derived as follows for the 4-parameter affine model

(10)

e. For the affine coding block, the CP MV in the lower right corner is derived and stored in the lower right MV unit, which is 4×4 in VVC. The stored lower right MV can be used for motion prediction, including affine model inheritance for subsequent encoded PU/CUs, or motion prediction or deblocking filtering processes for subsequent pictures.

i. The CP MV in the lower right corner (denoted as mv ₃ = ( mv ₃ ^h , mv ₃ ^v )) is derived as follows for the 4-parameter affine model:

(11)

ii. The CP MV in the lower right corner is derived as follows for the 6-parameter affine model:

(12)

iii. The CP MV in the lower right corner is derived as follows for both the 4-parameter affine model and the 6-parameter affine model:

a. (13)

b. If mv ₂ = ( mv ₂ ^h , mv ₂ ^v ) in the 4-parameter model is calculated as (10). Figure 8 shows an example of affine inheritance by derivation from two right-hand CPs of neighboring blocks.

3. Problems solved by the embodiments

Some previous designs could only support inheritance of 4-parameter affine models, which could result in coding performance loss when enabling 6-parameter affine models.

4. Exemplary embodiments

We propose several methods to inherit 6-parameter affine models while reducing storage requirements.

The following detailed description of the invention should be considered as examples to explain the general concepts. These inventions should not be interpreted narrowly. Furthermore, these inventions can be combined in any way. Combinations between this invention and other inventions are also suitable.

In the following discussion, it is assumed that the coordinates of the upper left corner/upper right corner/lower left corner/lower right corner of the adjacent CU above or left of the affine encoding are (LTNx, LTNy)/(RTNx, RTNy)/(LBNx, LBNy)/ respectively. (RBNx, RBNy); The coordinates of the upper left corner/upper right corner/lower left corner/lower right corner of the current CU are (LTCx, LTCy)/(RTCx, RTCy)/(LBCx, LBCy)/(RBCx, RBCy); affine The width and height of the adjacent CU above or to the left of the encoding are w' and h' respectively; the width and height of the current CU of the affine encoding are w and h respectively.

1. Affine model inheritance by using MVs in columns directly above the current block (e.g., MVs associated with blocks adjacent to the current block) in different ways depending on the above neighbor of the affine encoding Whether the block adopts a 4-parameter affine model or a 6-parameter affine model.

a. In one example, the inheritance method is applied when the affine-encoded upper neighbor CU adopts a 4-parameter affine model.

b. In one example, if the affine-coded upper neighbor block adopts a 6-parameter affine model, then the lower left MV and lower right MV of the affine-coded upper neighbor block as shown in Figure 9 and an auxiliary The MVs (i.e., mv ₀ ^N , mv ₁ ^N and the auxiliary MV associated with the auxiliary points) derive the CPMV of the current block for 6-parameter affine model inheritance.

i. In one example, the proposed 6-parameter affine model inheritance is performed only if the upper neighboring block of the affine encoding adopts a 6-parameter affine model and w’ > Th0 (for example, Th0 is 8).

ii. Alternatively, when the upper block is encoded with an affine mode, whether it is a 4- or 6-parameter affine mode, the proposed 6-parameter affine model inheritance can be called.

c. In one example, the auxiliary MV is derived via an affine model of the affine-encoded upper neighbor block using the auxiliary position.

i. The auxiliary position is predetermined;

ii. Alternatively, the auxiliary position is adaptive. For example, the auxiliary position depends on the size of the neighboring block above.

iii. Instead, the secondary location is signaled from the encoder to the decoder in VPS/SPS/PPS/stripe header/CTU/CU.

iv. Alternatively, the auxiliary positions are (LTNx + (w'>>1), LTNy + h' + Offset). Offset is an integer. For example, Offset= ^2K . In another example, Offset=-2 ^K . In some examples, K may be 1, 2, 3, 4, or 5. Specifically, the auxiliary positions are (LTNx + (w'>>1), LTNy + h' + 8).

v. The auxiliary MV is stored in one of the bottom columns of the basic unit block (such as the 4x4 block in VVC) of the upper neighboring block of affine coding, as shown in Figure 10. The basic unit block in which the auxiliary MV is to be stored is named the auxiliary block.

(a) In one example, the auxiliary MV cannot be stored in the lower left and lower right basic unit blocks of the affine-encoded upper neighbor block.

(b) The bottom column of the basic unit block refers to B(0), B(1), ..., B(M-1) from left to right. In one example, the auxiliary MV is stored in basic unit block B (M/2).

a. Instead, the auxiliary MV is stored in the basic unit block B (M/2+1);

b. Instead, the auxiliary MV is stored in the basic unit block B (M/2-1);

(c) In one example, the stored auxiliary MV can be used for motion prediction or merge of subsequently encoded PU/CU.

(d) In one example, the stored auxiliary MV can be used for motion prediction or merging of subsequent pictures.

(e) In one example, the stored auxiliary MV can be used in a filtering process (e.g., deblocking filtering).

(f) Alternatively, additional buffers can be used to store auxiliary MVs instead of storing them in basic unit blocks. In this case, the stored auxiliary MV may be used only for affine motion inheritance, not for encoding the current slice/slice or later encoded blocks in different pictures, and not for filtering processes (eg, deblocking filtering).

vi. After decoding the affine-encoded upper neighboring block, the coordinates of the auxiliary position are taken as input (x, y), and the auxiliary MV is calculated by Eq. (2). And then the auxiliary MV is stored in the auxiliary block.

(a) In one example, the MV stored in the auxiliary block is not used to perform MC for the auxiliary block.

d. In an example shown in Figure 9, the three CPMVs of the current block, referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) and mv ₂ ^C = (mv ₂ ^Ch , mv ₂ ^Cv ), mv ₀ ^N = (mv ₀ ^Nh , mv ₀ ^Nv ), as affine coded from the MV at the lower left corner of the upper neighboring block as affine code The mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) of the MV at the lower right corner of the adjacent block above and the mv _A = (mv _A ^h , mv _A ^v ) as the auxiliary MV are derived as follows:

(14)

where (x0, y0) are the coordinates of the upper left corner of the current block, and (x’0, y’0) are the coordinates of the lower left corner of the adjacent block.

Alternatively, furthermore, the division operation in (14) can be replaced by a right shift, with or without the offset i added before the shift. For example, the number K in equation (14) depends on how the auxiliary positions are defined to get Auxiliary MV. In the example disclosed in 1.c.iv, the auxiliary positions are (LTNx+(w'>>1), LTNy+h'+Offset), where Offset=2 ^K. In the example, K = 3.

e. For example, the current block is deduced from the lower left MV and lower right MV of the affine-coded neighboring block only if the neighboring block is in the M×N area above (or upper right, or upper left) the M×N area containing the current block. CP MV of block for affine model inheritance.

i. For example, M×N area is CTU, such as 128×128 area;

ii. For example, M×N area is the pipeline size, such as 64×64 area.

f. For example, y ₀ = y' ₀ .

i. Alternatively, y ₀ = 1+y' ₀ .

Figure 9 shows an example of 6-parameter affine inheritance derived from MVs stored in the bottom column of the upper neighbor block of the affine encoding.

Figure 10 shows an example of the bottom column (shaded) of basic unit blocks in which auxiliary MVs can be stored.

2. Perform affine model inheritance in different ways by using the MV in the column to the left of the current block, depending on whether the affine-encoded left neighboring CU adopts a 4-parameter affine model or a 6-parameter affine model.

a. In one example, when the affine-coded left neighboring CU adopts a 4-parameter affine model, the previously disclosed method is applied.

b. In an example, if the affine-coded left neighboring block adopts a 6-parameter affine model, then the upper right MV and lower right MV of the affine encoded left neighboring block shown in Figure 11 and an auxiliary MV can be obtained Derive the CPMV of the current block and perform 6-parameter affine model inheritance.

i. In one example, 6-parameter affine model inheritance is possible only if the affine-coded left neighboring block adopts a 6-parameter affine model and h’ > Th1 (for example, Th1 is 8).

ii. Alternatively, when the left block is encoded with affine mode, regardless of whether it is 4 or 6-parameter affine mode, the proposed 6-parameter affine model inheritance can be called.

c. In one example, the auxiliary MV is derived through an affine model of the affine-encoded left neighboring block using the auxiliary position.

i. In one example, the auxiliary position is predetermined;

ii. In one example, the auxiliary position is adaptive. For example, the auxiliary position depends on the size of the neighboring block to the left.

iii. In one example, the secondary location is signaled from the encoder to the decoder in VPS/SPS/PPS/stripe header/CTU/CU.

iv. In one example, the auxiliary positions are (LTNx+w'+Offset), LTNy+(h'>>1)). Offset is an integer. For example, Offset= ^2K . In another example, Offset = -2 ^K . In some examples, K may be 1, 2, 3, 4, or 5. In particular, the auxiliary positions are (LTNx+w'+8, LTNy+(h'>>1)).

v. The auxiliary MV is stored in one of the right rows of the basic unit block (such as the 4x4 block in VVC) of the left neighboring block of affine coding, as shown in Figure 12. The basic unit block to store the auxiliary MV is named auxiliary block.

(a) In one example, the auxiliary MV cannot be stored in the upper-right and lower-right basic unit blocks of the left neighboring block of affine encoding.

(b) The right row of the basic unit block is referred to as B(0), B(1), ..., B(M-1) from top to bottom. In one example, the auxiliary MV is stored in basic unit block B (M/2).

a. Instead, the auxiliary MV is stored in the basic unit block B (M/2+1);

b. Instead, the auxiliary MV is stored in the basic unit block B (M/2-1);

(c) In one example, the stored auxiliary MV can be used for motion prediction or merge of subsequently encoded PU/CU.

(d) In one example, the stored auxiliary MV can be used for motion prediction or merge of subsequent pictures.

(e) In one example, the stored auxiliary MV can be used in the filtering process (e.g., deblocking filtering).

(f) Alternatively, additional buffers can be used to store auxiliary MVs instead of storing them in basic unit blocks. In this case, the stored auxiliary MV may be used only for affine motion inheritance, not for encoding the current slice/slice or later encoded blocks in different pictures, and not for filtering processes (eg, deblocking filtering).

vi. After decoding the affine-encoded left neighboring block, the coordinates of the auxiliary position are taken as input (x, y), and the auxiliary MV is calculated by Equation (2). And then the auxiliary MV is stored in the auxiliary block.

(a) In one example, the MV stored in the auxiliary block is not used to perform MC for the auxiliary block.

d. In an example as shown in Figure 11, the three CPMVs of the current block are referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) and mv ₂ ^C = (mv ₂ ^Ch , mv ₂ ^Cv ), mv ₀ ^N = (mv ₀ ^Nh , mv ₀ ^Nv ), as affine coded from the MV at the upper right corner of the left neighboring block mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) of the MV at the lower right corner of the left neighboring block and mv _A = (mv _A ^h , mv _A ^v ) as the auxiliary MV, are derived as follows:

(15)

Where (x ₀ , y ₀ ) is the coordinate of the upper left corner of the current block, (x' ₀ , y' ₀ ) is the coordinate of the upper right corner of the adjacent block.

Alternatively, furthermore, the division operation in (15) can be replaced by a shift with or without addition operation.

i. For example, the number K in equation (15) depends on how the auxiliary position is defined to obtain the auxiliary MV. In the example disclosed in 2.c.iv, the auxiliary positions are (LTNx+w'+Offset), LTNy+(h'>>1)) where Offset=2 ^K. In the example, K = 3.

e. For example, only if the neighboring block is in the M×N area to the left (or upper left, or lower left) of the M×N area containing the current block, the current block is deduced from the upper right MV and lower right MV of the affine-coded neighboring block. CP MV of block for affine model inheritance.

i. For example, M×N area is CTU, such as 128×128 area;

ii. For example, M×N area is the pipeline size, such as 64×64 area.

f. For example, x ₀ = x' ₀ .

i. Alternatively, x ₀ = 1+x' ₀ .

3. In one example, the current block only needs to access the MV stored in the basic unit (such as the 4×4 block in VVC) in the column above the current block and in the row to the left of the current block, as shown in Figure 13 shown. The coordinates of the upper left position of the current block are referred to as (x0, y0). The current block has access to a column of basic cells above the current block that starts with the basic cell with upper-left coordinates (xRS, yRS) and ends with the basic cell with upper-left coordinates (xRE, yRE) (referred to as the column above the desired one) MV in. Similarly, the current block can access the row of basic cells to the left of the current block that starts with the basic cell with upper-left coordinates (xCS, yCS) and ends with the basic cell with upper-left coordinates (xCE, yCE) (referred to as the desired left row) in the MV. The width and height of the current block are referred to as W and H respectively. Assuming that the basic unit size is B×B (for example, 4×4 in VVC), then yRS=yRE=y0-B and xCS=xCE=x0-B.

a. The range of the required upper columns and the required left rows can be limited.

i. In one example, xRS = x0-n×W-m. n and m are integers, such as n=0, m=0, n=0, m=1, n=1, m=1, or n=2, m=1;

ii. In one example, xRE = x0+ n×W+m. n and m are integers, such as n = 1, m = -B, n = 1, m = B, n = 2, m = -B, or n = 3, m = -B;

iii. In one example, yCS = y0-n×H-m. n and m are integers, such as n=0, m=0, n=0, m=1, n=1, m=1, or n=2, m=1;

iv. In an example, yCE = y0+ n×H+m. n and m are integers, such as n = 1, m = -B, n = 1, m = B, n = 2, m = -B, or n = 3, m = -B;

v. In one example, the current block does not require the required upper column;

vi. In one example, the current block does not require the required left line;

vii. In one example, the selection of xRS, xRE, yCS and yCE can depend on the location of the auxiliary block.

(a) In one example, the auxiliary block is always covered by the selected upper column or left row.

(b) Alternatively, in addition, (xRS, yRS), (xRE, yRE), (xCS, yCS) and (xCE, yCE) should not overlap with auxiliary blocks.

viii. In one example, the range of required upper columns and required left rows depends on the current block position.

(a) If the current block is at the top boundary of the P×Q area, that is, when y0%Q==0, where Q can be 128 (CTU area) or 64 (pipeline area).

a. In one example, the current block does not require the required upper columns;

b. In an example, xRS = x0;

(b) If the current block is at the left boundary of the P×Q area, that is, when x0%P==0, P can be 128 (CTU area) or 64 (pipeline area).

a. In one example, the current block does not require the required left row;

b. In an example, yCS = y0;

4. As an alternative to example 2.d, the three CPMVs of the current block, referred to as mv ₀ ^C = (mv ₀ ^Ch , mv ₀ ^Cv ), mv ₁ ^C = (mv ₁ ^Ch , mv ₁ ^Cv ) and mv ₂ ^C = (mv ₂ ^Ch _, mv ₂ ^Cv ₎ , ^mv ₀ of the MV ^at the upper right corner of the block as affine- ^coded left neighbor The mv ₁ ^N = (mv ₁ ^Nh , mv ₁ ^Nv ) of the MV at the lower right corner of the block and the mv _A = (mv _A h, mv _A v) as the auxiliary MV are derived as follows:

(16)

where (x0, y0) is the coordinate of the upper left corner of the current block, and (x’0, y’0) is the coordinate of the upper right corner of the adjacent block.

Alternatively, furthermore, the division operation in (16) can be replaced by a shift with or without addition operation.

5. In Example 1.c.v, the auxiliary MV is stored in the bottom column of basic unit blocks of the affine-coded upper neighbor block (such as VVC) only if the affine-coded upper neighbor block is encoded with a 6-parameter affine model. in one of the 4x4 blocks (such as the middle one).

a. Instead, the auxiliary MV is stored in one (such as the middle one) of the bottom column of basic unit blocks (e.g., 4x4 blocks in VVC) of the affine-coded upper neighbor block, regardless of whether the affine-coded upper neighbor block is Whether encoded using a 4-parameter affine model or a 6-parameter affine model.

6. In Example 2.c.v, the auxiliary MV is stored in the right row basic unit block of the affine-coded left neighboring block (such as VVC) only if the affine-coded left neighboring block is encoded with a 6-parameter affine model. in one of the 4x4 blocks (such as the middle one).

a. Instead, store the auxiliary MV in one of the right row basic unit blocks (e.g. 4x4 blocks in VVC) (such as the middle one) of the affine-coded left neighboring block, regardless of the affine-coded left neighboring block Whether it is encoded using a 4-parameter affine model or a 6-parameter affine model.

7. In an example, the lower left basic unit block of the current block always stores the CPMV at the lower left corner, regardless of whether the current block adopts a 4-parameter model or a 6-parameter model. For example, in Figure 3, block LB always stores mv2.

8. In an example, the lower right basic unit block of the current block always stores the CPMV at the lower right corner, regardless of whether the current block adopts a 4-parameter model or a 6-parameter model. For example, in Figure 3, block RB always stores mv3.

9. Whether the affine-coded neighboring blocks adopt a 4-parameter model or a 6-parameter model, affine inheritance is performed in the same way.

a. Using the 6-parameter affine model inheritance method, derive the CPMV of the current block at the upper left corner, upper right corner and lower left corner from the MV stored in the upper left basic block, upper right basic block and lower left basic unit of the affine coded neighboring block. (e.g. mv ₀ ^C , mv ₁ ^C and mv ₂ ^C in Figure 4(b)).

i. For example, the MVs stored in the upper left basic block, upper right basic block and lower left basic unit of the affine coded neighboring block are the CPMVs at the upper left corner, upper right corner and lower right corner of the affine coded neighboring block.

b. Derive the upper left and upper right corners of the current block from the MVs stored in the lower left basic unit, lower right basic unit and extra basic unit of the bottom column of the basic unit of the upper adjacent block of affine encoding in the manner defined in Example 1 and CPMV at the lower left corner (e.g. mv ₀ ^C , mv ₁ ^C and mv ₂ ^C in Figure 4(b)).

i. For example, the MVs stored in the lower left basic unit, lower right basic unit, and extra basic unit in the bottom column of the basic unit of the affine-coded upper neighboring block are the CPMV at the lower left corner of the affine-coded neighboring block, CPMV and auxiliary MV in the lower right corner.

c. Derive the upper left corner, upper right corner of the current block from the MVs stored in the upper right basic unit, lower right basic unit and extra basic unit in the right row of the basic unit of the left adjacent block of affine encoding in the manner defined in Example 2 CPMV at the corner and lower left corner (e.g. mv ₀ ^C , mv ₁ ^C and mv ₂ ^C in Figure 4(b)).

i. For example, the MVs stored in the upper right basic unit, the lower right basic unit, and the extra basic unit in the right row of the basic unit of the affine-coded left neighboring block are the CPMV at the upper right corner of the affine-coded neighboring block, CPMV and auxiliary MV in the lower right corner.

d. Inherited affine merge blocks are always marked "use 6 parameters".

e. When the affine model is inherited from the upper neighboring block, and the width of the upper neighboring block is not greater than 8, the auxiliary MV is calculated as the average of the MV stored in the lower left basic unit and the lower right basic unit.

i. For example, the MVs stored in the lower left basic unit and the lower right basic unit of the affine-coded upper neighbor block are the CPMV at the lower left corner and the CPMV at the lower right corner of the affine-coded upper neighbor block.

f. When the affine model is inherited from the left neighboring block, and the height of the left neighboring block is not greater than 8, the auxiliary MV is calculated as the average of the MV stored in the upper right basic unit and the lower right basic unit.

i. For example, the MVs stored in the upper right basic unit and the lower right basic unit of the left adjacent block of affine coding are the CPMV at the upper right corner and the CPMV at the lower right corner of the left adjacent block of affine coding.

10. In one example, MC for a sub-block is performed by the MV stored in the sub-block.

a. For example, the stored MV is CPMV;

b. For example, the stored MV is an auxiliary MV.

11. Regardless of whether the affine-coded neighboring blocks adopt a 4-parameter model or a 6-parameter model, affine inheritance is performed in the same way.

12. Each coding block can store more than 1 auxiliary MV.

a. The auxiliary MV can be used in the same way as above.

Figure 11 shows an example of 6-parameter affine inheritance derived from MVs stored in the right row of affine-encoded left neighbor blocks.

Figure 12 shows an example of the right row (shaded) of basic unit blocks in which auxiliary MVs can be stored.

Figure 13 shows an example of an MV store.

5. Other embodiments

This section discloses examples of embodiments of the proposed invention. It should be noted that this is only one of all possible embodiments of the proposed method and should not be understood in a narrow way.

Normalization (a, b) is as defined in equation (7).

Input:

The coordinates from the current block to the upper left corner are recorded as (posCurX, posCurY);

The coordinates of the upper left corner of the adjacent block are recorded as (posLTX, posLTY);

The coordinates of the upper right corner of the adjacent block are recorded as (posRTX, posRTY);

The coordinates of the lower left corner of the adjacent block are recorded as (posLBX, posLBY);

The coordinates of the lower right corner of the adjacent block are recorded as (posRBX, posRBY);

The width and height of the current block are recorded as W and H;

The width and height of adjacent blocks are recorded as W’ and H’;

The MV at the upper left corner of the adjacent block is recorded as (mvLTX, mvLTY);

The MV at the upper right corner of the adjacent block is recorded as (mvRTX, mvRTY);

The MV at the lower left corner of the adjacent block is recorded as (mvLBX, mvLBY);

The MV at the lower right corner of the adjacent block is recorded as (mvRBX, mvRBY);

Constant: Shift, which can be any positive integer, such as 7 or 8.

Output:

The MV at the upper left corner of the current block is recorded as (MV0X, MV0Y);

The MV at the upper right corner of the current block is recorded as (MV1X, MV1Y);

The MV at the lower right corner of the current block is recorded as (MV2X, MV2Y);

Affine model inherited routines: If posRBY is equal to (posCurY-1) { // Above neighborhood block iDMvHorX = (mvRBX – mvLBX) >> (shift – log2(W’)) iDMvHorY = (mvRBY – mvLBY) >> (shift – log2(W’)) iDMvVerX = -iDMvHorY; iDMvVerY = iDMvHorX; if the neighborhood block applies the 6-parameter affine model and W’ > 8{ AuxiliaryX = posLBx + (W’ >> 1); AuxiliaryY = posLBy; (mvAuxiliaryX, mvAuxiliaryY) is set to be the MV stored in the basic unit block containing (AuxiliaryX, AuxiliaryY); mvMidX = Normalize( mvRBX + mvLBX, 1); mvMidY = Normalize( mvRBY + mvLBY, 1); iDMvVerX = (mvAuxiliaryX - mvMidX) >> (shift - 3); iDMvVerY = (mvAuxiliaryY - mvMidY) >> (shift - 3); } iMvScaleHor = mvLBX >> shift; iMvScaleVer = mvLBY >> shift; posNeiX = posLBX; posNeiY = posLBY + 1; } else if posRBX is equal to (posCurX -1) {//Left neighborhood block iDMvHorX = (mvRBY– mvRTY) >> (shift – log2(H’)) iDMvHorY = -(mvRBX – mvRTX) >> (shift – log2(H’)) iDMvVerX = -iDMvHorY; iDMvVerY = iDMvHorX; if the neighborhood block applies the 6-parameter affine model and H’ > 8 { AuxiliaryX = posRTX; AuxiliaryY = posRTY + (H’>>1); (mvAuxiliaryX, mvAuxiliaryY) is set to be the MV stored in the basic unit block containing (AuxiliaryX, AuxiliaryY); mvMidX = Normalize( mvRBX + mvRTX, 1); mvMidY = Normalize( mvRBY + mvRTY, 1); iDMvHorX = (mvAuxiliaryX - mvMidX) >> (shift - 3); iDMvHorY = (mvAuxiliaryY - mvMidY) >> (shift - 3); } iMvScaleHor = mvRTX >> shift; iMvScaleVer = mvRTY >> shift; posNeiX = posRTX+1; posNeiY = posRTY; } else{ Return "Input is invalid!" } } horTmp0 = iMvScaleHor + iDMvHorX * (posCurX - posNeiX) + iDMvVerX * (posCurY - posNeiY); verTmp0 = iMvScaleVer + iDMvHorY * (posCurX - posNeiX) + iDMvVerY * (posCurY - posNeiY); MV0X = Normalize (horTmp0, shift); MV0Y = Normalize ( verTmp0, shift ); horTmp1 = iMvScaleHor + iDMvHorX * (posCurX + W - posNeiX) + iDMvVerX * (posCurY - posNeiY); verTmp1 = iMvScaleVer + iDMvHorY * (posCurX + W - posNeiX) + iDMvVerY * (posCurY - posNeiY); MV1X = Normalize (horTmp1, shift); MV1Y = Normalize ( verTmp1, shift ); if the neighborhood block applies the 6-parameter affine model{ The current block also applies the 6-parameter affine model; horTmp2 = iMvScaleHor + iDMvHorX * (posCurX - posNeiX) + iDMvVerX * (posCurY+H - posNeiY); verTmp2 = iMvScaleVer + iDMvHorY * (posCurX - posNeiX) + iDMvVerY * (posCurY+H- posNeiY); MV2X = Normalize (horTmp2, shift); MV2Y = Normalize ( verTmp2, shift ); }

14 is a block diagram illustrating an example of the architecture of a computer system or other control device 2600 that may be used to implement portions of the disclosed technology. In Figure 14, computer system 2600 includes one or more processors 2605 and memory 2610 connected via an interconnect 2625. Interconnect 2625 may represent any one or more individual physical busses, point-to-point connections, or both connected by appropriate bridges, adapters, or controllers. Thus, interconnect 2625 may include, for example, a system bus, a Peripheral Component Interconnect (PCI) bus, a HyperTransport or Industry Standard Architecture (ISA) bus, a Small Computer System Interface (SCSI) bus, a Universal Serial Bus (USB ), IIC (I2C) bus, or Institute of Electrical and Electronics Engineers (IEEE) Standard 674 bus, sometimes called "Firewire."

Processor(s) 2605 may include a central processing unit (CPU) to control the overall operation of, for example, a host computer. In some embodiments, processor(s) 2605 does this by executing software or firmware stored in memory 2610 . Processor(s) 2605 may be or may include one or more programmable general or special purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic device (PLD), etc., or a combination of these devices.

Memory 2610 may be or include the main memory of the computer system. Memory 2610 represents any suitable form of random access memory (RAM), read only memory (ROM), flash memory, etc., or combinations of these devices. In use, memory 2610 may contain, among other things, a set of machine instructions that, when executed by processor 2605, cause processor 2605 to perform operations to implement embodiments of the disclosed technology.

Also connected to processor(s) 2605 via interconnect 2625 is an (optional) network adapter 2615 . Network adapter 2615 provides computer system 2600 with the ability to communicate with remote devices (eg, storage clients) and/or other storage servers, and may be, for example, an Ethernet adapter or a Fiber Channel adapter.

15 illustrates a block diagram of an example embodiment of an apparatus 2700 that may be used to implement portions of the disclosed technology. Mobile device 2700 may be a laptop, smartphone, tablet, camcorder, or other type of device capable of processing video. Mobile device 2700 includes a processor or controller 2701 for processing data, and a memory 2702 in communication with processor 2701 to store and/or buffer data. For example, processor 2701 may include a central processing unit (CPU) or a microcontroller unit (MCU). In some implementations, processor 2701 may include a field programmable gate array (FPGA). In some implementations, mobile device 2700 includes or communicates with a graphics processing unit (GPU), video processing unit (VPU), and/or wireless communication unit for various visual and/or communication data processing functions of the smartphone device. For example, memory 2702 may contain and store processor executable code that, when executed by processor 2701, configures mobile device 2700 to perform various operations, such as receiving information, commands and/or data, processing information and data, and The processed information/data is sent or provided to another device such as an actuator or external display. In order to support various functions of the mobile device 2700, the memory 2702 can store information and data, such as instructions, software, values, images, and other data processed or referenced by the processor 2701. For example, various types of random access memory (RAM) devices, read only memory (ROM) devices, flash memory devices, and other suitable storage media may be used to implement the storage function of memory 2702. In some implementations, mobile device 2700 includes an input/output (I/O) unit 2703 for connecting processor 2701 and/or memory 2702 to other modules, units, or devices. For example, I/O unit 2703 may interface with processor 2701 and memory 2702 to utilize various types of wireless interfaces compatible with typical data communications standards, e.g., between one or more computers in the cloud and a user device . In some implementations, mobile device 2700 can interface with other devices using wired connections via I/O unit 2703 . Mobile device 2700 may also interface with other external interfaces (e.g., data memory) and/or visual or audio display device 2704 to retrieve and transmit information that may be processed by a processor, stored in memory, or in an output unit of display device 2704 or Data and information displayed on external devices. For example, display device 2704 may display video frames based on MVP modifications in accordance with the disclosed techniques.

Figure 16 is a flow diagram of a method 1600 for video processing. The method 1600 includes accessing (1602) a motion vector (MV) stored in at least one of the basic unit blocks adjacent to the current block; and, based on the accessed MV, between the current block and the bits of the current block. Video processing is performed (1604) between stream representations.

Various embodiments and techniques disclosed in this document may be described in the following list of examples.

1. Video processing methods, including:

access a motion vector (MV) stored in at least one of the basic unit blocks adjacent to the current block; and

Video processing is performed between the current block and the bitstream representation of the current block based on the accessed MV.

2. The method of Example 1, wherein the basic unit blocks form a column located above the current block, the column containing a leftmost basic unit block with upper left corner coordinates (xRS, yRS) and a leftmost basic unit block with upper left corner coordinates (xRS, yRS) At least one of the rightmost basic unit blocks of (xRE, yRE).

3. The method of example 1, wherein the basic unit blocks form a row located to the left of the current block, the row containing a top basic unit block with upper left corner coordinates (xCS, yCS) and a top left corner coordinate (xCS, yCS) xCE, yCE) at least one of the bottom basic unit blocks.

4. The method as described in Example 2, wherein yRS=yRE=y0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B.

5. The method as described in Example 3, wherein xCS=xCE=x0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B.

6. Method as described in Example 4 or 5, where B=4.

7. The method as described in Example 4, where xRS = x0-n×W-m, n and m are integers, and W represents the width of the current block.

8. The method as described in Example 5, where yCS = y0-n×H-m, n and m are integers, and H represents the height of the current block.

9. A method as described in Example 7 or 8, where n = 0, m = 0; n = 0, m = 1; n = 1, m = 1; or n = 2, m = 1.

10. The method as described in Example 4, where xRE = x0+ n×W+m, n and m are integers, and W represents the width of the current block.

11. The method as described in Example 5, where yCE = y0+ n×H+m, n and m are integers, and H represents the height of the current block.

12. A method as described in Example 10 or 11, where n = 1, m = -B; n = 1, m = B; n = 2, m = - B; or n = 3, m = -B.

13. The method of example 2, wherein columns above the current block are not required to predict the current block.

14. The method of example 3, wherein rows to the left of the current block are not required to predict the current block.

15. The method of example 4, wherein the value of at least one of xRS and xRE depends on a location of an auxiliary block storing an auxiliary MV for predicting the current block.

16. The method of example 5, wherein the value of at least one of yCS and yCE depends on the location of the auxiliary block storing the auxiliary MV used to predict the current block.

17. The method of example 15, wherein the auxiliary block is covered by the column located above the current block.

18. The method of example 16, wherein the auxiliary block is covered by the row located to the left of the current block.

19. The method as described in Example 15, wherein the auxiliary block does not overlap with any of the upper left corner coordinates (xRS, yRS) and the upper left corner coordinates (xRE, yRE).

20. The method as described in Example 16, wherein the auxiliary block does not overlap with any of the upper left corner coordinates (xCS, yCS) and the upper left corner coordinates (xCE, yCE).

21. The method of example 2, wherein the range of columns located above the current block depends on the position of the current block.

22. The method of example 21, wherein if the current block is determined to be at the top boundary of the P×Q region, columns located above the current block are not required to predict the current block.

23. The method as described in Example 22, wherein if y0%Q == 0, the current block is determined to be at the top boundary of the P×Q area, and the upper left corner coordinate of the current block is expressed as ( x0, y0).

24. The method as described in Example 21, where XRS == x0, the coordinates of the upper left corner of the current block are expressed as (x0, y0).

25. The method of example 3, wherein a range of rows to the left of the current block depends on the position of the current block.

26. The method of example 25, wherein if the current block is determined to be at the left boundary of the P×Q region, rows located to the left of the current block are not required to predict the current block.

27. The method as described in Example 26, wherein if x0%P == 0, the current block is determined to be at the left boundary of the P×Q area, and the upper left corner coordinate of the current block is expressed as ( x0, y0).

28. The method as described in Example 25, where yCS == y0, and the coordinates of the upper left corner of the current block are expressed as (x0, y0).

29. The method as described in Example 26, wherein Q=128 in the coding tree unit area and Q=64 in the pipeline area.

30. The method as described in Example 27, wherein P=128 in the coding tree unit area and P=64 in the pipeline area.

31. The method as described in Example 1, wherein the current block is divided into multiple basic unit blocks, the method further includes:

Store the control point (CP) MV at the lower left corner of the current block in the lower left basic unit block of the current block; and/or

The control point (CP) MV at the lower right corner of the current block is stored in the lower right basic unit block of the current block.

32. The method of example 31, wherein the current block is encoded with a 4-parameter affine model or a 6-parameter affine model.

33. The method as described in Example 1, further including:

The sub-blocks split from the current block are motion compensated by using the MVs stored in the sub-blocks.

34. The method as described in Example 33, where

The stored MV is the control point (CP) MV of the current block or the auxiliary MV used to predict the current block.

35. The method of any one of examples 1-34, wherein the video processing includes encoding the video block to and decoding from the bitstream representation of the video block at least one of the video blocks.

36. A video processing device, including a processor, configured to implement the method of any one of Examples 1 to 35.

37. A computer program product, stored in a non-transitory computer-readable medium, the computer program product comprising program code for performing the method in any one of Examples 1 to 35.

The disclosed and other embodiments, modules, and functional operations described in this document may be implemented as digital electronic circuits, or as computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or any of them. A combination of one or more. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer-readable medium for execution by or to control data processing equipment. operation. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of substances that implements a machine-readable propagation signal, or a combination of one or more of them. The term "data processing equipment" covers all equipment, devices and machines used for processing data, including for example a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may include code that creates the execution environment for the computer program in question, e.g., code that constitutes the processor firmware, protocol stack, database management system, operating system, or one or more of them combination. A propagated signal is an artificially generated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to a suitable receiver device.

A computer program (also called a program, software, software application, script, or code) may be written in any form of programming language, including a compiled or interpreted language, and may be deployed in any form, including as a stand-alone program or suitable for use in a computing environment module, component, subroutine or other unit. Computer programs do not necessarily correspond to files in a file system. A program may be stored as part of a file that contains other programs or data (for example, one or more scripts stored in a markup language file), in a single file dedicated to the program in question, or in multiple coordinated In a file (for example, a file that stores one or more modules, subroutines, or portions of code). A computer program may be deployed to execute on one computer or on multiple computers located at a site or distributed across multiple sites and interconnected by a communications network.

The processes and logic flows described in this document may be executed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. Processes and logic flows may also be executed by, and devices may also be implemented as, dedicated logic circuits such as FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit) ) or ASIC (Application Specific Integrated Circuit)).

By way of example, processors suitable for the execution of a computer program include general and special purpose microprocessors, and any processor or processors of any type of digital computer. Typically, a processor will receive instructions and data from read-only memory or random access memory, or both. The basic components of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, the computer will also include or be operably coupled to one or more mass storage devices to receive data from or transmit data to one or more mass storage devices. A mass storage device or both, such as a magnetic disk, a magneto-optical disk, or an optical disk. However, a computer does not necessarily require such a device. Computer-readable media suitable for storage of computer program instructions and data includes all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, For example, internal hard drives or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special purpose logic circuits.

Although this patent document contains many details, these details should not be construed as limitations on the scope of any invention or that may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although the above features may be described as functioning in certain combinations and may even be originally claimed as such, in some cases one or more features from the claimed combination may be removed from the combination and as claimed A protected combination can be directed against a sub-combination or a variation of a sub-combination.

Similarly, although operations are depicted in the drawings in a specific order, this should not be understood to mean that achieving desirable results requires that such operations be performed in the specific order shown, or in sequence, or that all illustrated operations are performed. Furthermore, the separation of various system components in the embodiments described in this patent document should not be construed as requiring such separation in all embodiments.

Only a few implementations and examples are described, and other implementations, enhancements, and changes may be made based on what is described and illustrated in this patent document.

1600: Methods 1602~1604: Steps 2600: Computer system 2605: Processor(s) 2610, 2702: Memory 2615: Network adapter 2625: Interconnect 2700: Mobile device 2701: Processor/controller 2703: I/O unit 2704: display device A, B, C, D, E, LT, RT, LB, RB: sub-block a, b, c, d: adjacent block A0, A1, A2, B0, B1, B2, B3: Candidate block CP ₀ , CP ₁ , CP ₂ : Control point Cur: Current block mv ₀ , mv0', mv ₀ ^C , mv 0 ^N , mv ₁ , _mv1 ', mv ₁ ^C , mv ₁ ^N , mv ₂ , mv2', mv ₂ ^C , mv ₂ ^N , mv ₃ , mv3': motion vector T: time domain position (x ₀ ,y ₀ ), (x ₀ ',y ₀ '), (x ₀ +w,y ₀ ), (x ₀ '+w',y ₀ '), (x ₀ ',y ₀ '+h'), (x ₁ ,y ₁ ), (x ₁ ',y ₁ '), (x ₂ , y ₂ ), (xCS,yCS), (xRS,yRS), (xRE,yRE), (xCE,yCE): coordinates

Figure 1 shows an example of sub-block based prediction calculation. Figures 2A-2B show examples of simplified affine motion models (a) 4-parameter affine model; (b) 6-parameter affine model. Figure 3 shows an example of the affine motion vector field (MVF) for each sub-block. Figures 4A-4B illustrate candidates for AF_MERGE mode. Figure 5 shows exemplary candidate locations for the affine merge pattern. Figure 6 shows an example of a coding unit (CU) with four sub-blocks (A-D) and its neighboring blocks (a–d). Figure 7 shows an example of affine inheritance derived from two right-hand CPs of neighboring blocks. Figure 8 via affine inheritance derived from two right CPs of neighboring blocks. Figure 9 shows an example of 6-parameter affine inheritance derived from MVs stored in the bottom columns of affine-encoded upper neighbor blocks. Figure 10 shows an example of the bottom column (shaded) of basic unit blocks in which auxiliary MVs can be stored. Figure 11 shows an example of 6-parameter affine inheritance derived from MVs stored in the right row of affine-coded left neighbor blocks. Figure 12 shows an example of the right row (shaded) of basic unit blocks in which auxiliary MVs can be stored. Figure 13 shows an example of an MV bank used. 14 is a block diagram illustrating an example of an architecture that may be used to form a computer system or other control device that implements portions of the disclosed technology. 15 illustrates a block diagram of an exemplary embodiment of portions of a mobile device that may be used to implement the disclosed technology. Figure 16 is a flowchart of an exemplary method of visual media processing.

1600:Method

1602~1604: Steps

Claims (33)

A method for video processing, comprising: accessing a motion vector (MV) stored in at least one of the basic unit blocks adjacent to a current block; and based on the accessed MV, between the current block and the current block. Video processing is performed between bitstreams of blocks, wherein the basic unit blocks form a column above the current block, the column containing the leftmost basic unit block with upper left corner coordinates (xRS, yRS) and the upper left corner At least one of the rightmost basic unit blocks of coordinates (xRE, yRE), where the value of at least one of xRS and xRE depends on the position of the auxiliary block storing the auxiliary MV used to predict the current block, where the basic the unit blocks form a row located to the left of the current block, the row including at least one of a top basic unit block having upper left corner coordinates (xCS, yCS) and a bottom basic unit block having upper left corner coordinates (xCE, yCE), The value of at least one of yCS and yCE depends on the location of the auxiliary block storing the auxiliary MV used to predict the current block. The method described in item 1 of the patent application, wherein yRS=yRE=y0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B. The method described in item 1 of the patent application, wherein xCS=xCE=x0-B, (x0, y0) represents the coordinates of the upper left corner of the current block, and the basic unit block has a size of B×B. The method described in item 2 or 3 of the patent application scope, wherein B=4. As described in the method described in item 2 of the patent application scope, where xRS=x0-n×W-m, n and m are integers, and W represents the width of the current block. For example, the method described in item 3 of the patent application scope, wherein yCS=y0-n×H-m, n and m are integers, and H represents the height of the current block. For example, the method described in item 5 or 6 of the patent application scope, wherein n=0, m=0; n=0, m=1; n=1, m=1; or n=2, m=1. For example, the method described in item 2 of the patent application scope, wherein xRE=x0+n×W+m, n and m are integers, and W represents the width of the current block. For example, the method described in item 3 of the patent application scope, wherein yCE=y0+n×H+m, n and m are integers, and H represents the height of the current block. The method described in item 8 or 9 of the patent application, wherein n=1, m=-B; n=1, m=B; n=2, m=-B; or n=3, m=-B . The method of claim 1, wherein columns located above the current block are not required to predict the current block. The method as described in claim 1, wherein rows located to the left of the current block are not required to predict the current block. The method as described in claim 1, wherein the auxiliary block is covered by the column located above the current block. The method as described in claim 1 of the patent application, wherein the auxiliary block is covered by the row located to the left of the current block. For the method described in item 1 of the patent application, the auxiliary block does not overlap with any of the upper left corner coordinates (xRS, yRS) and the upper left corner coordinates (xRE, yRE). For the method described in item 1 of the patent application, the auxiliary block does not overlap with any of the upper left corner coordinates (xCS, yCS) and the upper left corner coordinates (xCE, yCE). The method described in claim 1, wherein the range of columns located above the current block depends on the position of the current block. The method of claim 17, wherein if the current block is determined to be at the top boundary of the P×Q region, columns located above the current block are not required to predict the current block. The method as described in item 18 of the patent application, wherein if y0%Q==0, the current block is determined to be at the top boundary of the P×Q area, and the upper left corner coordinate of the current block represents is (x0, y0). For the method described in item 17 of the patent application, where XRS==x0, the coordinates of the upper left corner of the current block are expressed as (x0, y0). For the method described in claim 1 of the patent application, the range of rows located to the left of the current block depends on the position of the current block. The method as described in claim 21, wherein if the current block is determined to be at the left boundary of the P×Q region, a row located to the left of the current block is not required to predict the current block. The method as described in item 22 of the patent application, wherein if x0%P==0, the current block is determined to be at the left boundary of the P×Q area, and the coordinates of the upper left corner of the current block represent is (x0, y0). For example, in the method described in item 21 of the patent application, yCS==y0, the coordinates of the upper left corner of the current block are expressed as (x0, y0). For the method described in item 22 of the patent application, Q=128 in the codec tree unit area and Q=64 in the pipeline area. For the method described in item 23 of the patent application, P=128 in the codec tree unit area and P=64 in the pipeline area. The method as described in item 1 of the patent application, wherein the current block is divided into a plurality of basic unit blocks, the method further includes: storing the control point (CP) MV at the lower left corner of the current block in In the lower left basic unit block of the current block; and/or the control point (CP) MV at the lower right corner of the current block is stored in the lower right basic unit block of the current block. For the method described in item 27 of the patent application, the current block is encoded and decoded using a 4-parameter affine model or a 6-parameter affine model. The method as described in claim 1 of the patent application further includes: performing motion compensation for sub-blocks divided from the current block by using MVs stored in the sub-blocks. The method as described in claim 29, wherein the stored MV is a control point (CP) MV of the current block or an auxiliary MV used to predict the current block. The method of any one of claims 1 to 3, 5 to 6, 8 to 9, and 11 to 30, wherein the video processing includes encoding the video block into bits of the video block Stream and decode at least one of the video blocks from a bitstream of the video blocks. A video processing device, including a processor, is configured to implement the method described in any one of items 1 to 31 of the patent application scope. A computer program product, stored in a non-transitory computer-readable medium, the computer program product includes program code for executing the method in any one of items 1 to 31 of the patent application scope.

Images