CN113170148B

CN113170148B - Reducing CTU storage required for affine inheritance

Info

Publication number: CN113170148B
Application number: CN201980079467.0A
Authority: CN
Inventors: 张凯; 张莉; 刘鸿彬; 许继征; 王悦
Original assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Current assignee: Beijing ByteDance Network Technology Co Ltd; ByteDance Inc
Priority date: 2018-12-08
Filing date: 2019-12-09
Publication date: 2024-10-01
Anticipated expiration: 2039-12-09
Also published as: WO2020114515A1; CN113170148A; WO2020114517A1; CN113170159A; CN113170111B; CN113170111A; WO2020114516A1; CN113170159B

Abstract

The CTU stores required to reduce affine inheritance are described. One example method includes: associating a first set of Control Point Motion Vectors (CPMV) for determining inherited motion information of a block encoded after the first block with a second set of CPMV for deriving MVs of a sub-block of the first block or a third set of CPMV signaled for the first block, wherein the first set of CPMV is different from the second set of CPMV or the third set of CPMV; determining inherited motion information of a second block encoded and decoded after the first block based on the first group of CPMV; and performing conversion between the second block and the bit stream representation of the second block by using the inherited motion information.

Description

Reducing CTU storage required for affine inheritance

The present application is directed to the priority and benefit of international patent application No. PCT/CN2018/119946 filed on 12 th month 8 of 2018, international patent application No. PCT/CN2018/121118 filed on 14 th month 12 of 2018, international patent application No. PCT/CN2019/075094 filed on 14 th month 2 of 2019, and international patent application No. PCT/CN2019/075846 filed on 22 nd month 2 of 2019, in accordance with applicable patent laws and/or rules. The entire disclosures of international patent applications nos. PCT/CN2018/119946, PCT/CN2018/121118, PCT/CN2019//075094, and PCT/CN2019/075846 are incorporated by reference as part of the present disclosure.

Technical Field

This patent document relates to image and video coding (decoding) and decoding.

Background

Digital video occupies the largest bandwidth usage in the internet and other digital communication networks. As the number of networked user devices capable of receiving and displaying video increases, the bandwidth requirements of digital video usage are expected to continue to increase.

Disclosure of Invention

The disclosed techniques may be used by video decoder or encoder embodiments during video decoding or encoding using control point motion vectors and affine codec.

In one example aspect, a method of processing video is disclosed. The method comprises the following steps: associating a first set of Motion Vectors (MVs) for determining inherited motion information of other video blocks, a second set of MVs for deriving MVs for sub-blocks of the current video block, and a third set of MVs included in a bitstream representation of the current video block with the current video block; and performing conversion between the current video block and the bitstream representation using the first set of MVs, the second set of MVs, or the third set of MVs.

In another example aspect, another video processing method is disclosed. The method comprises the following steps: conversion between the current block and the bit stream representation of the current block is performed using an affine inherited Motion Vector (MV) of the current block, wherein the affine inherited MV is derived from (1) an MV stored for a neighboring basic block (denoted Badj) or (2) an affine history list.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: conversion between the current block and the bit stream representation of the current block is performed using an affine inherited Motion Vector (MV) of the current block, wherein the affine inherited MV is derived from a first MV stored in a first basic block adjacent to the current block and a second MV stored in a second basic block offset from the first building block.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: associating a first set of control point motion vectors (control point motion vector, CPMV) for determining inherited motion information of a block that is encoded after the first block with a second set of CPMV for deriving MVs of a sub-block of the first block or a third set of CPMV signaled for the first block, wherein the first set of CPMV is different from the second set of CPMV or the third set of CPMV; determining inherited motion information of a second block encoded and decoded after the first block based on the first group of CPMV; and performing conversion between the second block and the bit stream representation of the second block by using the inherited motion information.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: deriving an affine inherited Motion Vector (MV) of the first block of video based on the stored Motion Vector (MV) for conversion between the first block of video and the bit stream representation of the first block; and performing conversion by using affine inheritance MV.

In yet another example aspect, another method of video processing is disclosed. The method comprises the following steps: for conversion between a current block of video and a bitstream representation of the current block, deriving an affine inherited Motion Vector (MV) for the current block based on a first stored Motion Vector (MV) and a second stored MV different from the first stored MV, wherein the first stored MV is stored in a first basic block adjacent to the current block and the second stored MV is stored in a second basic block offset from the first basic block; and performing conversion by using affine inherited MVs of the current block.

In yet another example aspect, another video processing method is disclosed. The method comprises the following steps: deriving one or more parameters of an affine model parameter set associated with an affine model of the current block for conversion between the current block and a bitstream representation of the current block; shifting one or more parameters; and storing the shifted one or more parameters.

In yet another example aspect, a video encoder apparatus is disclosed. The video encoder apparatus includes a processor configured to implement the methods described herein.

In yet another example aspect, a video decoder apparatus is disclosed. The video decoder apparatus comprises a processor configured to implement the methods described herein.

In yet another aspect, a computer-readable medium having code stored thereon is disclosed. The code, when executed by a processor, causes the processor to implement the methods described in this document.

These and other aspects are described in this document.

Drawings

Fig. 1 shows an example of a derivation process of the Merge candidate list construction.

FIG. 2 illustrates example locations of airspace Merge candidates.

Fig. 3 shows an example of candidate pairs that consider redundancy checks for spatial Merge candidates.

Fig. 4A-4B illustrate example locations of the second PU for nx2n and 2nxn partitions.

Fig. 5 is a graphical representation of motion vector scaling of a temporal Merge candidate.

Fig. 6 shows candidate positions of the time domain Merge candidates C0 and C1.

Fig. 7 shows an example of combining bi-prediction Merge candidates.

Fig. 8 summarizes the derivation of motion vector prediction candidates.

Fig. 9 is an example illustration of motion vector scaling of spatial motion vector candidates.

Fig. 10 shows an example of an alternative motion vector predictor (ALTERNATIVE MOTION VECTOR PREDICTOR, ATMVP) motion prediction of a codec unit CU.

FIG. 11 shows an example of one CU with four sub-blocks (A-D) and their neighboring blocks (a-D).

Fig. 12 shows an example flowchart for encoding with different MV precision.

Fig. 13A-13B show 4 and 6 parameter reduced affine motion models, respectively.

Fig. 14 shows an example of affine MVF for each sub-block.

Fig. 15A shows an example of a 4-parameter affine model.

Fig. 15B shows an example of a 6-parameter affine model.

Fig. 16 shows an example of MVP for inheriting af_inter of affine candidates.

Fig. 17 shows an example MVP for af_inter of the constructed affine candidate.

Fig. 18A shows an example of candidates of af_merge in five adjacent block scenes.

Fig. 18B shows an example flow of the CPMV predictor development process.

FIG. 19 illustrates example candidate locations for an affine Merge mode.

Fig. 20 shows an example of affine inheritance in CTU rows.

FIG. 21 shows an example of MVs stored in adjacent neighboring basic blocks

Fig. 22 shows the positions in a4×4 basic block.

Fig. 23 shows an example of MVs of two adjacent neighboring blocks.

Fig. 24 shows an example of MVs for affine inheritance across CTU rows.

Fig. 25 is a flowchart of an example of a video processing method.

Fig. 26 is a block diagram of an example of a video processing apparatus.

Fig. 27 shows an exemplary flowchart of finding the first basic block and the second basic block (rectangular blocks indicate termination of the entire process).

Fig. 28 shows another exemplary flowchart of finding the first basic block and the second basic block (rectangular blocks indicate termination of the entire process).

Fig. 29 is a flowchart of an example of a video processing method.

Fig. 30 is a flowchart of an example of a video processing method.

Fig. 31 is a flowchart of an example of a video processing method.

Fig. 32 is a flowchart of an example of a video processing method.

Detailed Description

The present document provides various techniques that may be used by a decoder of a video bitstream to improve the quality of decompressed or decoded digital video or images. In addition, the video encoder may implement these techniques during the encoding process to reconstruct the decoded frames for further encoding.

For ease of understanding, chapter titles are used in this document and do not limit the embodiments and techniques to corresponding chapters. Thus, embodiments from one section may be combined with embodiments from other sections.

1. Summary of the invention

This patent document relates to video codec technology. In particular, it relates to motion vector coding in video coding. It can be applied to existing video codec standards such as HEVC, or standards to be finalized (general video codec). It may also be applicable to future video codec standards or video codecs.

2. Introduction to the theory

Video codec standards have evolved primarily through the development of the well-known ITU-T and ISO/IEC standards. ITU-T makes h.261 and h.263, ISO/IEC makes MPEG-1 and MPEG-4 vision, and these two organizations jointly make h.262/MPEG-2 video and h.264/MPEG-4 Advanced Video Codec (AVC) and h.265/HEVC standards. Since h.262, video codec standards have been based on hybrid video codec structures, where temporal prediction plus transform coding is utilized. To explore future video coding techniques beyond HEVC, a joint video exploration group (JVET) was established by VCEG and MPEG in 2015. Thereafter, JVET employed a number of new methods and entered them into reference software named Joint Exploration Model (JEM). In month 4 2018, a joint video expert group (JVET) was created between VCEG (Q6/16) and ISO/IEC JTC1 SC29/WG11 (MPEG) aimed at VVC standards with a 50% bit rate reduction compared to HEVC.

2.1 Inter prediction in HEVC/H.265

Each inter-predicted PU (prediction unit) has motion parameters of one or two reference picture lists. The motion parameters include a motion vector and a reference picture index. The use of one of the two reference picture lists may also be signaled using inter predidc. The motion vector may be explicitly encoded as an increment relative to the predictor.

When a CU is encoded in skip mode, one PU is associated with the CU and there are no significant residual coefficients, no motion vector delta or reference picture index for the encoding and decoding. The Merge mode is specified whereby the current PU's motion parameters, including spatial and temporal candidates, are obtained from neighboring PUs. The Merge mode is applicable not only to skip mode, but also to any inter prediction PU. An alternative to the Merge mode is explicit transmission of motion parameters, where motion vectors (more precisely, motion vector differences (motion vector difference, MVD) compared to motion vector predictors)), corresponding reference picture indices for each reference picture list, and the use of reference picture lists are explicitly signaled per PU. In this disclosure, this mode is named advanced motion vector prediction (Advanced motion vector prediction, AMVP). When the signaling indicates that one of the two reference picture lists is to be used, a PU is generated from a sample block. This is called "unidirectional prediction". Unidirectional prediction may be used for P-stripes and B-stripes.

When the signaling indicates that both reference picture lists are to be used, a PU is generated from both sample blocks. This is called "bi-prediction". Bi-prediction is only available for B slices.

Details of inter prediction modes specified in HEVC are provided below. The description will start in Merge mode.

2.1.1 Reference Picture List

In HEVC, the term inter prediction is used to refer to predictions derived from data elements (e.g., sample values or motion vectors) of reference pictures other than the currently decoded picture. As in h.264/AVC, pictures can be predicted from multiple reference pictures. The reference pictures for inter prediction are organized in one or more reference picture lists. The reference index identifies which reference picture in the list should be used to create the prediction signal.

A single reference picture list (list 0) is used for P slices and two reference picture lists (list 0 and list 1) are used for B slices. It should be noted that the reference pictures included in list 0/1 may be from past and future pictures in terms of capture/display order.

2.1 Merge mode

2.1.2.1 Derivation of candidate for Merge mode

When predicting a PU using the Merge mode, an index to an entry in the Merge candidate list is parsed from the bitstream and used to retrieve motion information. The construction of this list is specified in the HEVC standard and can be summarized according to the following series of steps:

Step 1: initial candidate derivation

Step 1.1: spatial candidate derivation

Step 1.2: redundancy check of airspace candidates

Step 1.3: time domain candidate derivation

Step 2: additional candidate inserts

Step 2.1: creating bi-prediction candidates

Step 2.2: inserting zero motion candidates

These steps are also schematically depicted in fig. 1. For spatial-domain Merge candidate derivation, a maximum of four Merge candidates are selected among the candidates located at five different positions. For time domain Merge candidate derivation, a maximum of one Merge candidate is selected among the two candidates. Since a constant number of candidates per PU is assumed at the decoder, additional candidates are generated when the number of candidates obtained from step 1 does not reach the maximum number of Merge candidates signaled in the slice header (MaxNumMergeCand). Since the number of candidates is constant, the index of the best Merge candidate is encoded using truncated unary binarization (Truncated Unarybinarization, TU). If the size of the CU is equal to 8, then all PUs of the current CU share a single Merge candidate list, which is the same as the Merge candidate list of the 2Nx2N prediction unit.

Next, the operations associated with the foregoing steps are described in detail.

2.1.2.2 Spatial candidate derivation

In the derivation of the spatial-domain Merge candidates, up to four Merge candidates are selected among candidates located at the positions depicted in fig. 2. The deduced sequences are a ₁、B₁、B₀、A₀ and B ₂. Position B ₂ is only considered when any PU of position a ₁、B₁、B₀、A₀ is not available (e.g., because it belongs to another slice or slice) or is intra-coded. After adding the candidates at the position a ₁, redundancy check is performed on the addition of the remaining candidates, which ensures that candidates having the same motion information are excluded from the list, thereby improving the codec efficiency. In order to reduce the computational complexity, all possible candidate pairs are not considered in the mentioned redundancy check. Instead, only the pair linked with the arrow in fig. 3 is considered, and if the corresponding candidates for redundancy check do not have the same motion information, the candidates are added to the list only. Another source of duplicate motion information is a "second PU" associated with a partition other than 2nx2n. As an example, fig. 4A-4B depict the second PU in the case of nx2n and 2nxn, respectively. When the current PU is partitioned into nx2n, the candidate at position a ₁ is not considered for list construction. In practice, adding this candidate will result in two prediction units having the same motion information, which is redundant for having only one PU in the coding unit. Similarly, when the current PU is partitioned into 2nxn, position B ₁ is not considered.

Fig. 2 shows the location of the spatial Merge candidate.

Fig. 3 shows candidate pairs that consider redundancy checks for the spatial Merge candidate.

Fig. 4A-4B illustrate the locations of the second PUs of the nx2n and 2nxn partitions.

2.1.2.3 Time domain candidate derivation

In this step, only one candidate is added to the list. Specifically, in the derivation of the temporal Merge candidate, a scaled motion vector is derived based on a collocated (co-located) PU that belongs to a picture that has a smallest POC (Picture Order Count ) difference from a current picture within a given reference picture list. The reference picture list to be used for deriving the collocated PU is explicitly signaled in the slice header. In fig. 5, a scaled motion vector for the temporal Merge candidate is obtained as shown by the dashed line, which is scaled from the motion vector of the collocated PU using POC distances tb and td, where tb is defined as the POC difference between the reference picture of the current picture and the current picture, and td is defined as the POC difference between the reference picture of the collocated picture and the collocated picture. The reference picture index of the temporal Merge candidate is set equal to zero. The actual implementation of the scaling procedure is described in the HEVC specification. For the B slices, two motion vectors (one for reference picture list 0 and the other for reference picture list 1) are obtained and combined to generate bi-prediction Merge candidates.

Fig. 5 is an example illustration of motion vector scaling of a temporal Merge candidate.

In the collocated PU (Y) belonging to the reference frame, the location of the time domain candidate is selected between candidates C ₀ and C ₁, as depicted in fig. 6. If the PU at position C ₀ is not available, is intra-coded, or is outside the current Coding Tree Unit (CTU), also known as the maximum codec Unit (largest Coding Unit, LCU)) line, position C ₁ is used. Otherwise, position C ₀ is used in the derivation of the time domain Merge candidate.

Fig. 6 shows examples of candidate locations C ₀ and C ₁ of the time domain Merge candidates.

2.1.2.4 Additional candidate insertions

In addition to the spatial and temporal Merge candidates, there are two additional types of Merge candidates: the bi-prediction Merge candidate and the zero Merge candidate are combined. A combined bi-predictive Merge candidate is generated by utilizing the space-time Merge candidate. The combined bi-predictive Merge candidate is only used for B slices. A combined bi-prediction candidate is generated by combining a first reference picture list motion parameter of an initial candidate with a second reference picture list motion parameter of another initial candidate. If the two tuples provide different motion hypotheses they will form new bi-prediction candidates. As an example, fig. 7 depicts the case when two candidates (with mvL0 and refIdxL0 or mvL1 and refIdxL 1) in the original list (on the left) are used to create a combined bi-prediction Merge candidate that is added to the final list (on the right). There are many rules that are considered for combining to generate these additional Merge candidates.

Zero motion candidates are inserted to fill the remaining entries in the Merge candidate list and thus reach MaxNumMergeCand capacity. These candidates have zero spatial displacement and a reference picture index that starts from zero and increases each time a new zero motion candidate is added to the list. Finally, no redundancy check is performed on these candidates.

2.1.3 AMVP

AMVP exploits the spatial-temporal correlation of motion vectors with neighboring PUs, which is used for explicit transmission of motion parameters. For each reference picture list, a motion vector candidate list is constructed by first checking the availability of the left, upper temporally adjacent PU locations, removing redundant candidates, and adding zero vectors to make the candidate list a constant length. The encoder may then select the best predictor from the candidate list and send a corresponding index indicating the selected candidate. Similar to the Merge index signaling, the index of the best motion vector candidate is encoded using truncated unary. In this case, the maximum value to be encoded is 2 (see fig. 8). In the following section, details about the derivation process of motion vector prediction candidates will be provided.

2.1.3.1 Derivation of AMVP candidates

Fig. 8 outlines the derivation of motion vector prediction candidates.

In motion vector prediction, two types of motion vector candidates are considered: spatial domain motion vector candidates and temporal motion vector candidates. For spatial domain motion vector candidate derivation, two motion vector candidates are ultimately derived based on the motion vector of each PU located at five different locations as depicted in fig. 2.

For temporal motion vector candidate derivation, one motion vector candidate is selected from two candidates derived based on two different collocated positions. After the first space-time selection list is generated, the repeated motion vector candidates in the list are removed. If the number of potential candidates is greater than two, motion vector candidates having reference picture indices greater than 1 within the associated reference picture list are removed from the list. If the number of space-time motion vector candidates is less than two, additional zero motion vector candidates are added to the list.

2.1.3.2 Construction of spatial motion vector candidates

In the derivation of spatial motion vector candidates, a maximum of two candidates are considered among five potential candidates derived from PUs located at the positions depicted in fig. 2, those positions being the same as the positions of the motion Merge. The derivation order on the left side of the current PU is defined as a ₀、A₁, and scale a ₀, scale a ₁. The derivation order of the upper side of the current PU is defined as B ₀、B₁、B₂, scale B ₀, scale B ₁, scale B ₂. Thus for each side there are four cases that can be used as motion vector candidates, two of which do not require spatial scaling and two of which use spatial scaling. Four different cases are summarized as follows:

no spatial scaling

- (1) Identical reference picture list, and identical reference picture index (identical POC)

- (2) Different reference picture lists, but the same reference picture (same POC)

Spatial domain scaling

- (3) Identical reference picture list, but different reference pictures (different POC)

- (4) Different reference picture list, different reference picture (different POC)

First the non-spatial scaling case is checked, then the spatial scaling case. Spatial scaling is considered when POC differs between the reference picture of the neighboring PU and the reference picture of the current PU, regardless of the reference picture list. If all PUs of the left candidate are not available or are intra-coded, scaling for the upper motion vector is allowed to aid in the parallel derivation of the left and upper MV candidates. Otherwise, spatial scaling is not allowed for the upper motion vector.

Fig. 9 is a diagram of motion vector scaling of spatial motion vector candidates.

In the spatial scaling process, the motion vectors of neighboring PUs are scaled in a similar manner as in the temporal scaling, as depicted in fig. 9. The main difference is that the reference picture list and the index of the current PU are given as inputs; the actual scaling procedure is the same as the time domain scaling procedure.

2.1.3.3 Temporal motion vector candidates

All procedures for deriving temporal Merge candidates are the same as those for deriving spatial motion vector candidates except for reference picture index derivation (see fig. 6). The reference picture index is signaled to the decoder.

2.2 Motion vector prediction method based on sub CU in JEM

In a JEM with QTBT (quadtrees plus binary trees, quadtree plus binary tree), there may be at most one set of motion parameters per CU per prediction direction. By dividing the large CU into sub-CUs and deriving the motion information of all sub-CUs of the large CU, two sub-CU level motion vector prediction methods are considered in the encoder. An optional temporal motion vector prediction (ATMVP) method allows each CU to extract multiple sets of motion information from multiple blocks smaller than the current CU in the collocated reference picture. In the space-time motion vector prediction (STMVP) method, the motion vectors of the sub-CUs are recursively derived by using time-domain motion vector prediction amounts and Spatial neighboring motion vectors.

In order to maintain a more accurate motion field for sub-CU motion prediction, motion compression of the reference frame is currently disabled.

Fig. 10 shows an example of ATMVP motion prediction for a CU.

2.2.1 Optional temporal motion vector prediction

In an Alternative Temporal Motion Vector Prediction (ATMVP) method, motion vector temporal motion vector prediction (Temporal Motion Vector Prediction, TMVP) is modified by extracting multiple sets of motion information (including motion vectors and reference indices) from a block smaller than the current CU. The sub-CU is a square nxn block (N is set to 4 by default). ATMVP predicts the motion vectors of sub-CUs within a CU in two steps. The first step is to identify the corresponding block in the reference picture with a so-called temporal vector. The reference picture is called a motion source picture. The second step is to divide the current CU into sub-CUs and obtain a motion vector and a reference index of each sub-CU from a block corresponding to each sub-CU.

In a first step, the reference picture and the corresponding block are determined from motion information of spatial neighboring blocks of the current CU. To avoid the iterative scanning process of neighboring blocks, the first Merge candidate in the Merge candidate list of the current CU is used. The first available motion vector and its associated reference index are set as the temporal vector and the index of the motion source picture. In this way, in an ATMVP, the corresponding block (sometimes referred to as a collocated block) may be more accurately identified than a TMVP, where the corresponding block is always in a lower right or center position relative to the current CU.

In a second step, by adding a temporal vector to the coordinates of the current CU, the corresponding block of the sub-CU is identified by the temporal vector in the motion source picture. For each sub-CU, the motion information of its corresponding block (the smallest motion grid covering the center sample) is used to derive the motion information of the sub-CU. After identifying the motion information of the corresponding nxn block, it is converted into a motion vector and a reference index of the current sub-CU in the same manner as TMVP of HEVC, where motion scaling and other processes are applicable. For example, the decoder checks whether a low latency condition is satisfied (i.e., POC of all reference pictures of the current picture is less than POC of the current picture), and may use motion vector MV _x (motion vector corresponding to reference picture list X) to predict motion vector MV _y for each sub-CU (where X is equal to 0 or 1 and Y is equal to 1-X).

2.2.2 Spatial motion vector prediction

In this method, the motion vectors of the sub-CUs are recursively deduced in raster scan order. Fig. 11 shows this concept. Let us consider an 8 x 8 CU, which contains 4 x 4 sub-CUs: A. b, C and D. The neighboring 4 x 4 blocks in the current frame are labeled a, b, c, and d.

The motion derivation of sub-CU a begins by identifying its two spatial neighbors (neighbors). The first neighbor is an nxn block (block c) above sub CU a. If this block c is not available or intra-coded, then other nxn blocks above sub-CU a are checked (from left to right, starting from block c). The second neighbor is the block to the left of sub-CU a (block b). If block b is not available or intra-coded, other blocks to the left of sub-CU a are checked (from top to bottom, starting from block b). The motion information obtained from the neighboring blocks of each list is scaled to the first reference frame of the given list. Next, the Temporal Motion Vector Prediction (TMVP) of sub-block a is derived by following the same procedure as the TMVP derivation specified by HEVC. The motion information of the collocated block at position D is extracted and scaled accordingly. Finally, after retrieving and scaling the motion information, all available motion vectors (up to 3) are averaged separately for each reference list. The average motion vector is assigned as the motion vector of the current sub-CU.

FIG. 11 shows an example of one CU with 4 sub-blocks (A-D) and their neighboring blocks (a-D).

2.2.3 Sub-CU motion prediction mode Signaling

The sub-CU modes are enabled as additional Merge candidates and these modes need not be signaled by additional syntax elements. Two additional Merge candidates are added to the Merge candidate list for each CU to represent the ATMVP mode and the STMVP mode. If the sequence parameter set indicates that ATMVP and STMVP are enabled, up to seven Merge candidates are used. The coding logic of the additional Merge candidate is the same as that of the Merge candidate in the HM, which means that for each CU in the P-slice or B-slice, two more RD (Rate-Distortion) checks are required for the two additional Merge candidates.

In JEM, all binary numbers of the Merge index are context-coded by CABAC. In HEVC, only the first binary number is context-coded and the remaining binary numbers are context-bypass-coded.

2.3 Inter prediction method in VVC

There are several new codec tools for inter Prediction improvement, such as adaptive motion vector difference resolution (Adaptive Motion Vector difference Resolution, AMVR), affine Prediction mode, triangle Prediction Mode (TPM), ATMVP, generalized Bi-Prediction (GBI), bi-directional optical flow (BIO) for signaling MVD.

2.3.1 Coding and decoding block structure in VVC

In VVC, a quadtree/binary tree/multi-way tree (QT/BT/TT) structure is used to divide a picture into square or rectangular blocks.

In VVC, a separate tree (also referred to as dual coding tree) may be employed for I frames in addition to QT/BT/TT. For separate trees, the codec block structure is signaled separately for the luma and chroma components.

2.3.2 Adaptive motion vector difference resolution

In HEVC, a motion vector difference (Motion Vector Difference, MVD) between the motion vector of the PU and the predicted motion vector) is signaled in units of quarter (quater) luma samples when use_integer_mv_flag in the slice header is equal to 0. In VVC, a Locally Adaptive Motion Vector Resolution (LAMVR) is introduced. In VVC, MVDs may be encoded and decoded in units of quarter-luminance samples, integer-luminance samples, or four-luminance samples (i.e., 1/4 pixel, 1 pixel, 4 pixel). The MVD resolution is controlled at a Coding Unit (CU) level and, for each CU having at least one non-zero MVD component, a MVD resolution flag is conditionally signaled.

For a CU with at least one non-zero MVD component, a first flag is signaled to indicate whether quarter-luma sample MV precision is used in the CU. When the first flag (equal to 1) indicates that the quarter-luminance sample MV precision is not used, another flag is signaled to indicate whether the integer-luminance sample MV precision or the four-luminance sample MV precision is used.

When the first MVD resolution flag of a CU is zero or no coding is performed for the CU (meaning that all MVDs in the CU are zero), a quarter luma sample MV resolution is used for the CU. When the CU uses integer luminance sample MV precision or four luminance sample MV precision, the MVP in the AMVP candidate list of the CU is rounded to the corresponding precision. In the encoder, a RD-check at the CU level is used to determine which MVD resolution is to be used for the CU. That is, for each MVD resolution, three CU-level RD checks are performed. In order to accelerate the encoder speed, the following coding scheme is applied in JEM:

During RD checking of a CU with normal quarter-luma sample MVD resolution, the motion information of the current CU (integer luma sample accuracy) is stored. The stored motion information (after rounding) is used as a starting point for further small-range motion vector refinement during RD checking for the same CU with integer luminance samples and 4 luminance samples MVD resolution, so that the time-consuming motion estimation process is not repeated three times.

● RD checking is conditionally invoked for CUs with 4 luma sample MVD resolution. For a CU, when the RD cost of the integer luminance sample MVD resolution is much greater than the RD cost of the quarter luminance sample MVD resolution, the RD check for the 4 luminance sample MVD resolution of the CU is skipped.

The encoding process is shown in fig. 12. First, 1/4 pixel MVs are tested, RD costs are calculated and denoted as RDCost0, and then integer MVs are tested, RD costs being denoted as RDCost1. If RDCost1< th RDCost0 (where th is a positive value), then 4 pixels MV are tested; otherwise, 4 pixels MV are skipped. Basically, when checking integer or 4-pixel MVs, motion information and RD costs, etc., are known for 1/4-pixel MVs, which can be reused (reused) to accelerate the encoding process of integer or 4-pixel MVs.

Fig. 12 is a flowchart of encoding at different MV accuracies.

2.3.3 Affine motion compensated prediction

In HEVC, only translational motion models are applied to motion compensated prediction (MotionCompensation Prediction, MCP). While in the real world there are various movements such as zoom in/out, rotation, perspective movement and other irregular movements. In VVC, reduced affine transformation motion compensation prediction is applied with a 4-parameter affine model and a 6-parameter affine model. As shown in fig. 13A and 13B, the affine motion field of a block is described by two control point motion vectors (controlpoint motion vector, CPMV) of a 4-parameter affine model and 3 CPMV of a 6-parameter affine model.

The Motion Vector Field (MVF) of the block is described by the following equations, respectively a 4-parameter affine model of equation (1) (where 4-parameter affine models are defined as variables a, b, e, and f) and a 6-parameter affine model of equation (2) (where 6-parameter affine models are defined as variables a, b, c, d, e and f):

Where (mv ^h ₀,mv^h ₀) is the motion vector of the upper left corner control point, and (mv ^h ₁,mv^h ₁) is the motion vector of the upper right corner control point, and (mv ^h ₂,mv^h ₂) is the motion vector of the lower left corner control point, all 3 motion vectors are referred to as Control Point Motion Vectors (CPMV), (x, y) represent the coordinates of the representative point (REPRESENTATIVE POINT) within the current block relative to the upper left sample point, and (mv ^h(x,y),mv^v (x, y)) is the motion vector derived for the sample point located at (x, y). The CP motion vector may be signaled (as in affine AMVP mode) or dynamically derived (as in affine Merge mode). w and h are the width and height of the current block. In practice, division is performed by right-shifting and rounding operations. In the VTM, the representative point is defined as the center position of the sub-block, for example, when the coordinates of the upper left corner of the sub-block with respect to the upper left sample point within the current block are (xs, ys), the coordinates of the representative point are defined as (xs+2, ys+2). For each sub-block (i.e. 4 x 4 in VTM), the representative points are used to derive the motion vector for the entire sub-block.

To further simplify motion compensated prediction, sub-block based affine transformation prediction is applied. In order to derive the motion vector of each mxn sub-block (M and N are both set to 4 in the current VVC), the motion vector of the center sample of each sub-block may be calculated according to equations (1) and (2) (as shown in fig. 14), and rounded to a 1/16 fractional precision. Then, a motion compensated interpolation filter is applied for 1/16 pixels to generate a prediction for each sub-block with the derived motion vector. Affine mode introduces a 1/16 pixel interpolation filter.

Fig. 14 shows an example of affine MVF for each sub-block.

After MCP, the high precision motion vector for each sub-block is rounded and saved to the same precision as the normal motion vector.

2.3.3.1 Signaling of affine predictions

Similar to the translational motion model, there are two modes for signaling side information (side information) generated due to affine prediction. They are AFFINE _INTER and AFFINE _MERGE modes.

2.3.3.2 Examples of AF_INTER modes

For CUs with width and height both greater than 8, the af_inter mode may be applied. Affine flags at the CU level are signaled in the bitstream to indicate whether af_inter mode is used.

In this mode, for each reference picture list (list 0 or list 1), the affine AMVP candidate list is constructed with three types of affine motion prediction values in the following order, with each candidate including the estimated CPMV of the current block. The difference between the best CPMV found at the encoder side (such as mv ₀mv₁mv₂ in fig. 17) and the estimated CPMV is signaled. In addition, the index of affine AMVP candidates from which the estimated CPMV is derived is further signaled.

1) Inherited affine motion prediction values

The order of checking is similar to the order of spatial MVPs in HEVC AMVP list construction. First, the left inherited affine motion prediction value derived from the affine-coded block { A1, A0} and having the same reference picture as the first block in the current block. Second, an over inherited affine motion prediction value is derived from the affine codec in { B1, B0, B2} and has the first block of the same reference picture as in the current block. Five blocks A1, A0, B1, B0, B2 are depicted in fig. 16.

Once the neighboring block is found to be encoded with affine patterns, the CPMV of the codec unit covering the neighboring block is used to derive the prediction value of the CPMV of the current block. For example, if A1 is codec with a non-affine mode and A0 is codec with a 4-parameter affine mode, then the left inherited affine MV prediction value will be derived from A0. In this case, CPMU covering CU of A0 (CPMU on the upper left is denoted as CPMU on the left as shown in FIG. 18B)For CPMV on the upper right, expressed as) The estimated CPMV used to derive the current block (upper left position (coordinates (x 0, y 0)), upper right position (coordinates (x 1, y 1)) and lower right position (coordinates (x 2, y 2)) for the current block are expressed as )。

2) Constructed affine motion prediction values

As shown in fig. 17, the constructed affine motion prediction value includes a Control Point Motion Vector (CPMV) derived from neighboring inter-frame codec blocks having the same reference picture. If the current affine motion model is a 4-parameter affine, the number of CPMV is 2, otherwise if the current affine motion model is a 6-parameter affine, the number of CPMV is 3. CPMV at upper leftIs derived from MVs at the first block in the set { a, B, C } that are affine-coded and have the same reference picture as in the current block. CPMV on the upper rightIs derived from MVs at the first block in the set D, E, which are inter-coded and have the same reference picture as in the current block. CPMV at lower leftIs derived from MVs at the first block in the set F, G, which are inter-coded and have the same reference picture as in the current block.

-If the current affine motion model is a 4-parameter affine, then only ifAndWhen both are established, the constructed affine motion prediction value is inserted into the candidate list, that is,AndIs used as an estimated CPMV of the upper left (coordinates (x 0, y 0)), upper right (coordinates (x 1, y 1)) position of the current block.

-If the current affine motion model is a 6-parameter affine, then only ifAndThe constructed affine motion prediction value is inserted into the candidate list when it is established, that is, AndIs used as an estimated CPMV for the upper left (coordinates (x 0, y 0)), upper right (coordinates (x 1, y 1)) and lower right (coordinates (x 2, y 2)) positions of the current block.

When the constructed affine motion prediction value is inserted into the candidate list, the pruning process is not applied.

3) Normal AMVP motion prediction

Before the number of affine motion prediction values reaches the maximum value, the following condition applies.

1) By setting all CPMVs equal toIf available, affine motion prediction values are derived.

2) By setting all CPMVs equal toIf available, affine motion prediction values are derived.

3) By setting all CPMVs equal toIf available, affine motion prediction values are derived.

4) Affine motion prediction values are derived by setting all CPMV equal to HEVC TMVP (if available).

5) Affine motion prediction values are derived by setting all CPMV to zero MV.

Note that affine motion prediction values that have been constructed are derived

Fig. 15A shows an example of a 4-parameter affine model. Fig. 15B shows an example of a 6-parameter affine model.

Fig. 16 shows an example of MVP inheriting af_inter of affine candidates.

Fig. 17 shows an example of MVP of af_inter of the constructed affine candidate. In the AF INTER mode, when a 4/6 parameter affine mode is used, 2/3 control points are required, and thus 2/3 MVDs need to be encoded and decoded for these control points, as shown in fig. 15A and 15B. MV is proposed to be derived from mvd ₀ by predicting mvd ₁ and mvd ₂.

Wherein, Mvd _i and mv ₁ are the predicted motion vector, motion vector difference, and motion vector of the upper left pixel (i=0), upper right pixel (i=1), or lower left pixel (i=2), respectively, as shown in fig. 15B. Note that the addition of two motion vectors (e.g., mvA (xA, yA) and mvB (xB, yB)) is equal to the respective sum of the two components, that is, newMV =mva+mvb, and the two components of newMV are set to (xa+xb) and (ya+yb), respectively.

2.3.3.3 AF_Merge mode

When a CU is applied in af_merge mode, it obtains the first block encoded with affine mode from the valid neighboring reconstructed blocks. And the selection order of the candidate blocks is from left, top right, bottom left to top left as shown in fig. 18A (indicated in turn by A, B, C, D, E). For example, if the neighboring lower left block is encoded in affine mode as shown by A0 in fig. 18B, control Point (CP) motion vectors mv ₀ ^N、mv₁ ^N and mv ₂ ^N including the upper left, upper right, and lower left corners of the neighboring CU/PU of the block a are extracted. And motion vectors mv ₀ ^C、mv₁ ^C and mv ₂ ^C (for the 6-parameter affine model only) for the top left/top right/bottom left on the current CU/PU are calculated based on mv ₀ ^N、mv₁ ^N and mv ₂ ^N. It should be noted that if the current block is affine codec, the sub-block located in the upper left corner (e.g., 4 x 4 block of VTM) stores mv0, and the sub-block located in the upper right corner may store mv1. If the current block is encoded and decoded by using a 6-parameter affine model, storing mv2 in a sub-block positioned at the lower left corner; otherwise (in the case of a 4-parameter affine model), LB stores mv2'. The other sub-blocks store MVs for the MC.

After deriving CPMVmv ₀ ^C、mv₁ ^C and mv ₂ ^C of the current CU from the reduced affine motion model in equations (1) and (2), the MVF of the current CU is generated. In order to identify whether the current CU is encoded in the af_merge mode, an affine flag is signaled in the bitstream when there is at least one neighboring block encoded in the affine mode.

Fig. 18A shows an example candidate of af_merge in the case of 5 adjacent blocks.

Fig. 18B shows an example of the CPMV predictor development process.

It is proposed to construct an affine Merge candidate list with the following steps:

1) Inserting inherited affine candidates

Inherited affine candidates refer to candidates that are derived from affine motion models whose significant neighbors are affine codec blocks. Up to two inherited affine candidates are derived from affine motion models of neighboring blocks and inserted into the candidate list. For the predicted value on the left, the scan order is { A0, A1}; for the upper predictor, the scan order is { B0, B1, B2}.

2) Affine candidates for insertion constructs

If the number of candidates in the affine Merge candidate list is less than MaxNumAffineCand (e.g., 5), then the constructed affine candidates are inserted into the candidate list. Constructing affine candidates refers to constructing candidates by combining neighboring motion information of each control point.

A) The motion information of the control point is first derived from the designated spatial and temporal neighbors shown in fig. 19. CPk (k=1, 2,3, 4) represents the kth control point. A0, A1, A2, B0, B1, B2, and B3 are spatial locations for predicting CPk (k=1, 2, 3); t is the time domain position used to predict CP 4.

The coordinates of CP1, CP2, CP3, and CP4 are (0, 0), (W, 0), (H, 0), and (W, H), respectively, where W and H are the width and height of the current block.

Fig. 19 shows an example of candidate positions of the affine Merge mode.

Motion information for each control point is obtained according to the following priority order:

for CP1, the check priority is B2- > B3- > A2. If B2 is available, then B2 is used.

Otherwise, if B2 is available, B3 is used. If neither B2 nor B3 is available, A2 is used.

If none of the three candidates is available, the motion information of CP1 cannot be obtained.

For CP2, the check priority is B1- > B0.

For CP3, the check priority is A1- > A0.

For CP4, T is used.

B) Second, affine Merge candidates are constructed using combinations of control points.

I. Constructing 6-parameter affine candidates requires motion information for three control points. The three control points may select one from the following four combinations: { CP1, CP2, CP4}, { CP1, CP2, CP3}, { CP2, CP3, CP4}, { CP1, CP3, CP4}. The combination CP1, CP2, CP3, { CP2, CP3, CP4}, { CP1, CP3, CP4} will be converted into a 6-parameter motion model represented by the upper left, upper right and lower left control points.

Constructing 4-parameter affine candidates requires motion information of two control points. The two control points may select one from two combinations ({ CP1, CP2}, { CP1, CP3 }). These two combinations will be converted into a 4-parameter motion model represented by the upper left and upper right control points.

Inserting combinations of constructed affine candidates into the candidate list in the following order:

{CP1,CP2,CP3}、{CP1,CP2,CP4}、{CP1,CP3,CP4}、{CP2,CP3,CP4},{CP1,CP2}、{CP1,CP3}

i. For each combination, the reference index of list X is checked for each CP, and if they are all the same, the combination has a valid CPMV for list X. If the combination has no valid CPMVs for both list 0 and list 1, the combination is marked invalid. Otherwise, it is active and puts the CPMV in the sub-block Merge list.

3) Filling with zero motion vectors

If the number of candidates in the affine Merge candidate list is less than 5, a zero motion vector with a zero reference index is inserted into the candidate list until the list is full.

More specifically, for the sub-block Merge candidate list, there are 4 parameter Merge candidates, where MV is set to (0, 0) and the prediction direction is set to unidirectional prediction (for P slices) and bi-prediction (for B slices) from list 0.

2.3.3.4 Storage needed for affine model inheritance

Memory required inside CTU

For affine model inheritance as shown in fig. 18B, additional information is stored in each 8×8 block inside the CTU.

1) Two reference lists have three CPMV, requiring 2 (2 reference lists) ×2 (x and y components) ×16 (one component 16 bits) ×3 (3 CPMV) =192 bits (or 24 bytes in software design).

2) The coordinates of the upper left corner of the CU require 2×13=26 bits (or 4 bytes in the software design).

3) The width and height of a CU requires 2×7=14 bits (or 2 bytes in software design).

Therefore, it is necessary to store 232 bits (or 30 bytes) of side information in total for each 8×8 block. We should note that in HEVC, the total amount of motion information required to be stored in each 8 x 8 block is 2 (2 reference lists) ×2 (x and y components) ×16 (one component 16 bits) ×4 (4 MVs of 4 x 4 blocks) =256 bits (or 32 bytes in a software design).

Thus, in a software design, the memory inside the CTU required for motion information increases by 232/256=90% or 30/32=94%. This greatly increases the cache memory requirements.

Memory for line buffers

For blocks at CTU row boundaries, it is proposed to reduce the line buffer memory required for affine inheritance. Fig. 20 shows an example when the current CU is at a CTU row boundary. Suppose an inheritance affine model is chosen from the 4 x 4 blocks of the overlay (xLE, y _LE1) and then find the neighboring CUs of the overlay (x _LE1,y_LE1). The MVs (labeled vLE0 and vLE1 in the figure) of the lower left 4 x 4 block and lower right block of the neighboring CU are found. The CPMV of the current block is calculated as follows:

Deriving a control point vector for the current CU by using a 4-parameter model and by And

And if the current CU uses a 6-parameter affine motion model, a control point vector is derived by

Because CPMV of neighboring blocks outside the current CTU row is not required, CPMV is not required to be stored in the line buffer. Furthermore, the y-component of the height and upper left corner coordinates need not be stored in a line buffer. However, the x-component of the width and upper left corner coordinates still needs to be stored in the line buffer.

Fig. 20 shows an example of affine inheritance at CTU rows.

For affine inheritance, it is proposed to store affine parameters a, b, c and d for affine inheritance instead of storing CPMV. For Historical Motion Vector Prediction (HMVP), it is proposed that affine parameters a, b, c, and d may be stored to generate Historical Motion Vector Predictions (HMVP) for affine Merge or affine inter-frame codec. The buffer/table/list for storing history-based affine models is referred to as affine HMVP buffer/table/list.

3. Examples of problems addressed by the embodiments and techniques described herein

The additional storage required for affine inheritance remains a major problem in current designs.

Furthermore, in the current design of affine inheritance across CTU rows, there is misalignment (misalignment) with respect to MVs that are associated with one location but are assumed to be associated with another location.

As shown in fig. 20, the control point vector of the current CU is derived by using the 4-parameter model and by the following equationAnd

In the above equation, (x _LE1,y_LE1) and (x _LE0,y_LE0) are used as representative positions of the lower right and lower left sample coordinates of the codec unit in affine mode, respectively.

Let v _LE0 be assigned to the lower left position of the neighboring CU (denoted as (x _E0,y₀)). However, v _LE0 is assigned to the center position of the lower left sub-block of the neighboring CU.

In addition, the coordinates of the affine-coded CU and the width of the CU need to be known, because two representative sub-blocks covering the lower left and lower right sample points of the CU need to be identified. In the above example, the sub-block of the overlay (x _LE0,y_LE0) is referred to as a first representative sub-block, and the sub-block of the overlay (x _LE1,y_LE1) is referred to as a second representative sub-block. Therefore, an additional line buffer is required to store CU width, coordinates, etc.

4. Examples of embodiments and techniques

The following detailed list of items should be considered as an example to explain the general concept. These embodiments should not be construed narrowly. Furthermore, these inventions may be combined in any manner.

In the following discussion SatShift (x, n) is defined as

Shift (x, n) is defined as Shift (x, n) = (x+offset 0) > > n.

In one example, offset0 and/or offset1 is set to (1 < < n) > >1 or (1 < < (n-1)). In another example, offset0 and/or offset1 is set to 0.

Clip3 (min, max, x) is defined as

The CTU stores required for affine inheritance are reduced.

1. The first set of CPMV for the block that affine inherits the subsequent block (located at representative points (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2), respectively), denoted MV ^F0、MV^F and MV ^F 2) may be different from the second set of CPMV for the block that derives the MV of each sub-block (located at representative points (x ^S0,y^S0)、(x^S1,y^S 1) and (x ^S2,y^S 2), denoted MV ^S0、MV^S and MV ^S, respectively) or the third set of CPMV signaled from the encoder to the decoder (located at representative points (x ^T0,y^T0)、(x^T1,y^T 1) and (x ^T2,y^T 2), denoted MV ^T0、MV^T 1 and MV ^T, respectively).

A) In one example, the second set of CPMV is the same as the third set of CPMV.

B) In one example, the first set of CPMV is derived from the second or third set of CPMV.

C) In one example, the first set of CPMV is stored after encoding/decoding the block.

D) In one example, the coordinates of the representative point, such as (x ^Fi,y^Fi)、(x^Si,y^Si)、(x^Ti,y^T i), are defined as coordinates relative to one sub-block used in the affine motion compensation process.

2. The relative offset between the representative points of the two CPMV's in the first group may not depend on the width or height of the block.

A) In one example, CPMVs (denoted MV ^F0、MV^F 1 and MV ^F 2) for block B at representative points (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2), respectively, are stored. In another example, CPMVs (denoted as MV ^F and MV ^F 1) of block B at locations (x ^F0,y^F 0) and (x ^F1,y^F 1), respectively, are stored. (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) may be internal to block B or they may be external thereto.

B) In one example, y ^F1＝y^F0,x^F1＝x^F +pw.

C) In one example, x ^F1＝x^F0,y^F1＝y^F 0+ ph.

D) In one example, y ^F2＝y^F0,x^F2＝x^F +pw.

E) In one example, x ^F2＝x^F0,y^F2＝y^F 0+ ph.

F) In one example, y ^F2＝y^F1,x^F2＝x^F 1+pw.

G) In one example, x ^F2＝x^F1,y^F2＝y^F 1+ph.

H) PW and PH are integers.

I. in one example, pw=2 ^M. For example, PW may be equal to 4, 8, 16, 32, 64, or 128.

In one example, pw= -2 ^M. For example, PW may be equal to-4, -8, -16, -32, -64, or-128.

In one example, ph=2 ^M. For example, the PH may be equal to 4, 8, 16, 32, 64, or 128.

In one example, ph= -2 ^M. For example, the pH may be equal to-4, -8, -16, -32, -64, or-128.

PW and PH are not stored.

1. In one example, they are fixed.

2. In another example, they are signaled in VPS/SPS/PPS/slice header/CTU.

3. In another example, they may be different in different standard levels (profiles)/levels/hierarchies.

4. In another example, they may depend on the maximum CU size or/and the minimum CU size of the slice/picture.

I) In one example, MV ^F0＝MV^S0,(x^F0,y^F0)＝(x^S0,y^S 0), or MV ^F0＝MV^T0,(x^F0,y^F0)＝(x^T0,y^T 0);

j) In one example, MV ^F0＝MV^S1,(x^F0,y^F0)＝(x^S1,y^S 1), or MV ^F0＝MV^T1,(x^F0,y^F0)＝(x^T1,y^T 1);

k) In one example, MV ^F0＝MV^S2,(x^F0,y^F0)＝(x^S2,y^S 2), or MV ^F0＝MV^T2,(x^F0,y^F0)＝(x^T2,y^T 2);

l) in one example, MV ^F0、MV^F and MV ^F 2 are derived from MV ^S 0 and MV ^S 1 by equation (1) with (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates.

M) in one example, MV ^F0、MV^F and MV ^F 2 are derived from MV ^S0、MV^S 1 and MV ^S 2 by equation (2) with (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates.

N) in one example, MV ^F0、MV^F and MV ^F 2 are derived from MV ^T 0 and MV ^T 1 by equation (1) with (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates.

O) in one example, MV ^F0、MV^F and MV ^F 2 are derived from MV ^T0、MV^T 1 and MV ^T 2 by equation (2) with (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates.

P) in one example, MV ^F 2 is calculated only when the current block is coded with a 6-parameter affine model.

I. Alternatively, MV ^F 2 is calculated regardless of whether the current block is encoded with a 4-parameter affine model or a 6-parameter affine model.

3. The difference between CPMV is stored instead of storing CPMV itself.

A) In one example, d1=mv ^F1-MV^F 0 is stored;

b) In one example, d2=mv ^F2-MV^F is stored;

c) In one example, both D1 and D2 are stored;

i. in one example, D2 is stored only when the current block is encoded with a 6-parameter affine model.

Alternatively, D2 is stored regardless of whether the current block is encoded with a 4-parameter affine model or a 6-parameter affine model.

D) In one example, the difference between the CPMV and the CPMV may be stored together. For example, MVs ^F 0, D1, and D2 are stored.

4. The stored CPMV or the difference between CPMV may be shifted before storage. Assuming that the MV is CPMV or the difference between CPMVs to be stored, then

A) For example, MV 'x= SatShift (MVx, n), and MV' y= SatShift (MVy, n). MV ' = (MV ' x, MV ' y) is stored instead of MV.

B) For example, MV 'x=shift (MVx, n), and MV' y=shift (MVy, n). MV ' = (MV ' x, MV ' y) is stored instead of MV.

C) For example, n is an integer such as 2 or 4;

i. In one example, n depends on the motion accuracy.

N may be different when storing CPMV or storing the difference between CPMV.

D) In one example, the stored MV' is first left shifted before being used for affine inheritance.

5. The CPMV to be stored or the difference between CPMV may be limited before storing. Assuming that the MV is CPMV or the difference between CPMVs to be stored, then

A) MV 'x=clip 3 (MinV, maxV, MVx), and MV' y=clip 3 (MinV, maxV, MVy). MV ' = (MV ' x, MV ' y) is stored instead of MV.

B) In one example, the MV is stored with K bits, then MinV = -2 ^K-1,MaxV＝2^K-1 -1. For example, when k=8, minV = -128, maxv=127.

I.K may vary depending on whether the MV is CPMV or the difference between CPMV.

C) In one example, the stored MV' is first shifted and then sliced before it is used for affine inheritance.

6. The MVs stored in the adjacent neighboring basic blocks denoted Badj are used to derive affine inherited MVs for the current block.

A) Fig. 21 shows an example of MVs stored in adjacent neighboring basic blocks: l, A, LB, AR and AL. In VTM, the basic block is a4×4 block.

B) The MV stored in the adjacent neighboring basic block Badj is represented as mva= (MV ^h _a,mv^v _a), then affine inherited MV of the current block at the position (x, y) to be represented as (MV ^h(x,y),mv^v (x, y)) is derived as

Or alternatively

Wherein, (x ₀,y₀) is a representative point of MVa.

I. (x ₀,y₀) can be any location inside basic block Badj. Fig. 22 shows an example. (x ₀,y₀) can be any one of Pij (i= … Wb-1, j= … Hb-1), where Wb and Hb are the width and height of the basic block. In this example, wb=hb=4. In one example, assuming that the coordinates of the upper left corner sample in Badj are (xTL, yTL), then (x ₀,y₀) may be any one of (xTL +i, yTL +j) (i= … Wb-1, j= … Hb-1).

1. For example, (x ₀,y₀) may be P22 in fig. 22.

2. Assuming that the coordinates of the left-hand sample of the current block are (xPos, yPos 00), the coordinates of the right-hand sample of the current block are (xPos, yPos 00), and the coordinates of the right-hand sample of the current block are (xPos, yPos 01), then in fig. 21:

a. The adjacent basic block L has a value (x ₀,y₀) of (xPos 00-2, yPos 01-1);

b. the adjacent neighboring basic block LB has (x ₀,y₀) of (xPos 00-2, yPos01+3);

c. the adjacent basic block A has a value (x ₀,y₀) of (xPos-1, yPos 00-2);

d. The adjacent neighboring basic block AR (x ₀,y₀) is (xPos +3, yPos 00-2);

e. The adjacent neighboring basic block AL (x ₀,y₀) is (xPos 00-2, yPos 00-2).

(X ₀,y₀) may be any location at the exterior or boundary of basic block Badj.

1. For example, assuming that the coordinates of the upper left corner sample in Badj are (xTL, yTL), then (x ₀,y₀) may be any of (xTL +i, yTL +j), where i may be-1, 0, wb-1, or Wb; j may be-1, 0, hb-1 or Hb.

Assume that the coordinates of the upper left corner sample in Badj are (xTL, yTL), and (x ₀,y₀) = (xTL +i, yTL +j).

I and j may depend on the position of Badj. For example, if Badj is block L in fig. 21, i=0, j=hb-1, but if Badj is block a in fig. 21, i=wb-1, j=0.

The i and j may depend on the width and height of the current block.

I and j may be signaled in VPS/SPS/PPS/slice header/slice group header/slice/CTU/CU.

4. In another example, i and j may be different in different standard levels/hierarchies.

The position (x, y) may be in a sub-block of the current block, then the MVs of the sub-block are inherited depending on MVa.

The position (x, y) may be an angle of the current block, and then inherits the CPMV of the current block depending on the MVa.

1. In one example, inherited CPMV may be used to predict signaled CPMV of the current block of affine inter-frame codec.

2. In one example, the inherited CPMV may be directly used as the CPMV of the current block of the affine Merge codec.

In one example, if the current block uses a 4-parameter affine model, equation (3) is applied. If the current block uses a 6-parameter affine model, equation (4) is applied.

Alternatively, equation (4) is applied regardless of whether the current block uses a 4-parameter affine model or a 6-parameter affine model.

C) The a, b, c and d in the equations (3) and (4) are calculated as follows

I. In one example, a, b, c, and d are derived from CPMVs stored in the second or third group of CUs overlaying adjacent neighboring base blocks Badj, as stated in bullets 1. Assuming that the CU covering the neighboring base block Badj is block Z, mv_t0＝(mv^h _t0,mv^v _t0),mv_t1＝(mv^h _t1,mv^v _t1),mv_t2＝(mv^h _t2,mv^v _t2) is CPMV in the second or third group of block Z. w _t and h _t are the width and height of block Z.

In one example, a, b, c, and d are derived from CPMVs stored in a first group of CUs covering neighboring basic blocks Badj, as stated in bullets 1. Assuming that the CU covering the neighboring base block Badj is block Z, mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1) and mv _t2＝(mv^h _t2,mv^v _t2) are CPMV in the first group of block Z. w _t and h _t are PW and PH as claimed in item symbol 2.

In one example, mv^h _t1-mv^h _t0、mv^v _t1-mv^v _t0、mv^h _t2-mv^h _t0、mv^v _t2-mv^v _t0, is fetched directly from storage as declared in bulleted 3.

Fig. 21 shows an example of MVs stored in adjacent neighboring basic blocks.

Fig. 22 shows the positions in a4×4 basic block.

The line buffering required for affine inheritance is reduced.

7. A first MV stored in a first basic block adjacent to the current block and a second MV stored in a second basic block having a known offset from the first basic block are used to derive a CPMV of the current block.

A) In one example, MVs stored in the first neighboring basic blocks are represented as mva= (MV ^h _a,mv^v _a), then affine inherited MVs of the current block at position (x, y), represented as (MV ^h(x,y),mv^v (x, y)), are derived by equation (3), and a, b are derived by equation (5).

I. In one example, MV _t0 and MV _t1 in equation (5) are set equal to the MV stored in the first basic block and the MV stored in the second basic block, respectively. w _t is set as the horizontal offset between the two basic blocks.

B) Alternatively, a, b may be derived as

Wherein h _t is set as the horizontal offset between the two basic blocks.

C) For example, w _t and h _t must be in the form of 2 ^N, such as 4, 8,16 … …

D) Assuming that (xLT 0, yLT 0) and (xLT 1, yLT 1) represent the coordinates of the upper left corner samples of the first and second basic blocks, respectively, then

I. the horizontal offset between the first basic block and the second basic block is defined as xLT-xLT 0;

the vertical offset between the first basic block and the second basic block is defined as yLT-yLT 0;

e) In one example, yLT-yLT 0 must be equal to 0 when the first basic block is above the current block (such as blocks A, AL and AR in fig. 23).

F) In one example, xLT-xLT 0 must be equal to 0 when the first basic block is to the left of the current block (such as blocks L, LB and AL in fig. 23).

G) How the second basic block is selected depends on the position of the first block.

I. For example, if xLT0% M is not equal to 0, yLT 1= yLT0 and xLT 1= xLT0-offset, where offset is a positive number such as4, 8 or 16, e.g., m=8 and offset=4.

For example, if xLT% M is equal to 0, yLT 1= yLT0 and yLT1 = yLT0+offset, where offset is a positive number such as 4, 8 or 16, e.g., m=8 and offset=4. For example, if xLT% M is equal to 0, yLT 1= yLT0 and xLT1 = xLT0+offset, where offset is a positive number such as 4, 8 or 16, e.g., m=8 and offset=4. For example, if xLT% M is not equal to 0, xLT 1= xLT0 and yLT1 = yLT0-offset, where offset is a positive number such as 4, 8 or 16, e.g., m=8 and offset=4.

V. for example, if xLT% M is equal to 0, xLT 1= xLT0 and yLT1 = yLT0+offset, where offset is a positive number such as 4, 8 or 16, e.g. m=8 and offset=4.

Fig. 23 shows an example of a pair of a first basic block and a second basic block: AL and AL ', a and a ', AR and AR ', L and L ', LB and LB '.

For example, if the first block is above left of the current block (such as block AL in fig. 23), yLT 1= yLT0 and xLT1 = xLT0+offset, where offset is a positive number such as 4, 8, or 16.

For example, if the first block is above right of the current block (such as block AR in fig. 23), yLT 1= yLT0 and xLT 1= xLT0-offset, where offset is a positive number such as4, 8, or 16.

For example, if the first block is above and to the left of the current block (such as block AL in fig. 23), and the left boundary of the current block is also the left boundary of the CTU, yLT = yLT and xLT1 = xLT +offset, where offset is a positive number such as 4, 8, or 16.

X. for example, if the first block is above right of the current block (such as block AR in fig. 23) and the right boundary of the current block is also the right boundary of the CTU, yLT = yLT and xLT1 = xLT0-offset, where offset is a positive number such as 4, 8 or 16.

For example, if xLT-offset < xLT _al, where xLT _al is the upper left coordinate of the neighboring base block above the left of the current block (such as block AL in fig. 23), yLT = yLT0 and xLT 1= xLT +offset, where offset is a positive number such as 4, 8, or 16.

For example, if xLT0+offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring base block (such as block AR in fig. 23) above the right of the current block, yLT 1= yLT0 and xLT 1= xLT0-offset, where offset is a positive number such as 4, 8, or 16.

For example, if xLT0-offset < xLT _al, where xLT _al is the upper left coordinate of the neighboring base block above the left of the current block (such as block AL in fig. 23), and the left boundary of the current block is also the left boundary of the CTU, yLT 1= yLT0 and xLT 1= xLT0+offset, where offset is a positive number such as 4, 8, or 16.

For example, if xLT0+offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring basic block (such as block AR in fig. 23) above the current block, and the right boundary of the current block is also the right boundary of the CTU, yLT 1= yLT0 and xLT 1= xLT0-offset, where offset is a positive number such as 4, 8, or 16.

H) The second basic block may be selected from a number of candidate basic blocks.

I. For example, the upper left position of the M candidate basic blocks is denoted as (xC ₀,yC₀)、(xC₁yC₁)、……、(xC_M-1,yC_M-1). The M candidate basic blocks are sequentially checked to find one candidate basic block that is inter-coded and has a reference picture that is the same as the reference picture to which the MV of the first basic block refers. The found candidate is selected as the second basic block.

1. In one example, m=2. yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT-0-offset, where offset is a positive number such as 4, 8 or 16.

2. In one example, m=2. xC ₀＝xLT0,yC₀＝yLT0–offset,xC₁＝xLT0,yC₁ = yLT0-offset, where offset is a positive number such as 4, 8 or 16.

3. In one example, m=2. yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT0+offset, where offset is a positive number such as 4, 8 or 16.

4. In one example, m=2. xC ₀＝xLT0,yC₀＝yLT0–offset,xC₁＝xLT0,yC₁ = yLT0+offset, where offset is a positive number such as 4, 8 or 16.

Whether and/or how to select the second basic block from the candidate basic blocks may depend on the position of the first basic block and/or the position of the current block.

1. For example, m=1. If the first block is above the left of the current block (such as block AL in fig. 23), yC ₀＝yLT0,xC₀ = xLT +offset, where offset is a positive number such as 4, 8, or 16.

A. For example, in other cases, m=2. yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT0+offset, where offset is a positive number such as 4, 8 or 16.

2. For example, m=1. If the first block is above the right of the current block (such as block AR in fig. 23), yC ₀＝yLT0,xC₀ = xLT0-offset, where offset is a positive number such as 4, 8, or 16.

3. For example, m=1. If the first block is above the left of the current block (such as block AL in fig. 23), and the left boundary of the current block is also the left boundary of the CTU, yC ₀＝yLT0,xC₀ = xLT +offset, where offset is a positive number such as 4, 8, or 16.

4. For example, m=1. If the first block is above and to the right of the current block (such as block AR in fig. 23), and the right boundary of the current block is also the right boundary of the CTU, yC ₀＝yLT0,xC₀ = xLT0-offset, where offset is a positive number such as 4, 8, or 16.

5. For example, m=1. If xLT0-offset < xLT _al, where xLT _al is the upper left coordinate of the neighboring base block above the left of the current block (such as block AL in fig. 23), yC ₀＝yLT0,xC₀ = xLT0+offset, where offset is a positive number such as 4, 8, or 16.

6. For example, m=1. If xLT +offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring base block (such as block AR in fig. 23) above the right of the current block, yC ₀＝yLT0,xC₀ = xLT0-offset, where offset is a positive number such as 4, 8, or 16.

7. For example, m=1. If xLT0-offset < xLT _al, where xLT _al is the upper left coordinate of the neighboring base block (e.g., block AL in fig. 23) above the left of the current block, and the left boundary of the current block is also the left boundary of the CTU, yC ₀＝yLT0,xC₀ = xLT0+offset, where offset is a positive number such as 4, 8, or 16.

8. For example, m=1. If xLT +offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring base block (such as block AR in fig. 23) above the right of the current block, and the right boundary of the current block is also the right boundary of the CTU, yC ₀＝yLT0,xC₀ = xLT0-offset, where offset is a positive number such as 4, 8, or 16.

I) Affine inheritance from the first basic block cannot be made if a second basic block cannot be found, which is inter-coded and has a reference picture that is the same as the reference picture to which the MV of the first basic block refers.

J) In one example, whether and how the method in the bullets is applied may depend on the location of the current block.

I. for example, the method in this bullets applies only when the affine model inherits from an adjacent block above and it is not in the current CTU or CTU row.

For example, the method in this bullets is only applied when the affine model inherits from an adjacent block above or to the left and it is not applied at the current CTU.

K) It is proposed to identify a first representative sub-block by deriving sub-blocks inherited by the affine model from which it was derived. In this case, the motion vector associated with one block from which the affine model is derived is used as (mv ^h _a,mv^v _a) in equation (3).

L) an embodiment as shown in fig. 24 is proposed.

I. If the current block inherits an affine model from a basic block BB (BB may be A, AR or AL, the top-left position of BB is (xBB, yBB), BB must be affine codec), then BB is considered to be the first neighboring basic block. And the following procedure is applicable to find the second neighboring basic block.

If xBB% 8= 0, the basic block BBR on the right of BB is checked first (BBR is AR if BB is A; BBR is AR' if BB is AR; BBR is AL if BB is AL); otherwise (xBB% 8 |=0), the basic block BBL to the left of BB is checked first (BBL is a ' if BB is a; BBL is a if BB is AR; BBL is AL ' if BB is AL ').

When checking the BBR first, if the BBR is affine codec and it has the same reference index as the BB of the given reference list, then the BBR is considered as the second adjacent neighboring basic block. Otherwise, the BBL will be considered as a second adjacent neighboring basic block.

When checking the BBL first, if the BBL is affine codec and it has the same reference index as the BB of a given reference list, then the BBL is considered as a second adjacent neighboring basic block. Otherwise, the BBR will be considered as a second adjacent neighboring basic block.

M) whether the second block is found from a plurality of candidates or from a predefined offset depends on the position of the first block and/or the position of the current block, for example.

I. An embodiment as shown in fig. 24 is presented.

1. If the current block inherits an affine model from a basic block BB (BB may be A, AR or AL, the top-left position of BB is (xBB, yBB), BB must be affine codec), then BB is considered to be the first neighboring basic block. And the following procedure is applicable to find the second neighboring basic block.

A. if BB is AL and the left boundary of the current block is the left boundary of the CTU, only the basic block BBR (AL ") to the right of BB is checked. If the BBR is affine-coded and it has the same reference index as BB of a given reference list, then the BBR is considered to be a second adjacent neighboring basic block. Otherwise, affine models inherited from BB are not available.

B. Otherwise, if xBB% s+=0, the basic block BBR to the right of BB is checked first (BBR is AR if BB is a; AR' if BB is AR; AL if BB is AL "); otherwise (xBB% S |=0), the basic block BBL to the left of BB (BBL is a 'if BB is a; BBL is a if BB is AR; BBL is AL' if BB is AL) is checked first. For example, S equals 8.

I. When checking the BBR first, if the BBR is affine codec and it has the same reference index as the BB of the given reference list, the BBR is considered as the second adjacent neighboring basic block. Otherwise, the BBL will be considered as a second adjacent neighboring basic block.

N) the offset between the first and second base units disclosed in this document is a positive integer.

I. In one example, the offset must be in the form of 2 ^K.

In one example, the offset may depend on the minimum allowed CU width.

In one example, the offset may depend on the minimum allowed CU height.

In one example, the offset may depend on the basic block width.

In one example, the offset may depend on the base block height.

In one example, the offset may depend on the minimum allowed width of the CU for which affine codec is applicable.

In one example, the offset may depend on the minimum allowed height of the CU for which affine codec is applicable.

In one example, the offset may be signaled from the encoder to the decoder.

O) if the basic block P is selected as the second block when the basic block Q is the first block, Q is not allowed to be selected as the second block when P is the first block.

I. in one example, P is not allowed to be the first block.

In one example, when P is the first block, if Q is to the right of P, then the second block can only be selected from the base units to the left of P.

In one example, when P is the first block, the second block can only be selected from the base units to the right of P if Q is to the left of P.

In one example, when P is the first block, the second block can only be selected from the base units above P if Q is below P.

In one example, when P is the first block, the second block can only be selected from the base units below P if Q is above P.

P) in one example, adjacent neighboring basic blocks may be on a row or column adjacent to the current block. For example, in fig. 23, AL ', AR ', LB ' may also be considered as neighboring blocks.

Q) in one example, a first basic block is considered "valid" if it satisfies one, several or all of the following conditions:

i. It is inter-frame codec;

it is not intra block copy codec;

it is affine codec;

it is affine mere codec;

v. it is affine inter-frame codec;

r) in one example, whether the second basic block is considered "valid" may depend on the information of the first basic block.

S) in one example, the second basic block is considered "valid" if it satisfies one, several or all of the following conditions:

i. It is inter-frame codec;

it is not intra block copy codec;

it is affine codec;

it is affine mere codec;

v. it is affine inter-frame codec;

it has the same inter prediction direction as the first basic block;

it has the same reference index of reference list 0 as the first basic block;

It has the same reference index of reference list 1 as the first basic block;

it has the same inter prediction direction and the same reference index as the first basic block;

x. it has the same Picture Order Count (POC) value of the reference pictures in reference list X (X is 0 and/or 1) as the first basic block;

t) in one example, each candidate above a neighboring basic block (such as AR, a, and AL 1in fig. 24) is checked to determine if it is the first basic block that is valid. For example, the order may be AR, a, AL1 or a, AR, AL1.

I. In one example, if one basic block BB (BB may be AR, a or AL 1) is checked and BB is a valid first basic block, then its left basic block and/or its right basic block is checked in order to find the corresponding second basic block.

1. In one example, the basic block BBR to the right of BB is checked first. Examples of detailed steps for determining the first basic block and the second basic block are given below:

-outputting BB and BBR as the first basic block and the second basic block if BBR is a valid second basic block;

-otherwise (BBR invalid), checking the basic block BBL to the left of BB;

i. outputting BB and BBL as the first basic block and the second basic block if BBL is the valid second basic block;

otherwise (BBL invalid), the next basic block is checked in order as the first basic block. In an alternative example, the valid first basic block and second basic block cannot be output.

2. Alternatively, the basic block BBL to the left of BB is checked first. Examples of detailed steps for determining the first basic block and the second basic block are given below:

-outputting BB and BBL as the first basic block and the second basic block if BBL is a valid second basic block;

-otherwise (BBL invalid), checking the basic block BBR to the right of BB;

i. Outputting BB and BBR as the first basic block and the second basic block if BBR is the valid second basic block;

otherwise (BBR invalid), the next basic block is checked in order to determine if it is the first basic block that is valid.

1. Alternatively, no valid first basic block and no valid second basic block may be output from above adjacent basic blocks.

In one example, if one basic block BB is checked (BB may be AR, A or AL 1) and BB is a valid first basic block, only the basic block BBL to the left thereof is checked to find the corresponding second basic block. Examples of detailed steps for determining the first basic block and the second basic block are given below:

-otherwise (BBL invalid), outputting BB and the basic block BBR to the right of BB as first basic block and second basic block;

a. Alternatively, BB is not used as the first basic block, and the next basic block is checked in order to determine whether it is a valid first basic block.

B. Alternatively, no valid first basic block and no valid second basic block may be output from above adjacent basic blocks.

Alternatively, if a basic block BB (BB may be AR, a or AL 1) is checked and BB is a valid first basic block, only the basic block BBR to the right thereof is checked to find the corresponding second basic block. Examples of detailed steps for determining the first basic block and the second basic block are given below:

-otherwise (BBR inactive), outputting BB and the basic block BBL to the left of BB as first basic block and second basic block;

Fig. 27 and 28 show two exemplary flowcharts of how the first basic block and the second basic block are selected.

In one example, which top neighboring basic block may be checked to determine that the first basic block may depend on the position of the current block and/or the sub-block size of the affine motion compensation.

1. For example, if the current block is at the left boundary of the CTU, the candidate basic blocks are AR, a, and AL in fig. 24. For example, if xPos% ctu_w= 0 (xPos 00 is the upper left coordinate of the current block, ctu_w is the width of the CTU), the current block is located at the left boundary of the CTU.

A. For example, if the current block is located at the left boundary of the CTU, the candidate basic blocks are AR, a, and AL "; otherwise, candidates are AR, a, and AL1.

Whether a basic block can be used as the first basic block may depend on the position of the current block.

1. For example, if the current block is located at the left boundary of the CTU, the basic block AL1 in fig. 24 cannot be used as the first basic block.

2. For example, if the current block is located at the left boundary of the CTU, the basic block AL' in fig. 24 cannot be used as the first basic block.

3. For example, if the current block is located at the right boundary of the CTU, the basic block AR in fig. 24 cannot be used as the first basic block. For example, if (xPos 00+w)% ctu_w= 0 (xPos is the upper left coordinate of the current block, W is the width of the current block, ctu_w is the width of the CTU), the current block is located at the right boundary of the CTU.

4. For example, if the current block is located at the right boundary of the CTU, the basic block AR' in fig. 24 cannot be used as the first basic block.

Whether a basic block can be used as the second basic block may depend on the position of the current block.

1. For example, if the current block is located at the left boundary of the CTU, the basic block AL1 in fig. 24 cannot be used as the second basic block.

2. For example, if the current block is located at the left boundary of the CTU, the basic block AL' in fig. 24 cannot be used as the second basic block.

3. For example, if the current block is located at the right boundary of the CTU, the basic block AR in fig. 24 cannot be used as the second basic block.

4. For example, if the current block is located at the right boundary of the CTU, the basic block AR' in fig. 24 cannot be used as the second basic block.

U) in one example, the first basic block and the second basic block may be exchanged.

I. in one example, the output first basic block and second basic block are exchanged first and then used to decode one block.

Alternatively, the determination of the first basic block and the second basic block described above may be exchanged. Fig. 23 shows an example of MVs of two adjacent neighboring blocks.

Extension of stored affine parameters

8. The stored affine parameters may be shifted before storage. Assuming m is to be stored (m may be a, b, c or d), then

A) For example, m' = SatShift (m, n). Store m' instead of m.

B) For example, m' =shift (m, n). Store m' instead of m.

C) For example, n is an integer such as 2 or 4;

i. In one example, n depends on the motion accuracy.

In one example, n may be different for different affine parameters.

I. in one example, n may be signaled in VPS/SPS/PPS// slice header/slice group header/slice/CTU/CU.

In another example, n may be different in different standard levels/hierarchies.

D) In one example, the stored affine parameters are first left shifted before being used for affine inheritance.

E) In one example, the stored m' is shifted first, then sliced before it is used for affine inheritance.

Extension to affine HMVP

9. In one example, CPMV in the first group as disclosed in bullets 1 and 2 may be stored in affine HMVP buffer/table/list to represent one history-based candidate affine model.

10. In one example, the difference between CPMV as disclosed in bulleted 3 may be stored in affine HMVP buffer/table/list to represent one history-based candidate affine model.

11. All methods presented herein, such as bullets 4 and 5, can be applied to affine HMVP buffers/tables/lists stored CPMV or differences between CPMV.

12. All the methods disclosed in bullets 6 can be applied when CPMV or the difference between CPMV is not from an adjacent block but from an affine history buffer/table/list.

More accurate MV position

13. It is proposed that the position of the MV for affine inheritance from a block should be aligned with the position of the MV for deriving the block.

A) In one example as shown in FIG. 20, the control point vector of the current CU is derived by using the 4-parameter model and byAnd

I. for example, offx=1 and offy= -1;

For example, offx=2 and offy= -2;

For example, offx=1 and offy= -2;

for example, offx=2 and offy= -1;

v. for example, offx=1 and offy= -3;

For example, offx=2 and offy= -3;

b) In one example as shown in FIG. 20, the control point vector of the current CU is derived by using the 4-parameter model and by And

I. for example, offx= -1 and offy= -1;

For example, offx= -2 and offy= -2;

For example, offx= -1 and offy= -2;

for example, offx= -2 and offy= -1;

v. for example, offx= -1 and offy= -3;

For example, offx= -2 and offy= -3;

5. Examples

In one embodiment, bullets 6 are applied to affine inheritance not at CTU row boundaries. An exemplary decoding process is specified as (section numbers herein refer to the current version of the VVC standard):

derivation of motion vector and reference index in 8.3.3.2 sub-block Merge mode

……

2. When sps_affine_enabled_flag equals 1, the derivation of the sample position (xNbA₀,yNbA₀)、(xNbA₁,yNbA₁)、(xNbA₂,yNbA₂)、(xNbB₀,yNbB₀)、(xNbB₁,yNbB₁)、(xNbB₂,yNbB₂)、(xNbB₃,yNbB₃) and variables numSbX and numSbY is as follows:

(xA₀,yA₀)＝(xCb-2,yCb+cbHeight+2) (8-309)

(xA₁,yA₁)＝(xCb-2,yCb+cbHeight-2) (8-310)

(xA₂,yA₂)＝(xCb-1,yCb) (8-311)

(xB₀,yB₀)＝(xCb+cbWidth+2,yCb-2) (8-312)

(xB₁,yB₁)＝(xCb+cbWidth-2,yCb-2) (8-313)

(xB₂,yB₂)＝(xCb-2,yCb-2) (8-314)

(xB₃,yB₃)＝(xCb,yCb-1) (8-315)

numSbX＝cbWidth>>2 (8-316)

numSbY＝cbHeight>>2 (8-317)

3. when sps_affine_enabled_flag is equal to 1, variable availableFlagA is set equal to FALSE, and the following applies to (xNbA _k,yNbA_k) from (xNbA ₀,yNbA₀) to (xNbA ₁,yNbA₁):

-invoking the availability derivation procedure for the block as specified in clause 6.4.X with the current luminance position (xCurr, yCurr) and the neighboring luminance position (xNbA _k,yNbA_k) set equal to (xCb, yCb) as inputs, and assigning an output to the block availability flag availableA _k.

When availableA _k is equal to TRUE and MotionModelIdc [ xNbA _k][yNbA_k ] is greater than 0 and availableFlagA is equal to FALSE, the following applies:

-variable availableFlagA is set equal to TRUE, motionModelIdcA to MotionModelIdc [ xNbA _k][yNbA_k ], (xNb, yNb) is set equal to (CbPosX [ xNbA _k][yNbA_k],CbPosY[xNbA_k][yNbA_k ]), nbW is set equal to CbWidth [ xNbA _k][yNbA_k ], nbH is set equal to CbHeight [ xNbA _k][yNbA_k ], and numCpMv is set equal to MotionModelIdc [ xNbA _k][yNbA_k ] +1.

For X replaced by 0 or 1, the following applies:

-when PredFlagLX [ xNbA _k][yNbA_k ] is equal to 1, taking as input the luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), neighboring luma codec block position (xNb, yNb), neighboring sub-block center position (xNbA _k,yNbA_k), neighboring luma codec block width and height (nbW, nbH) and the number numCpMv of control point motion vectors, taking as output the control point motion vector predictor candidate cpMvLXA [ cpIdx ] (where cpIdx = 0.. numCpMv-1), invoking the derivation procedure to derive luma affine control point motion vectors from neighboring blocks as specified in clause 8.3.3.5.

-The following allocation is made:

predFlagLXA＝PredFlagLX[xNbA_k][yNbA_k] (8-318)

refIdxLXA＝RefIdxLX[xNbAk][yNbAk] (8-319)

4. When sps_affine_enabled_flag is equal to 1, variable availableFlagB is set equal to FALSE, and the following applies to (xNbB _k,yNbB_k) from (xNbB ₀,yNbB₀) to (xNbB ₂,yNbB₂):

-invoking the availability derivation procedure for the block as specified in clause 6.4.X with the current luminance position (xCurr, yCurr) and the neighboring luminance position (xNbB _k,yNbB_k) set equal to (xCb, yCb) as inputs, and assigning an output to the block availability flag availableB _k.

When availableB _k is equal to TRUE and MotionModelIdc [ xNbB _k][yNbB_k ] is greater than 0 and availableFlagB is equal to FALSE, the following applies:

-variable availableFlagB is set equal to TRUE, motionModelIdcB is set equal to MotionModelIdc [ xNbB _k][yNbB_k ], (xNb, yNb) is set equal to (CbPosX [ xNbAB ] [ yNbB _k],CbPosY[xNbB_k][yNbB_k ]), nbW is set equal to CbWidth [ xNbB _k][yNbB_k ], nbH is set equal to CbHeight [ xNbB _k][yNbB_k ], and numCpMv is set equal to MotionModelIdc [ xNbB _k][yNbB_k ] +1.

For X replaced by 0 or 1, the following applies:

-when PredFlagLX [ xNbB _k][yNbB_k ] is equal to TRUE, taking as inputs the luma codec block position (xCb, yCb), the luma codec block width and height (cbWidth, cbHeight), the neighboring luma codec block position (xNb, yNb), the neighboring sub-block center position (xNbB _k,yNbB_k), the neighboring luma codec block width and height (nbW, nbH) and the number numCpMv of control point motion vectors, taking as outputs the control point motion vector predictor candidates cpMvLXB [ cpIdx ] (where cpIdx = 0.. numCpMv-1), invoking the derivation procedure to derive luma affine control point motion vectors from neighboring blocks as specified in clause 8.3.3.5.

-The following allocation is made:

8.3.3.5 derivation of luma affine control Point motion vector from neighboring blocks

The inputs of this process are:

A luminance location (xCb, yCb), specifying a top left sample of the current luma codec block relative to a top left luma sample of the current picture,

Two variables cbWidth and cbHeight, specifying the width and height of the current luma codec block,

A luminance location (xNb, yNb) specifying a left upsampling point of the neighboring luma codec block relative to the left upsampling point of the current picture,

A luminance position (xNbC, yNbC), specifying a center sample of an adjacent luminance coding sub-block relative to an upper left luminance sample of the current picture,

Two variables nNbW and nNbH, specifying the width and height of neighboring luma codec blocks,

The number of control point motion vectors numCpMv.

The output of this process is the luminance affine control point vector cpMvLX [ cpIdx ], where cpIdx =0.. numCpMv-1, and X is 0 or 1.

The variables isCTUboundary are derived as follows:

isCTUboundary is set equal to TRUE if all of the following conditions are TRUE:

- ((yNb + nNbH)% CtbSizeY) is equal to 0

-YNb + nNbH is equal to yCb

Otherwise isCTUboundary is set equal to FALSE.

The variables log2NbW and log2NbH are derived as follows:

log2NbW＝Log2(nNbW) (8-369)

log2NbH＝Log2(nNbH) (8-370)

The derivation of variables mvScaleHor, mvScaleVer, dHorX and dVerX is as follows:

-if isCTUboundary is equal to TRUE, the following applies:

mvScaleHor＝MvLX[xNb][yNb+nNbH-1][0]<<7 (8-371)

mvScaleVer＝MvLX[xNb][yNb+nNbH-1][1]<<7 (8-372)

dHorX＝(MvLX[xNb+nNbW-1][yNb+nNbH-1][0]-MvLX[xNb][yNb+nNbH-1][0])

<<(7-log2NbW) (8-373)

dVerX＝(MvLX[xNb+nNbW-1][yNb+nNbH-1][1]-MvLX[xNb][yNb+nNbH-1][1])

<<(7-log2NbW) (8-374)

Otherwise (isCTUboundary equals FALSE), the following applies:

mvScaleHor＝MvLX[xNbC][yNbC][0][0]<<7 (8-375)

mvScaleVer＝MvLX[xNbC][yNbC][0][1]<<7 (8-376)

dHorX＝(CpMvLX[xNb+nNbW-1][yNb][1][0]-CpMvLX[xNb][yNb][0][0])

<<(7-log2NbW) (8-377)

dVerX＝(CpMvLX[xNb+nNbW-1][yNb][1][1]-CpMvLX[xNb][yNb][0][1])

<<(7-log2NbW) (8-378)

variables dHorY and dVerY were derived as follows:

If isCTUboundary is equal to FALSE and MotionModelIdc [ xNb ] [ yNb ] is equal to 2, then the following applies:

dHorY＝(CpMvLX[xNb][yNb+nNbH-1][2][0]-CpMvLX[xNb][yNb][2][0])

<<(7-log2NbH) (8-379)

dVerY＝(CpMvLX[xNb][yNb+nNbH-1][2][1]-CpMvLX[xNb][yNb][2][1])

<<(7-log2NbH) (8-380)

Otherwise (isCTUboundary equals TRUE or MotionModelIdc [ xNb ] [ yNb ] equals 1), the following applies:

dHorY＝-dVerX (8-381)

dVerY＝dHorX (8-382)

The luminance affine control point motion vector cpMvLX [ cpIdx ] (where cpIdx =0.. numCpMv-1, and X is 0 or 1) is derived as follows:

-yNb is set equal to yCb when isCTUboundary is equal to TRUE. Then xNbC is set equal to xNb, yNbC is set equal to yNb.

The first two control point motion vectors cpMvLX [0] and cpMvLX [1] are derived as follows:

cpMvLX[0][0]＝(mvScaleHor+dHorX*(xCb-xNbC)+dHorY*(yCb-yNbC)) (8-383)

cpMvLX[0][1]＝(mvScaleVer+dVerX*(xCb-xNbC)+dVerY*(yCb-yNbC)) (8-384)

cpMvLX[1][0]＝(mvScaleHor+dHorX*(xCb+cbWidth-xNbC)+dHorY*(yCb-yNbC)) (8-385)

cpMvLX[1][1]＝(mvScaleVer+dVerX*(xCb+cbWidth-xNbC)+dVerY*(yCb-yNbC)) (8-386)

-if numCpMv is equal to 3, the third control point vector cpMvLX [2] is derived as follows:

cpMvLX[2][0]＝(mvScaleHor+dHorX*(xCb-xNbC)+dHorY*(yCb+cbHeight-yNbC)) (8-387)

cpMvLX[2][1]＝(mvScaleVer+dVerX*(xCb-xNbC)+dVerY*(yCb+cbHeight-yNbC)) (8-388)

-invoking a rounding procedure for a motion vector as specified in clause 8.3.2.12 with mvX set equal to cpMvLX [ cpIdx ], RIGHTSHIFT set equal to 7, and LEFTSHIFT set equal to 0 as inputs, and rounded cpMvLX [ cpIdx ] as output (where X is 0 or 1, and cpIdx =0.. numCpMv-1).

……

Derivation process of 8.3.3.7 brightness affine control point motion vector predicted value

The inputs of this process are:

……

1. the number of control point motion vector predictor candidates in list numCpMvpCandLX is set equal to 0.

2. Variables availableFlagA and availableFlagB are both set equal to FALSE.

3. Sample points (xNbA₀,yNbA₀)、(xNbA₁,yNbA₁)、(xNbA₂,yNbA₂)、(xNbB₀,yNbB₀)、(xNbB₁,yNbB₁) and (xNbB ₂,yNbB₂) are derived as follows:

(xA₀,yA₀)＝(xCb-2,yCb+cbHeight+2) (8-440)

(xA₁,yA₁)＝(xCb-2,yCb+cbHeight-2) (8-441)

(xB₀,yB₀)＝(xCb+cbWidth+2,yCb-2) (8-442)

(xB₁,yB₁)＝(xCb+cbWidth-2,yCb-2) (8-443)

(xB₂,yB₂)＝(xCb-2,yCb-2) (8-444)

4. The following applies to (xNbA _k,yNbA_k) from (xNbA ₀,yNbA₀) to (xNbA ₁,yNbA₁):

-invoking as input the availability derivation procedure for the block as specified in clause 6.4.X [ ed. (BB): neighbouring blocks availability checking process tbd ] with the current luminance position (xCurr, yCurr) and the neighboring luminance position (xNbA _k,yNbA_k) set equal to (xCb, yCb), and assigning an output to the block availability flag availableA _k.

-the variable (xNb, yNb) is set equal to (CbPosX [ xNbA _k][yNbA_k],CbPosY[xNbA_k][yNbA_k ]), nbW is set equal to CbWidth [ xNbA _k][yNbA_k ], and nbH is set equal to CbHeight [ xNbA _k][yNbA_k ].

If PredFlagLX [ xNbA _k][yNbA_k ] is equal to 1 and DiffPicOrderCnt (RefPicListX [ RefIdxLX [ xNbA _k][yNbA_k ]), refPicListX [ refIdxLX ]) is equal to 0, the following applies:

-variable availableFlagA is set to true

-Invoking a derivation procedure for deriving a luminance affine control point motion vector from an adjacent block as specified in clause 8.3.3.5 with a luminance codec block position (xCb, yCb), a luminance codec block width and height (cbWidth, cbHeight), an adjacent luminance codec block position (xNb, yNb), an adjacent luminance codec block width and height (nbW, nbH) and a number of control point motion vectors numCpMv as inputs, and with a control point motion vector predictor candidate cpMvpLX [ cpIdx ] (wherein cpIdx = 0.. numCpMv-1) as output.

-Invoking a rounding procedure for a motion vector as specified in clause 8.3.2.12 with mvX set equal to cpMvpLX [ cpIdx ], RIGHTSHIFT set equal to 2, and LEFTSHIFT set equal to 2 as inputs, and rounded cpMvpLX [ cpIdx ] as output (where cpIdx = 0.. numCpMv-1).

-The following allocation is made:

cpMvpListLX[numCpMvpCandLX][0]＝cpMvpLX[0] (8-445)

cpMvpListLX[numCpMvpCandLX][1]＝cpMvpLX[1] (8-446)

cpMvpListLX[numCpMvpCandLX][2]＝cpMvpLX[2] (8-447)

numCpMvpCandLX＝numCpMvpCandLX+1 (8-448)

Otherwise if PREDFLAGLY [ xNbA _k][yNbA_k ] (where y= | X) is equal to 1 and DiffPicOrderCnt (REFPICLISTY [ RefIdxLY [ xNbA _k][yNbA_k ] ], refPicListX [ refIdxLX ]) is equal to 0, then the following applies:

-variable availableFlagA is set to TRUE

-Invoking a derivation procedure of deriving a luma affine control point motion vector from an adjacent block as specified in clause 8.3.3.5 with the luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), adjacent luma codec block position (xNb, yNb), adjacent sub-block center position (xNbA _k,yNbA_k), adjacent luma codec block width and height (nbW, nbH) and the number of control point motion vectors numCpMv as inputs, with the control point motion vector predictor candidates cpMvpLY [ cpIdx ] (where cpIdx = 0.. numCpMv-1) as outputs.

-Invoking a rounding procedure for a motion vector as specified in clause 8.3.2.12 with mvX set equal to cpMvpLY [ cpIdx ], RIGHTSHIFT set equal to 2, and LEFTSHIFT set equal to 2 as inputs, and rounded cpMvpLY [ cpIdx ] as output (where cpIdx = 0.. numCpMv-1).

-The following allocation is made:

cpMvpListLX[numCpMvpCandLX][0]＝cpMvpLY[0] (8-449)

cpMvpListLX[numCpMvpCandLX][1]＝cpMvpLY[1] (8-450)

cpMvpListLX[numCpMvpCandLX][2]＝cpMvpLY[2] (8-451)

numCpMvpCandLX＝numCpMvpCandLX+1 (8-452)

5. The following applies to (xNbB _k,yNbB_k) from (xNbB ₀,yNbB₀) to (xNbB ₂,yNbB₂):

-invoking as input the availability derivation procedure for the block as specified in clause 6.4.X [ ed. (BB): neighbouring blocks availability checking process tbd ] with the current luminance position (xCurr, yCurr) and the neighboring luminance position (xNbB _k,yNbB_k) set equal to (xCb, yCb), and assigning an output to the block availability flag availableB _k.

-the variable (xNb, yNb) is set equal to (CbPosX [ xNbB _k][yNbB_k],CbPosY[xNbB_k][yNbB_k ]), nbW is set equal to CbWidth [ xNbB _k][yNbB_k ], and nbH is set equal to CbHeight [ xNbB _k][yNbB_k ].

If PredFlagLX [ xNbB _k][yNbB_k ] is equal to 1 and DiffPicOrderCnt (RefPicListX [ RefIdxLX [ xNbB _k][yNbB_k ]), refPicLi stX [ refIdxLX ]) is equal to 0, the following applies:

-variable availableFlagB is set to TRUE

-Invoking a derivation procedure of deriving a luma affine control point motion vector from an adjacent block as specified in clause 8.3.3.5 with the luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), adjacent luma codec block position (xNb, yNb), adjacent sub-block center position (xNbB _k,yNbB_k), adjacent luma codec block width and height (nbW, nbH) and the number of control point motion vectors numCpMv as inputs, with the control point motion vector predictor candidates cpMvpLX [ cpIdx ] (where cpIdx = 0.. numCpMv-1) as outputs.

-The following allocation is made:

cpMvpListLX[numCpMvpCandLX][0]＝cpMvpLX[0] (8-453)

cpMvpListLX[numCpMvpCandLX][1]＝cpMvpLX[1] (8-454)

cpMvpListLX[numCpMvpCandLX][2]＝cpMvpLX[2] (8-455)

numCpMvpCandLX＝numCpMvpCandLX+1 (8-456)

Otherwise, if PREDFLAGLY [ xNbB _k][yNbB_k ] (where y= | X) is equal to 1 and DiffPicOrderCnt (REFPICLISTY [ RefIdxLY [ xNbB _k][yNbB_k ] ], refPicListX [ refIdxLX ]) is equal to 0, then the following applies:

-variable availableFlagB is set to TRUE

-Invoking a derivation procedure for deriving a luma affine control point motion vector from an adjacent block as specified in clause 8.3.3.5 with as input luma codec block position (xCb, yCb), luma codec block width and height (cbWidth, cbHeight), adjacent luma codec block position (xNb, yNb), adjacent luma codec block width and height (nbW, nbH), and a number of control point motion vectors numCpMv, with as output a control point motion vector predictor candidate cpMvpLY [ cpIdx ] (where cpIdx = 0.. numCpMv-1).

-The following allocation is made:

cpMvpListLX[numCpMvpCandLX][0]＝cpMvpLY[0] (8-457)

cpMvpListLX[numCpMvpCandLX][1]＝cpMvpLY[1] (8-458)

cpMvpListLX[numCpMvpCandLX][2]＝cpMvpLY[2] (8-459)

numCpMvpCandLX＝numCpMvpCandLX+1 (8-460)

Fig. 26 is a block diagram of a video processing device 2600. The apparatus 2600 may be used to implement one or more methods described herein. The apparatus 2600 may be embodied in a smartphone, tablet, computer, internet of things (IoT) receiver, or the like. The apparatus 2600 may include one or more processors 2602, one or more memories 2604, and video processing hardware 2606. The processor(s) 2602 may be configured to implement one or more methods described in this document. Memory(s) 2604 may be used to store data and code for implementing the methods and techniques described herein. The video processing hardware 2606 may be used to implement some of the techniques described in this document in hardware circuitry.

Fig. 25 is a flow chart of an example method 2500 of video processing. The method 2500 includes: associating (2502) a first set of Motion Vectors (MVs) for determining inherited motion information of other video blocks, a second set of MVs for deriving MVs of sub-blocks of the current video block, and a third set of MVs included in a bitstream representation of the current video block with the current video block; and performing (2504) a conversion between the current video block and the bitstream representation using the first, second, or third set of MVs.

It should be appreciated that several techniques have been disclosed that would be beneficial to video encoder and decoder embodiments incorporated within video processing devices such as smartphones, laptops, desktops, and the like by allowing for a reduction in the amount of memory used to store CPMV during affine codec-based video codec and decoding. Various embodiments and techniques may be described using the following clause-based descriptions.

1. A method of video processing, comprising:

Associating a first set of Motion Vectors (MVs) for determining inherited motion information of other video blocks, a second set of MVs for deriving MVs for sub-blocks of the current video block, and a third set of MVs included in a bitstream representation of the current video block with the current video block; and

The conversion between the current video block and the bitstream representation is performed using the first set of MVs, the second set of MVs, or the third set of MVs.

2. The method of clause 1, wherein the first set of MVs includes Control Points MV (CPMV) MVF0, MVF1, and MVF2 located at representative points (xF 0, yF 0), (xF 1, yF 1), and (xF 2, yF 2), respectively.

3. The method of any of clauses 1-2, wherein the second set of MVs includes Control Points MV (CPMV) at representative points (xS 0, yS 0), (xS 1, yS 1), and (xS 2, yS 2), denoted as MVs0, MVs1, and MVs2, respectively.

4. The method according to any one of clauses 1-3, wherein the third set of MVs comprises Control Points MV (CPMV) located at representative points (xT 0, yT 0), (xT 1, yT 1) and (xT 2, yT 2), denoted MVT0, MVT1 and MVT2, respectively.

5. The method of any of clauses 1-4, wherein the second set of MVs is the same as the third set of MVs.

6. The method of any of clauses 1-5, wherein the first set of MVs is derived from the second set of MVs or the third set of MVs.

7. The method of any of clauses 1-2, wherein the relative offset between the representative points of the two control points MV in the first group is independent of the width or height of the current video block.

8. The method of clause 2, wherein:

yf1=yf0, xf1=xf0+pw, or (b)

Xf1=xf0, yf1=yf0+ph, or (b)

Yf2=yf0, xf2=xf0+pw, or (b)

Xf2=xf0, yf2=yf0+ph, or (b)

Yf2=yf1, xf2=xf1+pw, or (b)

Xf2=xf1, yf2=yf1+ph, or (b)

Where PW and PH are integers.

8A the method of any one of clauses 4-7, wherein

Mvf0=mvs0, (xF 0, yf0) = (xS 0, yS 0), or mvf0=mvt0, (xF 0, yf0) = (xT 0, yT 0); or alternatively

Mvf0=mvs1, (xF 0, yf0) = (xS 1, yS 1), or mvf0=mvt1, (xF 0, yf0) = (xT 1, yT 1); or alternatively

Mvf0=mvs2, (xF 0, yf0) = (xS 2, yS 2), or mvf0=mvt2, (xF 0, yf0) = (xT 2, yT 2).

9. The method of any of clauses 2-8, further comprising storing the difference d1=mvf1-mvf0.

10. The method of any of clauses 2-9, further comprising storing the difference d2=mvf2-mvf0.

11. The method of any of clauses 9-10, wherein storing comprises storing a bit-shifted version of the motion vector values.

12. The method of clause 11, wherein storing further comprises clipping the motion vector values prior to storing.

Other examples and embodiments of clauses 1-12 are described in section 4, e.g., clauses 1-3.

13. A method of video processing, comprising: conversion between the current block and the bitstream representation of the current block is performed using an affine inherited Motion Vector (MV) of the current block, wherein the affine inherited MV is derived from (1) MVs stored for neighboring basic blocks (denoted Badj) or (2) an affine history list.

14. The method of clause 13, wherein storing MVs for Badj comprises: l (left), a (upper), LB (lower left), AR (upper right), and AL (upper left), and wherein Badj is a 4 x 4 size block.

15. The method of any of clauses 13-14, wherein the MV at position (x, y) in the current block is calculated using the motion vector MVa of Badj at position (x 0, y 0), wherein (x 0, y 0) is one of: (a) Badj, or (b) a location on the outside or boundary of Badj.

16. The method of clause 15, wherein the position (x, y) is (1) in a sub-block of the current block or (2) at an angle of the current block.

17. The method of any of clauses 13-16, wherein the current block uses a 4-parameter affine model.

18. The method of any of clauses 13-16, wherein the current block uses a 6-parameter affine model.

Other examples and embodiments of clauses 13-18 are described in section 4, e.g., clauses 3-6.

19. A method of video processing, comprising: conversion between the current block and the bit stream representation of the current block is performed using an affine inherited Motion Vector (MV) of the current block, wherein the affine inherited MV is derived from a first MV stored in a first basic block adjacent to the current block and a second MV stored in a second basic block offset from the first building block.

20. The method of clause 19, wherein affine inherited MV is derived using linear weighting of x-differences and y-differences between motion vectors weighted with coefficients a and b, where a and b are derived from offsets.

21. The method of any of clauses 19-20, wherein the value of the offset is a function of the location of the first basic block.

22. The method of any of clauses 19-21, wherein the second basic block is selected from M candidate basic blocks by examining the M candidate basic blocks sequentially, where M is an integer.

23. The method of any of clauses 19-22, wherein the first and second basic building blocks are inter-coded and refer to the same reference picture.

24. The method of any of clauses 1-23, wherein the affine parameters of the current block are shifted prior to storage.

25. The method of clause 24, wherein the bit-shift amount varies (1) with the motion accuracy used during the conversion, or (2) is different for different affine parameters.

26. The method of clause 24, wherein the affine parameters are clipped prior to storage.

27. The method of any of clauses 1-12, wherein MVs in the first set of MVs are stored and used in a history-based motion vector predictor table of a history-based candidate affine model of the current block.

28. The method of clauses 9-10, wherein D1 and D2 are stored and used in a history-based motion vector predictor table of the history-based candidate affine model of the current block.

29. The method of clause 19, wherein the first basic block adjacent to the current block is one of a (top), AR (top right), or AL (top left), and wherein the top left sample of the first basic block is denoted as xBB.

30. The method of clause 29, wherein (xbb% 8 = 0), wherein the block to the right of the first basic block is affine encoded and has the same reference index as the reference index of the first basic block of the given reference list, and wherein the second basic block is the block to the right of the first basic block.

31. The method of clause 29, wherein (xbb% 8 = 0), wherein the block to the right of the first basic block is not affine-coded, or has a reference index of a given reference list that is different from the reference index of the first basic block, and wherein the second basic block is the block to the left of the first basic block.

32. The method of clause 29, wherein (xBB% 8 noteq0), wherein the block to the left of the first basic block is affine coded and has the same reference index as the first basic block of the given reference list, and wherein the second basic block is the block to the left of the first basic block.

33. The method of clause 29, wherein (xbb% 8 +.0), wherein the block to the left of the first basic block is not affine-coded or has a reference index of a given reference list that is different from the reference index of the first basic block, and wherein the second basic block is the block to the right of the first basic block.

34. The method of clause 19, wherein the first basic block adjacent to the current block is located in a row or column adjacent to the current block.

35. The method of clause 19, wherein the at least one affine inherited MV is aligned with the first MV or the second MV.

36. The method of clause 19, wherein the second basic block is selected depending on the position of the first basic block, wherein the position of the first basic block is at the upper left, upper right, left boundary or right boundary of the current block, and the corresponding offset is selected based on the position depending on the position.

37. The method of clause 19, wherein the second basic block is selected from the plurality of candidate blocks using a technique that depends on the location of the first basic block or the location of the current block.

38. The method of any of clauses 19-37, wherein the decision to find the second block from the plurality of candidates or from a predetermined offset is made based on the position of the first block or the position of the current block.

39. The method of clause 38, wherein in the case where the current block inherits the affine model from the basic block, then treating the basic block as a first neighboring basic block, and determining a second neighboring basic block according to a rule based on the position of the basic block.

40. The method of any of clauses 19-39, wherein, in the event that basic block P is selected as the second block and basic block Q is selected as the first block, then the method does not include using basic block P as the first block and basic block Q as the second block during the conversion of another video block.

41. The method of clause 41, wherein the other video block is in the same codec tree unit row or the same slice as the current video block.

Other features and examples of the techniques described in clauses 36-41 are provided in section 4, clauses 7 (g), 7 (h), 7 (m), 7 (n), and 7 (o).

42. The method of any of clauses 1-41, wherein the converting comprises generating a bitstream representation from pixel values of the current block.

43. The method of any of clauses 1-41, wherein the converting comprises generating pixel values of the current block from the bitstream representation.

44. A video encoder apparatus comprising a processor configured to implement the method of any one or more of clauses 1-43.

45. A video decoder apparatus comprising a processor configured to implement the method of any one or more of clauses 1-43.

46. A computer program product stored on a non-transitory computer readable medium, the computer program product comprising program code for performing the method according to any one of clauses 1 to 43.

Fig. 29 is a flow chart of a method 2900 of processing video. The method 2900 includes associating (2902) a first set of Control Point Motion Vectors (CPMV) for determining inherited motion information of a block that is encoded after a first block with a second set of CPMV for deriving MVs of a sub-block of the first block or a third set of CPMV signaled for the first block, wherein the first set of CPMV is different from the second set of CPMV or the third set of CPMV; determining (2904) inherited motion information of a second block that is encoded and decoded after the first block based on the first set of CPMV; conversion between the second block and the bitstream representation of the second block is performed (2906) by using inherited motion information.

In some examples, the first set of CPMV is derived from the second set of CPMV or the third set of CPMV.

In some examples, the method further comprises: a first set of CPMV is stored after the conversion of the first block.

In some examples, the second set of CPMV is the same as the third set of CPMV.

In some examples, the plurality of representative point coordinates of the first set of CPMV, the plurality of representative point coordinates of the second set of CPMV, and/or the plurality of representative point coordinates of the third set of CPMV are defined as coordinates relative to one block or sub-block used in the affine motion compensation process.

In some examples, the relative offset between the representative points of two CPMV in the first set of CPMV is independent of the width or height of the first block.

In some examples, the representative points of the first set of CPMV are inside the first block or outside the first block.

In some examples, yf1=yf0, xf1=xf0+pw, or xf1=xf0, yf1=yf0+ph, or yf2=yf0, xf2=xf0+pw, or xf2=xf0, yf2=yf0+ph, or yf2=yf1, xf2=xf1+pw, or xf2=xf1, yf2=yf1+ph, where (xF 0, yF 0), (xF 1, yF 1), (xF 2, yF 2) are coordinates of the plurality of representative points of the first group, and PW and PH are integers.

In some examples, pw=2m, or pw= -2M, or ph=2m, or ph= -2M, M is an integer in the range of 2 to 7.

In some examples, PW and PH are not stored.

In some examples, PW and PH are fixed.

In some examples, PW and PH are signaled in at least one of a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), a Picture Parameter Set (PPS), a slice header, a slice group header, a slice, or a CTU.

In some examples, PW and PH are different in different standard levels or hierarchies.

In some examples, PW and PH depend on a maximum Coding Unit (CU) size or/and a minimum CU size of a slice or picture.

In some examples, in the first group of CPMV, mvf0=mvs0, (xF 0, yf0) = (xS 0, yS 0), or mvf0=mvt0, (xF 0, yf0) = (xT 0, yT 0); or mvf0=mvs1, (xF 0, yf0) = (xS 1, yS 1), or mvf0=mvt1, (xF 0, yf0) = (xT 1, yT 1); or mvf0=mvs2, (xF 0, yf0) = (xS 2, yS 2), or mvf0=mvt2, (xF 0, yf0) = (xT 2, yT 2); wherein the motion vectors MVF0, MVF1, MVF2 are the CPMV of the first group of CPMV at the plurality of representative point coordinates (xF 0, yF 0), (xF 1, yF 1), (xF 2, yF 2), the motion vectors MVS0, MVS1, MVS2 are the CPMV of the second group of CPMV at the plurality of representative point coordinates (xS 0, yS 0), (xS 1, yS 1), (xS 2, yS 2), and the motion vectors MVT0, MVT1, MVT2 are the CPMV of the third group of CPMV at the plurality of representative point coordinates (xT 0, yT 0), (xT 1, yT 1), (xT 2, yT 2).

In some examples, motion vectors MVF0, MVF1, and MVF2 in the first group of CPMV are derived from motion vectors MVS0 and MVS1 in the second group of CPMV by using a 4-parameter affine model with coordinates (xF 0, yF 0), (xF 1, yF 1), and (xF 2, yF 2) as input coordinates for the affine model.

In some examples, motion vectors MVF0, MVF1, and MVF2 in the first group of CPMV are derived from motion vectors MVS0, MVS1, and MVS2 in the second group of CPMV by using a 6-parameter affine model with (xF 0, yF 0), (xF 1, yF 1), and (xF 2, yF 2) as input coordinates for the affine model.

In some examples, the motion vectors MVF0, MVF1, and MVF2 in the first group of CPMV are derived from the motion vectors MVT0 and MVT1 in the third group of CPMV by using a 4-parameter affine model with (xF 0, yF 0), (xF 1, yF 1), and (xF 2, yF 2) as input coordinates of the affine model.

In some examples, motion vectors MVF0, MVF1, and MVF2 in the first group of CPMV are derived from motion vectors MVT0, MVT1, and MVT2 in the third group of CPMV by using a 6-parameter affine model with (xF 0, yF 0), (xF 1, yF 1), and (xF 2, yF 2) as input coordinates of the affine model.

In some examples, motion vectors MVF2 in the first set of CPMV comprising motion vectors MVF0, MVF1, and MVF2 are calculated only when the first block is encoded with a 6-parameter affine model, or motion vectors MVF2 in the first set of CPMV comprising motion vectors MVF0, MVF1, and MVF2 are calculated regardless of whether the first block is encoded with a 4-parameter affine model or a 6-parameter affine model.

In some examples, the method further comprises: one or more differences (D1, D2) between CPMV in the first group of CPMV are stored.

In some examples, the first set of MVs includes motion vectors MVF0, MVF1, and MVF2, stores d1=mvf1-MVF 0, or stores d2=mvf2-MVF 0, or stores both D1 and D2.

In some examples, D2 is stored only when the first block is encoded with a 6-parameter affine model.

In some examples, D2 is stored when the first block is encoded with a 4-parameter affine model or a 6-parameter affine model.

In some examples, the method further comprises: one or more differences (D1, D2) between the first group of CPMV and the CPMV of the first group of CPMV are stored together.

In some examples, a shifting function is utilized to shift the plurality of CPMV's in the first group of CPMV's and/or one or more differences between CPMV's in the first group of CPMV's and store the shifted CPMV's and/or differences.

In some examples, the shift function SatShift (x, n) is defined as: where n is an integer and offset0 and/or offset1 is set to (1 < < n) > >1 or (1 < < (n-1)) or ((1 < < n) > > 1) -1, or offset0 and/or offset1 is set to 0.

In some examples, the shift function SatShift (x, n) is defined as: shift (x, n) = (x+offset0) > > n, where n is an integer, and offset0 is set to (1 < < n) > >1 or (1 < < (n-1)) or ((1 < < n) > > 1) -1, or offset0 is set to 0.

In some examples, n is2 or 4, or n depends on the motion accuracy.

In some examples, n in the case of storing CPMV in the first group of CPMV is different from n in the case of storing the difference between CPMV in the first group of CPMV.

In some examples, the stored CPMV moves left first before affine inheritance for blocks that are codec after the first block.

In some examples, the plurality of CPMV in the first group of CPMV and/or one or more differences between CPMV in the first group of CPMV are clipped with a clipping function and the clipped CPMV and/or differences are stored.

In some examples, clip function Clip3 (min, max, x) is defined as:

Where Min is the lower threshold of the clipping function and Max is the upper threshold of the clipping function.

In some examples, when CPMV or difference is stored with K bits, min= -2 ^K-1, and Max = 2 ^K-1 -1, K is an integer.

In some examples, K varies depending on whether CPMV or bad is to be stored.

In some examples, the plurality of CPMV in the first group of CPMV and/or the one or more differences between CPMV in the first group of CPMV are sequentially processed with a shifting function and a clipping function, and the processed CPMV and/or differences are stored.

In some examples, a plurality of CPMV's in the first set of CPMV's are stored in an affine (historical motion vector prediction) HMVP buffer or table or list to represent one history-based candidate affine model.

In some examples, one or more differences between CPMV's in the first set of CPMV's are stored into an affine (historical motion vector prediction) HMVP buffer or table or list to represent one history-based candidate affine model.

In some examples, one or more CPMV or one or more differences stored in a HMVP buffer or table or list are shifted with a shifting function and/or clipped with a clipping function.

Fig. 30 is a flow chart of a method 3000 of processing video. The method 3000 includes: deriving (3002) an affine inherited Motion Vector (MV) of the first block of video based on the stored Motion Vectors (MV) for conversion between the first block of video and the bit stream representation of the first block; the conversion is performed (3004) by using affine inherited MV.

In some examples, MVs are stored in adjacent neighboring base blocks.

In some examples, MVs are stored in an affine (historical motion vector prediction) HMVP buffer or table or list.

In some examples, MVs stored in adjacent neighboring base blocks include at least: the MVs stored in the adjacent base blocks (L) on the left, the MVs stored in the adjacent base blocks (A) on the upper side, the MVs stored in the adjacent base blocks (LB) on the lower left, the MVs stored in the adjacent base blocks (AR) on the upper right, and the MVs stored in the adjacent base blocks (AL) on the upper left.

In some examples, the neighboring basic blocks are 4 x 4 blocks.

In some examples, affine inherited MVs at positions (x, y) in the first block are derived by using the first mva= (MV ^h _a,mv^v _a)) of adjacent neighboring basic blocks at the representative point (x ₀,y₀) based on an affine model with (x-x ₀,y-y₀) as input coordinates of the affine model.

In some examples, the representative point (x ₀,y₀) is any location inside the basic block.

In some examples, the representative point (x ₀,y₀) is any location outside of the basic block or at a boundary.

In some examples, the coordinates of the representative point (x ₀,y₀) are determined based on the coordinates (xTL, yTL) of the top-left corner sample point in the adjacent neighboring base block and additional information including two variables (i, j).

In some examples, a first variable (i) of the two variables depends on the width of the basic block, and a second variable (j) of the two variables depends on the height of the basic block.

In some examples, the variable (i, j) depends on the location of the neighboring basic block.

In some examples, the coordinates of the representative point (x ₀,y₀) are determined based on the coordinates of the top left sample point of the first block (xPos, yPos 00), the coordinates of the top right sample point of the first block (xPos, yPos 00), and the coordinates of the bottom left sample point of the first block (xPos 00, yPos 01).

In some examples, the coordinates of the representative point (x ₀,y₀) of the adjacent base block L on the left are (xPos-2, yPos 01-1); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block (LB) adjacent to the lower left are (xPos 00-2, yPos01+3); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block (a) adjacent above are (xPos-1, ypos 00-2); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block (AR) adjacent to the upper right are (xPos +3, ypos 00-2); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block (AL) adjacent to the upper left are (xPos 00-2, yPos 00-2).

In some examples, the additional information depends on a location of or adjacent neighboring basic blocks or is signaled in at least one of a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), a Picture Parameter Set (PPS), a slice header, a slice group header, a slice, a Codec Tree Unit (CTU), a CU.

In some examples, the additional information is different in different standard levels or tiers.

In some examples, affine inherited MVs at positions (x, y) in sub-blocks of a first block are derived by using a first MV (MVa) of adjacent neighboring basic blocks at a representative point (x ₀,y₀) based on an affine model having (x-x ₀,y-y₀) as input coordinates of the affine model.

In some examples, affine inherited MVs at locations (x, y) in the sub-blocks are used to perform motion compensation on the sub-blocks.

In some examples, affine inherited MVs at positions (x, y) that are corners of the first block are derived as inherited Control Point Motion Vectors (CPMV) of the first block by using first MVs (MVa) of adjacent neighboring basic blocks at representative points (x ₀,y₀) based on an affine model with (x-x ₀,y-y₀) as input coordinates of the affine model.

In some examples, inherited CPMV is used to predict signaled CPMV of the first block of affine inter-frame codec.

In some examples, the inherited CPMV is directly used as the CPMV of the first block of the affine Merge codec.

In some examples, when the first block uses a 4-parameter affine model, the affine model used to derive affine inherited MVs at positions (x, y) in the first block is:

Where a and b are variables of the affine model.

In some examples, when the first block uses a 6-parameter affine model, the affine model used to derive affine inherited MVs at positions (x, y) in the first block is:

where a, b, c and d are variables of the affine model.

In some examples, variables a, b, or a, b, c, d are calculated as follows:

Wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1)、mv_t2＝(mv^h _t2,mv^v _t2) are CPMV at three representative points in a first set of CPMV covering a second block adjacent to the basic block, respectively, and w _t and h _t depend on a relative offset between the representative points of the second block, wherein the first set of CPMV is used to determine inherited motion information of a block encoded after the second block.

In some examples, variables a, b, or a, b, c, d are calculated as follows:

wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1)、mv_t2＝(mv^h _t2,mv^v _t2) is the CPMV at three representative points in a second or third group of CPMV covering a second block of an adjacent neighboring basic block, respectively, and w _t and h _t are the width and height of the second block, wherein the second group of CPMV is used to derive the MV of each sub-block of the second block, and the third group of CPMV is signaled from the encoder to the decoder.

In some examples, variables a, b, or a, b, c, d are calculated as follows:

Wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1)、mv_t2＝(mv^h _t2,mv^v _t2) are CPMV at three representative points, respectively, covering a second block of an adjacent neighboring basic block, and w _t and h _t are the width and height of the second block, wherein ,mv^h _t1-mv^h _t0、mv^v _t1-mv^v _t0、mv^h _t2-mv^h _t0、mv^v _t2-mv^v _t0 is taken directly from the storage means for storing the difference between the CPMV of the blocks.

In some examples, the conversion generates the first/second blocks of video from the bitstream representation.

In some examples, the conversion generates a bitstream representation from the first/second block of video.

Fig. 31 is a flow chart of a method 3100 of processing video. The method 3100 includes: deriving (3102) for conversion between a current block of video and a bitstream representation of the current block an affine inherited Motion Vector (MV) of the current block based on a first stored Motion Vector (MV) and a second stored MV different from the first stored MV, wherein the first stored MV is stored in a first basic block adjacent to the current block and the second stored MV is stored in a second basic block offset from the first basic block; and performing (3104) conversion by using affine inherited MVs of the current block.

In some examples, the first basic block adjacent to the current block includes at least one of: the adjacent basic block (L) on the left, the adjacent basic block (a) on the upper side, the adjacent basic block (LB) on the lower left, the adjacent basic block (AR, AR ') on the upper right, and the adjacent basic block (AL, AL', AL1, AL ") on the upper left.

In some examples, based on an affine model with parameters a and b, by using a first stored MV (mva= (MV ^h _a,mv^v _a)) stored in a first basic block at point (x ₀,y₀) or a first stored MV (mva= (MV ^h _a,mv^v _a)) associated with a sub-block of the first basic block at point (x ₀,y₀), an affine inherited MV ((MV ^h(x,y),mv^v (x, y)) at position (x, y) in the current block is derived, the motion vector ((MV ^h(x,y),mv^v (x, y)) is derived by:

In some examples, parameters a and b are calculated as follows:

Wherein MV _t0＝(mv^h _t0,mv^v _t0) and MV _t1＝(mv^h _t1,mv^v _t1) are set equal to the MV stored in the first basic block and the MV stored in the second basic block, respectively, and w _t is the horizontal offset between the first basic block and the second basic block.

In some examples, parameters a and b are calculated as follows:

Wherein MV _t0＝(mv^h _t0,mv^v _t0) and MV _t1＝(mv^h _t1,mv^v _t1) are set equal to the MV stored in the first basic block and the MV stored in the second basic block, respectively, and h _t is the vertical offset between the first basic block and the second basic block.

In some examples, w _t＝2^N and h _t＝2^M, where N and M are integers.

In some examples, when the coordinates of the left sample point of the first and second basic blocks are (xLT 0, yLT 0) and (xLT 1, yLT 1), respectively, the horizontal offset between the first and second basic blocks is defined as xLT1-xLT0, and/or the vertical offset between the first and second basic blocks is defined as yLT1-yLT0.

In some examples, the vertical offset is 0 when the first basic block is above the current block.

In some examples, the horizontal offset is 0 when the first basic block is to the left of the current block.

In some examples, the second basic block is selected depending on the position of the first basic block.

In some examples, the coordinates of the left-hand sample points of the first and second basic blocks are (xLT 0, yLT 0) and (xLT 1, yLT 1), respectively, and

Where M is an integer, yLT 1= yLT0 and xLT 1= xLT0-offset, or xLT 1= xLT0 and yLT 1= yLT0-offset, where offset is a positive number, when xLT% M is not equal to 0.

Where when xLT% M equals 0, M is an integer, yLT 1= yLT0 and xLT1 = xLT0+offset, or xLT 1= xLT0 and yLT 1= yLT0+offset, where offset is a positive number.

In some examples, the coordinates of the left-hand sample points of the first and second base blocks are (xLT, yLT 0) and (xLT, yLT 1), respectively, and when the first block is above-left of the current block, or when the first block is above-left of the current block and the left boundary of the current block is also the left boundary of the CTU, or when xLT-offset < xLT _al, wherein xLT _al is the upper-left coordinates of the adjacent base block above-left of the current block, or when xLT-offset < xLT _al, wherein xLT _al is the upper-left coordinates of the adjacent base block above-left of the current block, and the left boundary of the current block is also the left boundary of the CTU,

YLT 1= yLT0 and xLT 1= xLT0+offset, where offset is a positive number.

In some examples, the coordinates of the left-hand sample points of the first and second base blocks are (xLT 0, yLT 0) and (xLT 1, yLT 1), respectively, and when the first block is above-right of the current block, or when the first block is above-right of the current block and the right boundary of the current block is also the right boundary of the CTU, or when xLT0+offset > xrt_ar, where xLT _ar is the upper-right coordinate of the adjacent base block above-right of the current block, or when xLT0+offset > xrt_ar, where xLT _ar is the upper-right coordinate of the adjacent base block above-right of the current block, and the right boundary of the current block is also the right boundary of the CTU, yLT1 = yLT0 and xLT 1= xLT0-offset, where offset is a positive number.

In some examples, the second basic block is selected from M candidate basic blocks, M being an integer.

In some examples, the second basic block is selected as the second basic block by sequentially checking the M candidate basic blocks to determine one of the M candidate basic blocks that is inter-coded and has an MV that references the same reference picture as the MV of the first basic block.

In some examples, the coordinates of the top left sample point of the first basic block are (xLT 0, yLT 0) and the coordinates of the top left positions of the M candidate basic blocks are (xC ₀,yC₀)、(xC₁ yC₁)、…、(xC_M-1,yC_M-1), respectively, and when M is 2, yC ₀＝yLT0,xC₀ = xLT0-offset, and yC ₁＝yLT0,xC₁ = xLT0+offset, or xC ₀＝xLT0,yC₀ = yLT0-offset, and xC ₁＝xLT0,yC₁ = yLT0+offset, or yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT0+offset, or xC ₀＝xLT0,yC₀＝yLT0–offset,xC₁＝xLT0,yC₁ = yLT0+offset, wherein offset is a positive number.

In some examples, whether and/or how to select the second basic block from the M candidate basic blocks depends on the position of the first basic block and/or the position of the current block.

In some examples, the coordinates of the top left sample point of the first basic block are (xLT 0, yLT 0), and the coordinates of the top left positions of the M candidate basic blocks are (xC ₀,yC₀)、(xC₁ yC₁)、…、(xC_M-1,yC_M-1), respectively, and when M is 1, yC ₀＝yLT0,xC₀ = xLT0+offset if the first basic block is above left of the current block, or yC ₀＝yLT0,xC₀ = xLT0-offset if the first block is above right of the current block, or yC ₀＝yLT0,xC₀ = xLT +offset if the first block is above right of the current block, or yc7232= xLT-offset if the right boundary of the current block is also right of the CTU, or yC xLT-offset < xLT _al, where xLT _al is the upper left coordinate of the neighboring base block to the upper left of the current block, yC ₀＝yLT0,xC₀ = xLT0+offset, or if xLT0+offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring base block to the upper right of the current block, yC ₀＝yLT0,xC₀ =3932 0-offset, or if xLT0-offset < 3932_al, where 3932_al is the upper left coordinate of the neighboring base block to the upper left of the current block, and the left boundary of the current block is also the left boundary of the CTU, yc3932=3932 0+offset, or if xLT0+offset > xrt_ar, where xrt_ar is the upper right coordinate of the neighboring base block to the upper right of the current block, and the right boundary of the current block is also the right boundary of the CTU, yc3932=3932 0-offset, where the offset is a positive number.

In some examples, the coordinates of the top left point of the first base block are (xLT, yLT 0) and the coordinates of the top left position of the M candidate base blocks are (xC ₀,yC₀)、(xC₁ yC₁)、…、(xC_M-1,yC_M-1) respectively, and when M is 2, yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT0+offset if the first base block is top left of the current block, or yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT +offset if the first base block is top right of the current block, or yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ +offset if the left boundary of the current block is also top left of the CTU, or yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ +offset if the first base block is top right of the current block, and the right boundary of the current block is also the right boundary of the CTU, or yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ = xLT0+offset if xLT-offset xLT _al is top left of the current block, or wherein yC ₀＝yLT0,xC₀＝xLT0–offset,yC₁＝yLT0,xC₁ +0+75+offset is top left of the current block, or xLT +3+2 if the left boundary of the first base block is also top right of the current block, or the left boundary of the front is also top of the current block, and the left boundary of the front is equal to 3930+3930+39362+3, and the left boundary of the front is also the top 2+3+2+3+2 if the left boundary of the first base block is top right of the current block.

In some examples, when a second basic block of the inter-frame codec cannot be found and has a reference picture that is referenced to the same MV as that of the first basic block, an affine inherited Motion Vector (MV) of the current block cannot be derived from the first basic block.

In some examples, whether and/or how to derive an affine inherited Motion Vector (MV) of the current block depends on the location of the current block.

In some examples, affine models for a current block are inherited from neighboring blocks above the current block differently depending on whether neighboring blocks in the current block are located in a Coding Tree Unit (CTU) or CTU row in which the current block is located.

In some examples, when inheriting an affine model of a current block from a neighboring block located above or to the left of the current block that is not located in a CTU or CTU row in which the current block is located, an affine inherited Motion Vector (MV) of the current block is derived based on the first stored Motion Vector (MV) and the second stored MV.

In some examples, whether and/or how to select the second basic block from the plurality of candidates or according to a predefined offset depends on the location of the first block and/or the location of the current block, and wherein the second basic block is a second neighboring basic block.

In some examples, the affine model of the current block is inherited from a first basic block adjacent to the current block, the first basic block being affine codec and comprising at least one of: an upper adjacent basic block (a), an upper right adjacent basic block (AR), and an upper left adjacent basic block (AL), and wherein the upper left position of the first basic block is (xBB, yBB).

In some examples, when xBB% 8=0, the second neighboring basic block is selected by:

Checking whether a basic block to the right of the first basic block is affine-coded and has the same reference index as the first basic block of a given reference list; and if so, selecting the basic block on the right of the first basic block as a second adjacent basic block, otherwise, selecting the basic block on the left of the first basic block as the second adjacent basic block.

In some examples, when xBB% 8-! When=0, the second neighboring basic block is selected by:

Checking whether a basic block to the left of the first basic block is affine-coded and has the same reference index as the first basic block of a given reference list; and if so, selecting the block to the left of the first basic block as a second adjacent basic block, otherwise, selecting the block to the right of the first basic block as the second adjacent basic block.

In some examples, when the first basic block is AL and the left boundary of the current block is the left boundary of the CTU, the second neighboring basic block is selected by: checking only whether the basic block to the right of the first basic block is affine-coded and has the same reference index as the first basic block of a given reference list; and if so, selecting the basic block to the right of the first basic block as a second adjacent basic block, otherwise, the affine model of the current block cannot be inherited from the first basic block.

In some examples, when xBB% s=0, the second neighboring basic block is selected by: checking whether a basic block to the right of the first basic block is affine-coded and has the same reference index as the first basic block of a given reference list; and if so, selecting the basic block on the right of the first basic block as a second adjacent basic block, otherwise, selecting the basic block on the left of the first basic block as the second adjacent basic block.

In some examples, when xBB% S-! When=0, the second neighboring basic block is selected by: checking whether a basic block to the left of the first basic block is affine-coded and has the same reference index as the first basic block of a given reference list; and if so, selecting the block to the left of the first basic block as a second adjacent basic block, otherwise, selecting the block to the right of the first basic block as the second adjacent basic block.

In some examples, S is equal to 8.

In some examples, the offset between the first basic block and the second basic block is a positive integer.

In some examples, the offset is in the form of 2 ^K, K is an integer, or depends on the minimum allowed CU width and/or height, or on the width and/or height of the basic block, or on the minimum allowed width and/or height of the CU for which affine codec is applicable, or signaled from the encoder to the decoder.

In some examples, when basic block P is selected as the second basic block in the first affine inheritance process and when basic block Q is the first basic block, basic block Q is not allowed to be selected as the second basic block when basic block P is the first basic block in the second affine inheritance process.

In some examples, when basic block P is selected as the second basic block in the first affine inheritance process and when basic block Q is the first basic block, basic block P is not allowed as the first basic block in the second affine inheritance process.

In some examples, when the basic block P is the first block in the second affine inheritance process, the second basic block in the second affine inheritance process can only be selected from basic blocks to the left of the basic block P if the basic block Q is to the right of the basic block P, or can only be selected from basic blocks to the right of the basic block P if the basic block Q is to the left of the basic block P, or can only be selected from basic blocks above the basic block P if the basic block Q is below the basic block P, or can only be selected from basic blocks below the basic block P if the basic block Q is above the basic block P.

In some examples, the neighboring basic block is on a row or column adjacent to the current block.

In some examples, the first basic block is determined to be valid if the first basic block satisfies at least one of the following conditions: i. it is inter-frame codec; it is not intra block copy codec; it is affine codec; it is affine mere codec; it is affine inter-frame codec.

In some examples, whether the second basic block is determined to be valid depends on the information of the first basic block.

In some examples, the second basic block is determined to be valid if the second basic block satisfies at least one of the following conditions: i. it is inter-frame codec; it is not intra block copy codec; it is affine codec; it is affine mere codec; v. it is affine inter-frame codec; it has the same inter prediction direction as the first basic block; it has the same reference index of reference list 0 as the first basic block; it has the same reference index of reference list 1 as the first basic block; it has the same inter prediction direction and the same reference index as the first basic block; x. it has the Picture Order Count (POC) value of the same reference picture as the first basic block in reference list X, where X is 0 and/or 1.

In some examples, each basic block above a current block that is adjacent to the basic block is checked in a predetermined order to determine whether it is a valid first basic block.

In some examples, the top adjacent basic block of the current block is checked in left to right or right to left order.

In some examples, the upper adjacent basic block includes at least one of: an upper adjacent basic block (a), an upper right adjacent basic block (AR), and a first upper left adjacent basic block (AL 1).

In some examples, when one of the upper neighboring Basic Blocks (BB) is a valid first basic block, the basic blocks to the left and/or right of the valid first Basic Block (BB) are checked to determine a corresponding second basic block.

In some examples, the Base Block (BBR) to the right of the valid first Base Block (BB) is first checked to determine whether it is a valid second base block, and when the Base Block (BBR) is a valid second base block, the valid first Base Block (BB) and the valid second Base Block (BBR) are output as the first base block and the second base block.

In some examples, when a Basic Block (BBR) is invalid, a Basic Block (BBL) to the left of a valid first Basic Block (BB) is checked to determine whether it is a valid second basic block, and when the Basic Block (BBL) is a valid second basic block, the valid first Basic Block (BB) and the valid second Basic Block (BBL) are output as the first basic block and the second basic block.

In some examples, when a Basic Block (BBL) is invalid, the next basic block to the left of one Basic Block (BB) is checked in order to determine if it is the valid first basic block.

In some examples, when a Basic Block (BBL) is invalid, no valid first basic block and no valid second basic block are output from an upper neighboring basic block.

In some examples, the Base Block (BBL) to the left of the valid first Base Block (BB) is first checked to determine whether it is a valid second base block, and when the Base Block (BBL) is a valid second base block, the valid first Base Block (BB) and the valid second Base Block (BBL) are output as the first base block and the second base block.

In some examples, when a Basic Block (BBL) is invalid, a Basic Block (BBR) to the right of a valid first Basic Block (BB) is checked to determine whether it is a valid second basic block, and when the Basic Block (BBR) is a valid second basic block, the valid first Basic Block (BB) and the valid second Basic Block (BBR) are output as the first basic block and the second basic block.

In some examples, when a Basic Block (BBR) is invalid, the next basic block to the right of one Basic Block (BB) is checked in order to determine if it is the valid first basic block.

In some examples, when a Basic Block (BBR) is invalid, no valid first basic block and no valid second basic block are output from the upper neighboring basic blocks.

In some examples, only the Base Block (BBR) to the right of the valid first Base Block (BB) is checked to determine whether it is a valid second base block, and when the Base Block (BBR) is a valid second base block, the valid first Base Block (BB) and the valid second Base Block (BBR) are output as the first base block and the second base block.

In some examples, when a Basic Block (BBR) is invalid, a valid first Basic Block (BB) and a Basic Block (BBL) to the left of the valid first basic block are output as the first basic block and the second basic block.

In some examples, only the Base Block (BBL) to the left of the valid first Base Block (BB) is checked to determine whether it is a valid second base block, and when the Base Block (BBL) is a valid second base block, the valid first Base Block (BB) and the valid second Base Block (BBL) are output as the first base block and the second base block.

In some examples, when the Basic Block (BBL) is invalid, a valid first Basic Block (BB) and a Basic Block (BBR) to the right of the valid first basic block are output as the first basic block and the second basic block.

In some examples, the selection of the candidate basic block to be checked for determining the first basic block from the top neighboring basic blocks depends on the position of the current block and/or on the sub-block size of the affine motion compensation.

In some examples, when the current block is located at the left boundary of the CTU, the candidate basic block includes at least one of: an upper adjacent basic block (a), an upper right adjacent basic block (AR), and a first upper left adjacent basic block (AL 1), and when the current block is not located at the left boundary of the CTU, the candidate basic block includes at least one of: an upper adjacent basic block (a), an upper right adjacent basic block (AR) and a second upper left adjacent basic block (AL ").

In some examples, whether a candidate basic block from an upper neighboring basic block can be used as the first basic block depends on the location of the current block.

In some examples, when the current block is located at the left boundary of the CTU, the upper left neighboring basic blocks (AL 1, AL ") cannot be used as the first basic block.

In some examples, when the current block is located at the right boundary of the CTU, an upper right neighboring basic block (AR, AR') cannot be used as the first basic block.

In some examples, whether a candidate basic block from an upper neighboring basic block can be used as the second basic block depends on the location of the current block.

In some examples, when the current block is located at the left boundary of the CTU, the upper left neighboring basic block (AL 1, AL') cannot be used as the second basic block.

In some examples, when the current block is located at the right boundary of the CTU, the upper right neighboring basic block (AR, AR') cannot be used as the second basic block.

In some examples, the first basic block and the second basic block are exchangeable.

In some examples, the first basic block and the second basic block are exchanged first, and the conversion of the current block is performed by using the exchanged first basic block and second basic block.

In some examples, the determination of the first basic block and the second basic block is exchangeable.

In some examples, the conversion generates the current video block from the bitstream representation.

In some examples, the conversion generates a bitstream representation from the current video block.

Fig. 32 is a flow chart of a method 3200 of processing video. The method 3200 includes: deriving (3202) one or more parameters of an affine model parameter set associated with an affine model of a current block for a transition between the current block and a bitstream representation of the current block of the video; shifting (3204) one or more parameters; and storing (3206) the shifted one or more parameters.

In some examples, shifting the one or more parameters further includes shifting the one or more parameters with a first shift function SatShift (x, n), the first shift function being defined as:

Where x is one of the one or more parameters, n is an integer, and offset0 and/or offset1 is set to (1 < < n) >1 or (1 < < (n-1)) or ((1 < < n) > 1) -1, or offset0 and/or offset1 is set to 0.

In some examples, shifting the one or more parameters further includes shifting the one or more parameters with a second Shift function Shift (x, n), the second Shift function Shift (x, n) being defined as:

Shift(x,n)＝(x+offset0)>>n，

Where x is one of the one or more parameters, n is an integer, and offset0 is set to (1 < < n) > >1 or (1 < < (n-1)) or ((1 < < n) > > 1) -1, or offset0 is set to 0.

In some examples, n is 2 or 4.

In some examples, n depends on the motion accuracy, or n is different for different parameters in the affine model parameter set.

In some examples, n is signaled in at least one of a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), a Picture Parameter Set (PPS), a slice header, a slice group header, a slice, a Codec Tree Unit (CTU), a Codec Unit (CU).

In some examples, n is different in different standard levels or tiers.

In some examples, the stored parameters are first left shifted before affine inheritance of the block for encoding and decoding after the current block.

In some examples, the stored parameters are sequentially shifted with a shift function and clipped with a clip function prior to affine inheritance of the block for encoding and decoding after the current block.

In some examples, parameter a of the affine model parameter set passesIs calculated, where mv ^h ₀ is the horizontal motion vector component of the upper left corner control point of the current block, mv ^h ₁ is the horizontal motion vector component of the upper right corner control point of the current block, and w is the width of the current block.

In some examples, parameter b of the affine model parameter set passesIs calculated, where mv ^v ₀ is the vertical motion vector component of the upper left corner control point of the current block, mv ^v ₁ is the vertical motion vector component of the upper right corner control point of the current block, and w is the width of the current block.

In some examples, parameter c of the affine model parameter set passesIs calculated, where mv ^h ₀ is the horizontal motion vector component of the upper left corner control point of the current block, mv ^h ₂ is the horizontal motion vector component of the lower left corner control point of the current block, and h is the height of the current block.

In some examples, parameter d of the affine model parameter set passesIs calculated, where mv ^v ₀ is the vertical motion vector component of the upper left corner control point of the current block, mv ^v ₂ is the vertical motion vector component of the lower left corner control point of the current block, and h is the height of the current block.

In some examples, parameter e of the affine model parameter set passesIs calculated where mv ^h ₀ is the horizontal motion vector component of the upper left corner control point of the current block.

In some examples, parameter f of the affine model parameter set passesIs calculated where mv ^v ₀ is the vertical motion vector component of the upper left corner control point of the current block.

In some examples, parameters e and f of the affine model parameter set are calculated by (e, f) = (mv _xi,mv_yi), where (mv _xi,mv_yi) is the motion vector of any point.

In some examples, the width and height of the current block are noted as w and h, w and h being equal to 2 ^WB and 2 ^HB, where WB and HB are integers greater than 1.

In some examples, parameter a of the affine model parameter set passesWhere P is an integer and represents the calculation accuracy, mv ^h ₀ is the horizontal motion vector component of the upper left corner control point of the current block, and mv ^h ₁ is the horizontal motion vector component of the upper right corner control point of the current block.

In some examples, parameter b of the affine model parameter set passesWhere P is an integer and represents the calculation accuracy, mv ^v ₀ is the vertical motion vector component of the upper left corner control point of the current block, and mv ^v ₁ is the vertical motion vector component of the upper right corner control point of the current block.

In some examples, parameter c of the affine model parameter set passesWhere P is an integer and represents the calculation accuracy, mv ^h ₀ is the horizontal motion vector component of the upper left corner control point of the current block, and mv ^h ₂ is the horizontal motion vector component of the lower left corner control point of the current block.

In some examples, parameter d of the affine model parameter set passesWhere P is an integer and represents the calculation accuracy, mv ^v ₀ is the vertical motion vector component of the upper left corner control point of the current block, and mv ^v ₂ is the vertical motion vector component of the lower left corner control point of the current block.

In some examples, P is set to 7.

In some examples, the method further comprises: the one or more parameters are clipped before the one or more parameters are stored.

In some examples, if K bits are used to store one parameter X of the one or more parameters, x=clip 3 (-2 ^K-1,2^K-1 -1, X), where x=a or b or c or d, and K is an integer greater than 1.

In some examples, X is a, b, c, d, e or f.

In some examples, K is equal to 8.

In some examples, the affine model parameter set includes 6 variables (a, b, c, d, e, f) corresponding to a 6-parameter affine model given by the following formula

Where mv ^h (x, y) is a horizontal component of the motion vector of the current block, mv ^v (x, y) is a vertical component of the motion vector of the current block, and (x, y) represents coordinates of the representative point with respect to the left-hand sample point within the current block; (mv ^h ₀,mv^v ₀) is a motion vector of the upper left corner Control Point (CP) of the current block, and (mv ^h ₁,mv^v ₁) is a motion vector of the upper right corner control point of the current block, and (mv ^h ₂,mv^v ₂) is a motion vector of the lower left corner control point of the current block.

In some examples, the one or more parameters include a, b, c, and d.

In some examples, the affine model parameter set includes 4 variables (a, b, e, f) corresponding to a 4-parameter affine model given by the following formula

Where mv ^h (x, y) is a horizontal component of the motion vector of the current block, mv ^v (x, y) is a vertical component of the motion vector of the current block, and (x, y) represents coordinates of the representative point with respect to the left-hand sample point within the current block; (mv ^h ₀,m^hv ₀) is a motion vector of the upper left corner Control Point (CP) of the current block, and (mv ^h ₁,mv^v ₁) is a motion vector of the upper right corner control point of the current block.

In some examples, the one or more parameters include a and b.

In some examples, the one or more parameters include a, b, e, and f.

In some examples, the one or more parameters include a, b, c, d, e and f, and wherein the c= -b and d = a are constrained when the conversion between the current block and the bitstream representation of the current block is performed with a 4-parameter affine model.

In some examples, the one or more parameters include a, b, c, and d, and wherein c= -b and d = a are restricted when the conversion between the current block and the bitstream representation of the current block is performed with a 4-parameter affine pattern.

In some examples, the parameter c= -b when performing the conversion between the current block and the bitstream representation of the current block with a 4-parameter affine pattern.

In some examples, parameter d=a when the conversion between the current block and the bitstream representation of the current block is performed with a 4-parameter affine pattern.

In some examples, the method further comprises: based on the stored shifted one or more parameters, a transition between the bitstream representations of the block encoded after the current block and the block encoded after the current block is performed.

In some examples, the method further comprises: based on the stored one or more parameters, a transition between the current block and the bitstream representation of the current block is performed.

In some examples, the conversion generates blocks from the bitstream representation that are encoded after the current block.

In some examples, the conversion generates a bitstream representation from a block that is encoded and decoded after the current block.

Those skilled in the art will appreciate that techniques are disclosed for using motion candidate lists in various video codec scenarios. The video blocks may be encoded into a bitstream representation comprising non-contiguous bits placed in various headers or network adaptation layers, etc.

Implementations of the disclosed and other solutions, examples, embodiments, modules, and functional operations described in this document may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this document and their structural equivalents, or in combinations of one or more of them. The disclosed and other embodiments may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a combination of materials affecting a machine-readable propagated signal, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. In addition to hardware, an apparatus may include code that creates a runtime environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.

A computer program (also known as a program, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. The computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

The processes and logic flows described in this document can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (Field Programmable GATE ARRAY ) or an ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks. However, a computer does not require such a device. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disk; CD ROM and DVD-ROM discs. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

Although this patent document contains many specifics, these should not be construed as limitations on the scope of any subject matter or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular technologies. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

Similarly, although operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

Only some embodiments and examples are described and other embodiments, enhancements, and variations may be made based on what is described and shown in this patent document.

Claims

1. A method of video processing, comprising:

associating a first set of control point motion vectors, CPMV, for determining inherited motion information of a block encoded after a first block with a second set of CPMV for deriving MVs of a sub-block of the first block or a third set of CPMV signaled for the first block, wherein the first set of CPMV is different from the second set of CPMV or the third set of CPMV;

determining inherited motion information of a second block encoded and decoded after the first block based on the first group of CPMV; and

By using the inherited motion information to perform a conversion between the second block and the bit stream representation of the second block,

Wherein the first set of CPMVs is derived from the second set of CPMVs or the third set of CPMVs.

2. The method of claim 1, further comprising:

a first set of CPMV is stored after the conversion of the first block.

3. The method of any of claims 1-2, wherein the second set of CPMV is the same as the third set of CPMV.

4. The method of any of claims 1-2, wherein the plurality of representative point coordinates of the first set of CPMV, the plurality of representative point coordinates of the second set of CPMV, and/or the plurality of representative point coordinates of the third set of CPMV are defined as coordinates relative to a block or sub-block used in the affine motion compensation process.

5. The method of any of claims 1-2, wherein a relative offset between representative points of two CPMV in the first set of CPMV is independent of a width or height of the first block.

6. The method of any of claims 1-2, wherein the representative points of the first set of CPMV are inside the first block or outside the first block.

7. The method of any one of claims 1-2, wherein:

a) y ^F1＝y^F0,x^F1＝x^F +PW, or

B) x ^F1＝x^F0,y^F1＝y^F 0+PH, or

C) y ^F2＝y^F0,x^F2＝x^F +PW, or

D) x ^F2＝x^F0,y^F2＝y^F 0+PH, or

E) y ^F2＝y^F1,x^F2＝x^F 1+PW, or

f)x^F2＝x^F1,y^F2＝y^F1+PH，

Where (x ^F0,y^F0)、(x^F1,y^F1)、(x^F2,y^F 2) is the coordinates of multiple representative points of the first set of CPMVs, and PW and PH are integers.

8. The method of claim 7, wherein PW = 2 ^M, or pw= -2 ^M, or PH = 2 ^M, or ph= -2 ^M, M is an integer in the range of 2 to 7.

9. The method of claim 7, wherein PW and PH are not stored.

10. The method of claim 9, wherein PW and PH are fixed.

11. The method of claim 9, wherein PW and PH are signaled in at least one of a sequence parameter set SPS, a video parameter set VPS, a picture parameter set PPS, a slice header, a slice group header, a slice, or a CTU.

12. A method according to claim 9, wherein PW and PH are different in different standard classes or levels.

13. The method of claim 9, wherein PW and PH depend on a maximum codec unit CU size or/and a minimum CU size of a slice or picture.

14. The method of any of claims 1-2, wherein, in the first set of CPMV,

A) MV ^F0＝MV^S0,(x^F0,y^F0)＝(x^S0,y^S 0), or MV ^F0＝MV^T0,(x^F0,y^F 0) =

(X ^T0,y^T 0); or alternatively

B) MV ^F0＝MV^S1,(x^F0,y^F0)＝(x^S1,y^S 1), or MV ^F0＝MV^T1,(x^F0,y^F 0) =

(X ^T1,y^T 1); or alternatively

C) MV ^F0＝MV^S2,(x^F0,y^F0)＝(x^S2,y^S 2), or MV ^F0＝MV^T2,(x^F0,y^F 0) =

(x^T2,y^T2)；

Wherein the motion vector MV ^F0、MV^F1、MV^F is a plurality of CPMV of the first group of CPMV at a plurality of representative point coordinates (x ^F0,y^F0)、(x^F1,y^F1)、(x^F2,y^F 2), respectively, the motion vector MV ^S0、MV^S1、MV^S is a plurality of CPMV of the second group of CPMV at a plurality of representative point coordinates (x ^S0,y^S0)、(x^S1,y^S1)、(x^S2,y^S 2), respectively, and the motion vector MV ^T0、MV^T1、MV^T 2 is a plurality of CPMV of the third group of CPMV at a plurality of representative point coordinates (x ^T0,y^T0)、(x^T1,y^T1)、(x^T2,y^T 2), respectively.

15. The method of any of claims 1-2, wherein the motion vectors MV ^F0、MV^F 1 and MV ^F 2 in the first group of CPMV are derived from the motion vectors MV ^S and MV ^S 1 in the second group of CPMV by using a 4-parameter affine model with coordinates (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates for the affine model.

16. The method of any of claims 1-2, wherein the motion vectors MV ^F0、MV^F 1 and MV ^F 2 in the first group of CPMV are derived from the motion vectors MV ^S0、MV^S and MV ^S 2 in the second group of CPMV by using a 6-parameter affine model with coordinates (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates for the affine model.

17. The method of any of claims 1-2, wherein the motion vectors MV ^F0、MV^F 1 and MV ^F 2 in the first group of CPMV are derived from the motion vectors MV ^T and MV ^T 1 in the third group of CPMV by using a 4-parameter affine model with coordinates (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates for the affine model.

18. The method of any of claims 1-2, wherein the motion vectors MV ^F0、MV^F and MV ^F 2 in the first group of CPMV are derived from the motion vectors MV ^T0、MV^T and MV ^T 2 in the third group of CPMV by using a 6-parameter affine model with coordinates (x ^F0,y^F0)、(x^F1,y^F 1) and (x ^F2,y^F 2) as input coordinates for the affine model.

19. The method of any of claims 1-2, wherein motion vectors MV ^F 2 in the first set of CPMV comprising motion vectors MV ^F0、MV^F and MV ^F are calculated only when the first block is encoded with the 6-parameter affine model, or motion vectors MV ^F in the first set of CPMV comprising motion vectors MV ^F0、MV^F and MV ^F 2 are calculated regardless of whether the first block is encoded with the 4-parameter affine model or the 6-parameter affine model.

20. The method of claim 1, further comprising:

one or more differences between CPMV in the first set of CPMV are stored.

21. The method of claim 20, wherein the first set of CPMV includes motion vectors MV ^F0、MV^F and MV ^F 2, and stores d1=mv ^F1-MV^F 0, or stores d2=mv ^F2-MV^F 0, or stores both D1 and D2.

22. The method of claim 21, wherein D2 is stored only when the first block is encoded with a 6-parameter affine model.

23. The method of claim 21, wherein D2 is stored when the first block is encoded with a 4-parameter affine model or a 6-parameter affine model.

24. The method of claim 1, further comprising:

One or more differences between the first set of CPMV and the CPMV in the first set of CPMV are stored together.

25. The method of claim 2, wherein the plurality of CPMV in the first group of CPMV and/or one or more differences between CPMV in the first group of CPMV are shifted with a shifting function and the shifted CPMV and/or differences are stored.

26. The method of claim 25, wherein the shift function SatShift (x, n) is defined as:

where n is an integer and offset0 and/or offset1 is set to (1 < < n) > >1 or (1 < < (n-1)), or ((1 < < n) > > 1) -1, or offset0 and/or offset1 is set to 0.

27. The method of claim 25, wherein the Shift function Shift (x, n) is defined as:

Shift(x,n)＝(x+offset0)>>n，

Where n is an integer and offset0 is set to (1 < < n) > >1 or (1 < < < (n-1)), or ((1 < < n) > > 1) -1, or offset0 is set to 0.

28. The method of claim 26 or 27, wherein n is 2 or 4, or n depends on the motion accuracy.

29. The method of claim 26 or 27, wherein n in the case of storing CPMV of the first group of CPMV is different from n in the case of storing a difference between CPMV of the first group of CPMV.

30. The method of claim 26 or 27, wherein the stored CPMV is first left-shifted before affine inheritance of a block for encoding after the first block.

31. The method of claim 2, wherein the plurality of CPMV's in the first group of CPMV's and/or one or more differences between CPMV's in the first group of CPMV's are clipped with a clipping function and the clipped CPMV's and/or differences are stored.

32. The method of claim 31, wherein the Clip function Clip3 (min, max, x) is defined as:

33. The method of claim 32, wherein, when CPMV or difference is stored with K bits, min= -2 ^K-1, and Max = 2 ^K-1 -1, K is an integer.

34. The method of claim 33, wherein K varies depending on whether CPMV or difference is to be stored.

35. The method of any of claims 25-27, 31-34, wherein the plurality of CPMV in the first group of CPMV and/or the one or more differences between CPMV in the first group of CPMV are sequentially processed with a shifting function and a clipping function, and the processed CPMV and/or differences are stored.

36. The method of any of claims 1-2, wherein a plurality of CPMV's in the first set of CPMV's are stored in an affine history motion vector prediction HMVP buffer or table or list to represent one history-based candidate affine model.

37. The method of claim 20, wherein one or more differences between CPMV in the first set of CPMV are stored in an affine history motion vector prediction HMVP buffer or table or list to represent one history-based candidate affine model.

38. The method of claim 36, wherein one or more CPMV or one or more differences stored in the HMVP buffer or table or list are shifted with a shifting function and/or clipped with a clipping function.

39. The method of claim 1, further comprising:

deriving an affine inherited motion vector MV for a first block of video based on the stored motion vectors MV for conversion between the first block and a bit stream representation of the first block; and

The conversion is performed by using the affine inheritance MV.

40. The method of claim 39, wherein the MVs are stored in adjacent neighboring base blocks.

41. The method of claim 39, wherein the MVs are stored in an affine history motion vector prediction HMVP buffer or table or list.

42. The method of claim 40, wherein the MVs stored in the neighboring basic blocks include at least: the MVs stored in the left adjacent neighboring basic block L, the MVs stored in the upper adjacent basic block a, the MVs stored in the lower left adjacent neighboring basic block LB, the MVs stored in the upper right adjacent neighboring basic block AR, and the MVs stored in the upper left adjacent neighboring basic block AL.

43. The method of claim 40 or 42, wherein the neighboring basic blocks are 4 x4 blocks.

44. The method according to any one of claims 39-42, wherein affine inherited MVs at positions (x, y) in the first block are derived by using the first MVs of adjacent neighboring basic blocks at the representative point (x ₀,y₀) based on an affine model having (x-x ₀,y-y₀) as input coordinates of the affine model.

45. The method of claim 44, wherein the representative point (x ₀,y₀) is any location within a basic block.

46. The method of claim 44, wherein the representative point (x ₀,y₀) is any location outside or at a boundary of a basic block.

47. The method according to claim 45 or 46, wherein the coordinates of the representative point (x ₀,y₀) are determined based on the coordinates (xTL, yTL) of the upper left corner sample point in the adjacent neighboring basic block and additional information comprising two variables i and j.

48. The method of claim 47, wherein a first variable i of the two variables depends on a width of the basic block and a second variable j of the two variables depends on a height of the basic block.

49. The method of claim 47, wherein the variables i and j depend on the location of the neighboring basic block.

50. The method of claim 45, wherein the coordinates of the representative point (x ₀,y₀) are determined based on the coordinates of the top left sample point (xPos 00, yPos) of the first block, the coordinates of the top right sample point (xPos, yPos 00) of the first block, and the coordinates of the bottom left sample point (xPos 00, yPos 01) of the first block.

51. The method of claim 49 or 50, wherein the coordinates of the representative point (x ₀,y₀) of the adjacent base block L on the left are (xPos 00-2, ypos 01-1); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block LB adjacent to the lower left are (xPos 00-2, ypos01+3); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block a adjacent above are (xPos-1, ypos 00-2); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block AR adjacent to the upper right are (xPos 10+3, ypos 00-2); the coordinates of the representative point (x ₀,y₀) of the adjacent basic block AL adjacent to the upper left are (xPos-2, ypos 00-2).

52. The method of claim 47, wherein the additional information is signaled depending on a position of a neighboring basic block or a neighboring basic block or in at least one of a sequence parameter set SPS, a video parameter set VPS, a picture parameter set PPS, a slice header, a slice group header, a slice, a codec tree unit CTU, a CU.

53. The method of claim 47, wherein the additional information is different in different standard levels or tiers.

54. The method according to any one of claims 39-42, wherein affine inherited MVs at positions (x, y) in sub-blocks of the first block are derived by using first MVs of adjacent neighboring basic blocks at representative points (x ₀,y₀) based on an affine model having (x-x ₀,y-y₀) as input coordinates of the affine model.

55. The method of claim 54, wherein affine inherited MVs at locations (x, y) in sub-blocks are used to perform motion compensation on the sub-blocks.

56. The method according to any one of claims 39-42, wherein the affine inherited MV at the position (x, y) being the angle of the first block is derived as the inherited control point motion vector CPMV of the first block by using the first MV of the adjacent neighboring basic block at the representative point (x ₀,y₀) based on an affine model having (x-x ₀,y-y₀) as input coordinates of the affine model.

57. The method of claim 56, wherein the legacy CPMV is used to predict a signaled CPMV of a first block of affine inter-frame codec.

58. The method of claim 56, wherein the legacy CPMV is directly used as the CPMV of the first block of the affine Merge codec.

59. The method of any one of claims 39-42, wherein when the first block uses a 4-parameter affine model, the affine model used to derive affine inherited MVs at positions (x, y) in the first block is:

Where a and b are variables of the affine model.

60. The method of any one of claims 39-42, wherein when the first block uses a 6-parameter affine model, the affine model used to derive affine inherited MVs at positions (x, y) in the first block is:

where a, b, c and d are variables of the affine model.

61. The method of claim 59, wherein the variables a, b are calculated as follows:

wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1) are CPMV at two representative points in a first set of CPMV covering a second block adjacent to the basic block, respectively, and w _t depends on a relative offset between the representative points of the second block, wherein the first set of CPMV is used to determine inherited motion information of a block that is encoded after the second block.

62. The method of claim 60, wherein the variables a, b, c, d are calculated as follows:

63. The method of claim 59, wherein the variables a, b are calculated as follows:

Wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1) is the CPMV at two representative points in a second or third group of CPMV covering a second block of an adjacent neighboring basic block, respectively, and w _t is the width of the second block, wherein the second group of CPMV is used to derive the MV of each sub-block of the second block, and the third group of CPMV is signaled from the encoder to the decoder.

64. The method of claim 60, wherein the variables a, b, c, d are calculated as follows:

65. The method of claim 59, wherein the variables a, b are calculated as follows:

Wherein ,mv_t0＝(mv^h _t0,mv^v _t0)、mv_t1＝(mv^h _t1,mv^v _t1) is the CPMV at two representative points at a second block covering an adjacent neighboring basic block, respectively, and w _t is the width of the second block, wherein mv ^h _t1-mv^h _t0、mv^v _t1-mv^v _t0 is fetched directly from the storage means for storing the difference between the CPMV of the blocks.

66. The method of claim 60, wherein the variables a, b, c, d are calculated as follows:

67. The method of any of claims 1-2, 39-42, wherein the converting generates the first block or the second block of video from a bitstream representation.

68. The method of any of claims 1-2, 39-42, wherein the converting generates a bitstream representation from a first block or a second block of video.

69. An apparatus in a video system comprising a processor and a non-transitory memory having instructions thereon, wherein the instructions, when executed by the processor, cause the processor to implement a method according to any one of claims 1 to 68.

70. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the method of any one of claims 1 to 68.