JP7652742B2 - VIDEO ENCODING METHOD, VIDEO ENCODING APPARATUS, STORAGE MEDIUM AND COMPUTER PROGRAM - Patent application - Google Patents

Description

This application claims priority to Provisional Application No. 62/790,421, filed January 9, 2019, the entire contents of which are incorporated herein by reference.

This application relates to video coding and compression. More particularly, this application relates to a method and apparatus relating to a combined inter- and intra-prediction (CIIP) method for video coding.

Various video coding techniques may be used to compress the video data. Video coding may be performed according to one or more video coding standards. For example, video coding standards include Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (H.265/HEVC), Advanced Video Coding (H.264/AVC), Moving Picture Experts Group (MPEG)
Video coding generally involves prediction methods (e.g., inter-prediction, intra-prediction, etc.) that exploit redundancy present in a video image or sequence.
An important goal of video coding techniques is to compress video data into a format that uses a lower bit rate while avoiding or minimizing degradation of video quality.

Examples of the present disclosure provide methods for improving the efficiency of syntax signaling for merge-related modes.

According to an aspect of the present disclosure, a method of video encoding includes obtaining a first reference image associated with a current coding block of a current image, the first reference image being temporally before the current image and a second reference image being temporally after the current image; obtaining a first prediction based on a first motion vector from the current coding block to a reference block in the first reference image; obtaining a second prediction based on a second motion vector from the current coding block to a reference block in the second reference image; and calculating a bidirectional prediction of the current coding block based on at least the first prediction and the second prediction, wherein the calculating includes enabling bidirectional optical flow (BDOF) when calculating the bidirectional prediction of the current coding block under a condition that a composite inter-intra prediction (CIIP) is not applied to the calculation of the bidirectional prediction of the current coding block.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the present disclosure.

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate examples consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.
FIG. 2 is a block diagram of an encoder according to an example of the present disclosure. FIG. 2 is a block diagram of a decoder according to an example of the present disclosure. 1 is a flowchart illustrating a method for generating a combined inter-and-intra prediction (CIIP) according to an example of the present disclosure. 4 is a flowchart illustrating a method for generating a CIIP according to an example of the present disclosure. FIG. 2 illustrates a block partition in a multi-type tree structure according to an example of the present disclosure. FIG. 2 illustrates a block partition in a multi-type tree structure according to an example of the present disclosure. FIG. 2 illustrates a block partition in a multi-type tree structure according to an example of the present disclosure. FIG. 2 illustrates a block partition in a multi-type tree structure according to an example of the present disclosure. FIG. 2 illustrates a block partition in a multi-type tree structure according to an example of the present disclosure. FIG. 2 illustrates a diagram showing combined inter-and-intra prediction (CIIP) according to an example of the present disclosure. FIG. 2 illustrates a diagram showing combined inter-and-intra prediction (CIIP) according to an example of the present disclosure. FIG. 2 illustrates a diagram showing combined inter-and-intra prediction (CIIP) according to an example of the present disclosure. 1 is a flowchart of an MPM candidate list generation process according to an example of the present disclosure. 1 is a flowchart of an MPM candidate list generation process according to an example of the present disclosure. FIG. 2 illustrates a workflow of an existing CIIP design in a VVC according to an example of the present disclosure. FIG. 1 illustrates a workflow of the proposed CIIP method by removing BDOF according to an example of the present disclosure. FIG. 1 illustrates a workflow of a single-prediction-based CIIP that selects a list of predictions based on POC distance, according to an example of the present disclosure. 1 is a flowchart of a method when enabling a CIIP block for MPM candidate list generation according to an example of the present disclosure. 13 is a flowchart of a method for disabling a CIIP block for MPM candidate list generation according to an example of the present disclosure. FIG. 1 illustrates a computing environment coupled with a user interface according to an example of the present disclosure.

Reference will now be made in detail to examples of the present disclosure, examples of which are illustrated in the accompanying drawings. The following description refers to the accompanying drawings, in which the same numbers in different drawings represent the same or similar elements, unless otherwise noted. The embodiments described in the following description of examples of the present disclosure do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with aspects related to the present disclosure as set forth in the appended claims.

The terminology used in this disclosure is for the purpose of describing particular embodiments only.
It is not intended to limit the disclosure. As used in this disclosure and the appended claims, the singular forms "a,""an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It is also to be understood that the term "and/or" as used herein is intended to mean and include any and all possible combinations of one or more of the associated listed items.

Herein, terms such as "first,""second," and "third" may be used to describe various pieces of information, but it should be understood that the information should not be limited by these terms. These terms are used only to distinguish one category of information from another. For example, the first information may be referred to as the second information, and similarly, the second information may be referred to as the first information, without departing from the scope of this disclosure. As used herein, the term "if" may also be used as "when" or "in the event of" or "when," depending on the context.
may be understood to mean "at discretion."

The first version of the HEVC standard was finalized in October 2013, and it offers approximately 50% bitrate savings or equivalent perceptual quality compared to the previous generation video coding standard H.264/MPEG AVC. Although the HEVC standard offers significant coding improvements over its predecessor, there is evidence that superior coding efficiency can be achieved by adding additional coding tools to HEVC. Based on this, VCEG and MPEG AVC have agreed to work together to develop a standard that will allow for the creation of a standard that will provide a high level of coding efficiency.
Both the ITU-TVECG and the ISO/IE Group have begun work on investigating new coding technologies for future video coding standardization. In October 2015, the ITU-TVECG and the ISO/IE Group agreed to begin significant research into advanced technologies that could enable significant improvements in coding efficiency.
A Joint Video Exploration Team (JVET) was formed by C-MPEG. A reference software called the Joint Exploration Model (JEM) is a JVE-based test model that integrates some additional coding tools on top of the HEVC Test Model (HM).
It was maintained by T.

In October 2017, a Joint Call for Proposals (CfP) for video compression with capabilities beyond HEVC was issued by ITU-T and ISO/IEC. In April 2018, at the 10th JVET meeting, 23 CfP responses were received and evaluated, with approximately 40% more support than HEVC.
Based on these evaluation results, JVET has decided to develop a compression efficiency gain of 100% for Versatile.
In 2013, the National Instruments Board of Education launched a new project to develop a new generation video coding standard called Wide Video Coding (VVC). In the same month, a reference software code base called the VVC Test Model (VTM) was established to demonstrate a reference implementation of the VVC standard.

Like HEVC, VVC is built on a block-based hybrid video coding framework. Figure 1 (described below) gives a block diagram of a general block-based hybrid video coding system. The input video signal is processed block by block (called coding unit (CU)). In VTM-1.0,
U can be up to 128x128 pixels. However, unlike HEVC, which partitions blocks based only on quad trees, in VVC, one coding tree unit (CTU) is divided into CUs to accommodate different local characteristics based on quad/binary/ternary trees. In addition, the concept of multiple partition unit types in HEVC is removed, that is, the separation of CU, prediction unit (PU) and transform unit (TU) no longer exists in VVC, and instead, each CU is always used as the basic unit for both prediction and transformation without additional partitions. In the multi-type tree structure, one CTU is first partitioned by a quad tree structure. Then, each quad tree leaf node can be further partitioned by binary and ternary tree structures.
As shown in Figures 5A, 5B, 5C, 5D, 5E (described below), there are five partitioning types: quaternary partitioning, horizontal binary partitioning, vertical binary partitioning, horizontal ternary partitioning, and vertical ternary partitioning, respectively.

In FIG. 1 (described below), spatial and/or temporal prediction can be performed.
Spatial prediction (or "intra prediction") predicts a current video block using pixels from samples of already coded neighboring blocks in the same video picture/slice (called reference samples). Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction (also called "inter prediction" or "motion compensated prediction") predicts a current video block using pixels from samples of already coded neighboring blocks in the same video picture/slice (called reference samples).
) predicts a current video block using reconstructed pixels from already coded video pictures. Temporal prediction reduces the temporal redundancy inherent in video signals. The temporal prediction signal for a particular CU is typically signaled by one or more motion vectors (MVs), which indicate the amount and direction of motion between the current CU and its temporal references. Also, if multiple reference pictures are supported, one reference picture index is additionally transmitted, which is used to identify which reference picture in the reference picture store the temporal prediction signal comes from. After spatial and/or temporal prediction, a mode decision block in the encoder selects an optimal prediction mode, for example based on a rate-distortion optimization method. The prediction block is then subtracted from the current video block, and the prediction residual is decorrelated using transform and quantization.

The quantized residual coefficients are dequantized and inverse transformed to form a reconstructed residual, which is then added to the prediction block to form a reconstructed signal for the CU. Further in-loop filtering, such as a deblocking filter, sample adaptive offset (SAO), or adaptive in-loop filter (ALF), can be applied to the reconstructed CU before it is placed in the reference picture store and used for coding future video blocks. To form an output video bitstream, the coding mode (inter or intra), prediction mode information, motion information, and the quantized residual coefficients are all sent to an entropy coding unit for further compression and packing to form a bitstream.

FIG. 2 (described below) shows a general block diagram of a block-based video decoder. The video bitstream is first entropy decoded in an entropy decoding unit. The coding mode and prediction information are sent to either a spatial prediction unit (if intra-coded) or a temporal prediction unit (if inter-coded) to form a prediction block. The residual transform coefficients are sent to an inverse quantization unit and an inverse transform unit to reconstruct the residual block. The prediction block and the residual block are then added together. The reconstructed block may further go through in-loop filtering before being stored in a reference image store. The reconstructed video in the reference image store is then sent to drive a display device and is also used to predict future video blocks.

1 shows a typical encoder 100. The encoder 100 receives a video input 110,
It has motion compensation 112, motion estimation 114, intra/inter mode decision 116, block predictor 140, adder 128, transform 130, quantization 132, prediction related information 142, intra prediction 118, image buffer 120, inverse quantization 134, inverse transform 136, adder 126, memory 124, in-loop filter 122, entropy coding 138, and bitstream 144.

2 shows a block diagram of an exemplary decoder 200. The decoder 200 includes a bitstream 210, an entropy decoder 212, an inverse quantizer 214, an inverse transform 216, an adder 218, and an inverse decoder 219.
18 , intra/inter mode selection 220 , intra prediction 222 , memory 230 , in-loop filter 228 , motion compensation 224 , image buffer 226 , prediction related information 234 , and video output 232 .

FIG. 3 illustrates an example method 300 for generating a combined inter-and-intra prediction (CIIP) in accordance with this disclosure.

In step 310, a first reference image and a second reference image associated with the current prediction block are
2, where the first reference image is before the current image in display order, and the second
The reference image is after the current image in display order.

In step 312, a first prediction L0 is obtained based on a first motion vector MV0 from the current prediction block to a reference block in a first reference image.

In step 314, a second prediction L1 is obtained based on a second motion vector MV1 from the current prediction block to a reference block in a second reference image.

4 illustrates an exemplary method for generating a CIIP according to the present disclosure, for example, the method includes uni-prediction based inter prediction and MPM based intra prediction to generate a CIIP.

In step 410, a reference picture in the reference picture list associated with the current prediction block is obtained.

In step 412, an inter prediction is generated based on a first motion vector from the current picture to the first reference picture.

At step 414, the intra-prediction mode associated with the current prediction block is obtained.

At step 416, an intra prediction of the current predicted block is generated based on the intra prediction.

In step 418, the inter prediction and the intra prediction are averaged to generate a final prediction for the current predicted block.

In step 420, the current prediction block is selected based on the most probable mode (MPM).
For the base intra-mode prediction, specify whether it is treated as inter-mode or intra-mode.

FIG. 5A illustrates a diagram illustrating block quad partitions in a multi-type tree structure according to an example of the present disclosure.

FIG. 5B illustrates a diagram illustrating block vertical binary partitioning in a multi-type tree structure according to an example of the present disclosure.

FIG. 5C illustrates a diagram illustrating block horizontal binary partitioning in a multi-type tree structure according to an example of the present disclosure.

FIG. 5D illustrates a diagram illustrating block vertical ternary partitioning in a multi-type tree structure according to an example of the present disclosure.

FIG. 5E illustrates a diagram illustrating block horizontal ternary partitioning in a multi-type tree structure according to an example of the present disclosure.

Hybrid Inter and Intra Prediction As shown in FIG. 1 and FIG. 2, the inter and intra prediction method is used in the hybrid video coding scheme. Here, each PU is allowed to select inter prediction or intra prediction to exploit correlations in either the time domain or the spatial domain only, but not both. However, as pointed out in the prior art, the residual signals generated by the inter prediction block and the intra prediction block may exhibit very different characteristics from each other. Therefore, if the two kinds of prediction can be efficiently combined, one more accurate prediction can be expected to reduce the energy of the prediction residual and improve the coding efficiency. Furthermore, in natural video content, the motion of moving objects may be complicated. For example, there may be regions that contain both old content (e.g., objects included in the previously coded image) and new new content (e.g., objects excluded in the previously coded image). In such a scenario, neither inter prediction nor intra prediction can provide one accurate prediction of the current block.

To further improve prediction efficiency, the VVC standard adopts Combined Inter and Intra Prediction (CIIP), which combines intra prediction and inter prediction for one CU coded by merge mode. Specifically, for each merge CU, one additional flag is signaled to indicate whether CIIP is enabled for the current CU. For the luma component, CIIP supports four frequently used intra modes, including planar mode, DC mode, horizontal mode, and vertical mode. For the chroma component, DM (i.e., chroma reuses the same intra mode of the luma component) is always applied without additional signaling. In addition, the existing CIIP is signaled to indicate whether CIIP is enabled for the current CU.
In the IP design, a weighted average is applied to combine the inter- and intra-predicted samples of one CIIP CU. Specifically, equal weights (i.e., 0.5) are applied when planar or DC mode is selected. Otherwise (i.e., either horizontal or vertical mode is applied), the current CU is first divided horizontally (for horizontal mode) or vertically (for vertical mode) into four equal-sized regions.

Furthermore, in the current VVC operation specification, the intra mode of one CIIP CU can be used as a predictor to predict the intra mode of its neighboring CIIP CUs via a Most Probable Mode (MPM) mechanism.
For an IP CU, if its neighboring blocks are also CIIP CUs, the intra modes of those neighboring blocks are first selected from the following: planar mode, DC mode, horizontal mode,
and the nearest mode within the vertical mode, and then added to the MPM candidate list of the current CU. However, when constructing the MPM list of each intra CU, if one of its neighboring blocks is coded in CIIP mode, it is considered as unavailable. That is, the intra mode of one CIIP CU is not allowed to predict the intra mode of its neighboring intra CU. Figures 7A and 7B (described below) compare the MPM list generation process of intra CU and CIIP CU.

Here, shift and _offset are 15-BD and 1<<(14-BD)+2*(1<<13
) which are the right shift and offset values applied to combine the L0 and L1 prediction signals of dual prediction.

FIG. 6A shows a diagram illustrating hybrid inter and intra prediction in horizontal mode according to an example of this disclosure.

FIG. 6B shows a diagram illustrating hybrid inter and intra prediction in vertical mode according to an example of this disclosure.

FIG. 6C shows a diagram illustrating hybrid inter and intra prediction for planar and DC modes according to an example of this disclosure.

FIG. 7A illustrates a flowchart of an MPM candidate list generation process for intra-CUS according to an example of the present disclosure.

FIG. 7B illustrates a flowchart of an MPM candidate list generation process for a CIIP CU according to an example of the present disclosure.

Improvements to CIIP Although CIIP can improve the efficiency of conventional motion compensated prediction, its design can be further improved. Specifically, the following problems in existing CIIP designs in VVC are identified in this disclosure:

First, as explained in the "Combined Inter and Intra Prediction" section, CIIP is
To combine inter and intra prediction samples, each CIIP CU needs to generate a prediction signal using its reconstructed neighboring samples.
This means that the decoding of a P CU depends on the complete reconstruction of its neighboring blocks. Due to such interdependence, in a practical hardware implementation, CIIP needs to be performed in the reconstruction stage, where neighboring reconstructed samples become available for intra prediction. Since the decoding of CUs in the reconstruction stage must be performed sequentially (i.e., one by one), the number of computation operations (e.g., multiplications, additions, bit shifts) involved in the CIIP process cannot be too high to ensure sufficient throughput for real-time decoding.

As mentioned in the "Bidirectional Optical Flow" section, BDOF is enabled to improve prediction quality when one inter-coded CU is predicted from two reference blocks from both forward and backward temporal directions. As shown in Figure 8 (described below), in current VVC, BDOF is also involved to generate inter prediction samples for CIIP mode. Considering the additional complexity due to BDOF, such a design requires a hardware codec encoding/decoding scheme when CIIP is enabled.
Decoding throughput may be significantly reduced.

Secondly, in the current CIIP design, when one CIIP CU references one merge candidate that is bi-predicted, it is necessary to generate motion compensation prediction signals for both lists L0 and L1. In the case where one or more MVs are not integer precision, an additional interpolation process must be invoked to interpolate samples at partial sample positions.
Such a process not only increases the computational complexity but also increases the memory bandwidth if more reference samples need to be accessed from external memory.

Then, as discussed in the "Combined Inter and Intra Prediction" section, the current CI
In the IP design, the intra modes of CIIP CUs and intra modes of intra CUs are treated differently when constructing the MPM lists of their neighboring blocks. Specifically, if one current CU is coded in CIIP mode, its neighboring CIIP CUs are considered as intra, i.e., the intra modes of the neighboring CIIP CUs can be added to the MPM candidate list. However, if the current CU is coded in intra mode, its neighboring CIIP CUs are considered as inter, i.e., the intra modes of the neighboring CIIP CUs are excluded from the MPM candidate list. Such a non-uniform design may not be optimal for the final version of the VVC standard.

FIG. 8 illustrates a diagram showing a workflow of an existing CIIP design in a VVC according to an example of the present disclosure.

Simplification of CIIP In this disclosure, a method is provided for simplifying the existing CIIP design to facilitate hardware codec implementation. In general, the main aspects of the technology proposed in this disclosure are summarized as follows:

First, to improve the CIIP coding/decoding throughput, it is proposed to exclude BDOF from the generation of inter-prediction samples in CIIP mode.

Next, in order to reduce computational complexity and memory bandwidth consumption, one CIIP
In the case where a CU is bi-predicted (ie, has both L0 and L1 MVs), a method is proposed to convert the block from bi-prediction to uni-prediction to generate inter-predicted samples.

Then, two methods are proposed to harmonize intra-CU and CIIP intra-modes when forming MPM candidates for neighboring blocks.

CIIP without BDOF
As pointed out in the "Problem Statement" section, BDOF is a
When U is bi-predicted, it is always enabled to generate inter-predicted samples for CIIP mode. Due to the additional complexity of BDOF, existing CIIP designs may suffer from significantly reduced encoding/decoding throughput, especially making real-time decoding difficult for VVC decoders. On the other hand, CIIP
For a CIIP CU, its final predicted sample is generated by averaging the inter-predicted sample and the intra-predicted sample. In other words, the improved predicted sample by BDOF is not directly used as the predicted signal of the CIIP CU. Therefore, compared with the traditional bi-predicted CU (where BDOF is directly applied to generate the predicted sample), the corresponding improvement obtained from BDOF is less efficient for the CIIP CU. Therefore, based on the above circumstances, it is proposed to disable BDOF when generating the inter-predicted sample for the CIIP mode. Figure 9 (described below) shows the corresponding workflow of the proposed CIIP process after removing BDOF.

FIG. 9 shows a diagram illustrating the workflow of the proposed CIIP method by removing BDOF according to an example of the present disclosure.

CIIP based on single prediction
As mentioned above, when a merge candidate referenced by one CIIP CU is bi-predicted, both L0 and L1 prediction signals are generated to predict samples in the CU. In order to reduce memory bandwidth and interpolation complexity, in one embodiment of the present disclosure,
Only inter-predicted samples generated using uni-prediction (even if the current CIIP CU is bi-predicted) will be used to combine with intra-predicted samples in CIIP mode. Specifically, when the current CIIP CU is uni-predictive, the inter-predicted samples are
The inter-predicted samples used by the CIIP are directly combined with the intra-predicted samples. Otherwise (i.e., when the current CU is bi-predicted), the inter-predicted samples used by the CIIP are generated based on single prediction from one prediction list (L0 or L1). Different methods can be applied to select the prediction list. The first method proposes to always select the first prediction (i.e., list L0) for any CIIP block predicted by two reference pictures.

In the second method, it is proposed to always select the second prediction (i.e., list L1) for any CIIP block predicted by two reference pictures. In the third method, an adaptive method is applied in which the prediction list associated with one reference picture with a small picture order count (POC) distance from the current picture is selected. Figure 10 (described below) shows the workflow of a single prediction-based CIIP that selects a prediction list based on the POC distance.

Finally, the last method proposes to enable the CIIP mode only if the current CU is uni-predicted.
The signaling of the P enable/disable flag depends on the prediction direction of the current CIIP CU. In the case where the current CU is mono-predicted, the CIIP flag is signaled in the bitstream to indicate whether CIIP is enabled or disabled. Otherwise (i.e., the current CU is bi-predicted), the signaling of the CIIP flag is skipped and always inferred as false, i.e., CIIP is always disabled.

FIG. 10 illustrates a diagram showing a workflow of a single-prediction based CIIP that selects a prediction list based on POC distance according to an example of the present disclosure.

Harmonization of intra-CU and CIIP intra-mode for MPM candidate list construction As mentioned above, current CIIP designs are not unified with respect to how intra-CU and CIIP CU intra-modes are used to form MPM candidate lists for their neighboring blocks. Specifically, both intra-CU and CIIP CU intra-modes can predict the intra-modes of neighboring blocks coded in CIIP mode.
However, only the intra mode of the intra CU can predict the intra mode of the intra CU. To achieve another unified design, two methods are proposed in this section to harmonize the usage of intra CU and intra mode of CIIP for MPM list construction.

The first method proposes treating the CIIP mode as an inter mode for MPM list construction. Specifically, one CIIP CU or one intra CU
When generating any of the MPM lists, if the neighboring block is coded in CIIP mode, the intra mode of the neighboring block is marked as disabled.
In this method, the intra mode of the CIIP block cannot be used to construct the MPM list. Conversely, in the second method, it is proposed to treat the CIIP mode as an intra mode for the purpose of constructing the MPM list.
For the intra mode of a P CU, the intra modes of both the neighboring CIIP blocks and intra blocks can be predicted. Figures 11A and 11B (described below) show the MPM candidate list generation process when the above two methods are applied.

Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the present disclosure disclosed herein. This application is in accordance with its general principles and includes any modifications, uses, and variations of the present disclosure, including departures from the present disclosure that are within known or customary practice in the art.
It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the specific examples described above and illustrated in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof, which is intended to be limited only by the scope of the appended claims.

FIG. 11A illustrates a flowchart of a method when enabling a CIIP block for MPM candidate list generation according to an example of the present disclosure.

FIG. 11B illustrates a flowchart of a method for disabling CIIP blocking for MPM candidate list generation according to an example of the present disclosure.

FIG. 12 illustrates a computing environment 1260 coupled with a user interface 1260.
12. The computing environment 1210 may be part of a data processing server. The computing environment 1210 includes a processor 1220, memory 1240, and I/O.
and an O interface 1250.

The processor 1220 typically controls the overall operation of the computing environment 1210, such as operations related to display, data acquisition, data communication, and image processing. The processor 1220 may include one or more processors that execute instructions for performing all or some of the steps of the methods described above.
220 and other components.
The processor may be a central processing unit (CPU), a microprocessor, a single chip machine, a GPU, or the like.

Memory 1240 is configured to store various types of data to support the operation of computing environment 1210. Examples of such data include instructions for any application or method operating in computing environment 1210, video data, image data, etc. Memory 1240 may be any type of volatile or non-volatile memory device, or combination thereof, such as, for example, static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), etc.
, erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The I/O interface 1250 provides an interface between the processor 1220 and a peripheral interface module such as a keyboard, a click wheel, buttons, including but not limited to a home button, a start scan button, and a stop scan button. The I/O interface 1250 can be coupled to an encoder and a decoder.

In one embodiment, a non-transitory computer readable storage medium is also provided that includes a number of programs, such as those contained in memory 1240, executable by processor 1220 in computing environment 1210 to perform the methods described above. For example, the non-transitory computer readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

A non-transitory computer-readable storage medium has stored therein a plurality of programs for execution by a computing device having one or more processors, the plurality of programs, when executed by the one or more processors, causing the computing device to perform the method for predicting behavior described above.

In one embodiment, the computing environment 1210 may include one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), and/or other processors for performing the methods described above.
P), digital signal processing device (DSPD), programmable logic device (PL
D) may be implemented by a field programmable gate array (FPGA), a graphical processing unit (GPU), a controller, a microcontroller, a microprocessor, or other electronic components.

Claims (6)

obtaining a first reference image that is associated with a current coding block of the current image and that is temporally preceding the current image and a second reference image that is temporally following the current image;
obtaining a first prediction based on a first motion vector from the current coding block to a reference block in the first reference image;
obtaining a second prediction based on a second motion vector from the current coding block to a reference block in the second reference image; and calculating a bidirectional prediction of the current coding block based on at least the first prediction and the second prediction, the calculating including: enabling a bidirectional optical flow (BDOF) when calculating the bidirectional prediction of the current coding block under a condition that a composite inter-intra prediction (CIIP) is not applied to the calculation of the bidirectional prediction of the current coding block; and disabling BDOF when calculating the bidirectional prediction of the current coding block in response to a determination that a CIIP is applied to the calculation of the bidirectional prediction of the current coding block, and calculating a bidirectional prediction of the current coding block based on an average of the first prediction and the second prediction .
The intra prediction modes include planar modes,
The BDOF operation is as follows:
calculating a first horizontal gradient value {(∂I ⁽⁰⁾ /∂x)(i,j)} and a first vertical gradient value {(∂I ⁽⁰⁾ /∂y)(i,j)} for a prediction sample associated with the first prediction, and calculating a second horizontal gradient value {(∂I ⁽¹⁾ /∂x)(i,j)} and a second vertical gradient value {(∂I ⁽¹⁾ /∂y)(i,j)} for a prediction sample associated with the second prediction, where I ⁽⁰⁾ (i,j) represents the prediction sample at sample position (i,j) associated with the first prediction and I ⁽¹⁾ (i,j) represents the prediction sample at sample position (i,j) associated with the second prediction;
calculating a bi-directional prediction of the current coding block based on the first prediction, the second prediction, the first horizontal gradient value, the first vertical gradient value, the second horizontal gradient value, and the second vertical gradient value.
Video encoding method. Calculating a bi-prediction of the current coding block in response to enabling the BDOF includes:
calculating a motion improvement value for each sub-block by minimizing a difference between a prediction sample associated with a first prediction and a prediction sample associated with a second prediction; and calculating a bidirectional prediction of the current coding block based on the motion improvement value, the first prediction, the second prediction, the first horizontal gradient value, the first vertical gradient value, the second horizontal gradient value, and the second vertical gradient value.
The method of claim 1 . Calculating a bi-prediction of the current coding block in response to enabling the BDOF includes:
calculating a BDOF value based on the motion improvement value, the first horizontal gradient value, the first vertical gradient value, the second horizontal gradient value, and the second vertical gradient value; and calculating a bi-directional prediction of the current coding block based on the first prediction, the second prediction, and the BDOF value.
The method of claim 2 . 1. A video encoding device comprising: one or more processors; and one or more storage devices coupled to the one or more processors,
Configured to carry out the method according to any one of claims 1 to 3 ,
Video encoding equipment. A non-transitory computer-readable storage medium storing a plurality of programs to be executed by a computing device having one or more processors,
The plurality of programs are executed by the one or more processors to cause the computing device to perform the method of any one of claims 1 to 3 .
Storage medium. A computer program stored on a computer readable storage medium comprising instructions which, when executed by a processor, perform the method of any one of claims 1 to 3 .