WO2022141282A1 - Video processing method and encoding device - Google Patents

Abstract

The present application provides a video processing method and an encoding device, capable of accurately determining a final reconstructed pixel and a final prediction mode of an image block to be encoded, and capable of improving the encoding efficiency. The method comprises: taking an initial reconstructed pixel of a neighboring block of an image block to be encoded as a reference pixel, and determining an optimal intra-frame prediction mode of said image block; determining an optimal prediction mode of said image block from the optimal intra-frame prediction mode of said image block and an inter-frame prediction mode of said image block; and if the optimal prediction mode is the optimal intra-frame prediction mode of said image block, taking a final reconstructed pixel of the neighboring block as the reference pixel, and determining a final prediction mode of said image block according to the optimal intra-frame prediction mode of said image block, wherein a reconstructed pixel in the final prediction mode of said image block is a final reconstructed pixel of said image block.

Description

Video processing method and encoding device

Copyright notice

The disclosure of this patent document contains material that is subject to copyright protection. This copyright belongs to the copyright owner. The copyright owner has no objection to the reproduction by anyone of the patent document or the patent disclosure as it exists in the official records and archives of the Patent and Trademark Office.

technical field

The present application relates to the field of video coding and decoding, and more particularly, to a video processing method and coding device.

Background technique

In the current technology, the video coding process mainly includes steps such as prediction, transformation, quantization, and entropy coding. Among them, prediction includes two types of intra-frame prediction and inter-frame prediction. In addition, in the currently commonly used coding techniques, the current frame to be coded will first be divided into blocks to generate multiple non-overlapping coding regions, and each coding region will be divided into multiple coding blocks. Perform operations such as prediction, transformation, quantization, and entropy coding.

However, when performing intra prediction on a coding block, it is necessary to rely on the reconstructed pixel values of the adjacent blocks of the coding block. In order to obtain the reconstructed pixels of the adjacent blocks of the coding block, a relatively simple implementation is to code each coding region in turn. However, this method has low parallel processing capability, it is difficult to realize real-time encoding and decoding of high-resolution video, and the encoding performance is poor.

SUMMARY OF THE INVENTION

The present application provides a video processing method and an encoding device, which can more accurately determine the final reconstructed pixels and final prediction mode of an image block to be encoded, and can improve encoding efficiency.

In a first aspect, a video processing method is provided, comprising: using initial reconstructed pixels of adjacent blocks of a to-be-coded image block as reference pixels, and determining an optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block, wherein , the reconstructed pixels of the to-be-coded image block in the optimal intra-frame prediction mode of the to-be-coded image block are the initial reconstructed pixels of the to-be-coded image block;

From the optimal intra prediction mode of the to-be-coded image block and the inter-frame prediction mode of the to-be-coded image block, determine the optimal prediction mode of the to-be-coded image block;

If the optimal prediction mode is the optimal intra prediction mode of the image block to be encoded, the final reconstructed pixel of the adjacent block is used as a reference pixel, and the optimal intra prediction mode of the image block to be encoded is used as the reference pixel and the final reconstructed pixel to determine the final prediction mode of the image block to be encoded, wherein the reconstructed pixel in the final prediction mode of the image block to be encoded is the final reconstructed pixel of the image block to be encoded.

In a second aspect, an encoding device is provided, comprising: a memory and a processor, wherein,

the memory is used to store computer programs;

The processor invokes the computer program, and when the computer program is executed, performs the following operations:

Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, determine the optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block, wherein the to-be-coded image block is in the to-be-coded image The reconstructed pixels in the optimal intra prediction mode of the block are the initial reconstructed pixels of the to-be-coded image block;

From the optimal intra prediction mode of the to-be-coded image block and the inter-frame prediction mode of the to-be-coded image block, determine the optimal prediction mode of the to-be-coded image block;

If the optimal prediction mode is the optimal intra prediction mode of the image block to be encoded, the final reconstructed pixel of the adjacent block is used as a reference pixel, and the optimal intra prediction mode of the image block to be encoded is used as the reference pixel , determining the final prediction mode and final reconstructed pixels of the image block to be encoded, wherein the reconstructed pixels in the final prediction mode of the image block to be encoded are the final reconstructed pixels of the image block to be encoded.

A third aspect provides a computer-readable storage medium on which codes for executing the method of the first aspect are stored.

In a fourth aspect, a computer program product is provided, comprising code for performing the method of the first aspect.

Description of drawings

FIG. 1 is a schematic diagram of an encoding method.

FIG. 2 is a schematic flowchart of a video processing method provided by the present application.

FIG. 3 is a schematic diagram of a specific implementation manner of some steps of the video processing method provided by the present application.

FIG. 4 is a schematic diagram of a division manner of an image block to be encoded provided by the present application.

FIG. 5 is a schematic diagram of another division manner of an image block to be encoded provided by the present application.

FIG. 6 is a schematic diagram of another division manner of an image block to be encoded provided by the present application.

FIG. 7 is a schematic diagram of staged processing of the video processing method provided by the present application.

FIG. 8 is a schematic diagram of a pipeline structure of the video processing method provided by the present application.

FIG. 9 is a schematic structural diagram of an encoding device provided by the present application.

Detailed ways

The technical solutions of the embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs. Terms used in the specification of the present application are for the purpose of describing specific embodiments only, and are not intended to limit the present application. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

This application can be applied to various video coding standards, such as H.264, high efficiency video coding (HEVC), universal video coding (VVC), audio video coding standard (audio video coding standard, AVS), AVS+, AVS2 and AVS3, etc.

Hereinafter, video coding will be briefly described. Generally speaking, the video coding process mainly includes the steps of prediction, transformation, quantization and entropy coding. In the currently commonly used coding technologies, the current frame to be coded is firstly subjected to block processing to generate multiple non-overlapping coding regions, and then these coding regions are separately coded during coding. Exemplarily, the coding region may also be referred to as a coding tree unit (coding tree unit, CTU). The size of the CTU may be, for example, 64×64, or 128×128 (the unit is a pixel, and the unit is omitted in the similar description below), and the like. When coding each coding region, the coding region can be further divided into square or rectangular image blocks, and operations such as prediction, transformation, quantization and entropy coding are performed on each image block respectively. Exemplarily, the image block may also be referred to as a coding unit (coding unit, CU). Hereinafter, the image block to be encoded is referred to as the image block to be encoded.

The prediction step in video coding includes two types of intra prediction and inter prediction. The intra-frame prediction mainly uses the information of the image in this frame as prediction data to remove the spatial redundancy information of the image block to be encoded. The method calculates the predicted value to generate a predicted block, and subtracts the corresponding pixel values of the image block to be encoded and the predicted block to obtain a residual. Inter-frame prediction is to use the information of the reference frame as prediction data to remove the temporal redundancy information of the image block to be encoded. The residuals are obtained by subtracting the corresponding pixel values of the encoded image block and the predicted block, and the residuals corresponding to the obtained image blocks to be encoded are combined together to obtain the residuals of the to-be-encoded area. After the residual error is generated through the prediction step, operations such as transformation, quantization and entropy coding can be used to further improve the coding efficiency.

When performing intra prediction, the prediction block can be calculated in Planar mode, DC mode or angular mode. Specifically, the reconstructed pixels of the left adjacent and upper adjacent blocks of the image block to be coded are used as reference pixels, and all prediction modes are traversed to perform prediction to generate prediction blocks, and finally, the bits consumed by each prediction mode and the distortion situation are optimized by rate distortion. The method selects the best prediction mode and the corresponding prediction block.

In the prior art, for intra-frame prediction, since it is necessary to refer to the information of adjacent blocks, a relatively simple implementation is to encode each encoding region in the image sequentially. However, such an encoding method has low parallel processing capability, and it is difficult to realize real-time encoding and decoding of high-resolution video.

In order to achieve the purpose of real-time encoding and decoding, the pipeline structure shown in Figure 1 is generally used to realize parallel encoding of the encoding area. However, there may be cases where the coding information of adjacent coding regions is not available. Referring to Figure 1, in the intra-frame prediction stage of the to-be-encoded region 2, the to-be-encoded region 1 only goes to the mode decision stage. At this time, the reconstructed pixel value of the to-be-encoded region 1 cannot be obtained, so the left adjacent encoding cannot be obtained. Reference information for the area. In one way, when performing intra-frame prediction of a certain image block to be coded in region 2 to be coded, if the reference information of adjacent blocks is not available, the DC value (usually 2 ^bitdepth-1 , where bitdepth is the pixel bit depth) can be used. ) as the predicted value, or use the original pixel of the adjacent block as the reference pixel for prediction, obtain the reconstructed pixel value in the subsequent pipeline stage and then use the reconstructed pixel value of the adjacent block as the reference pixel for correction.

However, because the DC value is used as prediction or the original pixel is used as a reference for prediction in the previous intra-frame prediction, the best prediction mode that is decided may be inaccurate. In this case, the inaccurate best prediction mode is used for correction. It will also lead to poor prediction effect, resulting in low video encoding quality.

In view of this, the present application provides a video processing method, which can improve video encoding quality while realizing real-time encoding. The method provided by the present application will be described below.

FIG. 2 is a schematic flowchart of a video processing method provided by the application. The method may include S210 to S230.

S210, using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, determine the optimal intra-frame prediction mode of the to-be-coded image block and the initial reconstructed pixels of the to-be-coded image block.

The reconstructed pixels of the image block to be encoded in the optimal intra prediction mode are the initial reconstructed pixels of the image block to be encoded. That is to say, the initial reconstructed pixels of adjacent blocks of the image block to be encoded are used as reference pixels, and the image block to be encoded is reconstructed based on the best intra prediction mode of the image block to be encoded (for example, including intra prediction, transform , quantization, inverse quantization, inverse transformation, and reconstruction) are the initial reconstructed pixels of the to-be-coded image block.

It should be understood that the initial reconstructed pixels of the adjacent blocks of the image block to be encoded are the reconstructed pixels of the adjacent blocks of the image block to be encoded in the optimal intra prediction mode of the adjacent blocks of the image block to be encoded. The optimal intra-frame prediction mode of the adjacent blocks of the image block to be coded takes the initial reconstructed pixels of the adjacent blocks of the adjacent blocks of the image block to be coded as reference pixels, and the intra-frame prediction of the adjacent blocks of the to-be-coded image block is performed. owned.

It should also be understood that the size of the image block to be encoded is not limited in this application, for example, the size of the image block to be encoded may be 64×64, 32×32, 16×16, or 8×8.

S220: Determine the optimal prediction mode of the image block to be encoded from the optimal intra prediction mode of the image block to be encoded and the inter prediction mode of the image block to be encoded.

S230, if the best prediction mode of the image block to be coded is the best intra prediction mode of the image block to be coded, take the final reconstructed pixel of the adjacent block of the image block to be coded as a reference pixel, according to the best prediction mode of the image block to be coded Intra prediction mode, which determines the final prediction mode of the image block to be encoded and the final reconstructed pixels of the image block to be encoded. The reconstructed pixels in the final prediction mode of the image block to be encoded are the final reconstructed pixels of the image block to be encoded.

To sum up, according to the method provided by the present application, each image block to be encoded can be encoded in parallel, and since the initial reconstruction pixels of adjacent blocks of the image block to be encoded are used to determine the optimal intra prediction mode of the image block to be encoded, the determined The best intra prediction mode is more accurate. In addition, since the final prediction mode of the to-be-coded image block is determined by using the final reconstructed pixels of the adjacent blocks of the to-be-coded image block, the determined final prediction mode is more accurate.

The steps in the method 200 are described in detail below.

Exemplarily, S210 may be implemented in two manners (ie, manner 1 and manner 2) that will be described below.

Optionally, in the case that the image block to be encoded can be divided into multiple sub-image blocks, mode 1 can be adopted, and in the case that the image block to be encoded cannot be divided into smaller sub-blocks, mode 2 can be adopted.

Alternatively, in the case that the image block to be encoded can be divided into multiple sub-image blocks, the second method can also be adopted.

The first mode and the second mode are respectively described below.

method one

Referring to FIG. 3 , the first way may include steps S310 to S330.

S310: Determine the first intra prediction mode and the first initial reconstructed pixels of the to-be-coded image block by using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels.

Optionally, S310 may specifically include: using the initial reconstructed pixels of adjacent blocks of the image block to be encoded as reference pixels, using N1 intra prediction modes to perform intra prediction on the image block to be encoded, and determining the first image block to be encoded. Intra prediction mode and first initial reconstructed pixel. where N1 is a positive integer.

Specifically, the initial reconstructed pixels of adjacent blocks of the image block to be encoded are used as reference pixels, and N1 intra-frame prediction modes are used to perform intra-frame prediction on the to-be-coded image block, and it is determined that the image block to be encoded is predicted in the N1 types of intra-frame predictions respectively. The encoding cost #1 in the mode, that is to say, N1 encoding costs #1 can be obtained. The intra prediction mode corresponding to the smallest encoding cost #1 among the N1 encoding costs #1 may be used as the first intra prediction mode of the image block to be encoded.

Exemplarily, the coding cost #1 may be a Hadamard (Hadamard, HAD) cost (cost), but this application does not limit this, for example, the coding cost #1 may also be the sum of the absolute difference (Sum of Absolute Difference, SAD). ) cost, absolute transform difference sum (Sum of Absolute Transformed Difference, STAD) cost or rate-distortion (rate-distortion, RD) cost, etc.

S310 is described by taking the encoding cost #1 as the HAD cost as an example. In S310, for any one of the N1 intra-frame prediction modes, the initial reconstructed pixels of adjacent blocks of the image block to be encoded are used as reference pixels, and the image block to be encoded is framed using the intra-frame prediction mode. Intra prediction, the prediction block can be obtained. Then, subtract the prediction block from the image block to be coded to obtain the residual, and perform HAD transformation on the residual to obtain the HAD cost. The N1 HAD costs obtained according to the N1 intra prediction modes are compared, and the intra prediction mode corresponding to the HAD cost with the smallest HAD cost is used as the first intra prediction mode of the image block to be encoded. The residual is obtained by subtracting the prediction block corresponding to the first intra-frame prediction mode of the image block to be encoded and the original pixels of the image block to be encoded, and then performing transform quantization and inverse quantization and inverse transformation to obtain the reconstruction residual. The prediction blocks corresponding to the first intra prediction mode of the encoded image block are added to obtain the first initial reconstructed pixel.

It should be understood that how to calculate HAD cost, SAD cost, STAD cost, RD cost, etc. according to the residual can refer to the prior art, which will not be repeated in this application.

Exemplarily, in one manner, the N1 intra-frame prediction modes may include one or more modes from a total of 35 intra-frame prediction modes of 33 angle modes, DC mode and Planar mode.

In another way, the N1 intra prediction modes may be determined by the following steps:

Step (A1), according to the gradient information of the original pixel of the to-be-coded image block, determine P1 intra-frame prediction modes from Q1 kinds of intra-frame prediction modes, Q1>P1, and both Q1 and P1 are integers;

Step (A2): Determine the N1 intra prediction modes according to the P1 intra prediction modes.

Optionally, P1=N1. For example, P1=N1=5.

Optionally, the Q1 intra prediction modes may include 33 angular modes (or, angular prediction modes), DC mode and Planar mode.

Optionally, step (A1) can be realized by the following process:

S11: Determine multiple subsets corresponding to the Q1 intra prediction modes. Wherein, each subset in the plurality of subsets includes multiple intra-frame prediction modes in the Q1 intra-frame prediction modes. The number of intra prediction modes in each subset may be the same or different. The prediction modes in each subset may or may not contain prediction modes in other subsets.

S12: Determine a subset from the multiple subsets according to the gradient information of the original pixels of the image block to be encoded.

S13: Use the original pixels of the adjacent blocks of the image block to be encoded as reference pixels, and use each intra prediction mode in the determined subset to perform intra prediction on the image block to be encoded, and determine that the image block to be encoded is in the determined image block. Coding cost #2 for each intra prediction mode in the resulting subset. The P1 intra prediction modes with the smallest coding cost #2 in the determined subset are the P1 intra prediction modes.

The above process will be described below by taking as an example that the Q1 intra-frame prediction modes are 33 angular modes, a DC mode, and a total of 35 intra-frame prediction modes.

Example 1, in step S11, the 35 intra prediction modes may be divided into multiple subsets according to the direction of the prediction mode.

For example, 35 intra prediction modes can be divided into 4 subsets as shown in Table 1.

Table 1

subset 1 0° {0,1,6,7,8,9,10,11,12,13,14} Subset 2 45° {0,1,14,15,16,17,18,19,20,21,22}

Subset 3 90° {0,1,22,23,24,25,26,27,28,29,30} Subset 4 135° {0,1,2,3,4,5,30,31,32,33,34}

For another example, the 35 intra prediction modes can be divided into 8 subsets as shown in Table 2.

Table 2

subset 1 -22.5° {0,1,4,5,6,7,8} Subset 2 0° {0,1,8,9,10,11,12} Subset 3 22.5° {0,1,12,13,14,15,16} Subset 4 45° {0,1,16,17,18,19,20} Subset 5 67.5° {0,1,20,21,22,23,24} Subset 6 90° {0,1,24,25,26,27,28} Subset 7 112.5° {0,1,28,29,30,31,32} Subset 8 135° {0,1,2,3,32,33,34}

Example 2, in step S11, 33 angle patterns may be divided into multiple groups of 2, 3, 4, 5, 6, 7, etc., according to the adjacent serial numbers. Take three groups as an example, the details are as follows:

Group 1: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

Group 2: 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23

Group 3: 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34

Three groups correspond to three subsets, and each subset may include a DC mode and/or a Planar mode in addition to the prediction mode in its corresponding group.

In step S12, it is assumed that the size of the image block to be encoded is 2Nx2N. In an example, 5 NxN sub-blocks can be taken from the image block to be encoded to calculate the mean values pa to pe respectively (four are obtained by dividing the quadtree, and then one is taken from the center), and the calculation can be performed by dividing the image block to be encoded along the 0°, 45° °, 90°, 135° (corresponding to a subset respectively) gradient information GI, compare the GI of the four directions, and select a subset corresponding to the direction with the smallest GI. in:

0°: GI=|pb-pa|+|pd-pc|,

45°: GI=|pc-pe|+|pe-pb|,

90°: GI=|pc-pa|+|pd-pb|,

135°: GI=|pd-pe|+|pe-pa|.

For example, if the multiple subsets in step S11 are the 4 subsets shown in Table 1, then the to-be-coded image block can be divided into 5 sub-blocks as shown in FIG. D and E. Then the gradient information (Gradient Information) of 4 different prediction directions (0°, 45°, 90° and 135°) can be calculated to obtain 4 gradient information. The size of each sub-block is 1/4 of the image block to be encoded. A, B, C, and D are the upper left, upper right, lower left, and lower right sub-blocks of the image block to be encoded, respectively, and E is the sub-block at the center of the image block to be encoded (as shown by the dotted line in FIG. 4 ).

For each of the five sub-blocks A, B, C, D, and E, the average value of the pixel values of all the pixels in the sub-block can be calculated, and the average value here can be obtained by arithmetic mean, geometric mean, or other methods The average value of , which is not limited in this application. Assuming that the average value of the pixel values of the 5 sub-blocks A, B, C, D and E can be expressed as a, b, c, d and e, respectively, then the four 0°, 45°, 90° and 135° can be calculated. The gradient information of different prediction directions is as follows:

GI1=|b-a|+|d-c|,

GI2=|c-e|+|e-b|,

GI3=|c-a|+|d-b|,

GI4=|d-e|+|e-a|.

Then, the four gradient information GI1, GI2, GI3, and GI4 can be compared to determine the smallest gradient information. Assuming that the minimum gradient information is GI1 and its corresponding prediction direction is 0°, the subset determined in step S12 is the subset corresponding to 0°.

In an example, if the multiple subsets in step S11 are the 8 subsets shown in Table 2, then the to-be-coded image block can be divided into 9 sub-blocks, except for the 5 sub-blocks shown in FIG. 4 In addition to blocks A, B, C, D and E, it also includes four sub-blocks F, G, H, and I as shown in FIG. 5 .

The size of each sub-block is 1/4 of the image block to be encoded. F and I are located above and below the center position of the image block to be encoded, respectively, where F is shown as the shaded area in FIG. 5 . G and H are located to the left and right of the center position of the image block to be encoded, respectively.

Similarly, for each of the 9 sub-blocks, the average value of the pixel values of all the pixels in the sub-block can be calculated, and the average value here can be an arithmetic average, a geometric average, or an average obtained in other ways, and this application applies to this Not limited. Denote the average value of the pixel values of these 9 sub-blocks as a, b, c, d, e, f, g, h and i in turn, then the gradient information of 8 different prediction directions can be calculated subsequently.

Wherein, on the basis of the 5 sub-blocks in FIG. 4, 4 gradient information of 4 different prediction directions of 0°, 45°, 90° and 135° can be obtained similarly to the above example, namely GI1, GI2, GI3 and GI4. And, 4 gradient information of 4 different prediction directions of -22.5°, 22.5°, 67.5° and 112.5° can be obtained similarly by combining the sub-blocks shown in Fig. 4 and Fig. 5, which are as follows:

GI5=|b-g|+|h-c|,

GI6=|d-g|+|h-a|,

GI7=|d-f|+|i-a|,

GI8=|c-f|+|i-b|.

Then, these 8 gradient information GI1, GI2, GI3, GI4, GI5, GI6, GI7, and GI8 can be compared to determine which of them is the smallest.

In step S13, the original pixels of adjacent blocks of the image block to be encoded are used as reference pixels, and the 11 intra prediction modes in the determined subset are used to perform intra prediction on the image block to be encoded, and the image block to be encoded is determined. Coding cost #2 in these 11 intra prediction modes. Finally, the intra-frame prediction modes corresponding to the smallest P1 (for example, 5) coding costs #2 among the 11 coding costs #2 may be selected.

Exemplarily, encoding cost #2 may be the same as encoding cost #1, or may be different. For example, encoding cost #2 can be HAD cost, SAD cost, STAD cost or RD cost.

Optionally, step (A2) may specifically include: according to the F1 intra prediction modes and the P1 intra predictions in the initial most probable mode (Most Probable Mode, MPM) list of adjacent blocks of the image block to be encoded mode, which determines the N1 intra prediction modes. The initial MPM list of the adjacent blocks of the image block to be encoded is determined according to the first intra-frame prediction mode of the adjacent blocks of the image block to be encoded, and F1 is a positive integer. It should be understood that the manner of determining the first intra prediction mode of the adjacent blocks of the image block to be encoded is similar to that of determining the first intra prediction mode of the image block to be encoded, and details are not repeated here.

In step (A2), N1 intra prediction modes may be determined from F1+P1 intra prediction modes. For example, N1-F1 intra-frame prediction modes that are different from the F1 intra-frame prediction modes and the F1 intra-frame prediction modes may be used as the N1 intra-frame prediction modes. forecast mode.

Exemplarily, F1=3. Taking F1=3 and P1=N1=5 as an example, in step (A2), the 3 intra-frame prediction modes in the above initial MPM list and the 5 intra-frame prediction modes determined in step (A1) can be used. Among the modes, two intra-prediction modes different from the three intra-prediction modes are referred to as the N1 intra-prediction modes.

It should be noted that, the MPM list involved in this application may be determined by any method of determining the MPM list in the prior art. For example, it is exemplified that the initial MPM list includes 3 intra prediction modes. If the first intra prediction mode of the left adjacent block of the image block to be encoded is the DC mode, the three intra prediction modes in the initial MPM list are the DC mode, the Planar mode and the angle prediction 26 mode. If the first intra prediction mode of the left adjacent block of the image block to be encoded is not the DC mode, the three intra prediction modes in the initial MPM list are: the first intra prediction mode of the left adjacent block of the image block to be encoded Mode; DC mode; Planar, DC, Angle prediction 26, the first of these three modes is different from the first intra prediction mode and DC mode of the left adjacent block of the image block to be encoded.

S320, using the initial reconstructed pixels of adjacent blocks of each sub-image block in the V sub-image blocks included in the image block to be encoded as reference pixels, and determining the second intra-frame prediction mode and the second initial reconstructed pixels of each sub-image block, V is an integer greater than 1.

The V sub-image blocks are obtained by dividing the to-be-coded image block according to the first division manner. The first division manner may be, for example, a binary tree, a quadtree, or an octree division. For example, picture (A) in FIG. 6 is a to-be-coded unit, and the to-be-coded image block in the to-be-coded unit may be divided into the structure shown in (B) of FIG. 6 . That is, the to-be-coded image block includes 4 sub-image blocks.

In this application, S310 and S320 may be executed in parallel.

Optionally, S320 may specifically include: using the initial reconstructed pixels of adjacent blocks of each sub-image block as reference pixels, and using N2 intra-frame prediction modes corresponding to each sub-image block to perform intra-frame prediction on each sub-image block, A second intra prediction mode and second initial reconstructed pixels are determined for each sub-image block. N2 is a positive integer, and N2 can be equal to or unequal to N1.

For example, for any sub-image block in the V sub-image blocks, the initial reconstructed pixels of the adjacent blocks of the sub-image block can be used as reference pixels, and the N2 intra prediction modes corresponding to the sub-image block can be used for this sub-image block. Perform intra-frame prediction on the sub-image block, and select an intra-frame prediction mode with the smallest corresponding coding cost #3 from the N2 kinds of intra-frame prediction modes according to the N2 kinds of coding costs #3 under the N2 kinds of intra-frame prediction modes , as the second intra prediction mode of the sub-image block. The residual is obtained by subtracting the prediction block corresponding to the second intra-frame prediction mode of the sub-image block and the original pixel of the sub-image block, and then performing the transformation and quantization and inverse quantization and inverse transformation process to obtain the reconstruction residual. The prediction blocks corresponding to the second intra prediction mode of the sub-image block are added to obtain the second initial reconstructed pixel of the sub-image block.

The coding cost #3 and the coding cost #1 may be the same or different, for example, the coding cost #3 may be HAD cost, SAD cost, STAD cost or RD cost.

Exemplarily, the manner of determining the N2 intra-frame prediction modes corresponding to each sub-image block may be similar to the manner of determining the N1 intra-frame prediction modes corresponding to the image block to be encoded. A method for determining the N2 intra prediction modes corresponding to the sub image block #1 will be described below. It should be understood that the sub-image block #1 may be any sub-image block in the V sub-image blocks.

Determine the N2 intra prediction modes corresponding to sub-image block #1 include:

Step (B1), according to the gradient information, determine P2 intra-frame prediction modes from Q2 intra-frame prediction modes, Q2>P2, and both Q2 and P2 are integers;

Step (B2): Determine the N2 intra prediction modes according to the P2 intra prediction modes.

Optionally, P2=N2. For example, P2=N2=5.

Optionally, the Q2 intra prediction modes may include 33 angle modes, DC mode and Planar mode.

Optionally, step (B1) can be achieved by the following process:

S21: Determine multiple subsets corresponding to the Q2 intra prediction modes, where each subset in the multiple subsets includes multiple intra prediction modes in the Q2 intra prediction modes. The number of intra prediction modes in each subset may be the same or different. The prediction modes in each subset may or may not contain prediction modes in other subsets.

S22: Determine a subset from the plurality of subsets according to the gradient information of the original pixels of the sub-image block #1.

S23: Using the original pixels of the adjacent blocks of the sub-image block #1 as reference pixels, using each intra prediction mode in the determined subset, perform intra-frame prediction on the sub-image block #1, and determine the sub-image block # 1 Coding cost #4 in each intra prediction mode in the determined subset. The P2 intra prediction modes with the smallest coding cost #4 in the determined subset are the P2 intra prediction modes.

The above process will be described below by taking as an example that the Q2 intra-frame prediction modes are 33 angular modes, a DC mode, and a total of 35 intra-frame prediction modes.

For example, in step S21, the 35 intra prediction modes may be divided into 4 subsets according to the direction of the prediction mode, that is, the 35 intra prediction modes correspond to 4 subsets, and each subset includes 11 intra predictions model. One way of dividing is as follows:

Subset 1 = {0,1,6,7,8,9,10,11,12,13,14}

Subset 2 = {0,1,2,3,4,5,30,31,32,33,34}

Subset 3 = {0,1,22,23,24,25,26,27,28,29,30}

Subset 4 = {0,1,14,15,16,17,18,19,20,21,22}

In step S22, one subset is determined from the four subsets by calculating the gradient information of the original pixels of the sub-image block #1. For this step, reference may be made to the description of step S12, which will not be repeated here.

In step S23, using the original pixels of the adjacent blocks of sub-image block #1 as reference pixels, and using 11 intra-frame prediction modes in the determined subset, perform intra-frame prediction on sub-image block #1, and determine the sub-image block #1. Coding cost #4 of image block #1 in these 11 intra prediction modes. Finally, the intra prediction modes corresponding to the smallest P2 (for example, 5) coding costs #4 among the 11 coding costs #4 may be selected. The selected intra-frame prediction mode is the P2 intra-frame prediction modes.

Exemplarily, encoding cost #4 may be the same as encoding cost #3, or may be different. Encoding cost #4 can be the same as encoding cost #2, or it can be different. For example, encoding cost #4 can be HAD cost, SAD cost, STAD cost or RD cost.

Optionally, step (B2) may include: determining the N2 intra prediction modes according to the F2 intra prediction modes and the P2 intra prediction modes in the initial MPM list of the adjacent blocks of the sub-image block #1 forecast mode. The initial MPM list of the adjacent blocks of the sub-image block #1 is determined according to the second intra prediction mode of the adjacent blocks of the sub-image block #1, and F2 is a positive integer.

That is, in step (B2), N2 intra prediction modes may be determined from F2+P2 intra prediction modes.

For example, N-F intra prediction modes that are different from the F2 intra prediction modes among the P2 intra prediction modes and the F2 intra prediction modes may be used as the N2 intra prediction modes .

Exemplarily, F2=3. Taking F2=3, P2=N2=5 as an example, in step (B2), the 3 intra-frame prediction modes in the above-mentioned MPM list and the 5 intra-frame prediction modes determined in step (B1) can be used. Among them, two intra prediction modes different from the three intra prediction modes are used as the N2 intra prediction modes.

It should be noted that, the MPM list involved in this application may be determined by any method of determining the MPM list in the prior art. For example, it is exemplified that the initial MPM list of adjacent blocks of sub-image block #1 includes 3 intra prediction modes. If the upper adjacent block of sub-image block #1 does not belong to the image block to be coded, in step (B2), it is considered that the second intra prediction mode of the upper adjacent block of sub-image block #1 is DC mode, otherwise, in step (B2) In step (B2), the second intra-frame prediction mode of the upper adjacent block of sub-image block #1 is considered to be the actually obtained second intra-frame prediction mode. If the second intra prediction modes of the upper and left adjacent blocks of sub-image block #1 are equal: (1) If both are Palnar mode, the initial MPM list of the adjacent blocks of sub-image block #1 includes Palnar mode, DC mode and angle prediction 26 mode; (2) if all are DC mode, the initial MPM list of adjacent blocks of sub-image block #1 includes DC mode, Planar mode and angle prediction 26 mode; (3) if all is the angle prediction mode, then the initial MPM list of the adjacent blocks of sub-image block #1 includes the second intra prediction mode of the left adjacent block (or upper adjacent block) of sub-image block #1, and sub-image block #1 The two angle prediction modes to which the second intra prediction mode of the left adjacent block (or upper adjacent block) of 1 is closest. If the second intra prediction modes of the upper and left adjacent blocks of sub-image block #1 are not equal, the initial MPM list of the adjacent blocks of sub-image block #1 is: the left phase of sub-image block #1 The second intra prediction mode of the adjacent block; the second intra prediction mode DC mode of the upper adjacent block of the sub-image block #1; Planar, DC, angle prediction 26, the first of these three modes is related to the sub-image block. A mode in which the second intra prediction mode of the left adjacent block of #1 and the upper adjacent block are different.

S330, according to the first coding cost of the to-be-coded image block in the first intra-frame prediction mode and the first coding cost of each sub-image block in the V sub-image blocks in the second intra-frame prediction mode, determine the value of the to-be-coded image block Best intra prediction mode and initial reconstructed pixels.

Optionally, in S330, if the first encoding cost of the image block to be encoded in the first intra prediction mode is greater than (or, greater than or equal to) the V sub image blocks in the second intra prediction mode If the sum of the first encoding costs is given below, the V second intra-frame prediction modes and the second initial reconstruction pixels corresponding to the V sub-image blocks may be determined as the best intra-frame prediction mode and initial reconstruction pixels of the to-be-coded image block, In this case, the image block to be encoded needs to be divided into smaller blocks, and intra-frame prediction is performed in units of smaller blocks. On the contrary, the first intra prediction mode and the first initial reconstruction pixel are determined as the best intra prediction mode and initial reconstruction pixel of the image block to be encoded. In this case, the image block to be encoded does not need to be smaller , and perform intra-frame prediction in the unit of the image block to be encoded.

Optionally, in S330, if the first encoding cost of the image block to be encoded in the first intra prediction mode is greater than (or, greater than or equal to) the V sub image blocks in the second intra prediction mode The sum of the products of the first encoding cost and the first preset weight value under the condition of Intra prediction mode and initial reconstructed pixels. On the contrary, the first intra-frame prediction mode and the first initial reconstruction pixel are determined as the best intra-frame prediction mode and initial reconstruction pixel of the image block to be encoded.

Or, in S330, if the product of the first encoding cost and the second preset weight of the image block to be encoded in the first intra prediction mode is greater than (or, greater than or equal to) each sub-image block in the V sub-image blocks The sum of the first coding costs in the second intra-frame prediction mode, the V second intra-frame prediction modes and the second initial reconstruction pixels corresponding to the V sub-image blocks may be determined as the best intra-frame frame of the to-be-coded image block Prediction mode and initial reconstructed pixels. On the contrary, the first intra-frame prediction mode and the first initial reconstruction pixel are determined as the best intra-frame prediction mode and initial reconstruction pixel of the image block to be encoded.

It should be noted that the second intra prediction modes corresponding to each sub-image block may be the same or different.

Exemplarily, the first preset weight may be a positive number less than 1, such as 0.5.

Exemplarily, the second preset weight may be a value greater than 1 and less than 2, such as 1.5.

Method 2

Using the initial reconstructed pixels of adjacent blocks of the image block to be encoded as reference pixels, use N intra-frame prediction modes to perform intra-frame prediction on the image block to be encoded, and determine the optimal intra-frame prediction mode of the image block to be encoded and its initial reconstructed pixels. . where N is a positive integer.

The optimal intra-frame prediction mode of the image block to be encoded here may be the same as the first intra-frame prediction mode in the first mode. Therefore, with regard to the second mode, the description of step (1) in the first mode may be referred to, and details are not described herein again.

Taking the encoding cost #5 as the RD cost as an example, step S220 will be described.

If S210 adopts mode 1, then in S220:

If the best intra prediction mode of the image block to be encoded is the first intra prediction mode, then compare the RD cost of the image block to be encoded in the first intra prediction mode with the RD cost of the image block to be encoded in the inter prediction mode cost. If the RD cost corresponding to the first intra-frame prediction mode is small, the best prediction mode of the image block to be encoded is the first intra-frame prediction mode. The best prediction mode is the inter prediction mode. It should be understood that the RD cost of the image block to be encoded in the first intra prediction mode is determined by using the initial reconstructed pixels of the adjacent blocks of the image block to be encoded as reference pixels.

If the best intra prediction mode of the image block to be encoded is the second intra prediction mode corresponding to each of the V sub image blocks, then compare the sum of the RD cost corresponding to the V sub image blocks with the image block to be encoded in the inter prediction mode RD cost. If the sum of the RD cost corresponding to the V sub-image blocks is small, then the best prediction mode of the image block to be encoded is V second intra prediction modes, otherwise the best prediction mode of the image block to be encoded is the inter prediction mode . It should be understood that the RD cost corresponding to any sub-image block is determined by using the initial reconstructed pixels of adjacent blocks of the sub-image block as reference pixels.

If S210 adopts Mode 2, then in S220:

If the best intra prediction mode of the image block to be encoded is the first intra prediction mode, then compare the RD cost of the image block to be encoded in the first intra prediction mode with the RD cost of the image block to be encoded in the inter prediction mode cost. If the RD cost corresponding to the first intra prediction mode is small, the best prediction mode of the image block to be encoded is the first intra prediction mode, otherwise the best prediction mode of the image block to be encoded is the inter prediction mode. It should be understood that the RD cost of the image block to be encoded in the first intra prediction mode is determined by using the initial reconstructed pixels of the adjacent blocks of the image block to be encoded as reference pixels.

If the first mode is adopted in S210, then, in S230, if the best prediction mode of the image block to be encoded is the first intra prediction mode, the final reconstructed pixel of the adjacent block of the image block to be encoded is used as the reference pixel, According to the first intra prediction mode, the final prediction mode of the image block to be encoded is determined. If the optimal prediction mode of the image block to be encoded is V second prediction modes, then the final reconstructed pixel of the adjacent block of each sub-image block is used as a reference pixel, and according to the second intra-frame prediction mode of each sub-image block, determine The final prediction mode for each sub-tile.

If the second mode is adopted in S210, then, in S230, if the best prediction mode of the image block to be encoded is the first intra prediction mode, the final reconstructed pixel of the adjacent block of the image block to be encoded is used as the reference pixel, According to the first prediction mode, the final prediction mode of the image block to be encoded is determined.

Optionally, the determining the final prediction mode of the to-be-coded image block according to the first intra-frame prediction mode using the final reconstructed pixels of the adjacent blocks of the to-be-coded image block as reference pixels includes: taking the phase of the to-be-coded image block The final reconstructed pixel of the adjacent block is used as a reference pixel to determine the encoding cost #5 of the image block to be encoded in the first intra prediction mode and M1 intra prediction modes; the first intra prediction mode and M1 intra prediction modes are used. Among them, the prediction mode with the smallest encoding cost #5 is determined as the final prediction mode of the image block to be encoded. M1 is a positive integer.

The encoding cost #5 may be the RD cost, but this application does not limit it.

Exemplarily, the M1 intra prediction modes include one or more of the following: one or more of the W1 intra prediction modes in the final MPM list of the image block to be encoded; or, the N1 One or more intra-prediction modes with the smallest coding cost #1 among the intra-prediction modes except the first intra-prediction mode. The final MPM list of the image block to be encoded is generated according to the final prediction mode of the adjacent blocks of the image block to be encoded and/or the best prediction mode of the adjacent blocks of the image block to be encoded. W1 is a positive integer, for example, W1=3, which is not limited in this application.

For example, the M1 intra prediction modes may include the first intra prediction mode, or the second or third intra prediction mode among the W1 intra prediction modes. Alternatively, the final reconstructed pixels of adjacent blocks of the image block to be encoded can also be used as reference pixels to calculate the RD cost, SAD cost, or HAD cost of the image block to be encoded in W1 intra-frame prediction modes, and compare the corresponding costs. A small intra prediction mode is used as one of M1 intra prediction modes. For example, in step (A1), encoding cost #1 corresponding to N1 intra prediction modes can be obtained. The intra prediction mode corresponding to the coding cost #1 of the N1 encoding cost #1 times smaller (greater than the first intra prediction mode, but smaller than other intra prediction modes) can be used as one of the M1 intra prediction modes .

It should be understood that the final MPM list of the image blocks to be encoded may be determined by any method of determining the MPM list in the prior art.

For example, if the image block to be coded has an upper adjacent block and a left adjacent block and the best prediction mode is the best intra prediction mode, the first two MPMs in the final MPM list of the image block to be coded are the upper adjacent blocks The optimal intra prediction mode of the block and the left adjacent block; if the image block to be coded has no upper adjacent block or left adjacent block or the optimal prediction mode is not the optimal intra prediction mode, the final prediction mode of the image block to be coded The first MPM in the MPM list is DC mode. If the optimal intra prediction modes of the upper adjacent block and the left adjacent block are equal: (1) If both are Palnar mode, the final MPM list of the image block to be encoded includes Palnar mode, DC mode and angle prediction 26 mode; ( 2) If all are DC mode, then the final MPM list of the image block to be encoded includes DC mode, Planar mode and angle prediction 26 mode; (3) If all are the angle prediction mode, then the final MPM list of the image block to be encoded includes the left The optimal intra prediction mode of the adjacent block (or the upper adjacent block), and the two angle prediction modes closest to the optimal intra prediction mode of the left adjacent block (or the upper adjacent block). If the intra prediction modes of the upper adjacent block and the left adjacent block are not equal, the final MPM list of the image block to be encoded is: the intra prediction mode of the left adjacent block; the DC mode of the upper adjacent block; Planar, DC , Angle prediction 26 The first of these three modes is different from the best intra prediction mode of the left adjacent block and the upper adjacent block.

Optionally, taking the final reconstructed pixels of adjacent blocks of each sub-image block as reference pixels, and determining the final prediction mode of each sub-image block according to the second prediction mode of each sub-image block, including: The final reconstructed pixels of adjacent blocks are used as reference pixels to determine the encoding cost #5 of each sub-image block in the corresponding second prediction mode and the corresponding M2 intra prediction modes; the second prediction mode corresponding to each sub-image block is used. and the prediction mode with the smallest coding cost #5 among the corresponding M2 intra-frame prediction modes, is determined as the final prediction mode of the sub-image block. M2 is a positive integer.

That is to say, for any sub-image block #1 in the V sub-image blocks, the final reconstructed pixel of the adjacent block of the sub-image block #1 is used as a reference pixel to determine the sub-image block #1 in its corresponding second prediction mode and the corresponding encoding cost #5 in M2 intra prediction modes. The obtained M2+1 encoding costs #5 are compared, and the intra prediction mode corresponding to the smallest encoding cost #5 is the final prediction mode of the sub-image block.

It should be noted that, in step S230, if encoding cost #5 is RD cost, RDO reconstruction needs to be performed first when calculating RD cost. Through the RDO reconstruction process, M1 intra-frame prediction modes or M1 intra-frame prediction modes can be obtained the corresponding reconstructed pixels. Wherein, the reconstructed pixel corresponding to the intra prediction mode with the smallest RD cost (that is, the final prediction mode of the image block to be encoded) is the final reconstructed pixel of the image block to be encoded. In step S230, if the encoding cost #5 is not the RD cost, after determining the final prediction mode of the image block to be encoded, the final reconstructed pixel of the adjacent block is used as the reference pixel, and the final prediction mode of the image block to be encoded is used to carry out RDO reconstruction, the obtained reconstructed pixels are the final reconstructed pixels of the image block to be encoded.

After obtaining the final prediction mode of the image block to be encoded in step S230, the method may further include: encoding the final prediction mode of the image block to be encoded, and sending the encoded code stream to the decoding end. Then, the decoding end can decode the transmitted video according to the final prediction mode of the image block to be encoded.

Optionally, as shown in FIG. 7 , the method provided by the present application may be divided into three stages: a mode preliminary screening stage, a mode decision stage, and a mode correction stage.

Mode preliminary screening stage: Select several optimal intra-frame prediction modes from multiple intra-frame prediction modes.

For example, 11 intra-frame prediction modes may be selected from 35 intra-frame prediction modes first, and then 5 better intra-frame prediction modes may be selected from the 11 intra-frame prediction modes.

For example, for the above-mentioned way 1, the 35 intra prediction modes may be firstly divided into 4 subsets according to the direction of the prediction modes, and each subset includes 11 intra prediction modes. Determine a mode set by calculating the gradient information of the original pixels of the current block to be coded, and then use the 11 prediction modes in the mode set to perform intra-frame prediction (the reference pixels used in prediction are the original pixels of adjacent blocks) to obtain the predicted block , and then subtract the prediction block from the original pixel block to get the residual, and then perform Hadamard transform to calculate the HAD cost, and select the 5 modes with the smallest HAD cost to transmit to the mode decision stage. For each sub-image block in the V sub-image blocks in the current block to be encoded, the 35 intra prediction modes are firstly divided into 4 subsets according to the direction of the prediction mode, and each subset contains 11 intra prediction modes. A mode set is determined by calculating the gradient information of the original pixels of the current sub-image block, and then the 11 prediction modes in the mode set are used for intra-frame prediction (the reference pixels used in the prediction are the original pixels of the adjacent blocks of the sub-image block). pixel) to obtain the prediction block, and then subtract the prediction block from the original pixel block to obtain the residual, perform Hadamard transform to calculate HAD cost, and select the 5 modes with the smallest HAD cost to transmit to the mode decision stage.

For the above-mentioned mode 2, the 35 intra prediction modes may be firstly divided into 4 subsets according to the direction of the prediction modes, and each subset includes 11 intra prediction modes. Determine a mode set by calculating the gradient information of the original pixels of the current block to be coded, and then use the 11 prediction modes in the mode set to perform intra-frame prediction (the reference pixels used in prediction are the original pixels of adjacent blocks) to obtain the predicted block , and then subtract the prediction block from the original pixel block to get the residual, and then perform Hadamard transform to calculate the HAD cost, and select the 5 modes with the smallest HAD cost to transmit to the mode decision stage.

Mode decision stage: Combine several better intra prediction modes obtained in the mode preliminary screening stage with the intra prediction modes in the MPM list (ie, the initial MPM list) generated by adjacent blocks in the mode decision stage to obtain a variety of modes. Intra-frame prediction mode, and then select the optimal one from these multiple prediction modes to perform the RDO reconstruction process to obtain RD cost. Then compare the RD cost with the RD cost of inter-frame prediction to determine the encoding mode (select the intra-frame prediction mode or select the inter-frame prediction mode).

For example, corresponding to the above method 1, for the current block to be coded, the five intra-frame prediction modes obtained in the mode preliminary screening stage and the MPM list generated in the mode decision stage of the adjacent blocks of the current block to be coded (that is, the current block to be coded The 3 intra-frame prediction modes in the initial MPM list of adjacent blocks) are combined to obtain new 5 intra-frame prediction modes, and then the optimal one is selected from these 5 prediction modes for the RDO reconstruction process. RD cost. For each sub-image block, compare the five intra-frame prediction modes obtained in the mode preliminary screening stage with the MPM list generated in the mode decision stage of the adjacent blocks of the sub-image small block (that is, the initial MPM of the adjacent blocks of the sub-image block). The 3 intra prediction modes in the list) are combined to obtain new 5 intra prediction modes, and then the optimal one is selected from these 5 prediction modes to perform the RDO reconstruction process to obtain the RD cost. Compare the sum of the RD cost of the V sub-image blocks with the size of the RD cost of the current block to be coded, and compare the smaller RD cost with the RD cost of inter-frame prediction to determine the coding mode.

For example, corresponding to the above method 2, compare the five intra-frame prediction modes obtained in the mode preliminary screening stage with the MPM list generated by the adjacent blocks of the current block to be coded in the mode decision stage (that is, the initial The 3 intra-frame prediction modes in the MPM list) are combined to obtain 5 new intra-frame prediction modes, and then the optimal one is selected from these 5 prediction modes to perform the RDO reconstruction process to obtain the RD cost. The coding mode is determined by comparing the RD cost with the RD cost of inter-frame prediction.

Mode correction stage: If the coding mode determined in the mode decision stage is the intra-frame coding mode, the real reconstructed pixels of adjacent blocks (ie, the final reconstructed pixels of adjacent blocks) and the coding mode are used for intra-frame prediction in the mode modification stage. correction.

For example, using the intra prediction mode decided in the mode decision stage to perform the RDO reconstruction process to obtain the reconstructed pixels and RD cost, where the reference pixels used in intra prediction are the reconstructed pixels of adjacent blocks in the mode correction stage (that is, adjacent the final reconstructed pixels of the block). At the same time, 3 intra prediction modes in the MPM list (ie, the final MPM list) are generated according to the coding mode and prediction mode information of the adjacent blocks in the mode modification stage, and the first and mode are selected from the 3 intra prediction modes. Different intra-frame prediction modes passed from the decision-making stage perform the RDO reconstruction process to obtain reconstructed pixels and RD cost. Compare the RD cost of the above two intra prediction modes, select the intra prediction mode with the smaller RD cost as the optimal intra prediction mode in the mode correction stage (that is, the final prediction mode of the image block to be encoded), and save its reconstructed pixels. As the final encoded reconstructed pixel (ie, the final reconstructed pixel of the image block to be encoded). Wherein, if the final prediction mode of the adjacent block is the intra prediction mode, the prediction mode information is the intra prediction mode, and if the final prediction mode of the adjacent block is the inter prediction mode, no the prediction mode information.

It should be understood that, if in the mode decision stage, the sum of the RD cost of the V sub-image blocks is smaller than the size of the RD cost of the current block to be encoded, the mode correction stage is directed to each sub-image block. For example, the final MPM list is the final MPM list of sub-image blocks.

It should be understood that, in an example, the above steps (A1) and (B1) correspond to the mode preliminary screening stage; (A2), (B2), S330 and S210 correspond to the mode decision stage; and S230 corresponds to the mode correction stage.

In another example, the above steps (A1) correspond to the mode preliminary screening stage; (A2) and S210 correspond to the mode decision stage; and S230 corresponds to the mode correction stage.

Optionally, the mode preliminary screening stage corresponds to the R-2 flow level; the mode decision stage corresponds to the R-1 flow stage; and the mode correction stage corresponds to the R-th flow stage.

Further, after the mode correction stage is completed, loop filtering and entropy coding can also be performed. Regarding how to perform in-loop filtering and entropy coding, reference may be made to the prior art.

Optionally, the loop filtering may correspond to the R+1th stage pipeline, and the entropy encoding may correspond to the R+2th stage pipeline.

Exemplarily, FIG. 8 shows a schematic diagram of a pipeline structure based on the method shown in FIG. 2 . Referring to FIG. 8 , one-stage pipelines are staggered between adjacent coding regions. For example, when coding region 1 goes to R-1 pipeline level, its adjacent coding region 2 goes to R-2 level pipeline. It should be understood that the different stages of pipeline work in parallel.

Those skilled in the art can understand that the N1 intra-frame prediction modes mentioned above are obtained from the R-2 pipeline stage; the optimal intra-frame prediction mode and the initial reconstructed pixels are obtained from the R-1 pipeline stage; the final prediction mode and The final reconstructed pixels are obtained by the R pipeline stage.

The method embodiments of the present application are described in detail above with reference to FIGS. 1 to 8 , and the apparatus embodiments of the present application are described in detail below with reference to FIG. 9 . It should be understood that the descriptions of the method embodiments correspond to the descriptions of the apparatus embodiments. Therefore, for the parts not described in detail, reference may be made to the foregoing method embodiments.

FIG. 9 is a schematic structural diagram of an encoding device provided by the present application. The apparatus 900 may be a physical device or a chip. The device may be a camera, an unmanned aerial vehicle, a mobile phone, or other equipment with a video codec processing function. The apparatus includes: a memory 910 and a processor 920 .

Memory 910 may be used to store code. The processor 920 may be configured to read the code in the memory to perform the following operations: using the initial reconstructed pixels of the adjacent blocks of the image block to be encoded as reference pixels, determine the optimal intra prediction mode of the image block to be encoded and initial reconstructed pixels, wherein the reconstructed pixels of the to-be-coded image block in the optimal intra prediction mode of the to-be-coded image block are the initial reconstructed pixels of the to-be-coded image block; In the best intra prediction mode of the to-be-coded image block and the inter-frame prediction mode of the to-be-coded image block, determine the best prediction mode of the to-be-coded image block; if the best prediction mode is the best prediction mode of the to-be-coded image block the optimal intra prediction mode, taking the final reconstructed pixels of the adjacent blocks as reference pixels, and determining the final prediction mode and the final reconstructed pixels of the to-be-coded image block according to the best intra-prediction mode of the to-be-coded image block , wherein the reconstructed pixels in the final prediction mode of the to-be-coded image block are the final reconstructed pixels of the to-be-coded image block.

Optionally, the determining the optimal intra prediction mode and the initial reconstructed pixels of the image block to be encoded by using the initial reconstructed pixels of the adjacent blocks of the image block to be encoded as reference pixels includes: The initial reconstructed pixels of adjacent blocks of the block are used as reference pixels to determine the first intra prediction mode and the first initial reconstructed pixels of the to-be-coded image block; each sub-image block of the V sub-image blocks included in the to-be-coded image block is The initial reconstructed pixels of adjacent blocks of the image block are used as reference pixels to determine the second intra-frame prediction mode and the second initial reconstructed pixels of each sub-image block, and the V sub-image blocks are the image blocks to be encoded according to Obtained by the first division method, V is a positive integer; according to the first coding cost of the to-be-coded image block in the first intra-frame prediction mode and each sub-image block in the V sub-image blocks in the The sum of the first coding costs in the second intra-frame prediction mode determines the optimal intra-frame prediction mode and initial reconstructed pixels of the image block to be coded.

Optionally, the determining the first intra prediction mode and the first initial reconstructed pixel of the image block to be encoded by using the initial reconstructed pixels of the adjacent blocks of the image block to be encoded as reference pixels includes: The initial reconstructed pixels of adjacent blocks of the to-be-coded image block are used as reference pixels, and N1 intra-frame prediction modes are used to perform intra-frame prediction on the to-be-coded image block, and the first intra-frame prediction mode of the to-be-coded image block is determined. and the first initial reconstructed pixel, N1 is a positive integer.

Optionally, determining the second intra prediction mode of each sub-image block by using the initial reconstructed pixels of adjacent blocks of each sub-image block in the V sub-image blocks included in the to-be-coded image block as reference pixels and a second initial reconstructed pixel, comprising: using the initial reconstructed pixel of the adjacent block of each sub-image block as a reference pixel, and using N2 intra prediction modes corresponding to each sub-image block The block performs intra-frame prediction to determine the second intra-frame prediction mode and the second initial reconstruction pixel of each sub-image block, and N2 is a positive integer.

Optionally, the processor is further configured to perform the following operation: according to the gradient information of the original pixels of the image block to be encoded, determine P1 intra prediction modes from Q1 intra prediction modes, Q1>P1, And Q1 and P1 are both integers; the N1 intra prediction modes are determined according to the P1 intra prediction modes.

Optionally, PI=N1.

Optionally, determining the P1 intra prediction modes from the Q1 intra prediction modes according to the gradient information of the original pixels of the to-be-encoded image block includes: determining a plurality of P1 intra prediction modes corresponding to the Q1 intra prediction modes. subsets, each of the subsets includes multiple intra prediction modes in the Q1 intra prediction modes; according to the gradient information of the original pixels of the image block to be encoded, from the multiple subsets Determining a subset; using the original pixels of adjacent blocks of the image block to be encoded as reference pixels, and using each intra prediction mode in the determined subset to perform intra prediction on the image block to be encoded, determining the first encoding cost of the image block to be encoded in each intra-frame prediction mode in the determined subset; determining the P1 intra-frame prediction modes with the smallest first encoding cost in the determined subset is the P1 intra prediction mode.

Optionally, the multiple subsets are 4 subsets.

Optionally, the number of intra prediction modes included in each subset is the same.

Optionally, the Q1 intra-frame prediction modes include Planar mode, DC mode, and 33 angular modes, and/or the types of intra-frame prediction modes included in each subset are 11 types.

Optionally, determining the N1 intra prediction modes according to the P1 intra prediction modes includes: according to the F1 intra prediction modes in the initial MPM list of adjacent blocks of the to-be-coded image block and The P1 intra prediction modes determine the N1 intra prediction modes, wherein the initial MPM list is determined according to the first intra prediction mode of the adjacent block of the to-be-coded image block, and F1 is positive integer.

Optionally, the N1 intra prediction modes are determined according to the F1 intra prediction modes and the P1 intra prediction modes in the initial MPM list of the adjacent blocks of the to-be-coded image block, including: : take N1-F1 intra-frame prediction modes that are different from the F1 intra-frame prediction modes and the F1 intra-frame prediction modes as the N1 intra-frame prediction modes .

Optionally, P1=N1, F1=3.

Optionally, the processor 720 may further perform the following operation: for each sub-image block, according to the gradient information of the sub-image block, determine P2 intra prediction modes from Q2 intra prediction modes, Q2>P2, and both Q2 and P2 are integers; N2 intra prediction modes of each sub image block are determined according to the P2 intra prediction modes of each sub image block.

Optionally, P2=N2.

Optionally, determining P2 intra-frame prediction modes from the Q2 intra-frame prediction modes according to the gradient information of the sub-image block, including: according to the direction and/or type of the intra-frame prediction mode, converting the Q2 kinds of frame The intra prediction mode is divided into multiple subsets, and each subset in the multiple subsets includes multiple intra prediction modes in the Q2 intra prediction modes; according to the gradient information of the original pixels of the sub image block, from A subset is determined from the plurality of subsets; the original pixels of the adjacent blocks of the sub-image block are used as reference pixels, and each intra-frame prediction mode in the determined subset is used to perform a frame rate on the sub-image block. Intra-prediction, determining the first coding cost of the sub-image block in each intra-frame prediction mode in the determined subset; performing the P2 intra-frame predictions with the smallest first coding cost in the determined subset The mode is determined to be the P2 intra prediction modes.

Optionally, the multiple subsets are 4 subsets.

Optionally, the number of intra prediction modes included in each subset is the same.

Optionally, the Q2 intra-frame prediction modes include Planar mode, DC mode, and 33 angle modes, and/or, the types of intra-frame prediction modes included in each subset are 11 types.

Optionally, determining N2 intra-frame prediction modes of each sub-image block according to the P2 intra-frame prediction modes of each sub-image block, including: F2 intra-frame prediction modes in the initial MPM list and P2 intra-frame prediction modes of each sub-image block are determined, and N2 intra-frame prediction modes of each sub-image block are determined, wherein the initial MPM list is based on It is determined by the second intra-frame prediction mode of the adjacent blocks of the sub-image block, and F2 is a positive integer.

Optionally, according to the F2 intra-frame prediction modes in the initial MPM list of adjacent blocks of each sub-image block and the P2 intra-frame prediction modes of each sub-image block, determine the each sub-image. N2 intra-frame prediction modes for blocks, including: for each sub-image block, assigning the P2 kinds of intra-frame prediction modes of the sub-image block different from the F2 kinds of intra-frame prediction modes of the sub-image block N2-F2 intra prediction modes, and the F2 intra prediction modes of the sub image block are used as the N2 intra prediction modes of the sub image block.

Optionally, P2=N2, F2=3.

Optionally, the first encoding cost is a Hadamard cost

Optionally, determining the final prediction mode of the to-be-coded image block according to the best intra-frame prediction mode of the to-be-coded image block by using the final reconstructed pixel of the adjacent block as a reference pixel, including: if The best intra-frame prediction mode of the image block to be encoded is the first intra-frame prediction mode, and the final reconstructed pixel of the adjacent block of the image block to be encoded is used as a reference pixel, and it is determined that the image block to be encoded is in The second coding cost under the optimal intra prediction mode and M1 intra prediction modes, and the minimum intra prediction of the second coding cost in the optimal intra prediction mode and M1 intra prediction modes. mode, determined as the final prediction mode of the to-be-coded image block, M1 is a positive integer; or, if the best intra-frame prediction mode of the to-be-coded image block is the second intra frame of the V sub-image blocks Prediction mode, for each sub-image block, the final reconstructed pixel of the adjacent block of the sub-image block is used as a reference pixel to determine the second intra-frame prediction mode of the sub-image block and the second coding cost in the M2 intra prediction modes of the sub image block, and the second intra prediction mode of the sub image block and the M2 intra prediction modes of the sub image block The second intra-frame prediction mode with the least coding cost is determined as the final prediction mode of the sub-image block, and M2 is a positive integer.

Optionally, the M1 intra prediction modes include one or both of the following: one or more of the W1 intra prediction modes in the final most probable mode MPM list of the image block to be encoded. The final MPM list of the image block to be encoded is generated according to the final prediction mode of the adjacent blocks of the image block to be encoded and/or the best prediction mode of the adjacent blocks of the image block to be encoded , W1 is a positive integer; among the N1 intra-frame prediction modes, one or more intra-frame prediction modes in which the second coding cost is the smallest except for the first intra-frame prediction mode.

Optionally, M1=1, and the M1 intra prediction modes include the first or second or W1th in the final MPM list of the image block to be encoded and the first frame of the image block to be encoded. Intra prediction modes with different intra prediction modes.

Optionally, the M2 intra-frame prediction modes of the sub-image block include one or both of the following: one of the W2 intra-frame prediction modes in the final MPM list of the sub-image block. One or more, wherein the final MPM list of the sub-image block is generated according to the final prediction mode of the adjacent blocks of the sub-image block and/or the best prediction mode of the adjacent blocks of the to-be-coded image block , W2 is a positive integer; among the N2 intra prediction modes of the sub-image block, one or more intra-frames with the smallest second coding cost except the second intra-frame prediction mode of the sub-image block forecast mode.

Optionally, M2=1, and the M2 intra prediction modes include the first or the second or the W2th in the final MPM list of the sub image block and the second intra prediction of the sub image block. Intra prediction modes with different modes.

Optionally, the second coding cost is a rate-distortion cost.

Optionally, the determining the optimal intra prediction mode and the first initial reconstruction pixel of the to-be-coded image block by using the initial reconstructed pixels of the adjacent blocks of the to-be-coded image block as reference pixels includes: The initial reconstructed pixels of adjacent blocks of the coded image block are used as reference pixels, and N1 intra-frame prediction modes are used to perform intra-frame prediction on the to-be-coded image block, and the optimal intra-frame prediction mode and initial frame of the to-be-coded image block are determined. Reconstructed pixel, N1 is a positive integer.

Optionally, the processor 920 may further perform the following operations: encode the final prediction mode of the image block to be encoded, and send the encoded code stream to the decoding end.

In addition, the present application also provides a computer-readable storage medium, on which a computer program (or referred to as a computer instruction) is stored, when the computer program is run on a computer, the computer executes the method to implement The video processing method in the example.

The present application further provides a computer program product, the computer program product includes computer program code, when the computer program code is run on a computer, the computer program code causes the computer to execute the video processing method in the method embodiment of the present application.

The present application also provides a chip including a processor. The memory for storing the computer program is provided independently of the chip, and the processor is used for executing the computer program stored in the memory to execute the video processing method in the method embodiment.

Optionally, the chip further includes the memory.

Further optionally, the chip further includes a communication interface. The communication interface may be a transceiver, an input/output interface, a pin or a circuit, or the like.

Optionally, the above-mentioned processors may be one or more, and the memory may be one or more.

Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

Optionally, the processor may be a logic circuit, an integrated circuit, or the like.

The present application also provides a chip, which includes an R-2 flow stage, an R-1 flow stage, and an R flow stage. The R-2 flow level, the R-1 flow level, and the R flow level may refer to the foregoing description, and will not be repeated here.

Optionally, the R-2 flow stage, the R-1 flow stage and the R flow stage work in parallel.

In the above embodiments, the processor may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a ready-made Programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, microprocessors, or one or more integrated circuits for controlling the execution of programs in the present application, etc. . For example, a processor may include a digital signal processor device, a microprocessor device, an analog-to-digital converter, a digital-to-analog converter, and the like. The processor may distribute the control and signal processing functions of the mobile device among the devices according to their respective functions. Additionally, the processor may include functionality to operate one or more software programs, which may be stored in memory. The functions of the processor may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above functions.

The memory can be read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of storage devices that can store information and instructions Dynamic storage device. It can also be electrically erasable programmable read-only memory (electrically erasable programmable read-only memory, EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, Laser Disc, Optical Disc, Digital Versatile Disc, Blu-ray Disc, etc.), magnetic disk storage medium or other magnetic storage device, or any other capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer medium, but not limited to this.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line, DSL) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, digital video disc (DVD)), or semiconductor media (eg, solid state disk (SSD)), etc. .

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or Can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the technical solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (66)

A video processing method, comprising: Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, determine the optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block, wherein the to-be-coded image block is in the to-be-coded image The reconstructed pixels in the optimal intra prediction mode of the block are the initial reconstructed pixels of the to-be-coded image block; From the optimal intra prediction mode of the to-be-coded image block and the inter-frame prediction mode of the to-be-coded image block, determine the optimal prediction mode of the to-be-coded image block; If the optimal prediction mode is the optimal intra prediction mode of the image block to be encoded, the final reconstructed pixel of the adjacent block is used as a reference pixel, and the optimal intra prediction mode of the image block to be encoded is used as the reference pixel , determining the final prediction mode and final reconstructed pixels of the image block to be encoded, wherein the reconstructed pixels in the final prediction mode of the image block to be encoded are the final reconstructed pixels of the image block to be encoded. The method according to claim 1, wherein the initial reconstructed pixels of adjacent blocks of the to-be-coded image block are used as reference pixels to determine the optimal intra-frame prediction mode and the initial reconstructed pixels of the to-be-coded image block ,include: Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, determining the first intra-frame prediction mode and the first initial reconstructed pixels of the to-be-coded image block; Using the initial reconstructed pixels of adjacent blocks of each sub-image block in the V sub-image blocks included in the to-be-coded image block as reference pixels, determining the second intra-frame prediction mode and the second initial reconstructed pixels of each sub-image block , the V sub-image blocks are obtained by dividing the to-be-coded image block according to the first division method, and V is a positive integer; according to the first encoding cost of the image block to be encoded in the first intra prediction mode and the first encoding cost of each sub image block in the V sub image blocks in the second intra prediction mode and, determining the optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block. The method according to claim 2, wherein the initial reconstruction pixel of the adjacent block of the to-be-coded image block is used as a reference pixel to determine the first intra-frame prediction mode and the first intra-frame prediction mode of the to-be-coded image block. An initial reconstructed pixel, including: Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, adopting N1 intra-frame prediction modes to perform intra-frame prediction on the to-be-coded image block, and determine the first intra-frame of the to-be-coded image block The prediction mode and the first initial reconstructed pixel, N1 is a positive integer. The method according to claim 2 or 3, wherein the initial reconstructed pixels of adjacent blocks of each sub-image block in the V sub-image blocks included in the to-be-coded image block are used as reference pixels to determine the The second intra prediction mode and the second initial reconstructed pixels of each sub-image block, including: Using the initial reconstructed pixels of adjacent blocks of each sub-image block as reference pixels, using N2 intra-frame prediction modes corresponding to each sub-image block to perform intra-frame prediction on each sub-image block, and determining the The second intra prediction mode and the second initial reconstructed pixel of each sub-image block, N2 is a positive integer. The method according to claim 3 or 4, wherein before determining the first intra prediction mode of the image block to be encoded, the method further comprises: According to the gradient information of the original pixels of the to-be-coded image block, determine P1 intra-frame prediction modes from Q1 intra-frame prediction modes, where Q1>P1, and both Q1 and P1 are integers; The N1 intra prediction modes are determined according to the P1 intra prediction modes. The method of claim 5, wherein PI=N1. The method according to claim 5 or 6, wherein the determining, according to the gradient information of the original pixels of the to-be-coded image block, the P1 intra prediction modes from the Q1 intra prediction modes, comprising: determining multiple subsets corresponding to the Q1 intra prediction modes, each subset in the multiple subsets includes multiple intra prediction modes in the Q1 intra prediction modes; determining a subset from the plurality of subsets according to the gradient information of the original pixels of the image block to be encoded; Using the original pixels of adjacent blocks of the to-be-coded image block as reference pixels, using each intra-frame prediction mode in the determined subset, perform intra-frame prediction on the to-be-coded image block, and determine the to-be-coded image block. the first coding cost of the image block in each intra prediction mode in the determined subset; The P1 intra-frame prediction modes with the smallest first coding cost in the determined subset are determined as the P1 intra-frame prediction modes. The method of claim 7, wherein the plurality of subsets is 4 subsets. The method according to claim 7 or 8, wherein the number of intra prediction modes included in each subset is the same. The method of claim 9, wherein the Q1 intra-frame prediction modes include Planar mode, DC mode, and 33 angle modes, and/or the types of intra-frame prediction modes included in each subset 11 species. The method according to any one of claims 5 to 10, determining the N1 intra prediction modes according to the P1 intra prediction modes, comprising: The N1 intra prediction modes are determined according to the F1 intra prediction modes and the P1 intra prediction modes in the initial MPM list of adjacent blocks of the image block to be encoded, wherein the initial MPM list is determined according to the first intra-frame prediction mode of the adjacent block of the image block to be encoded, and F1 is a positive integer. The method according to claim 11, wherein the determination of the F1 intra prediction modes and the P1 intra prediction modes in the initial MPM list of the adjacent blocks of the image block to be coded is performed. The N1 intra prediction modes, including: Among the P1 intra prediction modes, N1-F1 intra prediction modes that are different from the F1 intra prediction modes, and the F1 intra prediction modes are used as the N1 intra prediction modes. The method of claim 12, wherein P1=N1, F1=3. The method according to any one of claims 5 to 13, wherein before the determining the second intra prediction mode of each sub-image block, the method further comprises: For each sub-image block, according to the gradient information of the sub-image block, determine P2 intra-frame prediction modes from Q2 intra-frame prediction modes, where Q2>P2, and both Q2 and P2 are integers; According to the P2 intra prediction modes of each sub image block, N2 intra prediction modes of each sub image block are determined. The method of claim 14, wherein P2=N2. The method according to claim 14 or 15, wherein the determining, according to the gradient information of the sub-image block, P2 intra prediction modes from the Q2 intra prediction modes, comprising: determining multiple subsets corresponding to the Q2 intra prediction modes, each subset in the multiple subsets includes multiple intra prediction modes in the Q2 intra prediction modes; determining a subset from the plurality of subsets according to the gradient information of the original pixels of the sub-image block; Using the original pixels of adjacent blocks of the sub-image block as reference pixels, using each intra-frame prediction mode in the determined subset, perform intra-frame prediction on the sub-image block, and determine that the sub-image block is within the first coding cost under each intra prediction mode in the determined subset; The P2 intra-frame prediction modes with the smallest first coding cost in the determined subset are determined as the P2 intra-frame prediction modes. The method of claim 16, wherein the plurality of subsets is 4 subsets. The method of claim 16 or 17, wherein the number of intra prediction modes included in each subset is the same. The method of claim 18, wherein the Q2 intra prediction modes include Planar mode, DC mode and 33 angle modes, and/or the types of intra prediction modes included in each subset 11 species. The method according to any one of claims 14 to 19, wherein, according to the P2 intra prediction modes of each sub image block, determining N2 intra prediction modes of each sub image block, comprising: : According to the F2 intra-frame prediction modes in the initial MPM list of adjacent blocks of each sub-image block and the P2 intra-frame prediction modes of each sub-image block, determine the N2 intra-frame prediction modes of each sub-image block prediction mode, wherein the initial MPM list is determined according to the second intra prediction mode of the adjacent blocks of the sub-image block, and F2 is a positive integer. The method according to claim 20, wherein the F2 intra-frame prediction modes in the initial MPM list of adjacent blocks of each sub-image block and the P2 intra-frame prediction modes of each sub-image block are performed. Prediction mode, determining N2 intra-frame prediction modes of each sub-image block, including: For each sub-image block, the N2-F2 intra-frame prediction modes that are different from the F2 kinds of intra-frame prediction modes of the sub-image block among the P2 kinds of intra-frame prediction modes of the sub-image block, and the The F2 intra-frame prediction modes of the sub-image block are used as the N2 intra-frame prediction modes of the sub-image block. The method of claim 21, wherein P2=N2, F2=3. The method according to claim 7 or 16, wherein the first coding cost is a Hadamard cost. The method according to claim 4, wherein the final reconstructed pixel of the adjacent block is used as a reference pixel, and the to-be-encoded image is determined according to an optimal intra-frame prediction mode of the to-be-encoded image block The final prediction mode of the block, including: If the optimal intra prediction mode of the image block to be encoded is the first intra prediction mode, the final reconstructed pixel of the adjacent block of the image block to be encoded is used as a reference pixel, and the image block to be encoded is determined. the second coding cost in the best intra prediction mode and the M1 intra prediction modes, and the second coding cost in the best intra prediction mode and the M1 intra prediction modes is the smallest intra-frame Prediction mode, determined as the final prediction mode of the to-be-coded image block, M1 is a positive integer; or, If the optimal intra prediction mode of the image block to be encoded is the second intra prediction mode of the V sub-image blocks, for each sub-image block, the final reconstruction of the adjacent blocks of the sub-image block is performed. pixel as a reference pixel, determine the second coding cost of the sub-image block in the second intra-frame prediction mode of the sub-image block and the M2 intra-frame prediction modes of the sub-image block, and use The second intra-frame prediction mode of the sub-image block and the second intra-frame prediction mode with the least coding cost among the M2 intra-frame prediction modes of the sub-image block are determined as the final prediction of the sub-image block mode, M2 is a positive integer. The method of claim 24, wherein the M1 intra prediction modes include one or both of the following: One or more of W1 intra-frame prediction modes in the final MPM list of the image blocks to be encoded, wherein the final MPM list of the image blocks to be encoded is based on the The final prediction mode of the adjacent block and/or the optimal prediction mode of the adjacent block of the to-be-coded image block is generated, and W1 is a positive integer; One or more intra-frame prediction modes in which the second coding cost is the smallest among the N1 intra-frame prediction modes except the first intra-frame prediction mode. The method of claim 25, wherein M1=1, and the M1 intra-frame prediction modes include the first or the second or the W1th in the final MPM list of the to-be-coded image block and the Intra-frame prediction modes with different first intra-frame prediction modes of the image block to be encoded. The method according to any one of claims 24 to 26, wherein the M2 intra prediction modes of the sub-image block include one or both of the following: One or more of W2 intra prediction modes in the final MPM list of the sub-image block, wherein the final MPM list of the sub-image block is based on neighboring blocks of the sub-image block The final prediction mode and/or the best prediction mode of the adjacent block of the to-be-coded image block is generated, and W2 is a positive integer; Among the N2 intra-frame prediction modes of the sub-image block, one or more intra-frame prediction modes with the smallest coding cost except the second intra-frame prediction mode of the sub-image block. The method of claim 27, wherein M2=1, and the M2 intra prediction modes include the first or the second or the W2th in the final MPM list of the sub-image block and the The second intra-prediction mode of the sub-image block is an intra-prediction mode with a different intra-prediction mode. The method according to any one of claims 24 to 28, wherein the second coding cost is a rate-distortion cost. The method of claim 5, wherein the method is performed by a chip comprising an R-2 pipeline, an R-1 pipeline, and an R pipeline; Wherein, the following operations are performed by the R-2 pipeline stage: determining P1 intra prediction modes from Q1 intra prediction modes according to the gradient information of the image block to be encoded; sub-image blocks, and according to the gradient information of the sub-image blocks, determine P2 intra prediction modes from Q2 intra prediction modes; The following operations are performed by the R-1 pipeline stage: the N1 intra prediction modes are determined according to the P1 intra prediction modes; the P2 frames are determined according to the P2 frames of each sub-image block Intra-prediction mode, to determine N2 intra-frame prediction modes for each sub-image block; the initial reconstructed pixels of adjacent blocks of the image block to be encoded are used as reference pixels to determine the optimal intra-frame prediction mode of the image block to be encoded a prediction mode and an initial reconstructed pixel; and, determining the best prediction of the image block to be encoded from the optimal intra prediction mode of the image block to be encoded and the inter prediction mode of the image block to be encoded model; The following operations are performed by the R pipeline stage: taking the final reconstructed pixels of the adjacent blocks as reference pixels, and determining the final prediction of the to-be-coded image block according to the best intra-frame prediction mode of the to-be-coded image block mode and final reconstructed pixels. The method of claim 30, wherein the R-2 pipeline stage, the R-1 pipeline stage, and the R pipeline stage work in parallel. The method according to claim 1, wherein the optimal intra prediction mode and the initial reconstructed pixels of the to-be-coded image block are determined by using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels ,include: Using the initial reconstructed pixels of adjacent blocks of the image block to be encoded as reference pixels, use N1 intra prediction modes to perform intra prediction on the image block to be encoded, and determine the optimal intra frame of the image block to be encoded. Prediction mode and initial reconstructed pixels, N1 is a positive integer. The method of any one of claims 1 to 32, wherein the method further comprises: The final prediction mode of the image block to be encoded is encoded, and the encoded code stream is sent to the decoding end. An encoding device, comprising: a memory and a processor, wherein, the memory is used to store computer programs; The processor invokes the computer program, and when the computer program is executed, performs the following operations: Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, determine the optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block, wherein the to-be-coded image block is in the to-be-coded image block. The reconstructed pixels in the optimal intra-frame prediction mode of the block are the initial reconstructed pixels of the to-be-coded image block; From the optimal intra prediction mode of the to-be-coded image block and the inter-frame prediction mode of the to-be-coded image block, determine the optimal prediction mode of the to-be-coded image block; If the optimal prediction mode is the optimal intra prediction mode of the image block to be encoded, the final reconstructed pixel of the adjacent block is used as a reference pixel, and the optimal intra prediction mode of the image block to be encoded is used as the reference pixel , determining the final prediction mode and final reconstructed pixels of the image block to be encoded, wherein the reconstructed pixels in the final prediction mode of the image block to be encoded are the final reconstructed pixels of the image block to be encoded. The apparatus according to claim 34, wherein the initial reconstructed pixels of adjacent blocks of the to-be-coded image block are used as reference pixels to determine the optimal intra-frame prediction mode and the initial reconstructed pixels of the to-be-coded image block ,include: Using the initial reconstructed pixels of the adjacent blocks of the to-be-coded image block as reference pixels, determining the first intra-frame prediction mode and the first initial reconstructed pixels of the to-be-coded image block; Using the initial reconstructed pixels of adjacent blocks of each sub-image block in the V sub-image blocks included in the to-be-coded image block as reference pixels, determining the second intra-frame prediction mode and the second initial reconstructed pixels of each sub-image block , the V sub-image blocks are obtained by dividing the to-be-coded image block according to the first division method, and V is a positive integer; according to the first encoding cost of the image block to be encoded in the first intra prediction mode and the first encoding cost of each sub image block in the V sub image blocks in the second intra prediction mode and, determining the optimal intra-frame prediction mode and initial reconstructed pixels of the to-be-coded image block. The apparatus according to claim 35, wherein the initial reconstruction pixel of the adjacent block of the to-be-coded image block is used as a reference pixel to determine the first intra-frame prediction mode and the first intra-frame prediction mode of the to-be-coded image block. An initial reconstructed pixel, including: Using the initial reconstructed pixels of adjacent blocks of the to-be-coded image block as reference pixels, adopting N1 intra-frame prediction modes to perform intra-frame prediction on the to-be-coded image block, and determine the first intra-frame of the to-be-coded image block The prediction mode and the first initial reconstructed pixel, N1 is a positive integer. The apparatus according to claim 35 or 36, wherein the initial reconstructed pixels of adjacent blocks of each of the V sub-image blocks included in the to-be-coded image block are used as reference pixels to determine the The second intra prediction mode and the second initial reconstructed pixels of each sub-image block, including: Using the initial reconstructed pixels of adjacent blocks of each sub-image block as reference pixels, using N2 intra-frame prediction modes corresponding to each sub-image block to perform intra-frame prediction on each sub-image block, and determining the The second intra prediction mode and the second initial reconstructed pixel of each sub-image block, N2 is a positive integer. The apparatus of claim 36 or 37, wherein the processor is further configured to perform the following operations: According to the gradient information of the original pixels of the to-be-coded image block, determine P1 intra-frame prediction modes from Q1 intra-frame prediction modes, where Q1>P1, and both Q1 and P1 are integers; The N1 intra prediction modes are determined according to the P1 intra prediction modes. 39. The apparatus of claim 38, wherein PI=N1. The apparatus according to claim 38 or 39, wherein the determining of the P1 intra prediction modes from the Q1 intra prediction modes according to the gradient information of the original pixels of the to-be-coded image block comprises: determining multiple subsets corresponding to the Q1 intra prediction modes, each subset in the multiple subsets includes multiple intra prediction modes in the Q1 intra prediction modes; determining a subset from the plurality of subsets according to the gradient information of the original pixels of the image block to be encoded; Using the original pixels of adjacent blocks of the to-be-coded image block as reference pixels, using each intra-frame prediction mode in the determined subset, perform intra-frame prediction on the to-be-coded image block, and determine the to-be-coded image block. the first coding cost of the image block in each intra prediction mode in the determined subset; The P1 intra-frame prediction modes with the smallest first coding cost in the determined subset are determined as the P1 intra-frame prediction modes. The apparatus of claim 40, wherein the plurality of subsets is 4 subsets. The apparatus of claim 40 or 41, wherein the number of intra prediction modes included in each subset is the same. The apparatus of claim 42, wherein the Q1 intra-frame prediction modes include Planar mode, DC mode, and 33 angle modes, and/or the types of intra-frame prediction modes included in each subset 11 species. The apparatus according to any one of claims 38 to 43, determining the N1 intra prediction modes according to the P1 intra prediction modes, comprising: The N1 intra prediction modes are determined according to the F1 intra prediction modes and the P1 intra prediction modes in the initial MPM list of adjacent blocks of the image block to be encoded, wherein the initial MPM list is determined according to the first intra-frame prediction mode of the adjacent block of the image block to be encoded, and F1 is a positive integer. The apparatus according to claim 44, wherein the determination of the F1 intra prediction modes and the P1 intra prediction modes in the initial MPM list of the adjacent blocks of the image block to be coded is performed. The N1 intra prediction modes, including: Among the P1 types of intra-frame prediction modes, N1-F1 types of intra-frame prediction modes that are different from the F1 types of intra-frame prediction modes, and the F1 types of intra-frame prediction modes are used as the N1 types of intra-frame prediction modes. The apparatus of claim 45, wherein P1=N1 and F1=3. The apparatus of any one of claims 38 to 46, wherein the processor is further configured to perform the following operations: For each sub-image block, according to the gradient information of the sub-image block, determine P2 intra-frame prediction modes from Q2 intra-frame prediction modes, where Q2>P2, and both Q2 and P2 are integers; According to the P2 intra prediction modes of each sub image block, N2 intra prediction modes of each sub image block are determined. 48. The apparatus of claim 47, wherein P2=N2. The apparatus according to claim 47 or 48, wherein, according to the gradient information of the sub-image block, determining P2 intra prediction modes from Q2 intra prediction modes, comprising: determining multiple subsets corresponding to the Q2 intra prediction modes, each subset in the multiple subsets includes multiple intra prediction modes in the Q2 intra prediction modes; determining a subset from the plurality of subsets according to the gradient information of the original pixels of the sub-image block; Using the original pixels of adjacent blocks of the sub-image block as reference pixels, using each intra-frame prediction mode in the determined subset, perform intra-frame prediction on the sub-image block, and determine that the sub-image block is within the first coding cost under each intra prediction mode in the determined subset; The P2 intra-frame prediction modes with the smallest first coding cost in the determined subset are determined as the P2 intra-frame prediction modes. The apparatus of claim 49, wherein the plurality of subsets is 4 subsets. The apparatus of claim 49 or 50, wherein the number of intra prediction modes included in each subset is the same. The apparatus according to claim 51, wherein the Q2 intra prediction modes include Planar mode, DC mode and 33 angle modes, and/or the types of intra prediction modes included in each subset 11 species. The apparatus according to any one of claims 47 to 52, wherein, according to the P2 intra prediction modes of each sub image block, determining N2 intra prediction modes of each sub image block, comprising: : According to the F2 intra-frame prediction modes in the initial MPM list of adjacent blocks of each sub-image block and the P2 intra-frame prediction modes of each sub-image block, determine the N2 intra-frame prediction modes of each sub-image block prediction mode, wherein the initial MPM list is determined according to the second intra prediction mode of the adjacent block of the sub-image block, and F2 is a positive integer. The apparatus of claim 53, wherein the F2 intra-frame prediction modes in the initial MPM list of adjacent blocks of each sub-image block and the P2 intra-frame prediction modes of each sub-image block Prediction mode, determining N2 intra-frame prediction modes of each sub-image block, including: For each sub-image block, the N2-F2 intra-frame prediction modes that are different from the F2 kinds of intra-frame prediction modes of the sub-image block among the P2 kinds of intra-frame prediction modes of the sub-image block, and the The F2 intra-frame prediction modes of the sub-image block are used as the N2 intra-frame prediction modes of the sub-image block. The apparatus of claim 54, wherein P2=N2 and F2=3. The apparatus of claim 40 or 49, wherein the first coding cost is a Hadamard cost. The apparatus according to claim 37, wherein the final reconstructed pixel of the adjacent block is used as a reference pixel, and the image to be encoded is determined according to an optimal intra prediction mode of the image block to be encoded The final prediction mode of the block, including: If the optimal intra prediction mode of the image block to be encoded is the first intra prediction mode, the final reconstructed pixel of the adjacent block of the image block to be encoded is used as a reference pixel, and the image block to be encoded is determined. the second coding cost in the best intra prediction mode and the M1 intra prediction modes, and the second coding cost in the best intra prediction mode and the M1 intra prediction modes is the smallest intra-frame Prediction mode, determined as the final prediction mode of the to-be-coded image block, M1 is a positive integer; or, If the optimal intra prediction mode of the image block to be encoded is the second intra prediction mode of the V sub-image blocks, for each sub-image block, the final reconstruction of the adjacent blocks of the sub-image block is performed. pixel as a reference pixel, determine the second coding cost of the sub-image block in the second intra-frame prediction mode of the sub-image block and the M2 intra-frame prediction modes of the sub-image block, and use The second intra-frame prediction mode of the sub-image block and the second intra-frame prediction mode with the least coding cost among the M2 intra-frame prediction modes of the sub-image block are determined as the final prediction of the sub-image block mode, M2 is a positive integer. The apparatus of claim 57, wherein the M1 intra prediction modes include one or both of the following: One or more of W1 intra-frame prediction modes in the final MPM list of the image blocks to be encoded, wherein the final MPM list of the image blocks to be encoded is based on the The final prediction mode of the adjacent block and/or the optimal prediction mode of the adjacent block of the image block to be encoded is generated, and W1 is a positive integer; One or more intra-frame prediction modes in which the second coding cost is the smallest among the N1 intra-frame prediction modes except the first intra-frame prediction mode. The apparatus according to claim 58, wherein M1=1, and the M1 intra-frame prediction modes include the first or the second or the W1th in the final MPM list of the to-be-coded image block and the Intra prediction modes with different first intra prediction modes of the image block to be encoded. The apparatus according to any one of claims 57 to 59, wherein the M2 intra prediction modes of the sub-image block include one or both of the following: One or more of W2 intra prediction modes in the final MPM list of the sub-image block, wherein the final MPM list of the sub-image block is based on neighboring blocks of the sub-image block The final prediction mode and/or the best prediction mode of the adjacent block of the image block to be encoded is generated, and W2 is a positive integer; Among the N2 intra prediction modes of the sub-image block, one or more intra-frame prediction modes with the smallest coding cost except the second intra-frame prediction mode of the sub-image block. The apparatus of claim 60, wherein M2=1, and the M2 intra-frame prediction modes include the first or the second or the W2th in the MPM list of the sub-image block and the sub-image block. The second intra prediction mode of the image block is a different intra prediction mode. The apparatus of any one of claims 57 to 61, wherein the second encoding cost is a rate-distortion cost. The apparatus according to claim 34, wherein the initial reconstructed pixels of adjacent blocks of the to-be-coded image block are used as reference pixels to determine the optimal intra-frame prediction mode and the initial reconstructed pixels of the to-be-coded image block ,include: Using the initial reconstructed pixels of adjacent blocks of the image block to be encoded as reference pixels, use N1 intra prediction modes to perform intra prediction on the image block to be encoded, and determine the optimal intra frame of the image block to be encoded. Prediction mode and initial reconstructed pixels, N1 is a positive integer. The apparatus according to any one of claims 34 to 63, wherein the processor is further configured to perform the following operations: The final prediction mode of the image block to be encoded is encoded, and the encoded code stream is sent to the decoding end. A computer storage medium, characterized in that a computer program is stored thereon, and when the computer program is executed by a computer, the computer causes the computer to execute the method according to any one of claims 1 to 33 . A computer program product comprising instructions, wherein the instructions, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 33.

Images