CN112887712B - HEVC intra-frame CTU partitioning method based on convolutional neural network - Google Patents

Abstract

The invention belongs to the technical field related to video coding, and particularly relates to a convolutional neural network-based HEVC intra-frame CTU partitioning method, which comprises the following steps: processing a video frame by adopting a quad-tree neural network model to obtain all CTU quad-tree structures of the whole frame, coding and dividing all CTU quad-tree structures of a current frame by adopting an optimized coder, and dividing CTUs in an HEVC frame according to a dividing result; the method adopts the plurality of convolutional neural networks to optimize the quad-tree to form the quad-tree neural network model, all the CTUs in the HEVC frame can be divided through the model, the relevance of each division result is enhanced, the data processing efficiency is improved, and the coding time is reduced.

Description

A Convolutional Neural Network-based CTU Partitioning Method in HEVC Frame

technical field

The invention belongs to the related technical field of video coding, and in particular relates to a method for dividing CTUs in HEVC frames based on a convolutional neural network.

Background technique

With the gradual development of ultra-high-definition video, and the emergence of emerging video application methods such as short video, webcast, and web-on-demand, the storage and transmission of video is a huge challenge. Therefore, in 2013, the Joint Expert Group released a new generation of High Efficiency Video Coding (HEVC), which aims to effectively compress huge video data so that it can be stored and transmitted within a limited bandwidth. The compression rate is doubled compared to the previous generation video coding standard H.264/AVC. While improving the coding efficiency, HEVC adopts a more complex coding structure such as a quad-tree division method, which sharply increases the coding complexity and seriously affects the practicability of HEVC.

In the optimization of HEVC intra prediction, optimization is generally performed in the CU layer and the PU layer. The popular fast algorithms for HEVC intra prediction include: reducing the maximum depth of CU segmentation by using smooth regions, reducing the PU prediction direction based on texture, reducing the number of CU traversals based on reference, reducing the number of prediction modes according to the quadtree hierarchical structure, and predicting The directions are grouped to perform fast mode selection and to predict the prediction mode of the current block according to the linear interpolation law of the surrounding pixels of the current block. For example, the patent application number CN202010627907.6 "Video coding method and coding tree unit division method, system, device and readable storage medium" discloses a coding tree unit division method. A number of coding units are divided into training sets and test sets: training and testing a deep convolutional neural network, obtaining a deep convolutional neural network prediction model, and predicting the division of coding units through the deep convolutional neural network prediction model; to the coding tree All coding units within the unit complete the prediction, and the division result of each coding tree unit is obtained according to the prediction results of all coding units within each coding tree unit. The method can perform the coding tree unit division thread and the coding thread at the same time; when the coding tree unit division stage is performed, the coding tree unit division is performed by using the division result shared by the coding tree unit division thread, which effectively reduces the complexity of determining the coding unit division. , which reduces the overall encoding time of the video.

However, the convolutional neural network of this method is independent, so that the whole scheme can only determine whether the current CU can be divided, without considering the minimum rate-distortion value as a whole.

SUMMARY OF THE INVENTION

In order to solve the above problems existing in the prior art, the present invention proposes a method for dividing CTUs in HEVC frames based on a convolutional neural network. The method includes:

S1: Obtain video data and convert the video data into video frames;

S2: Input the video frame into the quadruple tree neural network model to obtain all the CTU quadruple tree structures of the entire frame;

S3: adopt the optimized encoder to read the structure of all CTU quadtrees of the current frame, and divide the coding unit CU of the current coding tree unit CTU according to the structure of the current CTU quadruple tree;

S4: Obtain the division result corresponding to the current coding unit CU. If the division result of the CU is 0, calculate the RD-cost of the current CU and stop recursive downward division; if the division result of the CU is 1, skip the RD-cost Calculate, and continue to recursively divide downward; if the division result of the CU is uncertain, the Gaussian discriminator is used to judge the division result of the CU, and this process is repeated until the recursion is stopped or the minimum CU size is divided;

S5: When the coding size of the CU is 8×8, the prediction unit PU is divided; if the current PU division result is 0, the size of the current PU is 8×8; if the PU division result is 1, the current PU is divided It is divided into four 4×4 PUs. If the division result is uncertain, a statistical method is used to determine the size of the current PU.

Preferably, the process of obtaining the structure of the CTU quadruped tree includes:

S21: Run the encoder HM, input the video frame data into the encoder HM, and calculate the quad-tree structure of all CTUs of the nth frame of the video frame data;

S22: The encoder saves the calculation result in a file, where the file is named by the number of frames, and outputs the pred_over_n.t×t flag file, where n represents the number of frames;

S23: When the encoder HM detects the pred_over_n.t×t flag file, it starts encoding from the nth frame. After the nth frame is encoded, it detects the flag file of the next frame, and continues encoding until all frames are encoded. .

Preferably, the quadtree neural network model includes 21 small-scale sub-convolutional neural networks.

Preferably, the specific process of dividing the coding unit CU includes:

S41: The size of each coding tree unit CTU is 64×64. When the division result is 1, the CTU is divided into four coding unit CUs of 32×32 size; when the division result is 0, the recursion is stopped and the current CTU is calculated. If the division result is uncertain, calculate the skewness value and kurtosis value of the CTU, if it satisfies the Gaussian distribution, divide it into four 32×32 size CUs, otherwise stop recursing downwards;

S42: Divide the CTU with a depth of 0 into four coding unit CUs with a size of 32×32, the encoder encodes the four CUs with a size of 32×32 one by one in a zigzag sequence, and the depth is set to 1; obtained from the quad-tree structure For the division result of each 32×32 size CU, if the division result of the current CU is 1, then divide the current CU into four 16×16 size CUs; if the division result of the current CU is 0, stop recursing downward and calculate RD-cost of the current 32×32 size CU; for uncertain division results, Gaussian judgment is used to determine the division result, if the division result is 1, the CTU with a depth of 0 is divided into four 32×32 size CUs, which is 0 then stop recursing downwards;

S43: Divide a CU with a depth of 1 into four 16×16-sized CUs, the encoder encodes the four 16×16-sized CUs one by one in the “Z” glyph order, and the depth is set to 2; obtain each CU from the quad-tree structure. If the division result of the current CU is 1, the current CU is divided into four 8×8 size CUs; if the division result is 0, the recursion is stopped and the current 16×16 is calculated. RD-cost of size CU; for uncertain division results, the division result is determined by Gaussian judgment. If the division result is 1, the CU with a depth of 2 is divided into four 8 × 8 size CUs, and if it is 0, the recursion is stopped. ;

S44: When encoding to a 8×8-sized CU, obtain the division result of the 8×8-sized prediction unit PU from the quad-tree structure. If the division result is 1, the current PU is divided into four 4×4-sized PUs; if If the division result is 0, it is a PU of 8×8 size; for an uncertain division result, the division result is determined by a statistical method. If the division result is 1, the current PU is divided into four 4×4 size PUs. If it is 0, it is a PU of 8×8 size.

Further, the process of calculating RD-cost includes:

Step 1: Calculate the rate-distortion cost J_SKIP in the SKIP mode, which is a skip mode of the PU;

Step 2: Calculate the rate-distortion cost of various modes during inter-frame prediction in turn, and find the minimum rate-distortion cost J_inter in these modes;

Step 3: Calculate the rate-distortion cost of various modes during intra-frame prediction, and find the smallest rate-distortion cost J_intra among these modes;

Step 4: Compare the sizes of J_SKIP, J_inter, and J_intra to find the smallest rate-distortion cost, which is RDCost.

Further, the formula for calculating the rate-distortion cost is:

J=B+λD

Preferably, the judging process of using the Gaussian discriminator for the uncertain CU division result includes:

Step 1: Calculate the residual of the coding unit CU;

Step 2: Calculate the central moment B _K of the k-order sample according to the residual;

Step 3: Calculate residual skewness estimation G ₁ and kurtosis estimation G ₂ according to B _K ;

Step 4: Since the skewness estimation G ₁ and the kurtosis estimation G ₂ obey the Gaussian distribution, G ₁ and G ₂ are converted into the form of the Gaussian distribution, and the mean and variance of the residual skewness and kurtosis are calculated according to the Gaussian distribution;

Step 5: set the critical value _zα ; set the Gaussian judgment condition according to the skewness estimation G ₁ , the kurtosis estimation G ₂ , the mean value, the variance and the critical value;

Step 6: If the skewness and kurtosis of the CU satisfy the Gaussian judgment condition, the CU is not divided, and if the condition is not met, the CU is divided.

Further, the Gaussian expressions of skewness estimation G ₁ and kurtosis estimation G ₂ are:

Further, the calculation formulas of the Gaussian distribution conditions μ ₁ and μ ₂ are:

Preferably, the process of statistically dividing the prediction unit PU includes:

Step 1: Calculate the prediction unit CU

Step 2: Calculate a priori detection threshold Q _step ;

是否满足先验假设；若满足，则预测单元PU的尺寸为8×8；否则将测单元PU划分为4个4×4尺寸PU。Step 3: Judging according to the prior detection threshold

Whether the prior hypothesis is satisfied; if satisfied, the size of the prediction unit PU is 8×8; otherwise, the measurement unit PU is divided into 4 PUs of 4×4 size.

In the present invention, multiple convolutional neural networks are used to optimize the quadtree, and a quadruple tree neural network model is formed. Through this model, all the CTUs in the HEVC frame can be divided, and the correlation of each division result can be achieved. Strengthen, improve the efficiency of processing data, reduce the coding time.

Description of drawings

Fig. 1 is the neural network structure schematic diagram of quadtree structure prediction of the present invention;

Fig. 2 is the neural network structure diagram of PU size prediction of the present invention;

Fig. 3 is the asynchronous operation flow chart of encoder and predictor of the present invention;

FIG. 4 is a flowchart of an optimized HM encoder intra-frame prediction CTU division method according to the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The present invention obtains the division quadtree structure of the entire CTU according to the image information and quantization parameters of the current CTU, and the probability value greater than 0.85 is regarded as 1, and the result less than 0.15 is regarded as 0, and the result between 0.15 and 0.85 is regarded as an uncertain result, For uncertain results, the division result is determined by calculating the skewness value and kurtosis value of the corresponding size CU, and then judging whether it satisfies the Gaussian distribution. The 8×8 size PU is divided in a similar way.

A method for dividing a HEVC intra-frame CTU based on a convolutional neural network, the method comprising:

S1: Obtain video data and convert the video data into video frames;

S2: Input the video frame into the asynchronously running predictor and encoder to obtain all the CTU quadtree structures of the entire frame;

S3: adopt the optimized encoder to read the structure of all CTU quadtrees of the current frame, and divide the coding unit CU of the current coding tree unit CTU according to the structure of the current CTU quadruple tree;

S4: Obtain the division result corresponding to the current coding unit CU. If the division result of the CU is 0, calculate the RD-cost of the current CU, and stop recursive downward division; if the division result of the CU is 1, skip the RD-cost Calculation, and continue to recursively divide downward; if the division result of CU is uncertain, use Gaussian discriminator to judge the division result of CU, and repeat this process until the recursion is stopped or the minimum CU size is divided;

S5: When the coding size of the CU is 8×8, the prediction unit PU is divided; if the current PU division result is 0, the size of the current PU is 8×8; if the PU division result is 1, the current PU is divided It is divided into four 4×4 PUs. If the division result is uncertain, a statistical method is used to determine the size of the current PU.

The main body of the encoder program is the HM (encoder), written in C++. In the present invention, the CNN is implemented by Tensorflow, the language is Python, and the communication between the two programs is used to indirectly realize the information transfer between the HM and the deep neural network. The encoding process and the prediction of CNN are carried out frame by frame, and the segmentation probability of each possible CU is obtained, and then the predicted CU segmentation result is used to simplify the RDO process and achieve complexity optimization.

As shown in Figure 1, it is a schematic diagram of the neural network structure for quad-tree structure prediction. Dense_1 represents a node of the quad-tree, and its predicted value represents the division result. What is predicted is the division result of the entire CTU, that is, the structure of the entire quadtree. The optimization of the entire neural network is to optimize the entire tree.

As shown in Figure 3, the process of obtaining the structure of the CTU quadruple tree includes:

S21: run the encoder HM, input the image frame data into the encoder HM, and obtain the quad-tree structure of all CTUs of the nth frame;

S22: The predictor calculates the quad-tree structure of all CTUs in the nth frame, saves the calculation results in a file, where the file is named by the number of frames, and outputs the pred_over_n.t×t flag file, where n represents the number of frames;

S23: When the encoder HM detects the pred_over_n.t×t flag file, it starts encoding from the nth frame. After the nth frame is encoded, it detects the flag file of the next frame, and continues encoding until all frames are completed. coding.

The encoder HM can only perform the read operation when it detects the flag file corresponding to the number of frames to prevent reading dirty data.

Each line in the result file is an array storage form of a quadtree, and HM can save the quadtree division structure of all CTUs of a frame by constructing a two-dimensional data. When encoding the corresponding CTU, only the corresponding quad-tree structure needs to be obtained, and then the division result of the entire CTU can be obtained.

The process of calculating the RD-cost includes: calculating the number of bits for CU size selection and mode selection; calculating the degree of image distortion; adding up to obtain the RD-cost, which is defined by the following formula. J=B+λD, where B is the number of bits for CU size selection and mode selection, λ refers to the Lagrange multiplier, and D refers to the degree of image distortion. Detailed steps (1): Calculate the rate-distortion cost J_SKIP in SKIP mode; (2) Calculate the rate-distortion cost of various modes in inter-frame prediction in turn, and find the minimum rate-distortion cost in these modes, which is J_inter; (3) Calculate The rate-distortion cost of various modes during intra-frame prediction, find the smallest rate-distortion cost in these modes, which is J_intra; (4) Compare the sizes of J_SKIP, J_inter, J_intra to find the smallest rate-distortion cost, which is RDCost.

As shown in Figure 1, the neural network structure predicted by the quadtree structure is: input layer, 4 first convolution layers, 4 first pooling layers, 16 second convolution layers, and 16 second pooling layers. layer, 4 first feature fusion layers, 4 first global average pooling layers, 16 first fully connected layers, 16 second feature fusion layers, 16 second connection layers, 16 first output layers, 5 third feature fusion layers, 5 third convolutional layers, 5 third pooling layers, 5 fourth convolutional layers, 5 fourth pooling layers, 5 second global average pooling layers, 5 fourth feature fusion layers, 5 third fully connected layers, 5 fifth feature fusion layers, 5 fourth fully connected layers, 5 second output layers: the first convolutional layer is 8 channels, the convolution The kernel size is set to 3 × 3; the first pooling layer is set to the Ma × pooling operation, the pooling window size is set to 2 × 2; the second convolution layer is 16 channels, and the convolution kernel size is set to 3 × 3; The second pooling layer is set to the Max×pooling operation, and the pooling window size is set to 2×2; the first feature fusion layer fuses the features between the previous layers as the output of the current layer; the first global average pooling layer and the second The global average pooling layer obtains the mean value of each feature map of the previous layer as the output of the current layer; the first fully connected layer is set to 32 neuron nodes; the second feature fusion layer uses the 32 neurons of the previous layer. The nodes and quantization parameters are fused together; the second fully connected layer is set to 16 neuron nodes; the output of the first output layer is a real number of [0,1], and the 1-4 outputs on the left correspond to the 5-8 nodes of the quadtree The division results of the left 5-8 outputs correspond to the division results of the 9-12 nodes of the quad tree, the right 9-12 outputs correspond to the division results of the 13-16 nodes of the quad tree, and the right 12-16 outputs correspond to the quad tree. The division results of 17-20 nodes; the third feature fusion layer fuses the feature maps of the four first pooling layers; the third convolution layer is 16 channels, and the convolution kernel size is set to 3 × 3; the third pooling layer is set For the Ma×pooling operation, the pooling window size is set to 3×3; the fourth convolutional layer is 32 channels, and the convolution kernel size is set to 3×3; the fourth pooling layer is set to the Ma×pooling operation, and the pooling window is The size is set to 3×3; the fourth feature fusion layer fuses the upper layer output and quantization parameters; the third fully connected layer is set to 32 neuron nodes; the fifth feature fusion layer fuses the upper layer output and quantization parameters; the fourth fully connected layer is set is 16 neuron nodes; the output of the second output layer is a real number of [0,1], the four outputs on the left are the division results of nodes 1-4 of the quadtree, and the one output on the right is the division result of the root node.

As shown in Figure 2, the neural network structure diagram of PU size prediction is: input layer, first convolution layer, first pooling layer, second convolution layer, first global average pooling layer, first feature fusion layer , the first fully connected layer, the second fully connected layer, and the output layer: the first convolution layer is 16 channels, and the convolution kernel size is set to 3 × 3; the first pooling layer is set to the Ma × pooling operation, the pooling window The size is set to 2 × 2; the second convolutional layer is 24 channels, and the size of the convolution kernel is set to 3 × 3; the first global average pooling layer calculates the mean of each feature map of the previous layer as the output of the current layer. ; The first feature fusion layer fuses the output and quantization parameters of the previous layer; the first fully connected layer is set to 256 neuron nodes; the second fully connected layer is set to 192 neuron nodes; the output of the output layer is [0, 1] is a real number.

The process of training a neural network includes:

Step 1: Obtain the training set for training the convolutional neural network;

When training the neural network, a large amount of data is required. By collecting other video sequences and letting the standard encoder encode it, each CTU division result is generated, and the corresponding division result is converted into a quadtree array form as the label of the data set. The input parameter data is the pixel points of the 64×64 size CTU corresponding to the Y component and the corresponding quantization parameters, which are 22, 27, 32, and 37, respectively. The training set is 80% of the total data set, the test set is 20% of the total data set, and the validation set is the official standard video sequence. The PU data set is also produced in the same way, and the corresponding neural network model can be obtained by training the designed PU size prediction model.

Step 2: Train the neural network;

Build a quadtree prediction neural network and set the loss function. Since the neural network for predicting the quadtree structure is a neural network formed by merging 21 sub-classifiers together. The 21 sub-classifiers correspond to 1 64×64, 4 32×32, and 16 16×16 child nodes respectively. 21 A classifier is fused together and affects each other. The weight of the loss function of each classifier is the same, and each classifier has the same importance. The optimization of the total loss value represents the optimization of the entire quadtree structure. The total loss function is defined by the following formula:

where loss _i is the loss function of each sub-classifier, and ω _i is the weight of each sub-classifier. loss _i is defined as follows:

loss _i =-[y _i *ln(a _i )+(1-y _i )*ln(1-a _i )]

where y _i is the true value and a _i is the predicted value.

Then set the corresponding parameters, and you can start training after building the neural network.

Build a PU size prediction neural network according to Figure 2. The loss function is the 2-category loss function provided by the deep learning framework, set the corresponding parameters, and start training after building the corresponding neural network.

For CU uncertain results, Gaussian discrimination is used to determine the results. In general, residual coefficients are usually modeled by Gaussian distribution or Laplace distribution. The final distribution to use for modeling needs to be determined experimentally. It is determined experimentally as a Gaussian distribution. It mainly studies the distribution of skewness and kurtosis, detects whether it obeys the Gaussian distribution, and the relationship between the coding blocks obeying the Gaussian distribution and the division results.

If x ₁ , x ₂ , . . . , x _n are residual coefficients, the skewness estimate G ₁ and the kurtosis estimate G ₂ are defined by the following formulas:

Among them, G ₁ represents the skewness estimation, B _k represents the K-th order sample center distance, and G ₂ represents the kurtosis estimation.

where B _K is defined by the following formula:

为x_i的均值，G₁和G₂通常服从高斯分布，所以可以写成以下高斯分布的形式：in

is the mean of x _i , G ₁ and G ₂ usually obey a Gaussian distribution, so it can be written in the form of the following Gaussian distribution:

表示偏度估计的平方差，n表示残差系数的个数，u₂表示峰度估计的均值，

表示峰度估计的平方差。where u ₁ represents the mean of the skewness estimate,

represents the squared difference of the skewness estimate, n represents the number of residual coefficients, u ₂ represents the mean of the kurtosis estimate,

Represents the squared difference of the kurtosis estimate.

According to the following formula, it can be judged whether the Gaussian distribution is satisfied:

where μ ₁ represents the statistical hypothesis test value of G ₁ , G ₁ represents the skewness estimate, σ ₁ represents the variance of the skewness estimate, μ ₂ represents the statistical hypothesis test value of G ₂ , G ₂ represents the kurtosis estimate, and u ₂ represents the The mean of the kurtosis estimate, and _σ2 represents the variance of the kurtosis estimate.

According to statistical hypothesis testing, the significance level α represents the probability of falsely rejecting the null hypothesis, i.e. the distribution is Gaussian. By examining the table Gaussian distribution, we can obtain the corresponding critical value test value z _α . The condition of obeying the Gaussian distribution is judged by the following formula:

|μ ₁ |≤z _α ,|μ ₂ |≤z _α

If the above two conditions are met, the residual coefficients are considered to follow a Gaussian distribution. In this case, it is determined not to divide. Obtaining the optimal α and the corresponding z _α is crucial to effectively increase the coding speed while maintaining coding efficiency. The z _α parameter values at different depths are different and can be embedded in the encoder to obtain the best parameter values through experiments. As shown in the table below, take values for each parameter at different depths.

For the uncertain result of the PU, a statistical method is used to determine the division result. Through modeling, it is found that the sum of the residual squared differences of the 8×8 size PU is less than a certain value, and it is judged as an 8×8 size, otherwise it is divided into four 4×4 size PUs.

The process of statistically dividing the prediction unit PU includes:

其中计算

的公式为：Step 1: Calculate the prediction unit CU

which calculates

The formula is:

表示残差平方差之和，OrgYuv表示原始像素点，pcPreYuv表示预测像素点。in,

Represents the sum of the residual squared differences, OrgYuv represents the original pixel, and pcPreYuv represents the predicted pixel.

Step 2: Calculate the prior detection threshold Q _step . The formula for calculating Q _step is:

Among them, Q _step represents a priori detection threshold, a _i represents the quantization parameter sum, QP represents the quantization parameter respectively, and i takes the value of i=QP%6, so i∈[0,5].

The values of a _i are: a ₀ =0.625, a ₁ =0.6875, a ₂ =0.8125, a ₃ =0.875, a ₄ =1, a ₅ =1.125.

是否满足先验假设；若满足，则预测单元PU的尺寸为8×8；否则将测单元PU划分为4个4×4尺寸PU。判断

是否满足先验假设的具体公式为：Step 3: Judging according to the prior detection threshold

Whether the prior hypothesis is satisfied; if satisfied, the size of the prediction unit PU is 8×8; otherwise, the measurement unit PU is divided into 4 PUs of 4×4 size. judge

The specific formula for whether the prior hypothesis is satisfied is:

Among them, SSD is the sum of squared differences of residuals, and β is a hyperparameter.

Video sequence size β hyperparameter value 416×240 12.0869 832×480 13.3564 1280×720 14.6832 1920×1080 15.0062

The following table shows the value of Q _step .

As shown in Figure 4, the specific process of the optimized HM encoder intra-frame prediction CTU division is:

Read the quadtree data structure in the result file, read the division result of the 64×64 node, and if the division result is greater than 0.85, divide it into four 32×32 size CUs if the obtained division result is 1, and skip the calculation of 64× The RD-cost of a 64-size CU, if the result is 0, stop recursing downwards, and calculate the RD-cost of the current 64×64-size CU. If the division result is uncertain, calculate the skewness and kurtosis values of the 64×64-size CU. If the Gaussian distribution is satisfied, it is divided into four 32×32 size CUs, otherwise the recursion is stopped.

If a CTU with a depth of 0 is divided into four 32×32 CUs, the encoder will encode the four 32×32 CUs one by one in a zigzag order, with the depth set to 1, and obtain each 32×32 CU from the result file. The division result of the CU, if the division result of the current CU is 1, the current CU is divided into four 16×16 size CUs, and the calculation of the RD-cost of the 32×32 size CU is skipped. The RD-cost of a 32×32 size CU is similarly determined by Gaussian judgment for uncertain division results. If the division result is 1, the CTU with a depth of 0 is also divided into four 32×32 size CUs, which is 0 Then stop recursing downwards.

If a CU with a depth of 1 is divided into four 16×16 CUs, the encoder encodes the four 16×16 CUs one by one in zigzag order, and the depth is set to 2, and each 16×16 CU is obtained from the result file. If the division result of the current CU is 1, the current CU is divided into four 8×8 size CUs and the calculation of the RD-cost of the 16×16 size CU is skipped. If it is 0, the recursion is stopped and the current 16 size is calculated. RD-cost of ×16 size CU. Similarly, for uncertain division results, the division result is determined by Gaussian judgment. If the division result is 1, the CU with depth of 2 is divided into four 8×8 size CUs. If it is 0, then Stop recursing downwards.

When encoding to a CU of 8×8 size, the division result of 8×8 size PU is obtained from the result file. If the division result is 1, the current PU is divided into 4 4×4 size PUs, and if it is 0, it is 8×8 For a PU of size, for an uncertain division result, the division result is determined by a statistical method. If the division result is 1, the current PU is divided into four 4×4 size PUs, and if it is 0, it is an 8×8 size PU. .

The above-mentioned embodiments further describe the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made to the present invention within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims (9)

1. a method for dividing CTU in HEVC frame based on convolutional neural network, is characterized in that, comprises: S1: Obtain video data and convert the video data into video frames; S2: Input the video frame into the quadruple tree neural network model to obtain all the CTU quadruple tree structures of the entire frame; S3: adopt the optimized encoder to read the structure of all CTU quadtrees of the current frame, and divide the coding unit CU of the current coding tree unit CTU according to the structure of the current CTU quadruple tree; S4: Obtain the division result corresponding to the CU of the current coding unit. If the division result of the CU is 0, calculate the RD-cost of the current CU, and stop recursive downward division; if the division result of the CU is 1, skip the RD-cost Calculation, and continue to recursively divide downwards; if the division result of the CU is uncertain, the Gaussian discriminator is used to judge the division result of the CU, and this process is repeated until the recursion is stopped or the minimum CU size is divided; The judging process of using the Gaussian discriminator to classify the uncertain CU includes: Step 1: Calculate the residual of the coding unit CU; Step 2: Calculate the central moment B _K of the k-order sample according to the residual; Step 3: Calculate residual skewness estimation G ₁ and kurtosis estimation G ₂ according to B _K ; Step 4: Since the skewness estimation G ₁ and the kurtosis estimation G ₂ obey the Gaussian distribution, G ₁ and G ₂ are converted into the form of the Gaussian distribution, and the mean and variance of the residual skewness and kurtosis are calculated according to the Gaussian distribution; Step 5: set the critical value _zα ; set the Gaussian judgment condition according to the mean value, variance and critical value of skewness estimation G ₁ and kurtosis estimation G ₂ ; Step 6: If the skewness and kurtosis of the CU satisfy the Gaussian judgment condition, the CU is not divided, and if the condition is not met, the CU is divided; S5: When the coding size of the CU is 8×8, the prediction unit PU is divided; if the current PU division result is 0, the size of the current PU is 8×8; if the PU division result is 1, the current PU is divided It is divided into four 4×4 PUs. If the division result is uncertain, a statistical method is used to determine the size of the current PU. 2. a kind of HEVC intra-frame CTU dividing method based on convolutional neural network according to claim 1, is characterized in that, the process that obtains the structure of CTU quadruple tree comprises: S21: Run the encoder HM, input the video frame data into the encoder HM, and calculate the quad-tree structure of all CTUs of the nth frame of the video frame data; S22: The encoder saves the calculation result in a file, where the file is named by the number of frames, and outputs the pred_over_n.t×t flag file, where n represents the number of frames; S23: When the encoder HM detects the pred_over_n.t×t flag file, it starts encoding from the nth frame. After the nth frame is encoded, it detects the flag file of the next frame, and continues encoding until all frames are encoded. . 3 . The method for dividing CTUs in HEVC frames based on a convolutional neural network according to claim 1 , wherein the four-branched tree neural network model comprises 21 small-scale sub-convolutional neural networks. 4 . 4. a kind of HEVC intra-frame CTU division method based on convolutional neural network according to claim 1, is characterized in that, the concrete process that coding unit CU is divided comprises: S41: The size of each coding tree unit CTU is 64×64. When the division result is 1, the CTU is divided into four coding unit CUs with a size of 32×32; when the division result is 0, the downward recursion is stopped and the current CTU is calculated. If the division result is uncertain, calculate the skewness value and kurtosis value of the CTU, if it satisfies the Gaussian distribution, divide it into four 32×32 size CUs, otherwise stop recursing downwards; S42: Divide the CTU with a depth of 0 into four coding unit CUs with a size of 32×32, the encoder encodes the four CUs with a size of 32×32 one by one in a zigzag sequence, and the depth is set to 1; obtained from the quad-tree structure For the division result of each 32×32 size CU, if the division result of the current CU is 1, the current CU is divided into four 16×16 size CUs; if the division result of the current CU is 0, the downward recursion is stopped and the calculation RD-cost of the current 32×32 size CU; for uncertain division results, Gaussian judgment is used to determine the division result, if the division result is 1, the CTU with a depth of 0 is divided into four 32×32 size CUs, which is 0 then stop recursing downwards; S43: Divide a CU with a depth of 1 into four 16×16-sized CUs, the encoder encodes the four 16×16-sized CUs one by one in the “Z” glyph order, and the depth is set to 2; obtain each CU from the quad-tree structure. If the division result of the current CU is 1, the current CU is divided into four 8×8 size CUs; if the division result is 0, the recursion is stopped and the current 16×16 is calculated. RD-cost of size CU; for uncertain division results, the division result is determined by Gaussian judgment. If the division result is 1, the CU with a depth of 2 is divided into four 8×8 size CUs, and if it is 0, the downward recursion is stopped. ; S44: When encoding to a 8×8-sized CU, obtain the division result of the 8×8-sized prediction unit PU from the quad-tree structure. If the division result is 1, the current PU is divided into four 4×4-sized PUs; if If the division result is 0, it is a PU of 8×8 size; for an uncertain division result, the division result is determined by a statistical method. If the division result is 1, the current PU is divided into four 4×4 size PUs. If it is 0, it is a PU of 8×8 size. 5. a kind of HEVC intra-frame CTU dividing method based on convolutional neural network according to claim 4, is characterized in that, the process of calculating RD-cost comprises: Step 1: Calculate the rate-distortion cost J_SKIP in the SKIP mode, which is a skip mode of the PU; Step 2: Calculate the rate-distortion cost of various modes during inter-frame prediction in turn, and find the minimum rate-distortion cost J_inter in the mode; Step 3: Calculate the rate-distortion cost of various modes during intra-frame prediction, and find the smallest rate-distortion cost J_intra among these modes; Step 4: Compare the sizes of J_SKIP, J_inter, and J_intra to find the smallest rate-distortion cost, which is RDCost. 6. a kind of HEVC intra-frame CTU dividing method based on convolutional neural network according to claim 5, is characterized in that, the formula of calculating rate-distortion cost is: J=B+λD Among them, B represents the number of bits for CU size selection and mode selection, λ represents the Lagrangian multiplier, and D represents the image distortion degree. 7. a kind of HEVC intra-frame CTU division method based on convolutional neural network according to claim 1 is characterized in that, the Gaussian expression of skewness estimation G ₁ and kurtosis estimation G ₂ is:

where u ₁ represents the mean of the skewness estimate,

represents the squared difference of the skewness estimate, n represents the number of residual coefficients, u ₂ represents the mean of the kurtosis estimate,

Represents the squared difference of the kurtosis estimate. 8. a kind of HEVC intra-frame CTU dividing method based on convolutional neural network according to claim 1, is characterized in that, the calculation formula of Gaussian distribution condition μ ₁ and μ ₂ is:

where μ ₁ represents the statistical hypothesis test value of G ₁ , G ₁ represents the skewness estimate, σ ₁ represents the variance of the skewness estimate, μ ₂ represents the statistical hypothesis test value of G ₂ , G ₂ represents the kurtosis estimate, and u ₂ represents the The mean of the kurtosis estimate, and _σ2 represents the variance of the kurtosis estimate. 9. The method for dividing a HEVC intra-frame CTU based on a convolutional neural network according to claim 1, wherein the process of statistically dividing the prediction unit PU comprises: Step 1: Calculate the sum of the residual squared differences of the prediction unit CU

Step 2: Calculate the prior detection threshold Qstep according to the quantization parameter and a _i ;

Step 3: Judging according to the prior detection threshold

Whether the a priori hypothesis is satisfied; if it is satisfied, the size of the prediction unit PU is 8×8; otherwise, the measurement unit PU is divided into four 4×4 size PUs; in,

Represents the sum of the residual squared differences, Q _step represents the a priori detection threshold, a _i represents the sum of the quantization parameters, and QP represents the quantization parameters respectively.

Images