CN114827630B - Method, system, device and medium for learning CU depth division based on frequency domain distribution - Google Patents

Abstract

The invention discloses a method, system, device and medium for learning CU depth division based on frequency domain distribution, wherein the method includes: dividing an image into several 64x64 blocks and performing DCT transformation to obtain a frequency domain coefficient distribution matrix _F64 ; W ₆₄ calculates the probability score p ₆₄ , if it is smaller than the division threshold α ₆₄ , the downward division will end, and if it is greater than the division threshold α ₆₄ , it will continue to be divided into four 32x32 sub-CU blocks according to the principle of quadtree. The domain coefficient matrix F ₃₂ is calculated with W ₃₂ to obtain the probability score p ₃₂ , and then the division threshold α ₃₂ is used to determine whether to continue division; and so on, until all CU blocks are stopped in advance or divided into the smallest 8x8 CU block. The present invention judges whether to continue to divide through the probability score and the division threshold, does not need to traverse and recurse all situations, reduces the complexity of CU depth division, saves a lot of encoding time, and can be widely used in the field of video encoding technology.

Description

Method, system, device and medium for learning CU depth division based on frequency domain distribution

Technical Field

The present invention relates to the fields of artificial intelligence and video coding technology, and in particular to a method, system, device and medium for CU depth division based on frequency domain distribution learning.

Background Art

In recent years, with the development of the Internet and communication technology, the rapid growth of video traffic has brought great challenges to video coding technology.

In the traditional coding framework (taking HEVC as an example), any coded frame usually needs to be divided into multiple CTU (Code Tree Unit) sequences before subsequent prediction transform quantization operations. CTU can be continuously divided into CU (Code Unit) of different sizes according to the quadtree principle, with the maximum size being 64x64 and the minimum being 8x8. The quality of CTU division determines the efficiency of subsequent coding.

In order to obtain the best CTU division method, the encoding program will use a traversal recursive form to continuously divide the CU from 64x64 to 8x8 CU, and predict each division method through the built-in rate-distortion cost function until one of the best predictions is selected.

This division method causes a huge waste of encoding time and computing resources, and the waste increases significantly with the increase of video image resolution. Therefore, how to reduce the complexity of CU depth division has become a hotly debated issue in the current industry.

Summary of the invention

In order to solve at least one of the technical problems existing in the prior art to a certain extent, the purpose of the present invention is to provide a method, system, device and medium for learning CU depth division based on frequency domain distribution.

The technical solution adopted by the present invention is:

A CU depth division method based on frequency domain distribution learning includes the following steps:

根据频域系数分布矩阵

获取概率分数

Get the video image, divide the video image into several 64x64 first CU blocks, and get the frequency domain coefficient distribution matrix of the DCT transform of the first CU block

According to the frequency domain coefficient distribution matrix

Get probability score

将第一CU块向下划分成4个32x32大小的第二CU块，获取第二CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第一CU块的划分；like

Divide the first CU block into four 32x32 second CU blocks, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the second CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the first CU block is terminated;

将第二CU块向下划分成4个16x16大小的第三CU块，获取第三CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第二CU块的划分；like

Divide the second CU block downward into four third CU blocks of size 16x16, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the third CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the second CU block ends;

将第三CU块向下划分成4个8x8大小的第四CU块；反之，结束第三CU块的划分；like

Divide the third CU block downward into four fourth CU blocks of 8x8 size; otherwise, end the division of the third CU block;

Among them, α _N is the division threshold, N=64, 32, 16.

通过以下公式计算获得：Furthermore, the probability score

Calculated by the following formula:

Wherein, W ₆₄ is a preset frequency domain distribution weight matrix, i represents the row coordinate of the matrix, and j represents the row coordinate of the matrix.

Further, the frequency domain distribution weight matrix W ₆₄ is obtained by:

表示第k个64x64大小CU块对应的DCT变换频域系数分布矩阵；L_k＝0、1，表示CU块是否继续向下划分，0表示否，1表示是；Get the training set and samples required by the network

Indicates the DCT transform frequency domain coefficient distribution matrix corresponding to the k-th 64x64 CU block; L _k = 0, 1, indicating whether the CU block continues to be divided downward, 0 means no, 1 means yes;

The network is trained according to a preset loss function to obtain a frequency domain distribution weight matrix W ₆₄ ;

The expression of the preset loss function is:

Further, the division threshold α ₆₄ is obtained by:

根据选取的样本计算概率分数Select the labeled samples with L=0 in the training set

Calculate the probability score based on the selected sample

获取划分阈值α₆₄。Based on the calculated probability score

Get the partition threshold α ₆₄ .

Furthermore, the threshold

Furthermore, the step of dividing the video image into a plurality of first CU blocks of 64×64 size includes:

The video image is divided into a number of first CU blocks of 64x64 size according to the luminance component.

Furthermore, the step of dividing the video image into a plurality of first CU blocks of 64×64 size includes:

After the video image is divided into a number of first CU blocks of 64x64 size, pixel interpolation is performed on the remaining pixel areas that are not divided into 64x64 size.

Another technical solution adopted by the present invention is:

A CU depth partitioning system based on frequency domain distribution learning, comprising:

根据频域系数分布矩阵

获取概率分数The first division module obtains a video image, divides the video image into a number of first CU blocks of 64x64 size, and obtains a frequency domain coefficient distribution matrix of the DCT transform of the first CU block

According to the frequency domain coefficient distribution matrix

Get probability score

将第一CU块向下划分成4个32x32大小的第二CU块，获取第二CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第一CU块的划分；The second partitioning module is used if

Divide the first CU block into four 32x32 second CU blocks, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the second CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the first CU block is terminated;

将第二CU块向下划分成4个16x16大小的第三CU块，获取第三CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第二CU块的划分；The third partitioning module is used if

Divide the second CU block downward into four third CU blocks of size 16x16, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the third CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the second CU block ends;

将第三CU块向下划分成4个8x8大小的第四CU块；反之，结束第三CU块的划分；The fourth partitioning module is used for

Divide the third CU block downward into four fourth CU blocks of 8x8 size; otherwise, end the division of the third CU block;

Among them, α _N is the division threshold, N=64, 32, 16.

Another technical solution adopted by the present invention is:

A CU depth division device based on frequency domain distribution learning, comprising:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

Another technical solution adopted by the present invention is:

A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to execute the method described above when executed by the processor.

The beneficial effects of the present invention are as follows: the present invention determines whether to continue partitioning by probability score and partitioning threshold, obtains a method of terminating partitioning in advance, does not need to traverse recursively for all situations, reduces the complexity of CU depth partitioning, and saves a lot of encoding time.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the embodiments of the present invention or the drawings of related technical solutions in the prior art are introduced below. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

FIG. 1 is a flowchart of a method for CU depth division based on frequency domain distribution learning in an embodiment of the present invention.

FIG2 is a schematic diagram of a video image in an embodiment of the present invention;

FIG3 is a diagram showing the distribution of 64x64 DCT transform frequency domain coefficients of the CU block of the video image of FIG2 ;

FIG4 is a schematic diagram of a flow chart of a method for fast CU depth division based on frequency domain distribution learning in an embodiment of the present invention;

FIG. 5 is a flow chart of network training for learning the frequency domain distribution weight matrix W ₆₄ and the partition threshold α ₆₄ in an embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and are not to be construed as limitations of the present invention. For the step numbers in the following embodiments, they are only provided for the convenience of explanation, and the order between the steps is not limited in any way, and the execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention in combination with the specific content of the technical solution.

As shown in FIG1 , this embodiment provides a method for learning CU depth division based on frequency domain distribution, including the following steps:

根据频域系数分布矩阵

获取概率分数

S101, obtain a video image, divide the video image into a number of first CU blocks of 64x64 size, and obtain a frequency domain coefficient distribution matrix of the DCT transform of the first CU block

According to the frequency domain coefficient distribution matrix

Get probability score

The brightness component of the video frame image is selected and divided into N CU blocks of 64x64 size. If there are remaining areas less than 64x64 pixels in size, pixel interpolation is performed on these areas.

k＝1,2,..,N。计算其划分概率：Perform DCT transformation on the kth 64x64 CU area to obtain the frequency domain coefficient matrix

k＝1,2,..,N. Calculate its partition probability:

将第一CU块向下划分成4个32x32大小的第二CU块，获取第二CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第一CU块的划分。S102, if

Divide the first CU block into four 32x32 second CU blocks, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the second CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the first CU block is ended.

则该CU区域提前停止划分；若

则该CU区域(称为LCU_64)按照四叉树原则继续向下划分成4个32x32大小的CU区域。like

Then the CU area stops dividing in advance; if

The CU area (called LCU_64) is further divided downward into four CU areas of 32x32 size according to the quadtree principle.

m＝1,2,3,4。计算其划分概率：Continue to divide LCU_64 downward to obtain the frequency domain coefficient matrix of the mth 32x32 CU area

m＝1,2,3,4. Calculate its partition probability:

将第二CU块向下划分成4个16x16大小的第三CU块，获取第三CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第二CU块的划分。S103, if

Divide the second CU block downward into four third CU blocks of size 16x16, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the third CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the second CU block is ended.

则该CU区域提前停止划分；若

则该CU区域(称为LCU_32)按照四叉树原则继续向下划分成4个16x16大小的CU区域。like

Then the CU area stops dividing in advance; if

The CU area (called LCU_32) is further divided downward into four CU areas of 16x16 size according to the quadtree principle.

n＝1,2,3,4。计算其划分概率：Similarly, continue to divide LCU_32 downward to obtain the frequency domain coefficient matrix of the nth 16x16 CU area

n＝1,2,3,4. Calculate the partition probability:

将第三CU块向下划分成4个8x8大小的第四CU块；反之，结束第三CU块的划分。S104, if

The third CU block is divided downward into four fourth CU blocks of 8x8 size; otherwise, the division of the third CU block is terminated.

则该CU区域提前停止划分；若

则该CU区域按照四叉树原则继续向下划分成4个8x8大小的CU区域，结束。like

Then the CU area stops dividing in advance; if

The CU area is then further divided into four 8x8 CU areas according to the quadtree principle, and the process ends.

As can be seen from the above, the above method can obtain a way to terminate the division in advance, without the need to traverse recursively for all situations, reducing the complexity of CU depth division and saving a lot of encoding time. At the same time, the probability score of the division calculated based on frequency domain learning is essentially just an operation on two matrices, which is simple and convenient for the processor to handle, and can save a lot of computing resources and time.

The above method is explained in detail below with reference to the accompanying drawings and specific embodiments.

As shown in Figure 4, Figure 4 is a flow chart of a method for fast CU depth division based on frequency domain distribution learning provided by an embodiment of the present invention. The method includes a frequency domain distribution learning network module and a CU depth division decision module. When making a CU depth division decision, it is necessary to first learn two important parameters, the frequency domain distribution weight matrix W _N and the division threshold α _N, through the frequency domain distribution learning network module, where N = 64, 32, 16.

5 shows a network training flow chart for learning the frequency domain distribution weight matrix W ₆₄ and the partition threshold α ₆₄ provided by an embodiment of the present invention. The training process is similar for the other frequency domain distribution weight matrices W ₃₂ , W ₁₆ and the partition thresholds α ₃₂ , α ₁₆ , and no additional figures are shown.

表示第k个64x64大小CU块对应的DCT变换频域系数分布矩阵；L_k＝0、1，表示该CU块是否继续向下划分，0表示否，1表示是；Step A1: Obtain the training set and samples required by the network

Indicates the DCT transform frequency domain coefficient distribution matrix corresponding to the k-th 64x64 CU block; L _k = 0, 1, indicating whether the CU block continues to be divided downward, 0 means no, 1 means yes;

训练网络，获得频域分布权重矩阵W₆₄；Step A2: Setting the loss function

Train the network to obtain the frequency domain distribution weight matrix W ₆₄ ;

计算划分概率分数

Step A3: Select the label sample with L=0 in the data set according to the currently learned frequency domain distribution weight matrix W ₆₄

Calculate the partition probability score

的分布，选择合适的方式设置划分阈值α₆₄。在本实施例中，选取

能够获得较好的实验结果。Step A4: Observe the probability scores

The distribution of , select a suitable method to set the segmentation threshold α _64. In this embodiment, select

Better experimental results can be obtained.

As shown in Figures 2 and 3, the area with more high-frequency components tends to be divided into smaller CU blocks, and the frequency domain distribution learning network module is to learn how to represent the relationship between the richness of high-frequency components (content richness) and the depth of CU division. This relationship is represented by the domain distribution weight matrix W _N and the division threshold α _N parameter.

根据

计算其对应概率分数；Step S1: Divide the brightness component of any video image into several 64x64 CU blocks and calculate the frequency domain coefficient distribution matrix of its DCT transform

according to

Calculate its corresponding probability score;

与划分阈值α₆₄之间的关系；Step S2: Decision probability score

Relationship with the partition threshold α ₆₄ ;

提前结束该CU块的划分；Step S3: If

End the division of the CU block in advance;

该CU块按照四叉树原则向下划分成4个32x32子CU块；Step S4: If

The CU block is divided downward into four 32x32 sub-CU blocks according to the quadtree principle;

计算概率分数

Step S5: Perform DCT transformation on the four 32x32 sub-CU blocks to obtain the frequency domain coefficient distribution matrix

Calculating probability scores

与划分阈值α₃₂之间的关系；Step S6: Determine the probability score

The relationship between the partition threshold α ₃₂ ;

提前结束该CU块的划分；Step S7: If

End the division of the CU block in advance;

该CU块按照四叉树原则向下划分成4个16x16子CU块；Step S8: If

The CU block is divided downward into four 16x16 sub-CU blocks according to the quadtree principle;

计算概率分数

Step S9: Perform DCT transformation on the four 16x16 sub-CU blocks to obtain the frequency domain coefficient distribution matrix

Calculating probability scores

与划分阈值α₁₆之间的关系；Step S10: Determine the probability score

The relationship between the partition threshold α ₁₆ ;

提前结束该CU块的划分；Step S11: If

End the division of the CU block in advance;

该CU块按照四叉树原则向下划分成4个8x8子CU块,结束此次划分。Step S12: If

The CU block is divided downward into four 8x8 sub-CU blocks according to the quadtree principle, ending this division.

It can be seen that steps S3, S7, and S11 all have the opportunity to end the division early, so in many cases it is possible to avoid traversing all the division methods of a CU block before making a decision, which can avoid wasting encoding time and computing resources. The divisions between different sub-CU blocks are independent and do not affect each other, so the program can be processed in parallel and make decisions on multiple sub-CU blocks at the same time, which greatly saves encoding time. Among them, S3 and S4, S7 and S8, S11 and S12 only execute one of the steps. Even if a 64x64CU block needs to be divided into a minimum 8x8CU block, only 9 steps are required, involving 3 DCT transformation steps, 3 matrix operations, and three judgments, which greatly reduces the demand for processor computing resources.

In the test experiment in this embodiment, a CU division depth result similar to that obtained by the traditional coding framework HEVC can be obtained, and the encoding time is greatly reduced without affecting the video quality and bit rate.

This embodiment also provides a CU depth division system based on frequency domain distribution learning, including:

根据频域系数分布矩阵

获取概率分数The first division module obtains a video image, divides the video image into a number of first CU blocks of 64x64 size, and obtains a frequency domain coefficient distribution matrix of the DCT transform of the first CU block

According to the frequency domain coefficient distribution matrix

Get probability score

将第一CU块向下划分成4个32x32大小的第二CU块，获取第二CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第一CU块的划分；The second partitioning module is used if

Divide the first CU block into four 32x32 second CU blocks, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the second CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the first CU block is terminated;

将第二CU块向下划分成4个16x16大小的第三CU块，获取第三CU块的DCT变换的频域系数分布矩阵

根据频域系数分布矩阵

获取概率分数

反之，结束第二CU块的划分；The third partitioning module is used if

Divide the second CU block downward into four third CU blocks of size 16x16, and obtain the frequency domain coefficient distribution matrix of the DCT transform of the third CU block

According to the frequency domain coefficient distribution matrix

Get probability score

Otherwise, the division of the second CU block ends;

将第三CU块向下划分成4个8x8大小的第四CU块；反之，结束第三CU块的划分；The fourth partitioning module is used for

Divide the third CU block downward into four fourth CU blocks of 8x8 size; otherwise, end the division of the third CU block;

Among them, α _N is the division threshold, N=64, 32, 16.

A CU depth division system based on frequency domain distribution learning in this embodiment can execute a CU depth division method based on frequency domain distribution learning provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

This embodiment also provides a CU depth division device based on frequency domain distribution learning, including:

at least one processor;

at least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the method shown in FIG. 1 .

A CU depth division device based on frequency domain distribution learning in this embodiment can execute a CU depth division method based on frequency domain distribution learning provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

The present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device can read the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method shown in FIG1.

This embodiment also provides a storage medium storing instructions or programs that can execute a CU depth division method based on frequency domain distribution learning provided by an embodiment of the method of the present invention. When the instructions or programs are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

In some selectable embodiments, the function/operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function/operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and/or features described may be integrated into a single physical device and/or software module, or one or more functions and/or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

The logic and/or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

In the above description of this specification, the description with reference to the terms "one embodiment/example", "another embodiment/example" or "certain embodiments/examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims (8)

1. The frequency domain distribution-based CU depth division learning method is characterized by comprising the following steps of:

acquiring a video image, dividing the video image into a plurality of first CU blocks with the size of 64x64, and acquiring a frequency domain coefficient distribution matrix of DCT (discrete cosine transform) of the first CU blocks

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

wherein ,

k on the table indicates a kth first CU block among the plurality of first CU blocks;

if it is

Dividing the first CU block into 4 second CU blocks with the size of 32x32 downwards to obtain a DCT transformed frequency domain coefficient distribution matrix of the second CU blocks +.>

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

Otherwise, ending the division of the first CU block; wherein (1)>

M above represents the mth m of the second CU divided downward by the first CU;

if it is

Dividing the second CU block into 4 third CU blocks with the size of 16x16 downwards to obtain a DCT transformed frequency domain coefficient distribution matrix of the third CU blocks>

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

Otherwise, ending the division of the second CU block; wherein (1)>

N on the upper represents the nth of the third CUs divided downward by the second CU;

if it is

Dividing the third CU block down into 4 fourth CU blocks of 8x8 size; otherwise, ending the division of the third CU block;

wherein ,α₆₄ 、α ₃₂ 、α ₁₆ All are dividing thresholds;

the probability score

Obtained by calculation by the following formula:

the probability score

Obtained by calculation by the following formula:

the probability score

Obtained by calculation by the following formula:

in the formula ,W₆₄ 、W ₃₂ 、W ₁₆ For a preset frequency domain distribution weight matrix, i represents row coordinates of the matrix, and j represents column coordinates of the matrix.

2. The CU depth partitioning method based on frequency domain distribution learning as set forth in claim 1, wherein the frequency domain distribution weight matrix W ₆₄ Obtained by:

acquiring a training set and a sample required by a network

Representing a DCT transformation frequency domain coefficient distribution matrix corresponding to a kth 64x64 size CU block; l (L) _k Indicating whether the kth 64x64 size CU block continues to partition downward, L _k The value 0 indicates no, L _k The value 1 indicates yes;

training the network according to a preset loss function to obtain a frequency domain distribution weight matrix W ₆₄ ；

The expression of the preset loss function is as follows:

where N represents the number of data in the training set, and N represents the size of the CU block.

3. The CU depth partitioning method based on frequency domain distribution learning as recited in claim 2, wherein the partitioning threshold α ₆₄ Obtained by:

select training set L _k Label sample of =0

Calculating probability scores from selected samples

From calculated probability scores

Acquiring the division threshold alpha ₆₄ ；

Division threshold

4. The method for frequency domain distribution based CU depth partitioning according to claim 1, wherein the partitioning the video image into a number of first CU blocks of 64x64 size comprises:

the video image is divided into first CU blocks of 64x64 size according to the luminance component.

5. The method for frequency domain distribution based CU depth partitioning according to claim 1, wherein the partitioning the video image into a number of first CU blocks of 64x64 size comprises:

after dividing the video image into a plurality of first CU blocks with 64x64 size, pixel interpolation is performed on the remaining pixel areas with the size of 64x 64.

6. A frequency domain distribution based learning CU depth partitioning system, comprising:

a first dividing module for obtaining video image, dividing the video image into a plurality of first CU blocks with 64x64 size, and obtaining DCT transformed frequency domain coefficient distribution matrix of the first CU blocks

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

wherein ,

K on the table indicates a kth first CU block among the plurality of first CU blocks;

a second dividing module for if

Dividing the first CU block into 4 second CU blocks with the size of 32x32 downwards to obtain a DCT transformed frequency domain coefficient distribution matrix of the second CU blocks +.>

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

Otherwise, ending the division of the first CU block; wherein (1)>

M above represents the mth m of the second CU divided downward by the first CU;

a third dividing module for if

Dividing the second CU block into 4 third CU blocks with the size of 16x16 downwards to obtain a DCT transformed frequency domain coefficient distribution matrix of the third CU blocks>

According to the frequency domain coefficient distribution matrix->

Acquiring probability score->

Otherwise, ending the division of the second CU block; wherein (1)>

N on the upper represents the nth of the third CUs divided downward by the second CU;

a fourth dividing module forIf it is

Dividing the third CU block down into 4 fourth CU blocks of 8x8 size; otherwise, ending the division of the third CU block;

wherein ,α₆₄ 、α ₃₂ 、α ₁₆ All are dividing thresholds;

the probability score

Obtained by calculation by the following formula:

the probability score

Obtained by calculation by the following formula:

the probability score

Obtained by calculation by the following formula:

in the formula ,W₆₄ 、W ₃₂ 、W ₁₆ For a preset frequency domain distribution weight matrix, i represents row coordinates of the matrix, and j represents column coordinates of the matrix.

7. A frequency domain distribution based learning CU depth partitioning apparatus, comprising:

at least one processor;

at least one memory for storing at least one program;

the at least one program, when executed by the at least one processor, causes the at least one processor to implement the method of any one of claims 1-5.

8. A computer readable storage medium, in which a processor executable program is stored, characterized in that the processor executable program is for performing the method according to any of claims 1-5 when being executed by a processor.

Images