CN107277506B - Motion vector accuracy selection method and device based on adaptive motion vector precision - Google Patents

Abstract

The invention discloses a kind of motion vector accuracy fast selecting methods and device based on adaptive motion vector precision, by judging the sum of corresponding two-dimensional orthogonal transformation absolute value of all pixels point in current prediction unit PU block: SATD_intAnd SATD_qterBetween corresponding MV correlation properties size relation, a possibility that look-ahead IMV, to skip its unnecessary inter predication process, in the case where guaranteeing that Subjective video quality decline is negligible, to further reduced the computation complexity of Video coding of new generation, the time of inter-prediction is significantly shortened, to save the scramble time；This method is simple and easy, is conducive to the industrialization promotion of video encoding standard of new generation.

Description

Motion vector precision selection method and device based on adaptive motion vector precision

technical field

The invention belongs to the field of video coding, in particular to a motion vector precision selection method and device based on adaptive motion vector precision.

Background technique

Adaptive motion vector accuracy (AMVR, Advanced Motion Vector Resolution) is in motion estimation. After the 1/4 pixel precision motion estimation is completed, the 1/4 pixel precision motion estimation will increase the number of bits, although the distortion may be reduced. However, the overall RDcost (rate-distortion cost) has not decreased, so it is necessary to perform an integer pixel motion estimation again and compare it with the optimal MV (motion vector) obtained before, and take the one with the smallest RDcost as the final optimal MV . The re-performed integer pixel motion estimation is called IMV (Integer Motion Vector, integer motion vector). Recently, the establishment of a new generation of video coding standards has introduced a large number of new coding tools, and IMV technology is one of them.

In recent years, the application range of video technology has become wider, such as video surveillance, online video conferencing, online video e-commerce, online government affairs, online shopping, online schools, telemedicine, online seminars, online showrooms, personal online Chat, visual consultation and other services. However, to realize all the above functions, video encoding and decoding must be carried out. The amount of data in the encoding and transmission process is very large. It is unrealistic to simply expand the memory capacity and increase the communication trunk line. At this time, special technology is needed to process the huge information Processing, data compression technology is an effective solution. Through efficient data compression technology, the amount of information data can be effectively reduced. Storage and transmission in a compressed form not only saves storage space and cost, but also improves the transmission efficiency of communication trunk lines and saves network bandwidth. Real-time processed audio and video information can be output and presented with almost lossless high-quality video and audio through decoding technology, so it can be seen that multimedia data compression is very necessary. However, at the same time, this trend of video application expansion puts forward stronger requirements for the next-generation video coding standard with higher coding efficiency than H.265/HEVC (High Efficiency Video Coding). It is against this background that the VCEG (Video Coding Experts Group) of ITU-TVCEG and the MPEG (Motion Picture Experts Group) of ISO/IEC established the JVET (Joint Video Exploration Team) in 2016. Discussing the development and formulation of a new generation of video coding standards.

The new generation of video coding standards still uses a hybrid coding framework, including modules such as transform, quantization, entropy coding, intra-frame prediction, inter-frame prediction, and loop filtering. However, in order to improve the video compression rate, the standard uses QTBT (Quadtreeplus binary tree , quadtree plus binary tree) division structure, replacing the quadtree division of HEVC. Under the QTBT structure, the concept of separation of multiple division types such as CU (coding unit), PU (prediction unit) and TU (transformation unit) is removed, and more flexible CU division types are supported to better match the local characteristics of video data. At the same time, a series of time-consuming new coding tools are introduced in each module, such as Affine (affine)-based merge (merge) technology and IMV (IntegerMotion Vector) technology. This increases the computational complexity of the encoder, which is not conducive to the industrialization of the new generation of video encoding standards. Therefore, in the case of ensuring that the subjective quality of the video is negligible, optimizing the encoder and reducing the encoding time has become one of the new problems that need to be studied and solved in the field of video encoding and decoding.

The IMV introduced in the new-generation video coding standard belongs to a part of motion estimation (Motion Estimation, ME for short). In the new standard, motion estimation is mainly divided into three steps, and the specific process is as follows:

Step 1: Perform motion estimation with integer pixel precision. If it is a type B slice, or if the fast search mode is not used, then perform a full search with integer pixel precision. If it is a P type slice or use the fast search mode, then perform a full search. Fast search of pixel precision, select the optimal MV by comparing the SAD of each MV and save the corresponding MV and SAD information;

Step 2: Perform motion estimation with sub-pixel precision, that is, motion estimation with 1/2 and 1/4 pixel precision. Perform 1/2 pixel precision motion estimation first, and then perform 1/4 pixel precision motion estimation, select the optimal MV by comparing the SATD of each MV and save the corresponding MV and SATD information;

Step 3: Perform motion estimation with integer pixel precision, that is, IMV. If the rate-distortion cost of the optimal MV with integer pixel precision is less than the rate-distortion cost of the current best MV, replace the optimal MV with the MV with integer pixel precision, and save Its rate-distortion cost and related information.

Through the test and analysis of the reference software JEM of the new generation of video coding standards, it is found that under the Lowdelay (low delay) configuration, the coding time of the entire inter-frame prediction accounts for 40% to 45% of the total coding time. Therefore, if relevant information can be passed Predicting whether to perform IMV in advance, so as to avoid unnecessary judgment and selection process will greatly improve the coding efficiency of the new generation video coding standard.

Contents of the invention

Aiming at the defect of low coding efficiency of the new generation of video, the present invention proposes a motion vector precision selection method and device based on adaptive motion vector precision, which skips unnecessary inter-frame prediction by predicting the possibility of IMV in advance In the process of ensuring that the subjective quality of the video is negligible, it reduces the computational complexity of the encoder, reduces the encoding time, and improves the encoding efficiency.

A motion vector precision selection method based on adaptive motion vector precision, comprising the following steps:

Step 1: When performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU (PredictionUnit) block of the video frame image, obtain the current prediction unit PU block after the whole-pixel precision motion estimation and sub-pixel precision motion estimation, each pixel Two-dimensional orthogonal transformation of the difference between the optimal MV value of the point and the corresponding predicted MV value;

The predicted MV value corresponding to each pixel in the current prediction unit PU block is obtained by using advanced motion vector prediction technology AMVP (Advanced motion vector prediction) during the encoding process;

Step 2: Based on the two-dimensional orthogonal transformation described in each pixel of the current prediction unit PU block obtained in step 1, calculate the sum of the absolute values of the two-dimensional orthogonal transformation corresponding to all pixels in the current prediction unit PU block: SATD _int and SATD _qter ;

SATD _int indicates the SATD value of the whole pixel MV, and SATD _qter indicates the SATD value of the sub-pixel MV;

The SATD represents the sum of absolute errors after the Hadaman transform;

Step 3: Determine whether the formula is true: SATD _qter <= K*SATD _int , if true, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode, otherwise, proceed to the IMV_2N×2N mode and perform the rate-distortion The cost determines the optimal MV and the best mode for subsequent execution of the current prediction unit PU block;

in, R _quarter indicates the number of bits of the optimal MV for sub-pixel precision motion estimation, R _integer indicates the number of bits of the optimal MV for integer pixel precision motion estimation, and λ is the parameter for calculating rate-distortion optimization.

Whether the IMV needs to be executed is judged by the correlation characteristics of the MV before the IMV, thereby skipping the unnecessary and time-consuming IMV, reducing the computational complexity of encoding, and reducing the encoding time.

Further, after the step 2, it is judged whether the optimal MV of the current prediction unit PU block is an integer pixel MV, if yes, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode; otherwise, enter step 3 .

The MV of each pixel in the prediction unit PU block is the same;

IMV is to re-perform 2NX2N prediction by changing the original MV to full-pixel MV. If the final MV is judged to be full-pixel MV, it is unnecessary to do IMV and reduce time.

A motion vector precision selection device based on adaptive motion vector precision, comprising:

Integer pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes integer pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value;

Sub-pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes sub-pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value;

The calculation module of SATD _int and SATD _qter first uses the current prediction unit PU block to go through the whole pixel precision motion estimation and sub-pixel precision motion estimation respectively, and obtain the optimal MV value of each pixel and the corresponding predicted MV value difference Two-dimensional orthogonal transformation, and then calculate the sum of the absolute values of the two-dimensional orthogonal transformation corresponding to all pixels in the current prediction unit PU block;

SATD _int indicates the SATD value of the whole pixel MV, and SATD _qter indicates the SATD value of the sub-pixel MV;

The SATD represents the sum of absolute errors after the Hadaman transform;

The selection module, according to the judgment conditions, selects the subsequent execution mode for the current prediction unit PU block:

If SATD _qter <=K*SATD _int holds true, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode, otherwise, proceeds to the IMV_2N×2N mode, and determines the optimal MV and current prediction according to the rate-distortion cost The best mode for the subsequent execution of the unit PU block;

in, R _quarter indicates the number of bits of the optimal MV for sub-pixel precision motion estimation, R _integer indicates the number of bits of the optimal MV for integer pixel precision motion estimation, and λ is the parameter for calculating rate-distortion optimization.

Further, the selection module first judges whether the optimal MV mean value of the current prediction unit PU block is an integer pixel MV, and if so, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode; otherwise, according to weight 3 The selection module described in selects a subsequent execution mode for the current prediction unit PU block.

Beneficial effect

The present invention provides a motion vector precision selection method and device based on adaptive motion vector precision, by judging the sum of absolute values of two-dimensional orthogonal transformations corresponding to all pixels in the current prediction unit PU block: the sum of SATD _int and SATD _qter The magnitude relationship between the corresponding MV correlation characteristics, the possibility of predicting the IMV in advance, to skip the unnecessary inter-frame prediction process, while ensuring that the subjective quality of the video is negligible, thus further reducing the new generation of video coding. The calculation complexity greatly shortens the time of inter-frame prediction, thereby saving the coding time; this method is simple and easy to implement, and is conducive to the industrialization and promotion of the new generation of video coding standards.

Description of drawings

Figure 1 is a flow chart of the method of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

In order to reduce encoding time and improve work efficiency, the technical solution specifically adopted in the present invention is: firstly, it is judged whether the optimal MV when the motion estimation with 1/4 pixel precision is performed is an integer pixel MV, and if so, skip the IMV process and continue to execute Encoding process below. Otherwise, first obtain the relevant information of the MV of the whole pixel, and calculate the SATD of the MV of the whole pixel. Then obtain the relevant information of the optimal MV and calculate the SATD of the optimal MV. Finally, it is judged whether the SATD of the optimal MV is smaller than K times of the whole pixel MV, and if so, the IMV process is directly skipped. Otherwise, continue to execute without skipping the IMV process.

As shown in Figure 1, a motion vector precision selection method based on adaptive motion vector precision, the specific steps are as follows:

Step 1: Integer pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU (PredictionUnit) block of the video frame image, after the current prediction unit PU block undergoes integer pixel precision motion estimation, obtain the integer pixel The relevant information of the optimal MV, including its corresponding bit number R _integer , etc., and calculate its SATD value, the formula is as follows:

Among them, i and j respectively represent the abscissa and vertical coordinates of the pixels of the current prediction unit PU block; N _hor represents the total number of pixels of the current prediction unit PU block in the horizontal direction, and N _ver represents the vertical direction of the current prediction unit PU block. The total number of pixels; CurrMV[i][j] and PredMV[i][j] respectively represent the MV value of the pixel at the abscissa of the current prediction unit PU block and the ordinate is j and its corresponding prediction MV value.

Step 2: The sub-pixel precision motion estimation module performs inter-frame 2Nx2N prediction sub-pixel precision motion estimation on the prediction unit PU block of the video frame image, and obtains the optimal MV value.

In the new-generation video coding standard, the value of MV is divided into horizontal value and vertical value, which are respectively m_iHor and m_iVer. When storing the value of MV, the value of MV of 1/2 pixel and the value of MV of 1/4 pixel are respectively stored by correspondingly shifting the value of MV of whole pixel. Therefore, when obtaining the value of sub-pixel precision MV, it can be obtained by the following method:

m_iHor'=m_iHor&1111

m_iVer'=m_iVer&1111

Among them, m_iHor' and m_iVer' indicate that the value of sub-pixel precision MV is divided into horizontal value and vertical value. After obtaining the value of sub-pixel precision MV, judge it, if its value is zero, that is:

m_iHor'==0&&m_iVer'==0

Then skip the next judgment and go directly to step 5, otherwise, go to the next step.

Step 3: Obtain the value of λ and the relevant information of the optimal sub-pixel MV of the current prediction unit PU block, calculate SATD _qter , and judge whether the following conditions are met:

SATD _qter <= K*SATD _int

The formula for calculating the threshold K is as follows:

Wherein, R _quarter represents the number of bits of the optimal MV for sub-pixel precision motion estimation, and R _integer represents the number of bits of the optimal MV for integer pixel precision motion estimation. If the judgment condition is met, skip the IMV and go to step 5, otherwise, go to the next step.

Step 4: The encoder performs the IMV_2N×2N mode, and determines the optimal MV and the best mode according to the rate-distortion cost.

Step 5: Make subsequent judgments on other modes.

A device for quickly selecting motion vector precision based on adaptive motion vector precision, comprising:

Integer pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes integer pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value;

Sub-pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes sub-pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value;

The calculation module of SATD _int and SATD _qter first uses the current prediction unit PU block to go through the whole pixel precision motion estimation and sub-pixel precision motion estimation respectively, and obtain the optimal MV value of each pixel and the corresponding predicted MV value difference Two-dimensional orthogonal transformation, and then calculate the sum of the absolute values of the two-dimensional orthogonal transformation corresponding to all pixels in the current prediction unit PU block;

SATD _int indicates the SATD value of the whole pixel MV, and SATD _qter indicates the SATD value of the sub-pixel MV;

The SATD represents the sum of absolute errors after the Hadaman transform;

The selection module, according to the judgment conditions, selects the subsequent execution mode for the current prediction unit PU block:

If SATD _qter <=K*SATD _int holds true, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode, otherwise, proceeds to the IMV_2N×2N mode, and determines the optimal MV and current prediction according to the rate-distortion cost The best mode for the subsequent execution of the unit PU block;

in, R _quarter indicates the number of bits of the optimal MV for sub-pixel precision motion estimation, R _integer indicates the number of bits of the optimal MV for integer pixel precision motion estimation, and λ is the parameter for calculating rate-distortion optimization.

The selection module first judges whether the optimal MV average value of the current prediction unit PU block is an integer pixel MV, if yes, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode; otherwise, according to the method described in weight 3 The selection module selects a subsequent execution mode for the current prediction unit PU block.

In order to verify the correctness and effectiveness of the above method, the present invention implements the method on the VisualStudio 2013 software based on the reference software JEM4.0. When testing the final experimental results, consider the instability of the experimental results (mainly referring to time) on one's own computer, so in order to ensure the stability of the experimental results, all experiments of the present invention are carried out on the high-performance computing platform of the school. The hardware of this platform is Dawning 5000, and its architecture is a hybrid cluster (Cluster) architecture. Computing nodes are composed of eight-way and four-core SMP fat nodes. The computing network uses Infinband high-speed switches, and the theoretical peak value of double-precision floating-point operations reaches 10TFlops ( Ten trillion times), the storage capacity is 20TB. The configuration of the specific encoding parameters of all experiments uses the JEM standard configuration file: encoder_lowdelay_jvet10.cfg and the standard configuration file corresponding to the test sequence.

Experimental results

In order to verify the performance of the method, this paper uses two indicators of BDBR (Bjotegaard Delta Bit rate) and ΔT to evaluate. Among them, BDBR is used to evaluate the impact of the method on the video quality. The larger the BDBR, the greater the impact of the method on the video quality, that is, the worse the performance of the method, which is mainly obtained by setting four sets of different quantization parameters QP. Different Bits and PSNR are used for calculation. ΔT reflects the improvement of the encoder efficiency by the current method, and its calculation formula is as follows:

Among them, T _JEM represents the time used for encoding by the original encoder without any fast method, T _prop represents the time required for encoding after the fast method is accelerated, and TR represents the percentage increase in the efficiency of the encoder after the fast method is accelerated.

By carrying out experimental simulation on a high-performance platform, the experimental results of the present invention are as shown in table 1, ΔBits% is compared with the traditional coder bit rate change percentage, ΔPSNR/dB is the peak signal to noise ratio compared with the traditional coder than change.

Table 1

It can be seen from Table 1 that the encoding with the fast method has achieved good results: the overall encoding time is reduced by 7.08%, while the increase of BDBR is only 0.57. From the experimental results, it can be seen that the present invention improves the coding efficiency under the premise of ensuring the subjective quality of the video, and achieves the purpose of the present invention.

The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which the present invention belongs can make various modifications or supplements to the described specific embodiments or adopt similar methods to replace them, but they will not deviate from the spirit of the present invention or go beyond the definition of the appended claims range.

Claims (4)

1. a motion vector precision selection method based on adaptive motion vector precision, is characterized in that, comprises the following steps: Step 1: When performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, obtain the current prediction unit PU block after the whole-pixel precision motion estimation and the sub-pixel precision motion estimation respectively, and obtain the maximum value of each pixel. Two-dimensional orthogonal transformation of the difference between the value of the optimal MV and the corresponding predicted MV value; The predicted MV value corresponding to each pixel in the current prediction unit PU block is obtained by using the advanced motion vector prediction technology AMVP during the encoding process; When storing the value of MV, the value of 1/2 pixel MV and the value of 1/4 pixel MV are respectively stored by correspondingly shifting the value of the whole pixel MV; The value of sub-pixel precision MV is obtained by the following method: m_iHor'=m_iHor&1111 m_iVer'=m_iVer&1111 Among them, m_iHor' and m_iVer' indicate that the value of sub-pixel precision MV is divided into horizontal value and vertical value; m_iHor and m_iVer are the horizontal value and vertical value of MV respectively; After obtaining the value of sub-pixel precision MV, judge it, if its value is zero, that is m_iHor'==0&&m_iVer'==0 Then skip the next judgment and directly proceed to the subsequent judgment of other modes, otherwise, go to the next step; Step 2: Based on the two-dimensional orthogonal transformation described in each pixel of the current prediction unit PU block obtained in step 1, calculate the sum of the absolute values of the two-dimensional orthogonal transformation corresponding to all pixels in the current prediction unit PU block: SATD _int and SATD _qter ; SATD _int indicates the SATD value of the whole pixel MV, and SATD _qter indicates the SATD value of the sub-pixel MV; The SATD represents the sum of absolute errors after the Hadaman transform; Step 3: Determine whether the formula is true: SATD _qter <= K*SATD _int , if true, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode, otherwise, proceed to the IMV_2N×2N mode and perform the rate-distortion The cost determines the optimal MV and the best mode for subsequent execution of the current prediction unit PU block; in, R _quarter indicates the number of bits of the optimal MV for sub-pixel precision motion estimation, R _integer indicates the number of bits of the optimal MV for integer pixel precision motion estimation, and λ is the parameter for calculating rate-distortion optimization. 2. The method according to claim 1, wherein after said step 1, it is judged whether the optimal MV value of the current prediction unit PU block is an integer pixel MV, if so, the current prediction unit PU block is skipped IMV_2N×2N mode, continue to follow-up mode; otherwise, go to step 3. 3. A motion vector precision selection device based on adaptive motion vector precision, characterized in that, comprising: Integer pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes integer pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value; Sub-pixel precision motion estimation module, when performing inter-frame 2Nx2N prediction motion estimation on the prediction unit PU block of the video frame image, after the current prediction unit PU block undergoes sub-pixel precision motion estimation, the optimal MV of the current prediction unit PU block is obtained value; When storing the value of MV, the value of 1/2 pixel MV and the value of 1/4 pixel MV are respectively stored by correspondingly shifting the value of the whole pixel MV; The value of sub-pixel precision MV is obtained by the following method: m_iHor'=m_iHor&1111 m_iVer'=m_iVer&1111 Among them, m_iHor' and m_iVer' indicate that the value of sub-pixel precision MV is divided into horizontal value and vertical value; m_iHor and m_iVer are the horizontal value and vertical value of MV respectively; After obtaining the value of sub-pixel precision MV, judge it, if its value is zero, that is m_iHor'==0&&m_iVer'==0 Then skip the selection module and directly proceed to the subsequent judgment of other modes, otherwise, use the selection module to select; The calculation module of SATD _int and SATD _qter first uses the current prediction unit PU block to go through the whole pixel precision motion estimation and sub-pixel precision motion estimation respectively, and obtain the optimal MV value of each pixel and the corresponding predicted MV value difference Two-dimensional orthogonal transformation, and then calculate the sum of the absolute values of the two-dimensional orthogonal transformation corresponding to all pixels in the current prediction unit PU block; SATD _int indicates the SATD value of the whole pixel MV, and SATD _qter indicates the SATD value of the sub-pixel MV; The SATD represents the sum of absolute errors after the Hadaman transform; The selection module, according to the judgment conditions, selects the subsequent execution mode for the current prediction unit PU block: If SATD _qter <=K*SATD _int is established, the current prediction unit PU block skips the IMV_2N×2N mode and continues to the subsequent mode, otherwise, proceed to the IMV_2N×2N mode, and determine the optimal MV and the current prediction unit according to the rate-distortion cost The best mode for the subsequent execution of the PU block; in, R _quarter indicates the number of bits of the optimal MV for sub-pixel precision motion estimation, R _integer indicates the number of bits of the optimal MV for integer pixel precision motion estimation, and λ is the parameter for calculating rate-distortion optimization. 4. The device according to claim 3, wherein the selection module first judges whether the optimal MV value of the current prediction unit PU block is an integer pixel MV, and if so, the current prediction unit PU block skips IMV_2N ×2N mode, continue to the subsequent mode; otherwise, select the subsequent execution mode for the current prediction unit PU block according to the selection module described in weight 3.