CN104394423B - The computational methods of macroblock motion vector in a kind of Spatial Resolution Video transcoding - Google Patents

Abstract

The computational methods of macroblock motion vector in a kind of Spatial Resolution Video transcoding, the present invention takes full advantage of the abundant information of source code flow, the quick motion vector computation for carrying out macro block, and ensures the correctness of result of calculation as far as possible；The calculating of motion vector is part most time-consuming in transcoding, and the present invention can be rapidly completed the contents of the section, so as to transcoding speed.Present invention employs the video code conversion of pixel domain, is not in drift error, would not also cause drift effect, so as to ensure that the picture quality after recompiling.

Description

A Calculation Method of Macroblock Motion Vector in Spatial Resolution Video Transcoding

technical field

The invention relates to a calculation method of a macroblock motion vector in spatial resolution video transcoding, and belongs to the technical field of multimedia signal processing.

Background technique

In video-on-demand applications, in order to provide rich video resources to a wider range of users, video content providers need to store video resources in various formats, such as high-definition quality, standard-definition quality, and general quality. In this way, the needs of various users, such as personal computer users and mobile phone users, can be met. In order to resolve differences caused by different user terminals, video transcoding technology may be used. In this technology, the image size, frame rate, image quality and other parameters in the video stream can be adjusted in real time, so as to meet the requirements of the access network and playback terminal. For example, in the video-on-demand described in Figure 1, a video transcoding module can be added to the video server, and users can complete the video-on-demand through the wireless terminal, thereby solving the problem that the wireless channel is too narrow to be able to order video.

The input of video transcoding is a bit stream format (such as image size, frame rate, code rate, encoding standard, etc.), and another output bit stream format can be obtained through the transcoding module. According to the input and output bitstream format, video transcoding is usually divided into two types: inter-standard transcoding and intra-standard transcoding. Inter-standard transcoding means that the input bitstream and the output bitstream belong to different standards. Standard internal transcoding means that the input bit stream and the output bit stream belong to the same standard, and it is often divided into three aspects: image size transcoding, frame rate transcoding, and bit rate transcoding.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a method for calculating motion vectors of macroblocks in spatial resolution video transcoding. The method is suitable for wireless video monitoring, Internet video monitoring, video on demand and other fields. The method can not only significantly speed up the transcoding speed, but also ensure the recoded image quality and meet the real-time requirement.

Technical scheme of the present invention is as follows:

Spatial resolution transcoding is also image size transcoding, that is, transcoding from a larger-sized image to a smaller-sized image. Image size scaling factors are generally divided into two types: integer and arbitrary values. The present invention supports the case of arbitrary value scaling factors, and stipulates that the image size before image size scaling is N1×M1, and the image size after image size scaling is N2×M2, so the horizontal and vertical image scaling factors are N1/N2 and M1/ M2.

Figure 2 shows the correspondence between the sub-blocks in the image before and after image size scaling. Fig. 2b is a sub-block after the image size is scaled, and its width and height are both B pixels, which is referred to as a B×B sub-block in this description. Figure 2a shows the position of the B×B sub-block corresponding to the image before image size scaling, where the shaded area, that is, the area marked as A2 in the figure, is the corresponding area of the B×B sub-block in the image before image size scaling, and B× The coordinates of the upper left corner of the sub-block B in the image after the image size scaling are (x, y), then the coordinates of the upper left corner of the area A2 in the image before the image size scaling are (x·N1/N2, y·M1/M2). Area A1 is defined as: its horizontal and vertical dimensions are integer multiples of B pixels, and it just completely includes area A2. In Figure 2a, the area surrounded by the thick black line is A1

A sub-block of 16×16 pixels in the image is selected as a macroblock, which is called the current macroblock. Each current macroblock is divided into 16 4×4 sub-blocks, and the division method is shown in Fig. 3 . The types of macroblocks involved in the present invention are P16×16, P16×8, P8×16, and P8×8, and the division diagram is shown in Fig. 4 .

A method for calculating motion vectors of macroblocks in spatial resolution video transcoding, comprising the following steps:

1) Estimate the motion vector for each 4×4 sub-block: the method is to calculate the intermediate value of all motion vectors in the region A1, where B=4; each 4×4 sub-block in the region A1 has a motion vector, which is decoded is extracted from the video stream; the calculated motion vectors are 16 in total, which are mv _i , i=0,1,...,15, and the calculation results are shown in Table 1:

Table 1 Calculation results of the motion vectors of each 4×4 sub-block in the macroblock

mv ₀ mv ₁ mv ₂ mv ₃ mv ₄ mv ₅ mv ₆ mv ₇ mv ₈ mv ₉ mv ₁₀ mv ₁₁ mv ₁₂ mv ₁₃ mv ₁₄ mv ₁₅

2) 16 motion vectors in step 1) are divided into 9 different collections, and division method is as follows:

MV ₁ ＝{mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ ,mv ₈ ,mv ₉ ,mv ₁₀ ,mv ₁₁ ,mv ₁₂ ,mv ₁₃ ,mv ₁₄ ,mv ₁₅ }

MV ₂ ={mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ }

MV ₃ = {mv ₈ , mv ₉ , mv ₁₀ , mv ₁₁ , mv ₁₂ , mv ₁₃ , mv ₁₄ , mv ₁₅ }

MV ₄ ＝{mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ ,mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ }

MV ₅ = {mv ₂ , mv ₃ , mv ₆ , mv ₇ , mv ₁₀ , mv ₁₁ , mv ₁₄ , mv ₁₅ }

MV ₆ ={mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ }

MV ₇ ={mv ₂ ,mv ₃ ,mv ₆ ,mv ₇ }

MV ₈ ={mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ }

MV ₉ ={mv ₁₀ ,mv ₁₁ ,mv ₁₄ ,mv ₁₅ }

3) Select a candidate type for the current macroblock, and the selection method adopts an existing public algorithm;

4) If the current macroblock type is selected as P16×16, where the P16×16 type includes an initial motion vector mv16×16 and a refinement step size re16×16, the calculation method is:

Among them, median{ } means to calculate the median value of all the values in the brackets;

5) If the current macroblock type is selected as P16×8, the P16×8 type includes two initial motion vectors, upper and lower, respectively: mv16×8 ₀ and mv16×8 ₁ , and each motion vector corresponds to a thinning Step size, respectively: re16×8 ₀ , re16×8 ₁ , the calculation method is:

6) If the current macroblock type is selected as P8×16, the P8×16 type includes two initial motion vectors, left and right, respectively: mv8×16 ₀ and mv8×16 ₁ , and each motion vector corresponds to a refinement Step size, respectively: re8×16 ₀ , re8×16 ₁ , the calculation method is:

7) If the current macroblock type is selected as P8×8, the P8×8 type includes four initial motion vectors of up, down, left and right, respectively: mv8×8 ₀ , mv8×8 ₁ , mv8×8 ₂ , mv8×8 ₃ , each motion vector corresponds to a refinement step, respectively: re8×8 ₀ , re8×8 ₁ , re8×8 ₂ , re8×8 ₃ , the calculation method is:

8) Use the initial motion vector calculated in the above steps, and use the existing public motion estimation algorithm to perform motion refinement on it.

The beneficial effects of the present invention are:

1) The present invention makes full use of the rich information of the original code stream, quickly calculates the motion vector of the macroblock, and ensures the correctness of the selection as much as possible; the calculation of the motion vector is the most time-consuming part in transcoding, and the present invention can quickly Complete this part of the content, so that the transcoding speed is faster.

2) The present invention adopts the video transcoding based on the pixel domain, so no drift error will occur, and no drift effect will be caused, thereby ensuring the image quality after re-encoding.

3) The applicable image scaling factors of the present invention are not limited to integer values, but arbitrary values, thereby expanding the application range of video transcoding.

4) The present invention uses an intermediate value filter in calculating the motion vector, so that the accuracy is higher; the initial motion vector is obtained through calculation, and only motion estimation is used in the motion refinement, so the speed is faster.

Description of drawings

Figure 1 is a schematic diagram of the application of transcoding in video-on-demand services;

Figure 2a corresponds to the sub-blocks in the image before and after transcoding, that is, the image before transcoding;

Figure 2b corresponds to the sub-blocks in the image before and after transcoding, that is, the sub-blocks after transcoding;

The division of the macroblock in Fig. 3;

Fig. 4 is a diagram of each macroblock type in the present invention.

Detailed ways

In order to facilitate the understanding and implementation of the present invention, the present invention will be further described in detail below in conjunction with an example of wireless video on demand, but not limited thereto.

As shown in Figures 1–4.

Embodiment 1,

In wireless video on demand, encoded video streams are stored on the video server, and these video streams are compressed under the premise of high bit rate, that is, the image size is large, the frame rate is high, and the image quality is good. When a user requests a certain video segment, the corresponding required parameters will be sent to the video server at the same time, these parameters include: image size, frame rate, bit rate, etc. The video server starts the transcoding module according to the requirements of these parameters, transcodes the encoded video stream into the required format, and sends the transcoded video stream to the user terminal in real time.

The video server starts the online transcoding module according to the parameter requirements of the client. In the online transcoding, a type needs to be selected for each macroblock. The specific implementation steps of this module are as follows:

(1) Online decoding. Start the decoder to completely decode the online real-time video stream to obtain pixel domain data.

(2) Extract information. Information such as macroblock type and residual data is extracted from the decoded information.

(3) Reduce the size of the image. The size of the image is reduced according to the needs of the terminal; the algorithm for reducing the size of the image described here may use a public algorithm.

(4) Macro block type selection. The type selection of the macroblock is completed by using the disclosed algorithm.

(5) Calculate the motion vector. Motion vectors are calculated for macroblocks using the algorithm of the present invention.

(6) RECODING. Using the selected macroblock type and the recalculated motion vector, the video is re-encoded and output.

A method for calculating motion vectors of macroblocks in spatial resolution video transcoding, comprising the following steps:

1) Estimate the motion vector for each 4×4 sub-block: the method is to calculate the intermediate value of all motion vectors in the region A1, where B=4; each 4×4 sub-block in the region A1 has a motion vector, which is decoded is extracted from the video stream; the calculated motion vectors are 16 in total, which are mv _i , i=0,1,...,15, and the calculation results are shown in Table 1:

Table 1 Calculation results of the motion vectors of each 4×4 sub-block in the macroblock

mv ₀ mv ₁ mv ₂ mv ₃ mv ₄ mv ₅ mv ₆ mv ₇ mv ₈ mv ₉ mv ₁₀ mv ₁₁ mv ₁₂ mv ₁₃ mv ₁₄ mv ₁₅

2) 16 motion vectors in step 1) are divided into 9 different collections, and division method is as follows:

MV ₁ ＝{mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ ,mv ₈ ,mv ₉ ,mv ₁₀ ,mv ₁₁ ,mv ₁₂ ,mv ₁₃ ,mv ₁₄ ,mv ₁₅ }

MV ₂ ={mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ }

MV ₃ = {mv ₈ , mv ₉ , mv ₁₀ , mv ₁₁ , mv ₁₂ , mv ₁₃ , mv ₁₄ , mv ₁₅ }

MV ₄ ＝{mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ ,mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ }

MV ₅ = {mv ₂ , mv ₃ , mv ₆ , mv ₇ , mv ₁₀ , mv ₁₁ , mv ₁₄ , mv ₁₅ }

MV ₆ ={mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ }

MV ₇ ={mv ₂ ,mv ₃ ,mv ₆ ,mv ₇ }

MV ₈ ={mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ }

MV ₉ ={mv ₁₀ ,mv ₁₁ ,mv ₁₄ ,mv ₁₅ }

3) Select a candidate type for the current macroblock, and the selection method adopts an existing public algorithm;

4) If the current macroblock type is selected as P16×16, where the P16×16 type includes an initial motion vector mv16×16 and a refinement step size re16×16, the calculation method is:

Among them, median{ } means to calculate the median value of all the values in the brackets;

5) If the current macroblock type is selected as P16×8, the P16×8 type includes two initial motion vectors, upper and lower, respectively: mv16×8 ₀ and mv16×8 ₁ , and each motion vector corresponds to a thinning Step size, respectively: re16×8 ₀ , re16×8 ₁ , the calculation method is:

6) If the current macroblock type is selected as P8×16, the P8×16 type includes two initial motion vectors, left and right, respectively: mv8×16 ₀ and mv8×16 ₁ , and each motion vector corresponds to a refinement Step size, respectively: re8×16 ₀ , re8×16 ₁ , the calculation method is:

7) If the current macroblock type is selected as P8×8, the P8×8 type includes four initial motion vectors of up, down, left and right, respectively: mv8×8 ₀ , mv8×8 ₁ , mv8×8 ₂ , mv8×8 ₃ , each motion vector corresponds to a refinement step, respectively: re8×8 ₀ , re8×8 ₁ , re8×8 ₂ , re8×8 ₃ , the calculation method is:

8) Use the initial motion vector calculated in the above steps, and use the existing public motion estimation algorithm to perform motion refinement on it.

Claims (1)

1. A method for computing macroblock motion vectors in spatial resolution video transcoding, comprising steps as follows: 1) Estimate the motion vector for each 4×4 sub-block: calculate the median value of all motion vectors in the area A1, wherein the area A1 is defined as: its horizontal and vertical dimensions are integer multiples of B pixels, where B= 4. Each 4×4 sub-block in area A1 has a motion vector, which is extracted from the video stream during decoding; a total of 16 motion vectors are calculated, which are mv _i , i=0,1,...,15 , and the calculation results are shown in Table 1: Table 1 Calculation results of the motion vectors of each 4×4 sub-block in the macroblock mv ₀ mv ₁ mv ₂ mv ₃ mv ₄ mv ₅ mv ₆ mv ₇ mv ₈ mv ₉ mv ₁₀ mv ₁₁ mv ₁₂ mv ₁₃ mv ₁₄ mv ₁₅ 2) 16 motion vectors in step 1) are divided into 9 different collections, and division method is as follows: MV ₁ ＝{mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ ,mv ₈ ,mv ₉ ,mv ₁₀ ,mv ₁₁ ,mv ₁₂ ,mv ₁₃ ,mv ₁₄ ,mv ₁₅ } MV ₂ ={mv ₀ ,mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ,mv ₅ ,mv ₆ ,mv ₇ } MV ₃ = {mv ₈ , mv ₉ , mv ₁₀ , mv ₁₁ , mv ₁₂ , mv ₁₃ , mv ₁₄ , mv ₁₅ } MV ₄ ＝{mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ ,mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ } MV ₅ = {mv ₂ , mv ₃ , mv ₆ , mv ₇ , mv ₁₀ , mv ₁₁ , mv ₁₄ , mv ₁₅ } MV ₆ ={mv ₀ ,mv ₁ ,mv ₄ ,mv ₅ } MV ₇ ={mv ₂ ,mv ₃ ,mv ₆ ,mv ₇ } MV ₈ ={mv ₈ ,mv ₉ ,mv ₁₂ ,mv ₁₃ } MV ₉ ={mv ₁₀ ,mv ₁₁ ,mv ₁₄ ,mv ₁₅ } 3) Select a candidate type for the current macroblock, and the selection method adopts an existing public algorithm; 4) If the current macroblock type is selected as P16×16, where the P16×16 type includes an initial motion vector mv16×16 and a refinement step size re16×16, the calculation method is: mv16×16=median{mv∈MV ₁ } <mrow><mi>r</mi><mi>e</mi><mn>16</mn><mo>&amp;times;</mo><mn>16</mn><mo>=</mo><mfrac><mn>1</mn><mn>16</mn></mfrac><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mo>mi><mo>=</mo><mn>0</mn></mrow><mn>15</mn></munderover><mo>|</mo><mi>m</mi><mi>v</mi><mn>16</mn><mo>&amp;times;</mo><mn>16</mn><mo>-</mo><mi>m</mi><mi>v</mi><mo>|</mo><mo>,</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><msub><mi>MV</mi><mn>1</mn></msub></mrow> Among them, median{ } means to calculate the median value of all the values in the brackets; 5) If the current macroblock type is selected as P16×8, the P16×8 type includes the upper and lower initial motion vectors, respectively: mv16×8 ₀ and mv16×8 ₁ , and each motion vector corresponds to a thinning Step size, respectively: re16×8 ₀ , re16×8 ₁ , the calculation method is: <mrow><mfenced open = "{" close = ""><mtable><mtr><mtd><mrow><mi>m</mi><mi>v</mi><mn>16</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>=</mo><mi>m</mi><mi>e</mi><mi>d</mi><mi>i</mi><mi>a</mi><mi>n</mi><mo>{</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><msub><mi>MV</mi><mrow><mi>k</mi><mo>+</mo><mn>2</mn></mrow></msub><mo>}</mo></mrow></mtd></mtr><mtr><mtd><mrow><mi>r</mi><mi>e</mi><mn>16</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>=</mo><mfrac><mn>1</mn><mn>8</mn></mfrac><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mi><mo>=</mo><mn>0</mn></mrow><mn>7</mn></munderover><mo>|</mo><mi>m</mi><mi>v</mi><mn>16</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>-</mo><mi>m</mi><mi>v</mi><mo>|</mo><mo>,</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><msub><mi>MV</mi><mrow><mi>k</mi><mo>+</mo><mn>2</mn></mrow></msub></mrow></mtd></mtr></mtable></mfenced><mo>,</mo><mi>k</mi><mo>=</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>;</mo></mrow> 6) If the current macroblock type is selected as P8×16, the P8×16 type includes two initial motion vectors, left and right, respectively: mv8×16 ₀ and mv8×16 ₁ , and each motion vector corresponds to a refinement Step size, respectively: re8×16 ₀ , re8×16 ₁ , the calculation method is: <mrow><mfenced open = "{" close = ""><mtable><mtr><mtd><mi>m</mi><mi>v</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>16</mn><mi>k</mi></msub><mo>=</mo><mi>m</mi><mi>e</mi><mi>d</mi><mi>i</mi><mi>a</mi><mi>n</mi><mo>{</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><mi>M</mi><msub><mi>V</mi><mrow><mi>k</mi><mo>+</mo><mn>4</mn></mrow></msub><mo>}</mo></mtd></mtr><mtr><mtd><mi>r</mi><mi>e</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>16</mn><mi>k</mi></msub><mo>=</mo><mfrac><mn>1</mn><mn>8</mn></mfrac><mstyle><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mi><mo>=</mo><mn>0</mn></mrow><mn>7</mn></munderover></mstyle><mo>|</mo><mi>m</mi><mi>v</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>16</mn><mi>k</mi></msub><mo>-</mo><mi>m</mi><mi>v</mi><mo>|</mo><mo>,</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><mi>M</mi><msub><mi>V</mi><mrow><mi>k</mi><mo>+</mo><mn>4</mn></mrow></msub></mtd></mtr></mtable></mfenced><mo>,</mo><mi>k</mi><mo>=</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>;</mo></mrow> 7) If the current macroblock type is selected as P8×8, the P8×8 type includes four initial motion vectors of up, down, left and right, respectively: mv8×8 ₀ , mv8×8 ₁ , mv8×8 ₂ , mv8×8 ₃ , each motion vector corresponds to a refinement step, respectively: re8×8 ₀ , re8×8 ₁ , re8×8 ₂ , re8×8 ₃ , the calculation method is: <mrow><mfenced open = "{" close = ""><mtable><mtr><mtd><mi>m</mi><mi>v</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>=</mo><mi>m</mi><mi>e</mi><mi>d</mi><mi>i</mi><mi>a</mi><mi>n</mi><mo>{</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><mi>M</mi><msub><mi>V</mi><mrow><mi>k</mi><mo>+</mo><mn>6</mn></mrow></msub><mo>}</mo></mtd></mtr><mtr><mtd><mi>r</mi><mi>e</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>=</mo><mfrac><mn>1</mn><mn>4</mn></mfrac><mstyle><munderover><mo>&amp;Sigma;</mo><mrow><mi>i</mi><mo>=</mo><mn>0</mn></mrow><mn>3</mn></munderover></mstyle><mo>|</mo><mi>m</mi><mi>v</mi><mn>8</mn><mo>&amp;times;</mo><msub><mn>8</mn><mi>k</mi></msub><mo>-</mo><mi>m</mi><mi>v</mi><mo>|</mo><mo>,</mo><mi>m</mi><mi>v</mi><mo>&amp;Element;</mo><mi>M</mi><msub><mi>V</mi><mrow><mi>k</mi><mo>+</mo><mn>6</mn></mrow></msub></mtd></mtr></mtable></mfenced><mo>,</mo><mi>k</mi><mo>=</mo><mn>0</mn><mo>,</mo><mn>1</mn><mo>,</mo>mo><mn>2</mn><mo>,</mo><mn>3</mn><mo>;</mo></mrow> 8) Use the initial motion vector calculated in the above steps, and use the existing public motion estimation algorithm to perform motion refinement on it.