CN100542298C - Block Size Calculation Method and Transmission Method for Cylindrical Panoramic Video Coding - Google Patents

Abstract

The present invention provides a block size calculation method and transmission method for cylindrical panoramic video coding, including a code rate of the obtained panoramic video coding without performing block-based direct precoding on the cylindrical panoramic video; the video is coded according to an optional block Pre-encode in blocks according to the size of the block to obtain the encoding bit rate under the block size; calculate the increment of the above two bit rates; calculate the impact factor of the video according to this increment; finally calculate the optimal encoding of the video based on this factor block size. According to the block size calculated by the above method, the present invention provides a method for encoding and transmitting the panoramic video in blocks. The present invention can calculate the optimal encoding block size through two pre-encodings, use the optimal block size to perform cylindrical panoramic video encoding, and can transmit the lowest amount of data under the condition of the same user visual quality.

Description

Block Size Calculation Method and Transmission Method for Cylindrical Panoramic Video Coding

technical field

The invention relates to the field of video and image coding and decoding, in particular to a block size calculation method of cylindrical panoramic video coding and a transmission technology of cylindrical panoramic video coding based on blocks.

Background technique

Panoramic video is an emerging media. Compared with traditional video, it has the characteristics of large field of view, high resolution, and large amount of data. With the panoramic video, the viewer's point of view remains unchanged, and the entire surrounding scene can be observed by changing the observation direction, while the ordinary two-dimensional video only reflects a certain part of the panoramic video.

Cylindrical panoramic video is a common panoramic video, which is equivalent to a virtual camera that projects three-dimensional objects in space onto a cylindrical surface. Cylindrical panoramic video can be generated by using a multi-camera or single-camera collection system. Since the projection surface is cylindrical, there are some blind spots in the up and down directions.

Since the field of view of the panoramic video is 5 to 6 times that of the ordinary video, the amount of data of the panoramic video is 5 to 6 times that of the ordinary video while providing users with the same visual quality. According to the traditional video transmission scheme, the use of panoramic video in the network environment becomes very difficult. However, because at the same time, what the user needs to see is only a certain part of the panoramic video, so block coding and transmission has become a common solution for panoramic video network transmission.

In the block coding of panoramic video, the size of the block has an important impact on the coding efficiency and transmission area of panoramic video, and these two factors directly determine the amount of data to be transmitted. If the coding block size is small, the transmission area is small, but the coding efficiency will be low; if the coding block size is large, the coding efficiency is high, but the transmission area is also large. Therefore, under the same visual quality, the amount of data to be transmitted is different for different encoding block sizes. However, in some systems at home and abroad, the size of the block is basically not considered, and the block size is generally determined according to experience, which leads to an increase in the amount of transmitted data.

Contents of the invention

The purpose of the present invention is to provide a fast and effective calculation method for the selection of cylindrical panoramic video coding block size, so that the amount of data to be transmitted is the smallest under the condition of the same partial visual quality.

In order to achieve the above object, the present invention provides a method for calculating the coding block size of a cylindrical panoramic video, which uses information such as the width of the coding panoramic video, the width of the involved panoramic video area, and the influence factor of the current video to calculate the optimal coding block size . Contains the following steps:

Cylindrical panoramic video is not directly pre-encoded in blocks, and the code rate R of the obtained panoramic video encoding _{is 0} ;

The cylindrical panoramic video is pre-encoded according to an optional block size a to obtain the coding rate R a under the block size _a ;

According to the formula ω _a =(R _a -R ₀ )/R ₀ , calculate the increment ω _a of the code rate R _a compared to the code rate R ₀ ;

According to formula C=ω _a · a · W/(Wa), calculate the impact factor C of this video, wherein W represents that the width of this video is W macroblocks;

According to the formula $a_{\mathrm{me}}=\left[\sqrt{\mathrm{CM}/(1-\frac{C}{W})}\right],$ Calculate the optimal encoding block size a _me of the video, where M indicates that the width of the target area is M macroblocks.

Based on the block size calculation method, the present invention also provides a cylindrical panoramic video coding transmission method, including the following steps:

According to the block size a _me calculated by the above method, the video is divided into blocks;

After chunking, encode the sequence of each chunk independently;

Select the desired encoded data for transmission.

Through the method for calculating the block size of the cylindrical panoramic video encoding provided by the present invention, a block size with higher compression efficiency of the panoramic video and a more appropriate transmission area can be obtained. This minimizes the amount of data that needs to be transferred.

Description of drawings

Hereinafter, embodiments of the present invention will be described in detail in conjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram of a cylindrical panoramic video.

Fig. 2 is a graph showing the relationship between ω _a and the number of coding blocks and between ω _a and the size of coding blocks.

Fig. 3 is an operation flow chart of a method for calculating an optimal coding block size of a cylindrical panoramic video.

Fig. 4 is a schematic diagram of block compression and transmission of cylindrical panoramic video.

Detailed ways

Partial view visual quality is determined by the panoramic video raw encoding quality. During the coding process, quantization parameters (QP) are used to control the quality of panoramic video coding. Under certain conditions of QP, it is necessary to calculate the block size a to minimize the amount of transmitted data, even if the transmission code rate r _a is the smallest block size a. Therefore, the functional relationship between the transmission code rate r _a and the block size a is firstly calculated.

Assume that the width of the target area is M macroblocks, and the block size is a macroblocks, as shown in FIG. 1 . Because the position where the target area falls in the panoramic video will change, the number of related blocks will also change accordingly. One case is that the right boundary of the target area is in the range of a', and the number of related blocks is N'; the other case is that the right border of the target area is in the range of a", and the number of related blocks is N" . but:

公式1

Formula 1

Let the probability of the target area fall in the range of a' be P', and the probability of the target area falling in the range of a" be P". Therefore, when the block size is a, the expected value of the relevant area width W _a can be calculated by the following formula:

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; N</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; P</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; N</span>}}^{<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; P</span>}}^{<span\; class="notranslate">\; \&prime;</span><span\; class="notranslate">\; \&prime;</span>}$

Formula 2

$<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; .</span>$

The transmission code rate is the sum of the code rates of all related blocks that need to be transmitted, and is calculated by the following formula.

$r_a=\frac{N_a}{N}\&Center\; Dot;R_a=\frac{W_a}{W}\&Center\; Dot;R_a$ Formula 3

Among them, r _a represents the transmission code rate when the block size is a, R _a represents the encoding code rate of the panoramic video when the block size is a, that is, the sum of the code rates of all block codes, N _a represents the panoramic video resolution The number of macroblocks containing the target area when the block size is a, N represents the total number of macroblocks in the panoramic video, W _a represents the sum of the widths of the macroblocks containing the target area when the block size of the panoramic video is a, and W represents the total number of macroblocks in the panoramic video width.

In order to calculate the optimal encoding block size a _me with the smallest transmission code rate when the original encoding quality is constant, the most direct method is to use the above formula to calculate the optimal encoding size by way of enumeration. The specific implementation method is to first encode each possible block size to obtain the panoramic video coding rate R _a , then calculate the width W of the relevant block according to formula 2, and then calculate the transmission bit rate according to formula 3 r _a , and finally by comparing the transmission code rates of different block sizes, it is obtained that the block size with the smallest transmission code rate is the optimal coding size. For example, the enumerated calculation results of the four sequences of forest, village, bridge and hall shown in Table 2.

It is impractical to use every possible block size to encode the panoramic video and find the optimal encoding block after comparison. The present invention proposes a fast optimal encoding block size selection method based on an inverse ratio model.

First define λ _a as the ratio of R _a and R ₀ .

${\mathrm{\&lambda;}}_a=\frac{R_a}{{\mathrm{<figure-callout\; id="0"\; label="R"\; filenames="C20071017544300122.png,C20071017544300131.png"\; state="\{\{state\}\}">R</figure-callout>}}_0}$ Formula 4

Where R _a is the panoramic video encoding bit rate when the encoding block size is a, and R ₀ is the encoding bit rate when the panoramic video is not divided into blocks. According to the relationship between the panoramic video encoding block size and encoding efficiency, λ _a is always a real number greater than or equal to 1.

From formula 3, formula 4 can get:

$r_a=\frac{W_a}{W}\&CenterDot;R_a={\mathrm{\&lambda;}}_a\&Center\; Dot;{\mathrm{<figure-callout\; id="0"\; label="R"\; filenames="C20071017544300122.png,C20071017544300131.png"\; state="\{\{state\}\}">R</figure-callout>}}_0\&Center\; Dot;\frac{W_a}{W}\&CenterDot;$ Formula 5

For the same encoding conditions and panoramic video, different encoding block sizes a, R ₀ and W are the same, so define:

P _a =λ _a ·W _a Formula 6

As long as the coding block size that makes P _a the smallest is calculated, the optimal coding block size can be obtained.

Further, define

${\mathrm{\&omega;}}_a=\frac{R_a-R_0}{{\mathrm{<figure-callout\; id="0"\; label="R"\; filenames="C20071017544300122.png,C20071017544300131.png"\; state="\{\{state\}\}">R</figure-callout>}}_0}={\mathrm{\&lambda;}}_a-1.$ Formula 7

Table 1 lists bridge sequences with different ω _a under different coding block sizes. For each coding block size, 4 different quantization parameters (QP) are applied for coding.

Table 1 Relationship between bridge sequence ω _a and coding block size

After organizing the data in the above table, the relationship between ω _a and the number of coding blocks and the size of coding blocks is shown in Figure 2a. There is a direct proportional relationship between ω _a and the number of coding blocks. Since there is an inverse proportional relationship between the number of coding blocks and the size of the coding block, there is an inverse proportional relationship between ω _a and the size of the coding block, as shown in Figure 2b.

Since ω _a is inversely proportional to the coding block size, the present invention establishes a model between the two to describe the relationship between the two.

${\mathrm{\&omega;}}_a=\frac{C}{a}-\frac{C}{W}.$ Formula 8

For the same video C is a constant, when the panoramic video is coded in blocks, a=W, at this time ω _a =0, which conforms to our definition of ω _a .

The calculation formula of C can be obtained from formula 8:

$C=\frac{{\mathrm{\&omega;}}_a\&Center\; Dot;a\&Center\; Dot;W}{W-a}.$ Formula 11

Thus, the video is coded with any coding block size a, and ω _a is calculated, then the constant C can be calculated.

Calculated by formulas 6, 7, 8:

$P_a=W_a\&CenterDot;{\mathrm{\&lambda;}}_a=(m+a)\&Center\; Dot;(1+{\mathrm{\&omega;}}_a)=(m+a)\&Center\; Dot;(1+\frac{C}{a}-\frac{C}{W}).$ Formula 9

Calculate the point where the derivative of P _a to a is 0, which is the extreme point of P _a , and the value of a at this time is the optimal coding block size.

${P_a}^{\&prime;}=0\&DoubleRightArrow;a_{\mathrm{me}}=\left[\sqrt{\frac{\mathrm{CM}}{1-\frac{C}{W}}}\right].$ Formula 10

Based on this, the optimal coding block size a _me can be calculated.

According to the above analysis, the coding block size is calculated according to information such as the width of the panoramic video and the width of the target area. The specific implementation of this calculation method as shown in accompanying drawing 3 is as follows:

First, the cylindrical panoramic video to be encoded is not divided into blocks, but pre-encoded as a whole, to obtain a coding rate R ₀ of the panoramic video. Then, arbitrarily select a block size a for the cylindrical panoramic video to be encoded, and pre-encode the panoramic video in blocks according to this block size, and obtain the coding rate R _a under the block size a. Then calculate the increment ω _a of the precoding code rate R _a compared to the code rate R ₀ according to the formula ω _a =(R _a −R ₀ )/R ₀ . Then, according to the formula C= _ωa ·a·W/(Wa), the influence factor C of the current cylindrical panoramic video is calculated, where W represents the width of the panoramic video. Finally, according to the formula $a_{\mathrm{me}}=\left[\sqrt{\mathrm{CM}/(1-\frac{C}{W})}\right],$ Compute the optimal coding block size a _me of the cylindrical panoramic video, where M is the width of the target region.

According to the above-mentioned block size calculation method of cylindrical panoramic video encoding, the transmission method of cylindrical panoramic video provided by the present invention, as shown in Figure 4, the specific steps are as follows:

Firstly, the panoramic video is divided into blocks according to the block size calculated by the above-mentioned block size calculation method of cylindrical panoramic video coding.

Then, according to the above block division, the sequence of each block is encoded independently.

Then select the required encoded data for transmission. Here data can be selected according to the user's current perspective. The transmission medium can be the Internet, a wireless network, a local area network, an optical network, other suitable transmission medium, or a suitable combination of these transmission mediums.

Finally, after receiving the data, the decoder performs independent decoding and projection transformation on these block sequences to obtain the desired image.

Table 2 Correspondence between coding block size and transmission code rate

Table 2 illustrates the impact of the coding block size on the actual transmission bit rate in the transmission of cylindrical panoramic video images. It is transmitted under the condition of quantization parameter QP=28 and target area M=20 with the resolution of 4 image sequences of 1920x352 as Example to illustrate. For the target images of forests, villages, bridges and halls, the encoding and transmission experiments were carried out with different block sizes, and the optimal block size of different images can be observed from the bit rate corresponding to each block size. For example, when the target image is a forest and the block size is equal to 2, the code rate is 194.89, which is the minimum value of the code rate for all different block sizes. The block size calculated by the method of the present invention is also 2, which is consistent with the optimal value of the enumeration test.

When table 3 is different QP, enumerate and calculate optimal result and the method result comparison table of the present invention

Table 3 has illustrated that under different quantization parameters QP, the most efficient block size obtained by using the actual enumeration block size coding test and the optimal block size obtained by the formula of the present invention are respectively with forest, village, bridge and hall as the 4 sequences of target images. Block size comparison result.

After analyzing the above tables, it is known that the most efficient block size calculated by the enumeration method is basically consistent with the block size calculated by the present invention. Only the asterisks in Table 3 are not completely consistent, but the two Also very close.

It should be noted and understood that various modifications and improvements can be made to the invention described in detail above without departing from the spirit and scope of the invention as claimed in the appended claims. Accordingly, the scope of the claimed technical solution is not limited by any particular exemplary teaching given.

Claims (4)

1. A block size calculation method of cylindrical panoramic video encoding, comprising the following steps: Cylindrical panoramic video is not directly pre-encoded in blocks, and the code rate R of the obtained panoramic video encoding _{is 0} ; The video is pre-encoded in blocks according to an optional block size a, and the coding rate _R a under the block size a is obtained; According to the formula ω _a =(R _a -R ₀ )/R ₀ , calculate the increment ω _a of the code rate R _a compared to the code rate R ₀ ; According to formula C=ω _a · a · W/(Wa), calculate the impact factor C of this video, wherein W represents that the width of panoramic video is W macroblocks; According to the formula $a_{\mathrm{me}}=\left[\sqrt{\mathrm{CM}/(1-\frac{C}{W})}\right],$ Calculate the optimal encoding block size a _me of the video, where M indicates that the width of the target area is M macroblocks. 2. A cylindrical panoramic video coding transmission method, comprising the following steps: The panoramic video is divided into blocks according to the block size a _me calculated by the method according to claim 1; After the block, the sequence of each block is encoded independently; Select the desired encoded data for transmission. 3. The method according to claim 2, wherein after the data is transmitted, the decoding end receives the data, and independently decodes and transforms the data. 4. The method of claim 2, wherein the required encoded data is selected according to the user's perspective.

Images