CN102075766B - Video coding and decoding methods and devices, and video coding and decoding system - Google Patents

Abstract

Embodiments of the present invention relate to a video encoding and decoding method, device, and video encoding and decoding system. The method includes: acquiring an original sequence of video image data; performing prediction, transformation, quantization, and Encoding to obtain the code stream of the first image; according to the prediction mode of the first image, predict, transform, and quantize the second image with the second size in the original sequence to obtain the transform domain data of the second image; if the second size is the same as The first sizes are different, predicting the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image, and encoding the predicted transform domain inter-layer prediction data of the second image, Get the code stream of the second image. The video encoding, decoding method, device and video encoding and decoding system provided by the embodiments of the present invention do not need to inversely transform the first image in the original sequence again in the transform domain, thus reducing the complexity of encoding.

Description

Video encoding, decoding method, device and video encoding and decoding system

technical field

Embodiments of the present invention relate to the field of communication technologies, in particular, a video encoding and decoding method, device, and video encoding and decoding system.

Background technique

With the rapid development of Internet technology, the application of video on the Internet is becoming more and more extensive. In order to better adapt to the needs of different clients, the Joint Video Team (JVT for short) incorporates Scalable Video Coding (SVC for short) into the relatively advanced video coding standard H.264/ Advanced Video Coding (Advanced Video Coding, referred to as: AVC) extension.

SVC code stream rewriting is a technology that converts SVC code streams into AVC code streams. There are three SVC scalable code stream rewriting technologies in the prior art, namely: scalable code stream rewriting for local playback, Scalable code stream rewriting for unicast and scalable code stream rewriting for broadcast/multicast. Since the hierarchical B frame has already solved the scalability of time resolution, the code stream rewriting technology does not need Consider the scalability of time; since JVT has already solved the scalability of quality and can be applied in practice, the code stream rewriting technology does not need to consider the scalability of quality.

The existing technology has listed quality-scalable code stream rewriting as a part of the SVC standard. The quality-scalable code stream rewriting technology performs inter-layer prediction in the transform domain by changing the code; , In the process of code stream rewriting and decoding, it is necessary to perform inter-layer prediction in the pixel domain. If it is necessary to generate a spatially scalable code stream, it is necessary to perform inter-layer prediction in the pixel domain. Therefore, in the process of code stream rewriting and decoding, it is necessary to Perform inverse transformation on the transformed data after entropy decoding, reconstruct the reference value in the pixel domain after inverse transformation, and then reconstruct the image; or re-predict, transform, quantize and encode after reconstructing the reference value to obtain a new code stream.

In the process of implementing the present invention, the inventor found that in the process of code stream rewriting and decoding, the decoded transformed data needs to be inversely transformed, thus increasing the complexity of code stream rewriting and decoding.

Contents of the invention

The purpose of the embodiments of the present invention is to provide a video encoding and decoding method, device and video encoding and decoding system, by encoding or decoding the original sequence of video image data in the transform domain, reducing the complexity of code stream rewriting and decoding Spend.

An embodiment of the present invention provides a video coding method, including:

Obtain a raw sequence of video image data;

Predicting, transforming, quantizing, and encoding a first image with a first size in the original sequence to obtain a code stream of the first image;

Predict, transform, and quantize a second image with a second size in the original sequence according to the prediction mode of the first image, to obtain transform domain data of the second image;

If the second size is different from the first size, predict the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image, and predict the predicted The transform domain inter-layer prediction data of the second image is encoded to obtain a code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code stream.

The embodiment of the present invention also provides a video code stream rewriting method, including:

Obtaining a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

In the process of rewriting the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain with the Using the transform domain data corresponding to the first coded texture of the first image as a reference, reconstructing the transform domain data of the second coded texture of the second image, and encoding the transform domain data of the second coded texture of the second image stream rewriting.

The embodiment of the present invention also provides a video decoding method, including:

Obtaining a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

In the process of decoding the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain, the The transform domain data corresponding to the first coded texture of an image is used as a reference, and the transform domain data of the second coded texture of the second image is reconstructed;

Inverse transform is performed on the transformation domain data of the second coded texture to obtain the second coded texture, and the second image is reconstructed by using the second coded texture.

The embodiment of the present invention also provides a video encoding device, including:

Obtaining module, for obtaining the original sequence of video image data;

A first encoding module, configured to predict, transform, quantize, and encode a first image with a first size in the original sequence to obtain a code stream of the first image;

A first processing module, configured to predict, transform, and quantize a second image with a second size in the original sequence according to the prediction mode of the first image, and obtain transform domain data of the second image;

A second encoding module, configured to predict the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image if the second size is different from the first size , encoding the predicted transform domain inter-layer prediction data of the second image to obtain the code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form Spatial scalable code stream.

The embodiment of the present invention also provides a video code stream rewriting device, including:

An acquisition module, configured to acquire a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

A code stream rewriting module, configured to rewrite the spatially scalable code stream, if the second image in the spatially scalable code stream has a second size different from the first size of the first image are the same, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the transform domain data of the second coded texture of the second image, and the second coded texture of the second image is reconstructed. The transform domain data of the encoded texture is codestream rewritten.

The embodiment of the present invention also provides a video decoding device, including:

An acquisition module, configured to acquire a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

A first reconstruction module, configured to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

The second reconstruction module is configured to decode the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then In the transform domain, using the transform domain data corresponding to the first coded texture of the first image as a reference, reconstructing the transform domain data of the second coded texture of the second image;

The third reconstruction module is configured to perform inverse transformation on the transformation domain data of the second coded texture, obtain the second coded texture, and use the second coded texture to reconstruct the second image.

An embodiment of the present invention also provides a video encoding and decoding system, including: a video encoding device, at least one video decoding device, at least one code stream rewriting device,

The video encoding device is configured to obtain an original sequence of video image data; perform prediction, transformation, quantization, and encoding on a first image with a first size in the original sequence to obtain a code stream of the first image; according to the The prediction mode of the first image, predict, transform, and quantize the second image with the second size in the original sequence, and obtain the transform domain data of the second image; if the second size is the same as the first size Not the same, the transform domain data of the second image is predicted in the transform domain with reference to the transform domain data of the first image, and the predicted transform domain inter-layer prediction data of the second image is encoded , acquiring the code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code stream;

The code stream rewriting device is used to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding; during the process of rewriting the spatially scalable code stream, if the spatially scalable code stream is If the second size of the second image is different from the first size of the first image, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the second image Transform domain data of the second coded texture, rewriting the code stream of the transform domain data of the second coded texture of the second image;

The video decoding device is configured to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding; during the process of decoding the spatially scalable code stream, if the second image in the spatially scalable code stream If the second size is not the same as the first size of the first image, the second encoded texture of the second image is reconstructed in the transform domain using the transform domain data corresponding to the first encoded texture of the first image as a reference The transform domain data of the second coded texture is inversely transformed to obtain the second coded texture, and the second coded texture is used to reconstruct the second image.

In the video encoding, decoding method, device and video encoding and decoding system provided by the embodiments of the present invention, the second image with the second size in the original sequence is compared in the transform domain with the transform domain data of the first image as a reference. Prediction is performed on the transform domain, and the predicted data of the second image is encoded to obtain the code stream of the second image. Since the first image in the original sequence does not need to be inversely transformed in the transform domain again, it reduces complexity of encoding and decoding.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings without any creative effort.

FIG. 1 is a schematic flowchart of an embodiment of a video encoding method of the present invention;

FIG. 2 is a schematic diagram of video encoding in the embodiment shown in FIG. 1;

FIG. 3 is another schematic diagram of video encoding in the embodiment shown in FIG. 1;

FIG. 4 is a schematic flow diagram of an embodiment of a method for rewriting a video code stream according to the present invention;

Fig. 5 is a schematic diagram of code stream rewriting in the embodiment shown in Fig. 4;

FIG. 6 is another schematic diagram of code stream rewriting in the embodiment shown in FIG. 4;

FIG. 7 is a schematic flowchart of an embodiment of a video decoding method of the present invention;

Fig. 8 is a schematic diagram of video decoding in the embodiment shown in Fig. 7;

FIG. 9 is another schematic diagram of video decoding in the embodiment shown in FIG. 7;

FIG. 10 is a schematic structural diagram of an embodiment of a video encoding device according to the present invention;

FIG. 11 is a schematic structural diagram of an embodiment of a video code stream rewriting device of the present invention;

FIG. 12 is a schematic structural diagram of an embodiment of a video decoding device according to the present invention;

FIG. 13 is a schematic structural diagram of an embodiment of the video encoding and decoding system of the present invention;

FIG. 14 is a schematic structural diagram of a code stream rewriting system applicable to an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

为第l层，第n帧，第i个编码块的编码端解码重建值。f_l，n ^i，pred为第l层，第n帧，第i个编码块的预测值，预测模式包括帧内预测和帧间预测。l-1层码流的模式为MODE(f_l-1，n ⁱ)，编码纹理为

U(·)表示上采样操作，即将l-1层的尺寸(或者分辨率)变换成和l-1层以上层同样的尺寸，例如：l、l+1、...、l+n(n为自然数)，本发明实施例以l-1层变换成l层为例进行说明；此外，U(·)不仅可以表示像素的上采样，还可以表示模式的上采样。DCT(·)表示对编码块进行离散余弦变换变换(Discrete Cosine Transform，简称：DCT)。In order to understand the embodiment of the present invention more easily, the embodiment of the present invention adopts the following basic symbols for description: Let f _{l, ni} ^be the lth layer, the nth frame (it can also be the nth field or the nth image), the nth The original pixel values of i coding blocks (l, n, i are all positive integers);

Decode the reconstructed value for the encoding end of the l-th layer, n-th frame, and i-th encoded block. f _{l, ni} ^{, pred} are the predicted values of the l-th layer, the n-th frame, and the i-th coding block, and the prediction mode includes intra-frame prediction and inter-frame prediction. The mode of the l-1 layer code stream is MODE(f _{l-1, n} ⁱ ), and the coded texture is

U( ) represents an upsampling operation, that is, transforming the size (or resolution) of the l-1 layer into the same size as the layer above the l-1 layer, for example: l, l+1, ..., l+n( n is a natural number), the embodiment of the present invention is described by taking l-1 layer transformation into l layer as an example; in addition, U(·) can not only represent the up-sampling of pixels, but also represent the up-sampling of patterns. DCT(·) represents performing discrete cosine transform (Discrete Cosine Transform, DCT for short) on the coding block.

编码块的整数DCT变换操作记作DCT(·)。In order to better understand the technical solutions of the embodiments of the present invention, the inter-layer prediction technology of multi-layer code stream coding is described in detail. When performing inter-layer prediction on layer l, it is necessary to completely decode and reconstruct the reference layer l-1 to obtain more coding information of the reference layer l-1. Considering factors such as the complexity of decoding and reconstruction, the reference layer Full decoding and reconstruction of layer l-1 is not the best way for inter-layer prediction, so it is necessary to choose between inter-layer prediction and temporal prediction, for example: sequences with slow motion and rich details will be more correlated in time, At this time, the method of time prediction is relatively simple. There are three inter-layer prediction modes for multi-layer stream coding: inter-layer motion prediction, inter-layer residual prediction, and inter-layer intra-frame prediction; among them, in the inter-layer motion prediction mode, when the coded block corresponding to the reference layer l-1 is When inter-layer coding (inter-coded), the coding block of enhancement layer 1 also adopts inter-layer coding (inter-coded). The coding block at the corresponding position of l-1 is inherited; in the inter-layer residual prediction method, all inter-layer coding blocks can adopt the inter-layer residual prediction method, assuming that the residual of the reference layer l-1 layer is The current coding block is f _{l, ni} ^{, and} the coding block searched by motion in the previous frame (or a certain reference frame) is

The integer DCT transform operation of a coded block is denoted DCT(•).

In addition, since the embodiment of the present invention is described by taking the prediction mode as an example of intra prediction or inter prediction, it needs to involve the code stream of the first image and the code stream of the second image. For the sake of simplicity of description, the code stream of the first image The code stream is called the l-1 layer code stream, and the code stream of the second image is called the l-layer code stream; if the second size of the second image is different from the first size of the first image, that is, l- If the resolutions (sizes) of the 1-layer code stream and the l-layer code stream are different, then the coding mode of the l-layer code stream is the same as the coding mode of the coding block at the corresponding position in the space of the l-1 layer code stream, for example: when When the l-layer code stream is the Common Intermediate Format (CIF for short) size, and the l-1 layer code stream is a quarter of the Common Intermediate Format (Quarter Common Intermediate Format, QCIF for short) size, an l- A coding block of layer 1 corresponds to four coding blocks of layer l, and the coding modes of the four coding blocks of layer l are all determined by the coding mode of the same coding block of layer l-1.

Fig. 1 is a schematic flow chart of an embodiment of the video coding method of the present invention, Fig. 2 is a schematic diagram of the video coding of the embodiment shown in Fig. 1, and Fig. 3 is another schematic diagram of the video coding of the embodiment shown in Fig. 1; 1, the embodiment of the present invention comprises the following steps:

Step 101, obtaining the original sequence of video image data;

Step 102. Predict, transform, quantize, and encode the first image with the first size in the original sequence to obtain the code stream of the first image;

Step 103. According to the prediction mode of the first image, perform prediction, transformation, and quantization on the second image with the second size in the original sequence to obtain transform domain data of the second image;

Step 104: If the second size is different from the first size, predict the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image, and predict Encoding the transform domain inter-layer prediction data of the second image to obtain the code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code flow.

Multiple images of different sizes can be decoded from the spatially scalable code stream in the embodiment of the present invention, and a larger-sized image is encoded with a smaller-sized image as a reference during encoding to obtain the code stream.

The video encoding method provided by the embodiment of the present invention uses the transform domain data of the first image as a reference to predict the second image in the transform domain, encodes the predicted data of the second image, and obtains the code stream of the second image , since the first image in the original sequence does not need to be inversely transformed in the transform domain again, thus reducing the complexity of coding.

Further, in the above step 101, the original sequence of video image data may be received from the media data acquisition device or locally stored video image data or through the device address, and the first image and the second image in the original sequence may be respectively The first image can also be obtained by post-processing the second image, such as by downsampling or scaling the second image; as long as it is video image data, the source of the video image data does not constitute a limitation to the embodiment of the present invention.

Further, in the above step 102, in order to be compatible with existing coding standards, the coding method for performing single-layer coding on the coding block of the first image with the first size in the original sequence can adopt the existing single-layer coding technology , for example: AVC coding technology; the existing single-layer coding technology does not constitute a limitation to the embodiment of the present invention, and any implementation of coding the original sequence data of the image that can adopt single-layer coding and meet the needs of the client is regarded as The method capable of forming the code stream of the first image described in the embodiment of the present invention; in this embodiment, for the convenience of description, the code stream of the first image is called the l-1 layer code stream, and the following is based on the l-1 layer The code stream of the second image formed by code stream encoding is called the l-layer code stream. Obviously, the l-layer code stream can also be used as the code stream of the first image, and the l+1 layer code stream can be used as the code stream of the second image , so the code stream of the first image and the code stream of the second image in the embodiment of the present invention are only relative conceptual descriptions; in addition, the first image referred to in the embodiment of the present invention refers to the The dimensions of the image are not the same as the reference image, so the first image is not limited to fixed-resolution images.

Further, with reference to FIG. 2 for exemplary illustration, if the prediction mode of the first image is an intra-frame prediction mode, the above step 103 may specifically be:

其中，

为第l-1层，第n帧，第i个编码块的编码端解码重建值，f_l-1，n ^i，pred为第l-1层，第n帧，第i个编码块的预测值；对l-1层的第一编码纹理

进行上采样处理得到对上采样后的数据进行DCT变换，然后在变换域上做量化和缩放处理，得到第一编码纹理的变换域数据为 Acquire the first coded texture of the first image, perform upsampling on the first coded texture, transform and quantize the upsampled data, and obtain the transform domain data of the first coded texture; for example: if the acquired l-1 layer ( The first encoded texture of the first image) is

in,

For the l-1th layer, the nth frame, the encoding end decodes the reconstruction value of the i-th coding block, f _{l-1, n} ^{i, pred} is the l-1th layer, the nth frame, the prediction of the i-th coding block Value; the first coded texture for layer l-1

Perform upsampling to get Perform DCT transformation on the upsampled data, and then perform quantization and scaling processing on the transform domain to obtain the transform domain data of the first coded texture as

其中，

为第l层，第n帧，第j个编码块的编码端解码重建值，f_l，n ^i，pred为第l层，第n帧，第i个编码块的预测值，则对第二编码纹理

进行DCT变换量化后，得到第二编码纹理的变换域数据

Obtain the second encoded texture of the second image, perform transformation and quantization on the second encoded texture, and obtain the transform domain data of the second encoded texture; for example: if the acquired second encoded texture of the second image (or the second residual )for

in,

is the lth layer, the nth frame, the coding end decoding reconstruction value of the jth coding block, f _{l, ni} ^{, pred} is the lth layer, the nth frame, the prediction value of the ith coding block, then for the second coded texture

After DCT transformation and quantization, the transformation domain data of the second coded texture is obtained

为参考，对所述第二编码纹理的变换域数据进行层间预测，得到第一变换域层间预测数据 $\mathrm{DCT}({\hat{f}}_{l,n}^j-f_{l,n}^{i,\mathrm{pred}})-\mathrm{DCT}\left(U({\hat{f}}_{l-1,n}^i-f_{l-1,n}^{i,\mathrm{pred}})\right);$ 对第一变换域层间预测数据进行熵编码，形成第二图像的码流。Performing inter-layer prediction on the transform-domain data of the second coded texture with reference to the transform-domain data of the first coded texture to obtain first transform-domain inter-layer predictive data, and performing entropy on the first transform-domain inter-layer predictive data Encoding to form the code stream of the second image, for example: the transform domain data of the first encoded texture

For reference, for the transform domain data of the second coded texture Perform inter-layer prediction to obtain the first transform domain inter-layer prediction data $\mathrm{DCT}({\hat{f}}_{l,\mathrm{no}}^j-f_{l,\mathrm{no}}^{i,\mathrm{pred}})-\mathrm{DCT}\left(u({\hat{f}}_{l-1,\mathrm{no}}^i-f_{l-1,\mathrm{no}}^{i,\mathrm{pred}})\right);$ Entropy coding is performed on the first transform domain inter-layer prediction data to form a code stream of the second image.

(残差)，对l-1层的第一编码纹理

进行DCT变换和量化得到

获得第二图像的第二编码纹理为

对第二编码纹理

进行DCT变换和量化后得到第二编码纹理的变换域数据

以第一编码纹理的变换域数据

为参考对第二编码纹理

的变换域数据

进行层间预测，得到第一变换域层间预测数据 $\mathrm{DCT}({\hat{f}}_{l,n}^i-{\hat{f}}_{l,n-1}^j)-\mathrm{DCT}\left(U({\hat{f}}_{l-1,n}^i-{\hat{f}}_{l-1,n-1}^j)\right);$ 对所述第一变换域层间预测数据进行熵编码，形成第二图像的码流。Alternatively, with reference to FIG. 3 for an exemplary illustration, if the prediction mode of the first image is an inter-frame prediction mode, then in the above step 103, the first coded texture of layer 1-1 (the first image) is acquired as

(residual), for the first encoded texture of layer l-1

Perform DCT transformation and quantization to get

Obtain the second encoded texture of the second image as

for the second encoded texture

Transform domain data of the second coded texture is obtained after DCT transformation and quantization

The transform domain data of the encoded texture in the first

as a reference to the second encoded texture

Transform domain data of

Perform inter-layer prediction to obtain the first transform domain inter-layer prediction data $\mathrm{DCT}({\hat{f}}_{l,\mathrm{no}}^i-{\hat{f}}_{l,\mathrm{no}-1}^j)-\mathrm{DCT}\left(u({\hat{f}}_{l-1,\mathrm{no}}^i-{\hat{f}}_{l-1,\mathrm{no}-1}^j)\right);$ Entropy coding is performed on the first transform-domain inter-layer prediction data to form a code stream of a second image.

Further, in the above step 103, according to the prediction mode of the first image, the following two processing methods can also be adopted:

First, the prediction mode of the first image is the processing method of the intra prediction mode:

其中，

为第l-1层，第n帧，第i个编码块的编码端解码重建值，f_l-1，n ^i，pred为第l-1层，第n帧，第i个编码块的预测值；对l-1层的第三编码纹理

进行DCT变换得到

对DCT变换后的数据进行上采样，获得第三编码纹理

的变换域数据

Obtain the third coded texture of the first image, transform and quantize the third coded texture, and upsample the transformed data to obtain the transform domain data of the third coded texture; for example: if the obtained l-1 layer (th One image) the third coded texture is

in,

For the l-1th layer, the nth frame, the encoding end decodes the reconstruction value of the i-th coding block, f _{l-1, n} ^{i, pred} is the l-1th layer, the nth frame, the prediction of the i-th coding block Value; third coded texture for layer l-1

Perform DCT transformation to get

Upsampling the DCT-transformed data to obtain the third coded texture

Transform domain data of

其中，

为第l层，第n帧，第j个编码块的编码端解码重建值，f_l，n ^i，pred为第l层，第n帧，第i个编码块的预测值，则对第四编码纹理

进行DCT变换量化后，得到第四编码纹理的变换域数据

Obtain the fourth encoded texture of the second image, perform transformation and quantization on the fourth encoded texture, and obtain the transform domain data of the fourth encoded texture; for example: if the acquired fourth encoded texture of the second image (or the second residual )for

in,

is the lth layer, the nth frame, the encoding end decoding reconstruction value of the jth coding block, f _{l, ni} ^{, pred} is the lth layer, the nth frame, the prediction value of the ith coding block, then for the fourth coded texture

After performing DCT transformation and quantization, the transformation domain data of the fourth coded texture is obtained

为参考，对第四编码纹理的变换域数据

进行层间预测，得到第二变换域层间预测数据 $\mathrm{DCT}({\hat{f}}_{l,n}^j-f_{l,n}^{i,\mathrm{pred}})-U\left(\mathrm{DCT}({\hat{f}}_{l-1,n}^i-f_{l-1,n}^{i,\mathrm{pred}})\right);$ 对第二变换域层间预测数据 $\mathrm{DCT}({\hat{f}}_{l,n}^j-f_{l,n}^{i,\mathrm{pred}})-U\left(\mathrm{DCT}({\hat{f}}_{l-1,n}^i-f_{l-1,n}^{i,\mathrm{pred}})\right);$ 进行熵编码，形成第二图像的码流。performing inter-layer prediction on the transform-domain data of the fourth coded texture by using the transform-domain data of the third coded texture as a reference to obtain second transform-domain inter-layer predictive data, and performing entropy coding on the second transform-domain inter-layer predictive data, Form the code stream of the second image, e.g. transform domain data with a third coded texture

For reference, the transform domain data for the fourth coded texture

Perform inter-layer prediction to obtain inter-layer prediction data in the second transform domain $\mathrm{DCT}({\hat{f}}_{l,\mathrm{no}}^j-f_{l,\mathrm{no}}^{i,\mathrm{pred}})-u\left(\mathrm{DCT}({\hat{f}}_{l-1,\mathrm{no}}^i-f_{l-1,\mathrm{no}}^{i,\mathrm{pred}})\right);$ For the second transform domain inter-layer prediction data $\mathrm{DCT}({\hat{f}}_{l,\mathrm{no}}^j-f_{l,\mathrm{no}}^{i,\mathrm{pred}})-u\left(\mathrm{DCT}({\hat{f}}_{l-1,\mathrm{no}}^i-f_{l-1,\mathrm{no}}^{i,\mathrm{pred}})\right);$ Entropy encoding is performed to form a code stream of the second image.

Second, the prediction mode of the first image is the processing method of the inter-frame prediction mode:

(残差)，对l-1层的第三编码纹理进行DCT变换和量化得到

对DCT变换后的数据

进行上采样

得到第三编码纹理的变换域数据

Get the third encoded texture of layer l-1 (first image) as

(residual), for the third encoded texture of layer l-1 Perform DCT transformation and quantization to get

For DCT transformed data

upsampling

Get the transform domain data of the third encoded texture

进行DCT变换和量化后得到第四编码纹理的变换域数据

以第三编码纹理的变换域数据

为参考对第四编码纹理

的变换域数据

进行层间预测，得到第二变换域层间预测数据对第二变换域层间预测数据进行熵编码，形成第二图像的码流。Get the fourth encoded texture of the second image For the fourth encoded texture

Transform domain data of the fourth coded texture is obtained after DCT transformation and quantization

Transform domain data of the encoded texture in a third

as a reference to the fourth coded texture

Transform domain data of

Inter-layer prediction is performed to obtain second transform-domain inter-layer prediction data, and entropy coding is performed on the second transform-domain inter-layer prediction data to form a code stream of a second image.

Further, if there are images of at least three sizes in the original sequence, the first image is used as a reference, or the second image is used as a reference and the second image is used as a first image, and the One of the remaining images of the at least three sizes of images is used as a second image, and if the second size of the second image in the original sequence is different from the first size, the first image is used as the second image. In order to refer to the predictive encoding of the second image in the transform domain, encode the data of the second image after predictive encoding, and obtain the code stream of the second image.

On the basis of the above-mentioned embodiments shown in Figures 1 to 3, it may further include: sending identification information for marking the spatially scalable code stream as transform-domain inter-layer prediction, so that the receiving device performs The spatially scalable code stream formed after the above coding is reconstructed in the transform domain;

Among them, the receiving device can be a network node capable of rewriting multi-layer code streams in the network, or a multi-layer code stream decoder capable of decoding multi-layer code streams, but the above two devices do not constitute Restricted by the embodiment of the invention, as long as the device capable of rewriting or decoding the received multi-layer code stream is the receiving device described in the embodiment of the invention.

Alternatively, the identification information for marking the multi-layer code stream as transform-domain inter-layer prediction may also be added to the multi-layer code stream, so that the receiving device performs transform domain reconstruction processing on the encoded multi-layer code stream according to the identification information.

By marking the multi-layer code stream as the identification information of transform domain inter-layer prediction in the marking information, specifically, the identification information can mark that the multi-layer code stream has the function of rewriting the spatial layer code stream, or mark the multi-layer code stream The code stream adopts the inter-layer predictive coding based on the transform domain in the encoding of multiple spatial layers; by transmitting the tag information to the decoding device or network node, or transmitting it to the decoding device or network node in a default way, Through the flag information, the decoding device or the network node knows that the multi-layer code stream has the function of rewriting the code stream of the spatial layer, or adopts the inter-layer predictive coding of the transform domain in the coding of multiple spatial layers. If the tag information is sent to the decoding device or network node in the form of transmission, the tag information can be carried through the code stream, or through related protocols (for example: Session Description Protocol (Session Description Protocol, referred to as: SDP)) or The message message (for example: Real-time Transport Control Protocol (Real-time Transport Control Protocol, RTCP for short) message) or data packet is transmitted to the network node or decoding device.

Fig. 4 is a schematic flow chart of an embodiment of the video code stream rewriting method of the present invention, Fig. 5 is a schematic diagram of the code stream rewriting in the embodiment shown in Fig. 4, and Fig. 6 is a code stream rewriting in the embodiment shown in Fig. 4 Another schematic diagram; as shown in Figure 4, the embodiment of the present invention includes the following steps:

Step 401. Obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

Step 402: In the process of rewriting the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain with the first The transform domain data corresponding to the first coded texture of the image is used as a reference, the transform domain data of the second coded texture of the second image is reconstructed, and the code stream is rewritten for the transform domain data of the second coded texture of the second image.

In the video code stream rewriting method provided by the embodiment of the present invention, the transform domain data corresponding to the first coded texture of the first image is reconstructed in the transform domain by using the transform domain data corresponding to the first coded texture of the first image as a reference, The code stream rewriting is performed on the transform domain data of the second coded texture of the second image. Since the code stream of the second image does not need to be inversely transformed in the transform domain again when rewriting the code stream of the second image, the code stream rewriting is reduced. Write complexity.

Further, in conjunction with what is shown in FIG. 5 , on the basis of the above-mentioned embodiment shown in FIG. 4 , step 402 may specifically include:

performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain first transform coefficients, performing upsampling on the first transform coefficients, and performing scaling processing in the transform domain to obtain the transform domain data of the first coded texture of the first image;

performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data;

reconstructing the transform domain data of the second coded texture of the second image from the transform domain data of the first coded texture and the third transform domain inter-layer prediction data;

Entropy coding is performed on the prediction mode in the spatially scalable code stream and the transform domain data of the second coded texture to obtain a rewritten code stream.

Further, in combination with what is shown in FIG. 6 , on the basis of the above embodiment shown in FIG. 4 , step 402 may specifically include:

performing entropy decoding on the code stream of the first image with a first size in the spatially scalable code stream to obtain first transformed data, performing inverse transformation on the first transformed data, and performing upsampling and summing on the inversely transformed data Transform and quantize to obtain transform domain data of the first coded texture of the first image;

performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data;

reconstructing the transform domain data of the second coded texture of the second image from the transform domain data of the first coded texture and the second transform domain inter-layer prediction data of the second image;

Entropy coding is performed on the transform domain data of the second coded texture and the prediction mode of the spatially scalable code stream to obtain a rewritten code stream.

Further, in the above-mentioned embodiments shown in FIGS. 4 to 6 , the code stream after rewriting is a single-layer code stream or a spatially scalable code stream obtained through transform-domain inter-layer predictive coding.

Fig. 7 is a schematic flowchart of an embodiment of the video decoding method of the present invention, Fig. 8 is a schematic diagram of video decoding in the embodiment shown in Fig. 7, and Fig. 9 is another schematic diagram of video decoding in the embodiment shown in Fig. 7; 7, the embodiment of the present invention includes the following steps:

Step 701. Obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding;

Step 702: In the process of decoding the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain, the first image The transform domain data corresponding to the first coded texture is used as a reference, and the transform domain data of the second coded texture of the second image is reconstructed;

Step 703: Perform inverse transformation on the transform domain data of the second coded texture to obtain the second coded texture, and use the second coded texture to reconstruct a second image.

The video decoding method provided by the embodiment of the present invention reconstructs the transform domain data of the second coded texture of the second image by using the transform domain data corresponding to the first coded texture of the first image as a reference in the transform domain, reducing the decoding cost. of complexity.

Further, it will be described in conjunction with FIG. 8 , in the above embodiment shown in FIG. 7 , the above step 702 may include:

performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain a first transform coefficient, performing upsampling on the first transform coefficient, and performing scaling processing in the transform domain to obtain the first transform coefficient transform domain data of a first coded texture of an image;

performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data;

The transform domain data of the second coded texture of the second image is reconstructed from the transform domain data of the first coded texture and the second transform domain inter-layer prediction data.

In the above process, since the transform domain data corresponding to the first coded texture of the first image is directly used as a reference in the transform domain to reconstruct the transform domain data of the second coded texture of the second image, the complexity of decoding is reduced.

Further, it will be described with reference to FIG. 9 , in the above embodiment shown in FIG. 7 , the above step 702 may also include:

performing entropy decoding on the code stream of the first image with a first size in the spatially scalable code stream to obtain a first transformation coefficient, performing inverse transformation on the first transformation coefficient, and performing upsampling and summing on the inversely transformed data transforming to obtain transform domain data of the first coded texture of the first image;

performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data;

The transform domain data of the second coded texture of the second image is reconstructed from the transform domain data of the first coded texture and the third transform domain inter-layer prediction data of the second image.

Further, on the basis of the above-mentioned embodiments shown in Figures 7-9, it may also include:

Reconstructing the transform domain data of the second coded texture of the second image in the transform domain according to the received identification information for marking the code stream with spatial scalability as transform domain inter-layer prediction;

By transmitting the marking information to the decoding device, or transmitting it to the decoding device or the network node in an applicable default manner, the decoding device can know that the multi-layer code stream has the function of rewriting the spatial layer code stream through the marking information, or in the Inter-layer predictive coding in the transform domain is used in the coding of multiple spatial layers; if the tag information is sent to the decoding device or network node in the form of transmission, the tag information can be carried in the code stream, or through the relevant The protocol (for example: SDP) or message message (for example: RTCP message) or data packet is transmitted to the network node or decoding device.

FIG. 10 is a schematic structural diagram of an embodiment of a video encoding device of the present invention. The embodiment of the present invention can realize the flow of the method embodiment shown in FIG. 1 above. As shown in FIG. 10, the embodiment of the present invention includes: an acquisition module 11, Encoding module 12, second encoding module 13, first processing module 14;

Wherein, the acquisition module 11 acquires the original sequence of video image data; the first encoding module 12 predicts, transforms, quantizes, and encodes the first image with the first size in the original sequence to obtain the code stream of the first image; The first processing module 14 predicts, transforms, and quantizes the second image with the second size in the original sequence according to the prediction mode of the first image, and obtains the transform domain data of the second image; if the original sequence The second image in the second image has a second size that is different from the first size, and the second encoding module 13 uses the transform domain data of the first image as a reference to predict the second image in the transform domain. Encoding the predicted transform domain inter-layer prediction data of the second image to obtain the code stream of the second image; wherein, the space formed by the code stream of the first image and the code stream of the second image can be Scalable code stream.

In the video encoding device provided by the embodiment of the present invention, the second encoding module 13 uses the transform domain data of the first image as a reference to predict the second image in the transform domain, encode the predicted data of the second image, and obtain the second image For the code stream of two images, since the first image in the original sequence does not need to be inversely transformed in the transform domain again, the complexity of encoding is reduced.

Further, on the basis of the above-mentioned embodiment shown in FIG. 10 , the second encoding module may further include: a first processing unit, a second processing unit, and a first encoding unit; the first processing unit acquires the first image of the first image. A coded texture, upsampling the first coded texture, transforming and quantizing the upsampled data to obtain transform domain data of the first coded texture; the second processing unit obtains the second coded texture of the second image , transform and quantize the second coded texture to obtain the transform domain data of the second coded texture; the first coding unit transforms the second coded texture with the transform domain data of the first coded texture as a reference Inter-layer prediction is performed on the domain data to obtain first transform-domain inter-layer prediction data, and entropy coding is performed on the first transform-domain inter-layer prediction data to form a code stream of the second image.

Further, on the basis of the above-mentioned embodiment shown in FIG. 10 , the second encoding module may further include: a third processing unit, a fourth processing unit, and a second encoding unit; wherein, the third processing unit acquires the first image the third coded texture, transform and quantize the third coded texture, and upsample the transformed data to obtain the transform domain data of the third coded texture; the fourth processing unit obtains the second image of the second image Four coded textures, performing transformation and quantization on the fourth coded texture to obtain transform domain data of the fourth coded texture of the second image; the second encoding unit uses the transform domain data of the third coded texture as a reference to the fourth coded texture. Perform inter-layer prediction on the transformation domain data of the four-coded texture to obtain second transformation domain inter-layer prediction data, and perform entropy coding on the second transformation domain inter-layer prediction data to form a code stream of the second image.

Further, on the basis of the above-mentioned embodiment shown in FIG. 10 , it further includes: a sending module, configured to send identification information for marking the spatially scalable code stream as transform-domain inter-layer prediction, so that the receiving device according to the The identification information performs decoding processing, transcoding processing, or code stream rewriting processing on the spatially scalable code stream formed after the encoding in the transform domain.

FIG. 11 is a schematic structural diagram of an embodiment of the video code stream rewriting device of the present invention. The embodiment of the present invention can realize the flow of the method embodiment shown in FIG. 4 above. As shown in FIG. 11 , the embodiment of the present invention includes: an acquisition module 111 , code stream rewriting module 112;

Wherein, the acquisition module 111 acquires the spatially scalable code stream obtained through transform domain inter-layer predictive coding; the code stream rewriting module 112 rewrites the spatially scalable code stream, if the spatially scalable code stream The second image in the second image has a second size different from the first size of the first image, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the second image The transform domain data of the second coded texture of the second image, performing code stream rewriting on the transform domain data of the second coded texture of the second image.

In the video code stream rewriting device provided by the embodiment of the present invention, the code stream rewriting module 112 reconstructs the second code of the second image by referring to the transform domain data corresponding to the first coded texture of the first image in the transform domain. The transform domain data of the texture, rewrite the code stream of the transform domain data of the second coded texture of the second image, because there is no need to inversely transform the image in the original sequence on the transform domain again, so the code stream rewriting is reduced Write complexity.

Further, on the basis of the above-mentioned embodiment shown in FIG. 11 , the code stream rewriting module may further include: a first processing unit, a second processing unit, a first reconstruction unit, and a first entropy encoding unit; wherein, the first processing A unit, configured to perform entropy decoding on a code stream of a first image having a first size in the spatially scalable code stream to obtain first transformed data, perform inverse transformation on the first transformed data, and perform inverse transformation on the inversely transformed data performing upsampling and transforming to obtain transform domain data of the first coded texture of the first image; a second processing unit configured to process a code stream of a second image having a second size in the spatially scalable code stream Entropy decoding to obtain second transform-domain inter-layer prediction data; a first reconstruction unit configured to reconstruct the second image according to the transform-domain data of the first coded texture and the second transform-domain inter-layer prediction data of the second image Transform domain data of the second coded texture; a first entropy coding unit configured to perform entropy coding on the transform domain data of the second coded texture and the prediction mode of the spatially scalable code stream to obtain a rewritten code stream.

Further, on the basis of the above-mentioned embodiment shown in FIG. 11 , the code stream rewriting module may further include: a third processing unit, a fourth processing unit, a second reconstruction unit, and a second entropy encoding unit; wherein, the second processing A unit, configured to perform entropy decoding on a code stream of a first image having a first size in the spatially scalable code stream to obtain a first transform coefficient, perform upsampling on the first transform coefficient, and perform scaling in a transform domain processing to obtain the transform domain data of the first coded texture of the first image; a fourth processing unit, configured to perform entropy decoding on the code stream of the second image having a second size in the spatially scalable code stream to obtain the first Three transform-domain inter-layer prediction data; a second reconstruction unit configured to reconstruct the transform-domain data of the second coded texture of the second image according to the transform-domain data of the first coded texture and the third transform-domain inter-layer prediction data ; A second entropy coding unit, configured to perform entropy coding on the prediction mode in the spatially scalable code stream and the transform domain data of the second coded texture to obtain a rewritten code stream.

FIG. 12 is a schematic structural diagram of an embodiment of a video decoding device of the present invention. The embodiment of the present invention can realize the flow of the method embodiment shown in FIG. 7 above. As shown in FIG. Reconstruction module 122 , second reconstruction module 123 ; third reconstruction module 124 .

Among them, the obtaining module 121 obtains the spatially scalable code stream obtained through transform-domain inter-layer predictive coding; the first reconstruction module 122 obtains the spatially scalable code stream obtained through transform-domain inter-layer predictive coding; During the decoding process of the scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, the first image of the first image in the transform domain The transform domain data corresponding to a coded texture is used as a reference to reconstruct the transform domain data of the second coded texture of the second image; the third reconstruction module 124 inversely transforms the transform domain data of the second coded texture to obtain the second coded texture texture, reconstructing the second image using the second coded texture.

In the video decoding device provided by the embodiment of the present invention, the first reconstruction module 122 reconstructs the second coded texture of the second image by referring to the transform domain data corresponding to the first coded texture of the first image in the transform domain For transform domain data, since the first image in the original sequence does not need to be inversely transformed in the transform domain again, the complexity of decoding is reduced.

Further, on the basis of the above-mentioned embodiment shown in FIG. 12 , the first reconstruction module may further include: a first processing unit, a first decoding unit, and a first reconstruction unit; wherein, the first processing unit is used to process the performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain first transform coefficients, performing upsampling on the first transform coefficients, and performing scaling processing in the transform domain to obtain the first Transform domain data of the first coded texture of the image; a first decoding unit configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data a first reconstruction unit, configured to reconstruct the transform domain data of the second coded texture of the second image according to the transform domain data of the first coded texture and the second transform domain inter-layer prediction data.

Further, on the basis of the above-mentioned embodiment shown in FIG. 12 , the first reconstruction module may further include: a second processing unit, a second decoding unit, and a second reconstruction unit; wherein, the second processing unit is used to process the performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain a first transformation coefficient, performing inverse transformation on the first transformation coefficient, and performing upsampling and transformation on the inversely transformed data to obtain Transform domain data of the first coded texture of the first image; a second decoding unit configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain a third transform domain inter-layer prediction data; a second reconstruction unit for reconstructing the transform domain of the second coded texture of the second image based on the transform domain data of the first coded texture and the third transform-domain inter-layer prediction data of the second image data.

FIG. 13 is a schematic structural diagram of an embodiment of the video encoding and decoding system of the present invention. As shown in FIG. 13 , the embodiment of the present invention includes: a video encoding device 131, a code stream rewriting device 132, and a video decoding device 133;

Wherein, the video encoding device 131 obtains the original sequence of video image data; performs prediction, transformation, quantization, and encoding on the first image with the first size in the original sequence to obtain the code stream of the first image; according to the first image A prediction mode of an image, performing prediction, transformation, and quantization on a second image with a second size in the original sequence to obtain transform domain data of the second image; if the second size is different from the first size , using the transform domain data of the first image as a reference to predict the transform domain data of the second image in the transform domain, encode the predicted transform domain inter-layer prediction data of the second image, and obtain The code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code stream;

The code stream rewriting device 132 obtains the spatially scalable code stream obtained through transform domain inter-layer predictive coding; during the process of rewriting the spatially scalable code stream, if the second image in the spatially scalable code stream If the second size is not the same as the first size of the first image, the second encoded texture of the second image is reconstructed in the transform domain using the transform domain data corresponding to the first encoded texture of the first image as a reference transform domain data of the second image, performing code stream rewriting on the transform domain data of the second coded texture of the second image;

The video decoding device 133 obtains a spatially scalable code stream obtained through transform-domain inter-layer predictive coding; during the process of decoding the spatially scalable code stream, if the second image in the spatially scalable code stream has a second If the size is different from the first size of the first image, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the transform domain data of the second coded texture of the second image ; Perform inverse transformation on the transform domain data of the second coded texture to obtain the second coded texture, and use the second coded texture to reconstruct the second image.

The video encoding and decoding system provided by the embodiment of the present invention performs the second image on the transform domain according to the prediction mode of the first image and takes the transform domain data of the first image as a reference for the second image with the second size in the original sequence. Perform prediction on the transform domain, encode the data of the second image after encoding and prediction, and obtain the code stream of the second image. Since the first image in the original sequence does not need to be inversely transformed in the transform domain again, so Reduced encoding and decoding complexity.

Fig. 14 is a schematic structural diagram of a code stream rewriting system applicable to the embodiment of the present invention. As shown in Fig. 14, the embodiment of the present invention includes: an encoder 141, a network node 142, and a decoder 143; wherein, depending on the specificity of the encoder 141 It may be the video encoding device in the embodiment shown in FIG. 10; the network node 142 may be specifically the video code stream rewriting device in the embodiment shown in FIG. 11; the decoder 143 may be specifically the video decoding device in the embodiment shown in FIG. 12.

Wherein, the encoder 141 obtains the original sequence of the video image data and performs encoding processing to form a multi-layer code stream. There are at least the following three situations in the network transmission process of the multi-layer code stream: 1. The multi-layer code stream is directly transmitted to the Decoder 143 for decoding the multi-layer code stream, 2. The multi-layer code stream is transmitted to the network node 142 capable of rewriting and transcoding the multi-layer code stream, and the network node 142 rewrites or transcodes the multi-layer code stream After forming a single-layer code stream, the single-layer code stream is transmitted to the decoder 143, and the decoder 143 decodes the received single-layer code stream, and continues to transmit the multi-layer code stream to subsequent network nodes, and waits for subsequent network The node rewrites the code stream until a single-layer code stream is input; 3. After the multi-layer code stream passes through the network node 142, a multi-layer code stream with a number of code stream layers smaller than the layer number of the multi-layer code stream input to the network node 142 is formed , and then transmitted to the subsequent network nodes, so that the subsequent network nodes rewrite or transcode the multi-layer code stream, and rewrite or convert it into a single-layer code stream or multi-layer code stream according to the needs of the terminal equipment.

In the code stream rewriting system to which the embodiment of the present invention is applicable, the second image with the second size in the original sequence is performed on the transform domain by using the transform domain data of the first image as a reference to the second image in the transform domain. Prediction: encode the data of the second image after encoding and prediction, and obtain the code stream of the second image. Since the first image in the original sequence does not need to be inversely transformed in the transform domain again, the encoding and decoding are reduced. of complexity.

The peak signal-to-noise ratio (Peak Signal-to-Noise Ratio, referred to as: PSNR) of the SVC encoding of the non-rewritable code stream in the embodiment of the present invention and the prior art is shown in Table 1. The embodiment of the present invention and the prior art The PSNR in Table 2 is shown in Table 2. The rewritable coding scheme and the comparison results of simulcast coding are shown in Table 3. Rewriting the four-layer SVC code stream into one layer of AVC code stream and directly encoding one layer of AVC code stream The comparison results are shown in Table 4, wherein deltaQP represents the comparison of results when the difference between the quantization parameter (Quantization Parameter, QP) of the embodiment of the present invention and SVC is 2, 3, or 6.

The PSNR gain comparison of the embodiment of the present invention and SVC of table 1

sequence type deltaQP=2 deltaQP=3 deltaQP=6 Bus -0.753684 -0.531987 -0.25914 Mobile -0.849045 -0.558285 -0.229195

Table 2 Comparison of the embodiment of the present invention and the PSNR gain of the simulcast code stream

sequence type deltaQP=2 deltaQP=3 deltaQP=6 Bus 0.71315 0.68764 0.53639 Mobile 0.48129 0.45957 0.3193

Table 3 Comparison of rewritable coding schemes and simulcast coding

sequence type deltaQP=2 deltaQP=3 deltaQP=6 Bus 0.71315 0.68764 0.53639 Mobile 0.48129 0.45957 0.3193

Table 4 The code stream of the embodiment of the present invention is compared with the code stream of a single layer of AVC code stream after the code stream is rewritten

sequence type deltaQP=2 deltaQP=3 deltaQP=6 Bus 4.53543 4.49629 3.86001 Mobile 3.57424 3.49251 3.05672

The above-mentioned technical solution provided by the embodiments of the present invention that can realize the code stream rewriting of the multi-layer code stream can comprehensively realize the code stream rewriting of the multi-layer quality code stream and the multi-layer space code stream. The code stream generated by the embodiment of the present invention The rate distortion (RateDistortion, referred to as: RD) performance of the multi-layer code stream is the same as that of the multi-layer code stream generated by the prior art; From the first image (or low-layer code stream) of the code stream to any number of second images as the enhancement layer code stream, the receiving end can conveniently and quickly rewrite the new multi-layer code stream after receiving the multi-layer code stream Or a single-layer code stream; the decoding result of the single-layer or multi-layer code stream after rewriting is the same as the decoding result of the multi-layer code stream before rewriting.

The DCT transform in the above-mentioned embodiments of the present invention may also be other transforms that implement discrete transform domain transforms, such as wavelet transforms. The embodiments of the present invention are only described with DCT transforms and do not constitute restrictions on the transform domain processing in the embodiments of the present invention. Any processing that can be realized through other transform domain methods is the technical solution described in the embodiments of the present invention.

Those of ordinary skill in the art can understand that all or part of the steps for realizing the above-mentioned embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the execution includes The steps of the above-mentioned method embodiments; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (17)

1. A video coding method, characterized in that, comprising: Obtain a raw sequence of video image data; Predicting, transforming, quantizing, and encoding a first image with a first size in the original sequence to obtain a code stream of the first image; Predict, transform, and quantize a second image with a second size in the original sequence according to the prediction mode of the first image, to obtain transform domain data of the second image; If the second size is different from the first size, predict the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image, and predict the predicted Encoding the transform domain inter-layer prediction data of the second image to obtain a code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code stream; Predicting the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image, and encoding the predicted transform domain inter-layer prediction data of the second image, Acquiring the code stream of the second image includes: Acquire a first coded texture of the first image, perform upsampling on the first coded texture, transform and quantize the upsampled data, and obtain transform domain data of the first coded texture; Obtaining a second coded texture of a second image, performing transform and quantization on the second coded texture, and obtaining transform domain data of the second coded texture; Performing inter-layer prediction on the transform-domain data of the second coded texture with reference to the transform-domain data of the first coded texture to obtain first transform-domain inter-layer prediction data, and performing inter-layer prediction on the first transform-domain inter-layer prediction data Entropy coding is performed to form a code stream of the second image. 2. The method according to claim 1, wherein the transform domain data of the second image is predicted in the transform domain with reference to the transform domain data of the first image, and the predicted The transformation domain inter-layer prediction data of the second image is encoded, and obtaining the code stream of the second image includes: Acquiring a third coded texture of the first image, performing transform and quantization on the third coded texture, performing upsampling on the transformed data, and obtaining transform domain data of the third coded texture; Obtaining a fourth encoded texture of the second image, performing transform and quantization on the fourth encoded texture, and obtaining transform domain data of the fourth encoded texture of the second image; Performing inter-layer prediction on the transform-domain data of the fourth coded texture with reference to the transform-domain data of the third coded texture to obtain second transform-domain inter-layer predictive data, and encoding the second transform-domain inter-layer predictive data Entropy encoding is performed on the data to form a code stream of the second image. 3. The method according to any one of claims 1-2, further comprising: Sending identification information for marking the spatially scalable code stream as transform-domain inter-layer prediction, so that the receiving device performs decoding processing or transcoding on the encoded spatially scalable code stream in the transform domain according to the identification information processing or stream rewriting processing. 4. The method according to any one of claims 1-2, further comprising: If there are images of at least three sizes in the original sequence, the first image is used as a reference, or the second image is used as a reference and the second image is used as a first image, and the at least three One of the remaining images in the images of this size is used as the second image, and if the second image in the original sequence has a second size different from the first size, then according to the prediction mode of the first image performing predictive coding on the second image in the transform domain with the first image as a reference, coding the predictively coded data of the second image, and acquiring a code stream of the second image. 5. A video code stream rewriting method is characterized in that, comprising: Obtaining a spatially scalable code stream obtained through transform domain inter-layer predictive coding; In the process of rewriting the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain with the Using the transform domain data corresponding to the first coded texture of the first image as a reference, reconstructing the transform domain data of the second coded texture of the second image, and encoding the transform domain data of the second coded texture of the second image stream rewrite; The transform domain data corresponding to the first encoded texture of the first image is used as a reference in the transform domain to reconstruct the transform domain data of the second encoded texture of the second image, and the second encoded texture of the second image The code stream rewriting of the transformation domain data of the texture includes: performing entropy decoding on the code stream of the first image with a first size in the spatially scalable code stream to obtain first transformed data, performing inverse transformation on the first transformed data, and performing upsampling and summing on the inversely transformed data transforming to obtain transform domain data of the first coded texture of the first image; performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data; reconstructing the transform domain data of the second coded texture of the second image from the transform domain data of the first coded texture and the second transform domain inter-layer prediction data of the second image; Entropy coding is performed on the transform domain data of the second coded texture and the prediction mode of the spatially scalable code stream to obtain a rewritten code stream. 6. The method according to claim 5, wherein the transform domain data corresponding to the first coded texture of the first image is used as a reference in the transform domain to reconstruct the second coded texture of the second image The transform domain data of the second image, and performing code stream rewriting on the transform domain data of the second coded texture of the second image includes: performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain first transform coefficients, performing upsampling on the first transform coefficients, and performing scaling processing in the transform domain to obtain the transform domain data of the first coded texture of the first image; performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data; reconstructing the transform domain data of the second coded texture of the second image from the transform domain data of the first coded texture and the third transform domain inter-layer prediction data; Entropy coding is performed on the prediction mode in the spatially scalable code stream and the transform domain data of the second coded texture to obtain a rewritten code stream. 7. The method according to any one of claims 5-6, wherein the rewritten code stream is a single-layer code stream or a spatially scalable code stream obtained through transform domain inter-layer predictive coding. 8. A video decoding method, characterized in that, comprising: Obtaining a spatially scalable code stream obtained through transform domain inter-layer predictive coding; In the process of decoding the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then in the transform domain, the The transform domain data corresponding to the first coded texture of the first image is used as a reference, and the transform domain data of the second coded texture of the second image is reconstructed; performing inverse transformation on the transform domain data of the second coded texture to obtain the second coded texture, and using the second coded texture to reconstruct the second image; Referring to the transform domain data corresponding to the first coded texture of the first image in the transform domain, reconstructing the transform domain data of the second coded texture of the second image includes: performing entropy decoding on the code stream of the first image with the first size in the spatially scalable code stream to obtain first transform coefficients, performing upsampling on the first transform coefficients, and performing scaling processing in the transform domain to obtain the transform domain data of the first coded texture of the first image; performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data; The transform domain data of the second coded texture of the second image is reconstructed from the transform domain data of the first coded texture and the second transform domain inter-layer prediction data. 9. The method according to claim 8, characterized in that, in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the second coded texture of the second image The transform domain data includes: performing entropy decoding on the code stream of the first image with a first size in the spatially scalable code stream to obtain a first transformation coefficient, performing inverse transformation on the first transformation coefficient, and performing upsampling and summing on the inversely transformed data transforming to obtain transform domain data of the first coded texture of the first image; performing entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data; The transform domain data of the second coded texture of the second image is reconstructed from the transform domain data of the first coded texture and the third transform domain inter-layer prediction data of the second image. 10. The method according to any one of claims 8-9, further comprising: Reconstructing the transform domain data of the second coded texture of the second image in the transform domain according to the received identification information for marking the code stream with spatial scalability as transform domain inter-layer prediction. 11. A video encoding device, comprising: Obtaining module, for obtaining the original sequence of video image data; A first encoding module, configured to predict, transform, quantize, and encode a first image with a first size in the original sequence to obtain a code stream of the first image; A first processing module, configured to predict, transform, and quantize a second image with a second size in the original sequence according to the prediction mode of the first image, and obtain transform domain data of the second image; A second encoding module, configured to predict the transform domain data of the second image in the transform domain with reference to the transform domain data of the first image if the second size is different from the first size , encoding the predicted transform domain inter-layer prediction data of the second image to obtain the code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form Spatial scalable code stream; The second coding module includes: A first processing unit, configured to acquire a first coded texture of the first image, perform upsampling on the first coded texture, transform and quantize the upsampled data, and obtain a transform domain of the first coded texture data; The second processing unit is configured to obtain a second coded texture of the second image, perform transform and quantization on the second coded texture, and obtain transform domain data of the second coded texture; The first coding unit is configured to use the transform domain data of the first coded texture as a reference to perform inter-layer prediction on the transform domain data of the second coded texture to obtain first transform domain inter-layer prediction data, and perform the inter-layer prediction on the second coded texture. Entropy encoding is performed on the inter-layer prediction data in the transform domain to form a code stream of the second image. 12. The device according to claim 11, wherein the second encoding module comprises: A third processing unit, configured to acquire a third coded texture of the first image, perform transform and quantization on the third coded texture, perform upsampling on the transformed data, and obtain transform domain data of the third coded texture ; A fourth processing unit, configured to obtain a fourth encoded texture of the second image, perform transform and quantization on the fourth encoded texture, and obtain transform domain data of the fourth encoded texture of the second image; The second coding unit is configured to perform inter-layer prediction on the transform domain data of the fourth coded texture by using the transform domain data of the third coded texture as a reference to obtain second transform domain inter-layer prediction data, and perform the interlayer prediction on the first transform domain data. Entropy encoding is performed on the inter-layer prediction data in the transform domain to form a code stream of the second image. 13. The device according to any one of claims 11-12, further comprising: A sending module, configured to send identification information for marking the spatially scalable code stream as transform-domain inter-layer prediction, so that the receiving device performs encoding in the transform domain on the encoded spatially scalable code stream according to the identification information Decoding processing or transcoding processing or code stream rewriting processing. 14. A video code stream rewriting device, characterized in that, comprising: An acquisition module, configured to acquire a spatially scalable code stream obtained through transform domain inter-layer predictive coding; A code stream rewriting module, configured to rewrite the spatially scalable code stream, if the second image in the spatially scalable code stream has a second size different from the first size of the first image are the same, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the transform domain data of the second coded texture of the second image, and the second coded texture of the second image is reconstructed. The code stream rewrites the transform domain data of the encoded texture; The code stream rewriting module includes: The first processing unit is configured to perform entropy decoding on a code stream of a first image having a first size in the spatially scalable code stream to obtain first transformed data, perform inverse transformation on the first transformed data, and inverse transform the Perform upsampling and transformation on the obtained data to obtain transform domain data of the first coded texture of the first image; A second processing unit, configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data; A first reconstruction unit, configured to reconstruct the transform domain data of the second coded texture of the second image according to the transform domain data of the first coded texture and the second transform domain inter-layer prediction data of the second image; The first entropy coding unit is configured to perform entropy coding on the transform domain data of the second coded texture and the prediction mode of the spatially scalable code stream to obtain a rewritten code stream. 15. according to claim 14, is characterized in that, described code stream rewriting module comprises: The third processing unit is configured to perform entropy decoding on the code stream of the first image having a first size in the spatially scalable code stream to obtain a first transformation coefficient, and after upsampling the first transformation coefficient, perform the transformation domain to perform scaling processing to obtain transform domain data of the first coded texture of the first image; A fourth processing unit, configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data; The second reconstruction unit is configured to reconstruct the transform domain data of the second coded texture of the second image according to the transform domain data of the first coded texture and the third transform domain inter-layer prediction data; The second entropy coding unit is configured to perform entropy coding on the prediction mode in the spatially scalable code stream and the transform domain data of the second coded texture to obtain a rewritten code stream. 16. A video decoding device, comprising: An acquisition module, configured to acquire a spatially scalable code stream obtained through transform domain inter-layer predictive coding; A first reconstruction module, configured to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding; The second reconstruction module is configured to decode the spatially scalable code stream, if the second size of the second image in the spatially scalable code stream is different from the first size of the first image, then In the transform domain, using the transform domain data corresponding to the first coded texture of the first image as a reference, reconstructing the transform domain data of the second coded texture of the second image; The third reconstruction module is configured to perform inverse transformation on the transform domain data of the second coded texture, obtain the second coded texture, and use the second coded texture to reconstruct the second image; The first reconstruction module includes: The first processing unit is configured to perform entropy decoding on a code stream of a first image having a first size in the spatially scalable code stream to obtain a first transformation coefficient, and after upsampling the first transformation coefficient, perform the transformation domain to perform scaling processing to obtain transform domain data of the first coded texture of the first image; A first decoding unit, configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain second transform domain inter-layer prediction data; The first reconstruction unit is configured to reconstruct the transform domain data of the second coded texture of the second image according to the transform domain data of the first coded texture and the second transform domain inter-layer prediction data. 17. according to claim 16, is characterized in that, described first reconstruction module comprises: The second processing unit is configured to perform entropy decoding on a code stream of a first image having a first size in the spatially scalable code stream to obtain a first transformation coefficient, perform an inverse transformation on the first transformation coefficient, and perform an inverse transformation on the first transformation coefficient Perform upsampling and transformation on the obtained data to obtain transform domain data of the first coded texture of the first image; A second decoding unit, configured to perform entropy decoding on a code stream of a second image having a second size in the spatially scalable code stream to obtain third transform domain inter-layer prediction data; The second reconstruction unit is configured to reconstruct the transform domain data of the second coded texture of the second image according to the transform domain data of the first coded texture and the third transform domain inter-layer prediction data of the second image. 18. A video codec system, characterized in that it includes: a video encoding device, at least one video decoding device, at least one code stream rewriting device, The video encoding device is configured to obtain an original sequence of video image data; perform prediction, transformation, quantization, and encoding on a first image with a first size in the original sequence to obtain a code stream of the first image; according to the The prediction mode of the first image, predict, transform, and quantize the second image with the second size in the original sequence, and obtain the transform domain data of the second image; if the second size is the same as the first size Not the same, the transform domain data of the second image is predicted in the transform domain with reference to the transform domain data of the first image, and the predicted transform domain inter-layer prediction data of the second image is encoded , acquiring the code stream of the second image; wherein, the code stream of the first image and the code stream of the second image form a spatially scalable code stream; The code stream rewriting device is used to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding; during the process of rewriting the spatially scalable code stream, if the spatially scalable code stream is If the second size of the second image is different from the first size of the first image, then in the transform domain, the transform domain data corresponding to the first coded texture of the first image is used as a reference to reconstruct the second image Transform domain data of the second coded texture, rewriting the code stream of the transform domain data of the second coded texture of the second image; The video decoding device is configured to obtain a spatially scalable code stream obtained through transform domain inter-layer predictive coding; during the process of decoding the spatially scalable code stream, if the second image in the spatially scalable code stream If the second size is not the same as the first size of the first image, the second encoded texture of the second image is reconstructed in the transform domain using the transform domain data corresponding to the first encoded texture of the first image as a reference The transform domain data of the second coded texture is inversely transformed to obtain the second coded texture, and the second coded texture is used to reconstruct the second image.

Images