CN1489391A - Novel Spatial Prediction Method and Device for Video Coding - Google Patents

Abstract

A new spatial prediction method and device for video coding. When encoding a code stream, the predicted image is transformed into the frequency domain for frequency domain quantization and then used as a predicted image; when decoding, the predicted image is quantized in the frequency domain and then compensated. onto the decoded residual image. The device is at least provided with an encoding module and a decoding module; the encoding module is provided with a prediction module, a residual coefficient calculation module, a scanning module and an entropy encoding module; the decoding module is provided with: an entropy decoding module, a compensation module, an inverse quantization module, and an inverse transformation module and transform quantization modules. The invention overcomes the defect of flicker phenomenon when the video coding method based on multi-directional spatial prediction is applied to the coding of continuous I frames, improves the coding efficiency of the video coding method of multi-directional spatial prediction; and provides continuous I frames with anti-flickering The encoding scheme and the pattern-based residual coefficient scanning scheme further ensure the encoding efficiency; a specific system structure for realizing the above method is provided.

Description

Novel Spatial Prediction Method and Device for Video Coding

Technical field :

The present invention relates to a novel spatial prediction method and device for video coding, specifically refers to the elimination of flicker generated by a full intra-frame (I frame) coded video stream in a video coding and decoding technology based on the JVT standard The invention relates to a novel space prediction method and a device for realizing the method for further improving codec efficiency and compensation; it belongs to the technical field of digital video processing.

Background technology :

Efficient video codec technology is the key to realizing multimedia data storage and transmission, and advanced video codec technology usually exists in the form of standards. At present, the typical video compression standards include the MPEG series of international standards launched by the Moving Picture Expert Group (MPEG) under the International Organization for Standardization (ISO) and the H.26x series of video compression standards proposed by the International Telecommunication Union (ITU). , and the JVT video coding standard being developed by the Joint Video Team (JVT for short) established by ISO and ITU. The JVT standard adopts a new type of coding technology, which has much higher compression efficiency than any existing coding standard. The official name of the JVT standard in the ISO is the tenth part of the MPEG-4 standard, and the official name in the ITU is the H.264 standard.

The video encoding process is the process of encoding each frame of the video sequence. In the JVT video coding standard, the coding of each frame of image takes macroblock as the basic unit. When coding each frame of image, it can be divided into intra-frame (I frame) coding, predictive (P frame) coding and bidirectional predictive (B frame) coding and so on. The characteristic of I frame coding is that it does not need to refer to other frames when encoding and decoding. Generally speaking: during encoding, I frame, P frame and B frame encoding are interspersed, for example: in the order of IBBPBBP. However, for some special applications, such as applications that require low computational complexity, low storage capacity, or real-time compression, only I frames can be used for encoding. In addition, full I-frame encoded video is also easy to edit. In I-frame coding, the redundancy in the macroblock is eliminated by orthogonal transformation, such as discrete cosine transformation (DCT), wavelet transformation and so on. However, when traditional video coding algorithms eliminate the redundancy between macroblocks, they usually use the prediction method in the coefficient domain of orthogonal transformation. However, this prediction can only be done on the DC component, so it is not efficient.

In I-frame coding, multi-directional spatial prediction is the mainstream of current research, and good results have been obtained. Intra-frame spatial prediction means: when performing I-frame encoding and decoding, first of all, according to a certain mode, the information in the frame (obtainable by the decoding end) (for example: adjacent reconstructed blocks) is used to generate the current block Prediction; then, subtract the predicted block from the actual block to be encoded to obtain a residual, and then encode the residual.

Multi-directional spatial prediction technology has been well applied in video coding. The JVT video coding standard adopts this technology. However, there are still two main shortcomings in the existing multi-directional spatial prediction technology: one is that when the existing technology is applied to continuous I-frame coding, it will produce serious flickering phenomenon, which affects the visual effect; the other is: the multi-directional spatial Prediction changes the probability distribution of the residual image in the coefficient domain, while the existing methods still use a fixed zigzag scan order of the transform coefficients, see Figure 4, the zigzag scan refers to the transformation quantized in the video coding scheme The coding order of the coefficients of the block has a great influence on the coding efficiency. In current encoding systems (jpeg, mpeg, etc.), this fixed scanning order is generally adopted for blocks of the same size, so the encoding efficiency is not optimal. JVT is an efficient video coding standard currently being formulated. It was first formulated by the ITU (International Telecommunications Union, International Telecommunications Union), and then adopted by the ISO/IEC International Standards Organization as the tenth part of ISO/IEC 14496 (MPEG4).

However, when JVT is used for full I-frame encoding, the reconstructed image will flicker during playback. After analysis and verification, it is considered that it is mainly caused by the variable block size and the relative randomness of intra-frame prediction during encoding.

A variable block size means that a coded macroblock can be subdivided into smaller sub-blocks according to the coding mode. Different division modes have different sizes of sub-blocks. The variable block size causes flickering. The main reason is that the block with the same position in the previous frame and the next frame and the content basically does not change. Different segmentation methods are used during encoding, resulting in very different reconstruction results. This part can be avoided by appropriately modifying the encoding strategy of the encoder without modifying the decoder. Referring to FIG. 1 , it is a schematic diagram of subdividing a macroblock into various fine blocks in the JVT.

Ordinary coding schemes generally only have inter-frame prediction, which is used to eliminate temporal redundancy, and spatial redundancy is eliminated by various transformations. JVT proposes intra-frame prediction, which is used together with transform coding to eliminate spatial redundancy, thereby greatly improving coding efficiency. Specifically, there are two types of Intra4×4 mode and Intra16×16 mode (Intra4×4 mode and Intra16×16 mode are two macroblock division modes, Intra4×4 mode has 9 prediction modes, Intra16×16 has 4 a predictive model). Referring to FIG. 2 , in the Intra4×4 mode, intra-frame prediction is performed on each sub-block in a macroblock. The middle pixel of each 4×4 small block will be predicted by the decoded 17 pixels in the adjacent block.

Referring to FIG. 3 , intra-frame prediction modes are divided into nine types (mode 0 to mode 8), wherein mode 2 is the DC prediction in the MPEG-4 standard.

In the Intra16×16 mode, it is assumed that the pixels of the block to be predicted are represented by P(x, y),

Among them, x, y=0...15,

Left critical block pixel P(-1, y) of the predicted block, y=0..15,

Critical block pixels P(x, -1) on the predicted block, x=0..15.

Four prediction modes are defined, namely: vertical prediction, horizontal prediction, DC prediction and planar prediction. There are also four prediction modes for chroma blocks, which are basically similar to luma blocks.

The relative randomness of intra-frame prediction means that due to the difference in the information in the frame used to generate the prediction and the difference in the prediction mode, the blocks with the same position and basically no change in the content of the two frames before and after the two frames generally have a lower prediction value. Are not the same. When continuous I-frame coding is performed for a system without intra-frame prediction, the small difference between the two corresponding blocks before and after will be removed by the quantization operation in the transform domain. The more similar the British and German macroblocks are, the greater the probability that this difference will be removed. If two patches are identical, the reconstructed image is also identical. However, for systems with intra prediction, the reconstructed image consists of the addition of two parts: the predicted image and the reconstructed residual image. Due to the quantization process in the frequency domain, the reconstructed residual has a frequency domain coefficient that satisfies an integer multiple of the quantization step size; and because the predicted image does not have this process, its frequency domain coefficient just satisfies the possibility of an integer multiple of the quantization step size Very small, in fact, if its frequency domain coefficient is divided by the quantization step size, the fractional part of the obtained coefficient can be considered as a random number between 0-1 (when the quantization step size is not very large), that is, it is equal to 0 The probability of any number between -1 (including 0) is equal.

Since the reconstructed image is composed of such two parts, the fractional part of the coefficient obtained by dividing its frequency domain coefficient by the quantization step size can also be regarded as a random number between 0-1. For two corresponding blocks with similar pixel values, due to the relative randomness of the reconstructed image in the frequency domain, unlike the system without intra-frame prediction, the similarity of their reconstructed images is equal to their own similarity The relationship is not very close. Even if they are identical themselves, the probability that the reconstructed images are identical is very small.

Referring to Figures 5 and 6, the existing encoding process is as follows: the reconstructed image is processed by the prediction module under the control of the mode selection module, and the predicted image is output. After the predicted image and the current encoded image are processed by the residual coefficient calculation module, After processing by the scanning module and entropy coding, the coded stream is finally output.

Referring to FIG. 7 , the signal flow of the decoding part of the multi-directional spatial predictive coding system in the prior art is as follows: the decoded code stream is subjected to entropy decoding, inverse quantization and inverse transformation, and then compensated by the predicted image before being output as a video stream.

The above encoding/decoding process cannot overcome the flickering phenomenon when the video coding method based on multi-directional spatial prediction is applied to continuous I frame coding, and the coding efficiency of the video coding method based on multi-directional spatial prediction cannot be improved due to the fixed scanning method .

Contents of the invention

The main purpose of the present invention is to provide a novel spatial prediction method for video coding, to overcome the defect that the video coding method based on multi-directional spatial prediction produces flickering phenomenon when it is applied to continuous I frame coding, and to improve the performance of multi-directional spatial prediction. The coding efficiency of the video coding method provides a flicker-resistant continuous I-frame coding scheme and a pattern-based residual coefficient scanning scheme for the video coding technology based on multi-directional spatial prediction, which ensures coding efficiency while reducing flicker.

Another object of the present invention is to provide a novel spatial prediction method for video coding, using the JVT standard as an implementation example, to provide the JVT standard with specific technical means to solve the problem of continuous I-frame coding flicker and improve coding efficiency.

Another object of the present invention is to provide a novel spatial prediction device for video coding, and provide a specific device for realizing the above method.

The purpose of the present invention is achieved through the following technical solutions:

A new spatial prediction method for video coding. When encoding the code stream, the prediction image is also transformed into the frequency domain by the transformation method used when processing the residual image, and quantized with the same quantization coefficient, and then used as Predicted image; when decoding, the predicted image is quantized in the frequency domain using the same processing method as that used during encoding, and then compensated to the decoded residual image.

The encoding process is specifically:

Step 100: According to the selected prediction mode, a predicted image is generated from the decoded images of adjacent blocks of the current coding block;

Step 101: Transform the predicted image into the frequency domain;

Step 102: Quantize the frequency domain coefficients of the predicted image, wherein the quantization coefficients are the same as the quantization coefficients used when processing the residual image; and the matrix of the quantized frequency domain coefficients satisfies the following formula:

Z＝Q(Y)＝(Y×Quant(Qp)+Qconst(Qp))＞＞Q_bit(Qp)

in,

Z is the quantized frequency domain coefficient matrix,

Y is the frequency domain coefficient matrix,

Qp is the quantization parameter,

Quant(Qp), Qconst(Qp), Q_bit(Qp) are quantization functions defined by JVT;

Step 103: Dequantize the matrix of frequency domain coefficients obtained in step 102 according to the following formula;

W＝DQ(Z)＝(Z×DQuant(Qp)+DQconst(Qp))＞＞Q_per(Qp)

DQuant(Qp)×DQconst(Qp)≈2 ^{Q_per(Qp)×Q_bit(Qp)}

in,

W is the frequency domain coefficient matrix after inverse quantization,

Z is the frequency domain coefficient matrix before inverse quantization after quantization,

Qp is the quantization parameter,

DQuant(Qp), DQconst(Qp), Q_bit(Qp), Q_per(Qp) are quantization functions defined by JVT;

Step 104: according to the method of steps 100-103, transform the encoded current block into the frequency domain to obtain a frequency domain image;

Step 105: subtracting the dequantized frequency domain coefficient matrix from the frequency domain image to directly obtain the frequency domain residual image;

Step 106: Quantize the frequency domain residual image to obtain the quantized frequency domain residual coefficient, the formula is the same as step 102;

Step 107: Perform coefficient scanning on the residual coefficients in the frequency domain, and entropy coding to obtain a code stream;

Step 108: Compensate the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula;

C=C+Z;

in,

C is the frequency domain residual coefficient,

Z is the frequency domain coefficient matrix;

Step 109: dequantize the residual coefficient in the frequency domain with the formula of JVT;

Step 110: Inversely transform the residual coefficients in the frequency domain according to the size and mode of the block to obtain a preliminary reconstructed image;

Step 111: Filter the reconstructed image to remove block effects to obtain an output image of the current block.

Before the above encoding, it further includes the processing of determining the scanning order based on the prediction mode. The specific process is: respectively counting the probability of non-zero coefficients of each frequency of the residual image in each mode, and according to the value of the probability, from large to large Generate a scan order table in order to replace a single zigzag scan table. The zigzag scanning refers to the encoding order of coefficients of transformed and quantized blocks in a video encoding scheme, and the order has a great influence on encoding efficiency. Referring to FIG. 4 , in current encoding systems (jpeg, mpeg, etc.), a fixed scanning order is generally adopted for blocks of the same size.

When coding and scanning, the scan order table is consulted according to the selected mode order, and the residual coefficients are scanned in the order of the found positions.

The described decoding process is:

Step 200: Obtain the prediction mode and frequency domain residual coefficients through entropy decoding;

Step 201: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block;

Step 202: Transform the predicted image into the frequency domain;

Step 203: Quantize the frequency-domain coefficients of the predicted image to obtain a frequency-domain coefficient matrix;

Step 204: Compensate the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula,

C=C+Z

in,

C is the frequency domain residual coefficient,

Z is the frequency domain coefficient matrix;

Step 205: dequantize the residual coefficient in the frequency domain;

Step 206: According to the block size and mode, inversely transform the residual coefficients in the frequency domain to obtain a preliminary reconstructed image;

Step 207: Filter the reconstructed image to remove block effects to obtain an output image of the current block.

The decoding process can also be:

Step 210: Obtain the prediction mode and frequency domain residual coefficient through entropy decoding;

Step 211: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block;

Step 212: Transform the predicted image into the frequency domain;

Step 213: Quantize the frequency-domain coefficients of the predicted image to obtain a matrix of frequency-domain coefficients;

Step 214: dequantize the frequency domain coefficient and the frequency domain residual coefficient respectively;

Step 215: Compensate the dequantized frequency domain coefficient matrix to the dequantized frequency domain residual coefficient;

Step 216: According to the block size and mode, inversely transform the residual coefficients in the frequency domain to obtain a preliminary reconstructed image;

Step 217: Filter the reconstructed image to remove block effects to obtain an output image of the current block.

Described decoding process can be again:

Step 220: Obtain the prediction mode and frequency-domain residual coefficients through entropy decoding;

Step 221: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block;

Step 222: Transform the predicted image into the frequency domain;

Step 223: Quantize the frequency-domain coefficients of the predicted image to obtain a matrix of frequency-domain coefficients;

Step 224: Dequantize the frequency domain coefficient and the frequency domain residual coefficient respectively;

Step 225: Inversely transform the frequency domain coefficients and frequency domain residual coefficients according to the block size and mode respectively;

Step 226: Compensate the dequantized and inversely transformed frequency-domain coefficient matrix to the frequency-domain residual coefficients to obtain a preliminary reconstructed image;

Step 227: Filter the reconstructed image to remove block effects to obtain an output image of the current block.

During the decoding and scanning described above, the scan order table is consulted according to the selected mode order, and the residual coefficients are scanned in the order of the found positions.

A novel spatial prediction device for video encoding, which at least includes an encoding module and a decoding module; wherein,

The encoding module is at least provided with: a prediction module, a module for calculating residual coefficients, a scanning module and an entropy encoding module; the prediction module processes the input reconstructed image to obtain a predicted image, and the predicted image is processed by the module for calculating residual coefficients Compensate the current encoded image, and then, after the compensated code stream is processed by the scanning module, it is encoded and output by the entropy encoding module;

The decoding module is at least provided with: an entropy decoding module, a compensation module, an inverse quantization module, an inverse transformation module, and a transformation and quantization module; entropy decoding, inverse quantization, and inverse transformation are performed on the input code stream signal in turn; the transformation and quantization module performs prediction on the image Processing to obtain compensation information in the decoding process.

The encoding module and/or decoding module is also provided with a mode selection module that determines the scanning order based on the prediction mode, which is used to control the mode selection in the prediction module and the scanning module, and improve the encoding and/or decoding efficiency; the scanning module counts each In this mode, the probability that the coefficient of each frequency of the residual image is non-zero, and according to the value of the probability, a scanning sequence table is generated in descending order to replace a single zigzag scanning table.

The codes described are specifically:

The prediction module generates a prediction image from the decoded images of adjacent blocks of the current coding block according to the prediction mode selected by the mode selection module;

The module for calculating residual coefficients transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image, wherein the quantized coefficients are the same as the quantized coefficients used when processing the residual image; and the matrix of the quantized frequency domain coefficients satisfies the following formula:

Z=Q(Y)=(Y×Quant(Qp)+Qconst(Qp))>>Q_bit(Qp)

in,

Z is the quantized frequency domain coefficient matrix;

Y is the frequency domain coefficient matrix;

Qp is the quantization parameter

Quant(Qp), Qconst(Qp), Q_bit(Qp) are quantization functions defined by JVT;

The residual coefficient calculation module dequantizes the matrix of obtained frequency domain coefficients according to the following formula;

W=DQ(Z)=(Z×DQuant(Qp)+DQconst(Qp))>>Q_per(Qp)

DQuant(Qp)×DQconst(Qp)≈2 ^{Q_per(Qp)×Q_bit(Qp)}

in,

W is the frequency domain coefficient matrix after inverse quantization,

Z is the frequency domain coefficient matrix before inverse quantization after quantization,

Qp is the quantization parameter,

DQuant(Qp), DQconst(Qp), Q_bit(Qp), Q_per(Qp) are quantization functions defined by JVT;

The calculation residual coefficient module adopts the above method to transform the encoded current block into the frequency domain to obtain a frequency domain image; further subtract the frequency domain coefficient matrix to directly obtain the frequency domain residual image; then quantize the frequency domain residual image , to obtain the quantized frequency domain residual coefficient;

The scanning module performs coefficient scanning on the residual coefficients in the frequency domain; the entropy encoding module encodes the scanned information to obtain a code stream;

The compensation module compensates the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula;

C=C+Z,

in,

C is the frequency domain residual coefficient;

Z is the frequency domain coefficient matrix;

The inverse quantization module dequantizes the residual coefficients in the frequency domain according to the JVT formula; the inverse transformation module inversely transforms the residual coefficients in the frequency domain according to the block size and mode to obtain a preliminary reconstructed image; the filtering module removes blocks from the reconstructed image Effect filtering to get the output image of the current block.

The decoding is specifically:

The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block;

The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix;

The compensation module compensates the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula,

C=C+Z

in,

C is the frequency domain residual coefficient;

Z is the frequency domain coefficient matrix;

The inverse quantization module performs inverse quantization processing on the frequency domain residual coefficient;

The inverse transform module inversely transforms the residual coefficients in the frequency domain according to the size and mode of the block to obtain a preliminary reconstructed image; finally, the filtering module filters the reconstructed image to remove the block effect to obtain the output image of the current block.

Described decoding can specifically also be:

The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block;

The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix;

An inverse quantization module located after the entropy decoding module and the transform quantization module respectively, respectively dequantizes frequency domain coefficients and frequency domain residual coefficients;

The compensation module compensates the dequantized frequency domain coefficient matrix to the dequantized frequency domain residual coefficient;

The inverse transformation module inversely transforms the residual coefficients in the frequency domain according to the size and mode of the block to obtain a preliminary reconstructed image; finally, filters the reconstructed image to remove the block effect to obtain the output image of the current block.

The decoding can be specifically described as:

The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block;

The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix;

An inverse quantization module located after the entropy decoding module and the transform quantization module respectively, respectively dequantizes frequency domain coefficients and frequency domain residual coefficients;

The inverse transformation module located after the inverse quantization module inversely transforms the frequency domain coefficients and the frequency domain residual coefficients according to the block size and mode, respectively;

The compensation module compensates the dequantized and inversely transformed frequency-domain coefficient matrix to the frequency-domain residual coefficients to obtain a preliminary reconstructed image; finally, filters the reconstructed image to remove block effects to obtain the output image of the current block.

When decoding, the decoding module also consults the scan order table according to the selected mode order, and scans the residual coefficients in the order of the found positions.

By analyzing the above technical solutions, the present invention has the following advantages:

1. Through the above-mentioned new spatial prediction method for video coding, it overcomes the defect that the video coding method based on multi-directional spatial prediction produces flicker when it is applied to continuous I frame coding, and improves the video coding method of multi-directional spatial prediction The coding efficiency provides a flicker-resistant continuous I-frame coding scheme and a pattern-based residual coefficient scanning scheme for the video coding technology based on multi-directional spatial prediction, which further ensures the coding efficiency while reducing the flickering phenomenon.

2. The present invention takes the JVT standard as an implementation example, provides a solution to the problem of continuous I-frame coding flicker for the JVT standard, and provides specific technical means for improving the coding efficiency of the standard.

3. The device provided by the present invention provides a specific system structure for realizing the above method, and a hardware module and a combination scheme for realizing the system.

Description of drawings :

FIG. 1 is a schematic diagram of macroblock subdivision in JVT.

FIG. 2 is a schematic diagram of prediction of pixels in each 4×4 small block in the Intra4×4 mode.

FIG. 3 is a schematic diagram of a prediction direction of an intra prediction mode.

FIG. 4 is a schematic diagram of a fixed scanning sequence commonly used in current coding systems.

FIG. 5 is a schematic diagram of an encoding process in the prior art.

Fig. 6 is a schematic diagram of an encoding device in the prior art.

FIG. 7 is a signal flow diagram of the decoding part of the multi-directional spatial predictive coding system in the prior art.

Fig. 8 is a flow chart of the new scanning mode of the present invention.

FIG. 9 is a schematic diagram of encoding in an embodiment of a new scanning mode of the present invention.

FIG. 10 is a flowchart of the anti-flicker decoding terminal of the present invention.

FIG. 11 is a block diagram of a decoding portion of an embodiment of the present invention.

Fig. 12 is a block diagram of a decoding section of another embodiment of the present invention.

Fig. 13 is a block diagram of the decoding part of still another embodiment of the present invention.

Detailed ways

Below in conjunction with specific embodiment the present invention is described in further detail:

The present invention provides a novel spatial prediction method and device for video coding, the purpose of which is to effectively alleviate the flickering phenomenon generated during continuous I-frame coding in a video coding method based on multi-directional spatial prediction; and according to the prediction mode The order of scanning is determined, which is used to effectively improve the coding efficiency of the video coding method based on multi-directional spatial prediction.

In the JVT coding standard, the following steps are adopted in an embodiment of the present invention to realize anti-flicker processing:

See Figure 8, Figure 9,

Processing on the encoding side:

1. Generate a predicted image: according to the selected prediction mode, generate a predicted image from the decoded image of the adjacent block of the current coding block; this step is the same as the original step of JVT;

2. Transform the predicted image to the frequency domain, and the transformation method is the same as the transformation method used when processing the residual image in JVT; for example, for a 4×4 block, if the input is X, the output Y is:

$\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 2</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 02</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 03</span>}}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 11</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 12</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 13</span>}}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 20</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; one</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; two</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; twenty\; three</span>}}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 30</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 31</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 32</span>}}& {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 33</span>}}\end{array}\right]\left[\begin{array}{cccc}\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 2</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 2</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

Among them, Y is the frequency domain coefficient of the predicted image, and X is the predicted image;

3. Quantize the frequency domain coefficient Y of the predicted image; the quantization coefficient Qp is the same as the Qp used when processing the residual image, and the matrix of the quantized frequency domain coefficient is set to Z; then the quantization formula is:

Z=Q(Y)=(Y×Quant(Qp)+Qconst(Qp))>>Q_bit(Qp)

4. Dequantize the matrix Z of the frequency domain coefficients to obtain W; the dequantization here is different from the dequantization in JVT: the dequantization in JVT is not the same as the quantization scale,

In JVT, its dequantization formula is:

W=DQ(Z)=(Z×DQuant(Qp)+DQconst(Qp))>>Q_per(Qp)

In order to restore the quantized coefficient to the scale before quantization, the inverse quantization formula must be redesigned: the inverse quantization formula of the present invention is:

W＝DQ’(Z)＝(Z×DQuant’(Qp)+DQcons t’(Qp))＞＞Q_per’(Qp)

And, the new formula must satisfy:

DQuant'(Qp)×Quant(Qp) is approximately equal to 2 ^{Q_per'(Qp)×Q_bit(Qp)}

5. Transform the current block I to be encoded into the frequency domain to obtain a frequency domain image F; the method is the same as above;

6. Subtract W from the frequency domain image F to directly obtain the frequency domain residual image S;

7. Quantize the frequency domain residual image S to obtain the quantized frequency domain residual coefficient C, the formula is the same as above;

8. Perform coefficient scanning on the residual coefficient C in the frequency domain, and entropy coding to obtain the code stream;

9. Compensate Z to the frequency domain residual coefficient C; that is, C=C+Z;

10. Dequantize the residual coefficient C in the frequency domain, using the formula of JVT;

11. Inversely transform the residual coefficient C in the frequency domain to obtain a preliminary reconstructed image B, and use the original inverse transformation method of JVT according to the block size and mode;

12. Perform filtering to remove the block effect on the reconstructed image B to obtain an output image 0 of the current block.

See Figure 10, Figure 11

One of the processing on the decoding end:

1. Entropy decoding to obtain the prediction mode and frequency domain residual coefficient C;

2. Generating a predicted image: According to the prediction mode obtained by entropy decoding, a predicted image is generated from the decoded images of adjacent blocks of the currently decoded block. This step is the same as the original step of JVT;

3. Transform the predicted image to the frequency domain, the method of transformation is the same as the transformation method adopted when processing the residual image in JVT, and this step is the same as the second step during encoding;

4. Quantize the frequency domain coefficient Y of the predicted image to obtain the matrix Z of the frequency domain coefficients. This step is the same as the third step during encoding;

5. Compensate the matrix Z of frequency domain coefficients to the frequency domain residual coefficient C; that is, C=C+Z; the same as step 9 of the encoding process;

6. Quantize the residual coefficient C in the frequency domain, using the formula of JVT; the same as step 10 of the encoding process;

7. Inversely transform the residual coefficient C in the frequency domain to obtain a preliminary reconstructed image B; use the original inverse transformation method of JVT according to the block size and mode; it is the same as the eleventh step of the encoding process;

8. Perform filtering to remove the block effect on the reconstructed image B to obtain the output image 0 of the current block; it is the same as the twelfth step of the encoding process;

Referring to Fig. 12 and Fig. 13, the second and third processing of the decoding end of the present invention are basically the same as one of the above-mentioned processing of the decoding end, the difference is that one of the processing of the decoding end is directly compensated after quantization, and the second processing of the decoding end is Compensation after inverse quantization is done, and the third processing at the decoding end is compensation after inverse transformation.

The above schemes 2 and 3 are similar to scheme 1, except that the position of compensation is different. According to the difference of compensation position, inverse quantization and inverse transformation should be performed on the quantized predicted image.

The method for determining the scanning order based on the prediction mode of the present invention can effectively improve the coding efficiency of the video coding method based on multi-directional spatial prediction. In the JVT coding standard, the following steps are used to realize the scanning module based on the prediction mode:

In the design stage: first, the probability of non-zero coefficients of each frequency of the residual image in each mode is counted separately; then, the scan order table is generated in order of probability from large to small (for example: with variable matrix T(m, i ) means; that is, the position of the coefficient of the i-th scan in the m-th mode is T(m, i)). Instead of a single zigzag scan table Z(i).

In the codec stage:

When scanning, according to the selected mode m, look up the scanning sequence table T in the increasing order of i, and scan the residual coefficients in the order of the found positions.

Referring to Fig. 9, the coding module in the device of the present invention is based on the device of the prior art, and a new scanning module is also provided between the calculation residual coefficient module and the entropy coding module, and the new scanning module is based on the mode selection module control, select a predetermined scan sequence, thereby improving the efficiency of processing.

Referring to Figures 11, 12, and 13, the decoding modules in the device of the present invention are all equipped with a transform and quantization module, and include an entropy decoding module, an inverse quantization module, an inverse transformation module, and a prediction compensation module. The difference is that in different In an embodiment, the position of the prediction compensation module may be respectively set after the entropy decoding module, or after the inverse quantization module, or after the inverse transformation module. Taking Fig. 11 as an example, the specific decoding process includes: after performing entropy decoding on the code stream. The predicted image processed by the transform and quantization module and the entropy-decoded code stream information are compensated in the prediction compensation module, and then processed by the inverse quantization module and the inverse transformation module in turn before being output. The difference from FIG. 11 is that the prediction and compensation positions shown in FIGS. 12 and 13 are respectively located after the inverse quantization module or the inverse transformation module.

Finally, it should be noted that: the above embodiments are only used to illustrate the present invention without limitation, although the present invention has been described in detail with reference to the preferred embodiments disclosed above, those of ordinary skill in the art should understand that: the present invention can still be modified Or an equivalent replacement; all technical solutions that do not depart from the spirit and scope of the present invention shall be covered by the scope of the claims of the present invention.

Claims (15)

1. A novel spatial prediction method for video coding, characterized in that: when encoding the code stream, the predicted image is also transformed into the frequency domain, and quantized with the same quantization coefficient, and then used as the predicted image; The predicted image is quantized in the frequency domain by the same processing method as that used for encoding, and then compensated to the decoded residual image. 2. The novel spatial prediction method for video coding according to claim 1, characterized in that: the coding process is specifically: Step 100: According to the selected prediction mode, a predicted image is generated from the decoded images of adjacent blocks of the current coding block; Step 101: Transform the predicted image into the frequency domain; Step 102: Quantize the frequency domain coefficients of the predicted image, wherein the quantization coefficients are the same as the quantization coefficients used when processing the residual image; and the matrix of the quantized frequency domain coefficients satisfies the following formula: Z＝Q(Y)＝(Y×Quant(Qp)+Qconst(Qp))＞＞Q_bit(Qp) in, Z is the quantized frequency domain coefficient matrix, Y is the frequency domain coefficient matrix, Qp is the quantization parameter, Quant(Qp), Qconst(Qp), Q_bit(Qp) are quantization functions defined by JVT; Step 103: Dequantize the matrix of frequency domain coefficients obtained in step 102 according to the following formula; W＝DQ(Z)＝(Z×DQuant(Qp)+DQconst(Qp))＞＞Q_per(Qp) DQuant(Qp)×DQconst(Qp)≈2 ^{Q_per(Qp)×Q_bit(Qp)} in, W is the frequency domain coefficient matrix after inverse quantization, Z is the frequency domain coefficient matrix before inverse quantization after quantization, Qp is the quantization parameter, DQuant(Qp), DQconst(Qp), Q_bit(Qp), Q_per(Qp) are quantization functions defined by JVT; Step 104: according to the method of steps 100-103, transform the encoded current block into the frequency domain to obtain a frequency domain image; Step 105: subtracting the dequantized frequency domain coefficient matrix from the frequency domain image to directly obtain the frequency domain residual image; Step 106: Quantize the frequency domain residual image to obtain the quantized frequency domain residual coefficient, the formula is the same as step 102; Step 107: Perform coefficient scanning on the residual coefficients in the frequency domain, and entropy coding to obtain a code stream; Step 108: Compensate the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula; C=C+Z; in, C is the frequency domain residual coefficient, Z is the frequency domain coefficient matrix; Step 109: dequantize the residual coefficient in the frequency domain with the formula of JVT; Step 110: Inversely transform the residual coefficients in the frequency domain according to the size and mode of the block to obtain a preliminary reconstructed image; Step 111: Filter the reconstructed image to remove block effects to obtain an output image of the current block. 3. The new spatial prediction method for video coding according to claim 1 or 2, characterized in that: before the coding, it further includes the processing of determining the scanning order based on the prediction mode, and the specific process is: respectively counting The probability that the coefficient of each frequency of the residual image in each mode is non-zero, according to the value of the probability, the scanning sequence table is generated in descending order. 4. The novel spatial prediction method for video coding according to claim 2, characterized in that: during coding and scanning, the scanning order table is consulted according to the selected mode order, and the residual is calculated according to the order of the found positions Coefficients are scanned. 5. The novel spatial prediction method for video coding according to claim 1, characterized in that: the decoding process is: Step 200: Obtain the prediction mode and frequency domain residual coefficients through entropy decoding; Step 201: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block; Step 202: Transform the predicted image into the frequency domain; Step 203: Quantize the frequency-domain coefficients of the predicted image to obtain a frequency-domain coefficient matrix; Step 204: Compensate the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula, C=C+Z in, C is the frequency domain residual coefficient, Z is the frequency domain coefficient matrix; Step 205: dequantize the residual coefficient in the frequency domain; Step 206: According to the block size and mode, inversely transform the residual coefficients in the frequency domain to obtain a preliminary reconstructed image; Step 207: Filter the reconstructed image to remove block effects to obtain an output image of the current block. 6. The novel spatial prediction method for video coding according to claim 1, characterized in that: the decoding process is: Step 210: Obtain the prediction mode and frequency domain residual coefficient through entropy decoding; Step 211: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block; Step 212: Transform the predicted image into the frequency domain; Step 213: Quantize the frequency-domain coefficients of the predicted image to obtain a matrix of frequency-domain coefficients; Step 214: dequantize the frequency domain coefficient and the frequency domain residual coefficient respectively; Step 215: Compensate the dequantized frequency domain coefficient matrix to the dequantized frequency domain residual coefficient; Step 216: According to the block size and mode, inversely transform the residual coefficients in the frequency domain to obtain a preliminary reconstructed image; Step 217: Filter the reconstructed image to remove block effects to obtain an output image of the current block. 7. The novel spatial prediction method for video coding according to claim 1, characterized in that: the decoding process is: Step 220: Obtain the prediction mode and frequency-domain residual coefficients through entropy decoding; Step 221: According to the prediction mode obtained by entropy decoding; generate a prediction image from the decoded image of the adjacent block of the currently decoded block; Step 222: Transform the predicted image into the frequency domain; Step 223: Quantize the frequency-domain coefficients of the predicted image to obtain a matrix of frequency-domain coefficients; Step 234: dequantize the frequency domain coefficient and the frequency domain residual coefficient respectively; Step 225: Inversely transform the frequency domain coefficients and frequency domain residual coefficients according to the block size and mode respectively; Step 226: Compensate the dequantized and inversely transformed frequency-domain coefficient matrix to the frequency-domain residual coefficients to obtain a preliminary reconstructed image; Step 227: Filter the reconstructed image to remove block effects to obtain an output image of the current block. 8. The novel spatial prediction method for video coding according to claim 5, 6 or 7, characterized in that: when decoding and scanning, the scanning order table is consulted according to the selected mode sequence, and the searched position is The residual coefficients are scanned sequentially. 9. A novel spatial prediction device for video coding, characterized in that it includes at least an encoding module and a decoding module; wherein, The encoding module is at least provided with: a prediction module, a module for calculating residual coefficients, a scanning module and an entropy encoding module; the prediction module processes the input reconstructed image to obtain a predicted image, and the predicted image is processed by the module for calculating residual coefficients Compensate the current encoded image, and then, after the compensated code stream is processed by the scanning module, it is encoded and output by the entropy encoding module; The decoding module is at least provided with: an entropy decoding module, a compensation module, an inverse quantization module, an inverse transformation module, and a transformation and quantization module; entropy decoding, inverse quantization, and inverse transformation are performed on the input code stream signal in turn; the transformation and quantization module performs prediction on the image Processing to obtain compensation information in the decoding process. 10. The novel spatial prediction device for video coding according to claim 9, characterized in that: the coding module and/or the decoding module is also provided with a mode selection module that determines the scanning order based on the prediction mode, and is used to control the prediction The mode selection in the module and the scanning module improves the efficiency of encoding and/or decoding; the scanning module separately counts the probability that the coefficient of each frequency of the residual image in each mode is non-zero, and according to the value of the probability, from large to small The order to generate the scan order table. 11. The novel spatial prediction device for video coding according to claim 9 or 10, characterized in that: the coding is specifically: The prediction module generates a prediction image from the decoded images of adjacent blocks of the current coding block according to the prediction mode selected by the mode selection module; The module for calculating residual coefficients transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image, wherein the quantized coefficients are the same as the quantized coefficients used when processing the residual image; and the matrix of the quantized frequency domain coefficients satisfies the following formula: Z=Q(Y)=(Y×Quant(Qp)+Qconst(Qp))>>Q_bit(Qp) in, Z is the quantized frequency domain coefficient matrix; Y is the frequency domain coefficient matrix; Qp is the quantization parameter Quant(Qp), Qconst(Qp), Q_bit(Qp) are quantization functions defined by JVT; The residual coefficient calculation module dequantizes the matrix of obtained frequency domain coefficients according to the following formula; W=DQ(Z)=(Z×DQuant(Qp)+DQconst(Qp))>>Q_per(Qp) DQuant(Qp)×DQconst(Qp)≈2 ^{Q_per(Qp)×Q_bit(Qp)} in, W is the frequency domain coefficient matrix after inverse quantization, Z is the frequency domain coefficient matrix before inverse quantization after quantization, Qp is the quantization parameter, DQuant(Qp), DQconst(Qp), Q_bit(Qp), Q_per(Qp) are quantization functions defined by JVT; The calculation residual coefficient module adopts the above method to transform the encoded current block to the frequency domain to obtain a frequency domain image; and further subtracts the frequency domain coefficient matrix to directly obtain the frequency domain residual image; then quantizes the frequency domain residual image , to obtain the quantized frequency domain residual coefficient; The scanning module performs coefficient scanning on the residual coefficients in the frequency domain; the entropy encoding module encodes the scanned information to obtain a code stream; The compensation module compensates the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula; C=C+Z, in, C is the frequency domain residual coefficient; Z is the frequency domain coefficient matrix; The inverse quantization module dequantizes the residual coefficients in the frequency domain according to the JVT formula; the inverse transformation module inversely transforms the residual coefficients in the frequency domain according to the block size and mode to obtain a preliminary reconstructed image; the filtering module removes blocks from the reconstructed image Effect filtering to get the output image of the current block. 12. The novel spatial prediction device for video coding according to claim 9, characterized in that: the decoding is specifically: The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block; The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix; The compensation module compensates the frequency domain coefficient matrix to the frequency domain residual coefficient according to the following formula, C=C+Z in, C is the frequency domain residual coefficient; Z is the frequency domain coefficient matrix; The inverse quantization module performs inverse quantization processing on the frequency domain residual coefficient; The inverse transformation module inversely transforms the frequency domain residual coefficients according to the size and mode of the block to obtain a preliminary reconstructed image; finally, the filtering module performs filtering to remove the block effect on the reconstructed image to obtain the output image of the current block. 13. The novel spatial prediction device for video coding according to claim 9, characterized in that: the decoding is specifically: The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block; The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix; An inverse quantization module located after the entropy decoding module and the transform quantization module respectively, respectively dequantizes frequency domain coefficients and frequency domain residual coefficients; The compensation module compensates the dequantized frequency domain coefficient matrix to the dequantized frequency domain residual coefficient, The inverse transform module inversely transforms the residual coefficients in the frequency domain according to the size and mode of the block to obtain a preliminary reconstructed image; finally, the reconstructed image is filtered to remove the block effect to obtain the output image of the current block. 14. The novel spatial prediction device for video coding according to claim 9, characterized in that: said decoding is specifically: The entropy decoding module decodes the input code stream to obtain the prediction mode and the residual coefficient in the frequency domain; then according to the prediction mode obtained by entropy decoding; the prediction image is generated from the decoded image of the adjacent block of the current decoding block; The transformation and quantization module transforms the predicted image into the frequency domain, and quantizes the frequency domain coefficients of the predicted image to obtain a frequency domain coefficient matrix; An inverse quantization module located after the entropy decoding module and the transform quantization module respectively, respectively dequantizes frequency domain coefficients and frequency domain residual coefficients; The inverse transformation module located after the inverse quantization module inversely transforms the frequency domain coefficients and the frequency domain residual coefficients according to the block size and mode, respectively; The compensation module compensates the dequantized and inversely transformed frequency-domain coefficient matrix to the frequency-domain residual coefficients to obtain a preliminary reconstructed image; finally, filters the reconstructed image to remove block effects to obtain the output image of the current block. 15. The new spatial prediction device for video coding according to claim 12, 13 or 14, characterized in that: when decoding, the decoding module also consults the scanning sequence table according to the selected mode sequence, and according to the found The order of positions scans the residual coefficients.

Images