视频编解码处理方法 技术领域 本发明涉及多媒体通信领域, 尤其涉及一种视频编解码处理方法。 TECHNICAL FIELD The present invention relates to the field of multimedia communications, and in particular, to a video codec processing method.
背景技术 Background technique
在多媒体通信系统中, 视频图像质量对多媒体通信性能起到了非常关 键的作用。 但是往往却由于传输信道质量的问题, 导致视频图像的编码码 流在传输过程中, 会出现丢包现象和误码现象, 尤其是无线传输信道的误 码率和丢包率都是较高的, 且带宽变化幅度较大, 这样就会使视频图象接 收方不能完全正确的接收到视频图象的编码码流, 而如果要获得较好的视 频通信效果,就需要对接收到的错误码流(包括误码码流和被丢弃码流等) 进行抗误码处理, 以使从视觉上能够正常观看到解码恢复出来的视频图 象。 In multimedia communication systems, video image quality plays a very important role in multimedia communication performance. However, often due to the problem of the quality of the transmission channel, the encoded image stream of the video image will cause packet loss and error during the transmission process, especially the wireless transmission channel has a high bit error rate and packet loss rate. And the bandwidth changes greatly, so that the video image receiver cannot completely receive the encoded video stream of the video image, and if you want to obtain a good video communication effect, you need to correct the received error code Streams (including errored code streams and discarded code streams, etc.) are subjected to error-resistant processing, so that the video images recovered from decoding can be viewed normally.
目前, 在运动图像专家组 MPEG (Motion picture experts group )标 准 MPEG- 4、低比特率视频通信编码 H.263标准和先进视频压缩标准 H.264 中都相应提出了对视频传输码流进行抗误码处理的方式, 如: At present, the motion picture experts group MPEG (Motion picture experts group) standard MPEG-4, the low-bit-rate video communication coding H.263 standard and the advanced video compression standard H.264 have all proposed corresponding anti-error for video transmission bitstreams. Code processing methods, such as:
重同步 ( Resynchronization )处理方式 ^ Resynchronization processing method ^
可逆变长编码(RVLC, Reversible Variable Length Code ) 处理方 式; Reversible Variable Length Code (RVLC) processing method;
数据分割 (Data Partition)处理方式; Data partition (Data Partition) processing method;
自适应条带排序 ( ASO, Adaptive SLICE Ordering)处理方式; Adaptive Striping (ASO, Adaptive SLICE Ordering) processing method;
灵活宏块排序 ( FMO, Flexible Macroblock Ordering )处理方式; 及 Flexible Macroblock Ordering (FMO) processing; and
冗余条带 (Redundant SLICEs )处理方式等; Redundant SLICEs processing method, etc .;
上述视频传输码流的抗误码处理方式都能够在一定程度上对由于传 输信道而引起的错误码流进行相应的补偿, 以使视频图像接收方能够较好 的进行解码恢复视频图像处理, 得到较好的视频通信效果。 The above-mentioned anti-error processing methods of the video transmission code stream can compensate the erroneous code stream caused by the transmission channel to a certain extent, so that the video image receiver can better decode and recover the video image processing to obtain Better video communication.
但是由于在视频图像编解码过程中,使用了视频传输码流的抗误码处 理方式,可能会适当降低编解码效率,但是在传输信道质量恶劣的情况下,
传输码流的抗误码处理对达到较好的视频通信效果是非常重要的, 所以往 往愿意适当降低编解码效率以能换取较好的抗误码和抗丟包性能, 以达到 较好的错误补偿目的, 获得较好的视频通信效果。 However, during the video image encoding and decoding process, the error-resistant processing method of the video transmission code stream is used, which may appropriately reduce the encoding and decoding efficiency. However, when the quality of the transmission channel is poor, The anti-error processing of the transmission stream is very important to achieve a good video communication effect. Therefore, it is often willing to appropriately reduce the encoding and decoding efficiency in order to obtain better anti-error and packet loss performance in order to achieve better errors. Compensation purpose to obtain better video communication effect.
其中在先进视频压缩标准 H. 264中, 其视频图像压缩标准是基于性能 逼近于离散数字余弦变换 ( DCT, Digi tal cos ine transform ) 的整数变 换方法, 且压缩针对的单位对象是视频图像中被划分开的每个 16 x 16 象 素大小的宏块(MB, Macroblock )。 参照图 1, 该图是现有技术中 H. 264 所采用的压缩处理和解压缩处理过程示意图, 其对视频图像进行压缩和解 压缩的处理过程如下: Among them, in the advanced video compression standard H.264, the video image compression standard is an integer transform method based on performance approximating a discrete digital cosine transform (DCT, Digit Cosine Transform), and the unit object targeted for compression is the video image. Each 16 x 16 pixel-sized macroblock (MB, Macroblock). Referring to FIG. 1, this figure is a schematic diagram of a compression process and a decompression process used by H. 264 in the prior art. A process of compressing and decompressing a video image is as follows:
视频图像发送方的图像压缩过程 Image compression process of video image sender
一对在视频图像中划分出的每个 16 x 16 象素大小的宏块进行运动估 计和运动补偿处理; Perform motion estimation and motion compensation on a pair of macroblocks each 16 x 16 pixels divided in the video image;
一再对每个宏块中的 4X4子块进行 DCT变换; DCT transform the 4X4 sub-blocks in each macro block repeatedly;
一对 DCT变换后的变换系数进行量化; Quantize a pair of DCT transformed transform coefficients;
一再对量化后的结果进行熵编码后, 组装成数据包发送至传输信道; 视频图像接收方的图像解压缩过程 After entropy encoding the quantized result repeatedly, it is assembled into a data packet and sent to the transmission channel; the image decompression process of the video image receiver
一对接收编码码流进行熵解码; Entropy decoding of a pair of received encoded code streams;
一对解码后的结果进行逆量化处理; Perform inverse quantization on a pair of decoded results;
—再对逆量化后的结果进行逆向离散数字余弦变换(IDCT, Inverse digi tal cos ine transform ); —The inverse discrete digital cosine transform (IDCT) is then performed on the inverse quantized result;
一对 IDCT变换后的结果进行运动补偿后恢复解码后的视频图像。 其中将一个视频图像划分为多个宏块 MB的过程包括: 先对整个视频 图像进行变换宏块划分处理, 以得到多个 16 x 16 象素的宏块, 再将宏块 进行分类处理, 而将每类宏块所组成的宏块组定义为一个视频片段 ( SLICE )。 其中对每一视频图像进行宏块划分处理得到的每个宏块 MB的 编码结构如图 2所示, 对于 H. 264而言, 每个宏块 MB内包括 16个 4X4象 素大小的亮度块 Y (编码 0至编码为 15的块区域), 4个 4X4象素大小的 色度块 Cb (编码 16至编码为 19的块区域)和 4个 4X4象素大小的色度块 Cr (编码 20至编码为 23的块区域)。
为了得到较好的视频通信效果, H. 264标准采用了 POLYCOM公司提出 的灵活宏块排序 (FM0, Flexible Macroblock Order ing )视频编码处理 方式。 参照图 3, 该图是现有技术中 FM0视频编码处理方式的处理原理示 意图; 在该图中, 假设标识为 "0" 的宏块 MB属于视频片段 SLICEO, 标识 为 "1"的宏块 MB属于视频片段 SLICE1 (当然除该图所示的以这两种宏块 排序方案来划分不同 SLICE的情况外, 还可以包括 居任意的宏块顺序来 划分不同 SLICE的各种情形)。 如该图所示, 在视频图像发送方首先对标 识为 "0" 的宏块 MB进行压缩编码处理, 然后封装成码流数据包发送; 再 对标识为 "1" 的宏块 MB进行压缩编码处理, 再封装成码流数据包发送。 在视频图像接收方, 首先对标识为 "0" 的码流数据包进行解码处理, 再 对标识为 "1" 的码流数据包进行解码处理; 而如果由于传输信道质量问 题而引起标识为 "0" 的码流数据包(即视频片段 SLICEO )在传输信道上 被丢掉, 则可以根据已解码恢复的相邻标识为 "1" 的宏块图象信息在相 应的标识为 "0" 的宏块中插入逼近图象信息, 从而达到对错误进行掩盖 的目的。 A pair of IDCT transformed results are motion compensated and the decoded video image is restored. The process of dividing a video image into multiple macroblocks MB includes: first performing transform macroblock division processing on the entire video image to obtain multiple macroblocks of 16 x 16 pixels, and then classifying the macroblocks, and A macroblock group composed of each type of macroblock is defined as a video clip (SLICE). The coding structure of each macroblock MB obtained by performing macroblock division processing on each video image is shown in Figure 2. For H.264, each macroblock MB includes 16 4 × 4 pixel-sized luminance blocks. Y (coded 0 to 15 block areas), 4 4X4 pixel-sized chroma blocks Cb (coded 16 to 19 block areas) and 4 4X4 pixel-sized chroma blocks Cr (coded 20 To block area coded as 23). In order to obtain better video communication effects, the H. 264 standard uses the Flexible Macroblock Ordering (FM0) video coding processing method proposed by POLYCOM. Referring to FIG. 3, which is a schematic diagram of the processing principle of the FM0 video encoding processing method in the prior art. In this figure, it is assumed that a macroblock MB identified by “0” belongs to the video segment SLICEO, and a macroblock MB identified by “1” It belongs to the video segment SLICE1 (of course, in addition to the case where the two kinds of macroblock ordering schemes are used to divide different SLICEs as shown in the figure, it can also include various cases where the arbitrary macroblock order is used to divide different SLICEs). As shown in the figure, the video image sender first performs compression encoding processing on the macroblock MB identified as "0", and then encapsulates it into a stream data packet for transmission; and then performs compression encoding on the macroblock MB identified as "1". Process, and then encapsulate and send it as a stream data packet. On the video image receiver side, first decode the stream data packet identified as "0", and then decode the stream data packet identified as "1"; if the transmission channel quality problem causes the flag to be " 0 "bitstream data packets (that is, the video segment SLICEO) are dropped on the transmission channel. According to the decoded and restored macroblock image information of the adjacent identifier" 1 "in the corresponding macro identified by" 0 " The approximate image information is inserted in the block, so as to achieve the purpose of masking errors.
由于 FM0视频编码处理技术是以宏块 MB为基本处理单位进行编解码■ 处理的, 尽管对由于传输信道质量而引起的误码码流具有较好的掩盖效 果, 但在视频图像恢复过程中, 其根据其他相邻宏块 MB的恢复图像信息 在产生了误码的宏块中插入的逼近图像在视觉角度上还是有较大的失真, 从而使视频通信不能达到预想效果。 Since the FM0 video encoding processing technology uses the macroblock MB as the basic processing unit for encoding and decoding, although it has a good concealment effect on the bit error code stream caused by the quality of the transmission channel, during the video image restoration process, According to the restored image information of other adjacent macroblocks MB, the approximated image inserted in the macroblock that caused the bit error still has a large distortion in the visual angle, so that the video communication cannot achieve the expected effect.
发明内容 Summary of the invention
本发明要解决的技术问题是提出一种视频编解码处理方法, 以更为准 确的对由于传输信道质量而引起的误码码流进行抗误码处理, 减小视频图 像在视觉上的失真, 提高视频通信的效果。 The technical problem to be solved by the present invention is to propose a video codec processing method to more accurately perform error-resistant processing on the bit error stream caused by the quality of the transmission channel, and reduce the visual distortion of the video image. Improve the effectiveness of video communications.
为解决上述问题,本发明提出了一种视频编解码处理方法, 包括步骤: To solve the above problems, the present invention provides a video codec processing method, which includes steps:
B、 分别对每个宏块组变换处理后的所有子块进行分类处理; B. Perform classification processing on all the sub-blocks after each macroblock group transformation process;
C、 发送方对每个宏块组头信息、 宏块头信息和每类子块组的变换系 数分别进行编码处理后发送;
D、 接收方分别对正确接收的码流进行解码处理, 以恢复出相应子块 的图像信息; 对于每个产生了误码的子块, 接收方则根据相应相邻子块的 图像信息进行插入逼近图像信息处理。 C. The sender encodes each macroblock group header information, the macroblock header information, and the transform coefficients of each type of subblock group and sends them separately; D. The receiver separately decodes the correctly received bitstream to recover the image information of the corresponding sub-block. For each sub-block that generates an error, the receiver inserts the image based on the image information of the corresponding neighboring sub-block. Approaching image information processing.
其中所述步骤 B之后还包括步骤: 类子块组分別归属于独立的宏块组, 以将原相应每个宏块组分裂为多个独 立的宏块组。 After the step B, the method further includes the following steps: Class sub-block groups belong to independent macro block groups, respectively, to split each corresponding macro block group into multiple independent macro block groups.
其中所述步骤 B之后还包括步骤: 类子块组分别归属于相应宏块组的子宏块组, 以使原相应每个宏块组包含 多个子宏块组。 After the step B, the method further includes the following steps: The sub-block groups of the class belong to the sub-macro block groups of the corresponding macro-block group respectively, so that each corresponding macro-block group originally includes multiple sub-macro block groups.
其中在步骤 C及 D中, 所述发送方和接收方还对属于同一类型的相邻 子块进行去方块处理。 In steps C and D, the sender and receiver also perform deblocking processing on adjacent sub-blocks belonging to the same type.
其中在步骤 C及 D中, 所述发送方和接收方还对属于同一类型的手块 中的图像信息进行帧内预测。 In steps C and D, the sender and receiver also perform intra prediction on the image information in the hand block belonging to the same type.
其中所述步骤 B中分别对每个宏块组变换处理后的所有子块进行分类 处理具体为: In the step B, classifying all the sub-blocks after each macroblock group transformation process is specifically performed as follows:
将每个宏块组变换处理后的所有子块按照子块变换系数划分为两种 类型的子块组。 All sub-blocks after each macroblock group transformation process are divided into two types of sub-block groups according to the sub-block transformation coefficients.
其中所述每个子块大小为 4 x 4 象素。 则每个宏块中两种类型的子块 组的排列模式包括: Wherein each sub-block is 4 x 4 pixels in size. The arrangement modes of the two types of sub-block groups in each macro block include:
分别属于不同类型的两个子块呈对角状态分布; Two sub-blocks belonging to different types are distributed diagonally;
分别属于不同类型的子块呈隔行状态分布; 或 Sub-blocks belonging to different types are distributed in an interlaced state; or
呈隔列状态分布。 Distributed in a state of columns.
其中所述每个子块大小为 8 x 8 象素。 则每个宏块中两种类型的子块 组的排列模式包括: Wherein each sub-block is 8 x 8 pixels in size. The arrangement modes of the two types of sub-block groups in each macro block include:
分别属于不同类型的两个子块呈对角状态分布; Two sub-blocks belonging to different types are distributed diagonally;
分别属于不同类型的子块呈行状态分布; 或 The sub-blocks belonging to different types are distributed in a row state; or
呈列状态分布。
本发明能够达到的有益效果如下: Stated distribution. The beneficial effects achieved by the present invention are as follows:
由于本发明视频编解码处理方法不再将宏块 MB 中的所有子块编码信 息归属于同一个宏块组 SLICE, 而是将 MB中的子块分类,使得各类子块编 码信息分别属于不同的宏块组 SLICE, 或者属于同一宏块组 SLICE的子宏 块组 SLICE, 这样在视频图像解码恢复过程中, 对于产生了误码的子块就 可以参照相邻的其他子块的图像信息, 插入相应的逼近图像信息, 从而实 现了插入的逼近信息精确度较高、 对错误掩盖效果较好的目的, 从而在视 觉效果上减小了由于传输信道质量而引起的错误码流所带来的图像失真, 提高了视频通信的效果。 Since the video codec processing method of the present invention no longer assigns all subblock coding information in the macroblock MB to the same macroblock group SLICE, but classifies the subblocks in the MB, so that the various types of subblock coding information belong to different Macroblock group SLICE, or a submacroblock group SLICE belonging to the same macroblock group SLICE, so that during the video image decoding and restoration process, for the subblocks that have generated errors, the image information of other neighboring subblocks can be referred to, Insert the corresponding approximation image information, thereby achieving the purpose of higher accuracy of the inserted approximation information and better error concealment effect, thereby reducing the visual effect of the error code stream caused by the quality of the transmission channel. The image is distorted, which improves the effect of video communication.
附图说明 BRIEF DESCRIPTION OF THE DRAWINGS
图 1 是现有技术中 H. 264 所采用的压缩处理和解压缩处理过程示意 图; FIG. 1 is a schematic diagram of a compression process and a decompression process used by H. 264 in the prior art;
图 2为现有技术中每个宏块 MB的编码结构示意图; 2 is a schematic diagram of a coding structure of each macroblock MB in the prior art;
图 3是现有技术中 FMO视频编码处理方式的处理原理示意图; 图 4是本发明视频编解码处理方法的实现过程流程图; 3 is a schematic diagram of a processing principle of an FMO video encoding processing method in the prior art; FIG. 4 is a flowchart of an implementation process of a video encoding and decoding processing method of the present invention;
图 5是本发明视频编解码处理方法中, 将每个 MB划分为 4 x 4象素 大小的两类子块的第一排列模式图; FIG. 5 is a first arrangement pattern diagram of dividing each MB into two types of sub-blocks of a size of 4 x 4 pixels in the video encoding and decoding processing method of the present invention;
图 6是本发明视频编解码处理方法中, 将每个 MB划分为 4 x 4象素 大小的两类子块的第二排列模式图; 6 is a second arrangement pattern diagram of dividing each MB into two types of sub-blocks of a size of 4 x 4 pixels in the video encoding and decoding processing method of the present invention;
图 Ί是本发明视频编解码处理方法中, 将每个 MB划分为 4 x 4象素 大小的两类子块的第三排列模式图; FIG. Ί is a third arrangement pattern diagram of dividing each MB into two types of sub-blocks of 4 × 4 pixels in the video encoding and decoding processing method of the present invention;
图 8是本发明视频编解码处理方法中, 将每个 MB划分为 8 x 8象素 大小的两类子块的第一排列模式图; FIG. 8 is a first arrangement pattern diagram of dividing each MB into two types of sub-blocks of 8 × 8 pixels size in the video encoding and decoding processing method of the present invention; FIG.
图 9是本发明视频编解码处理方法中, 将每个 MB划分为 8 x 8象素 大小的两类子块的第二排列模式图; FIG. 9 is a second arrangement pattern diagram of dividing each MB into two types of sub-blocks of 8 × 8 pixels in the video encoding and decoding processing method of the present invention;
图 10是本发明视频编解码处理方法中, 将每个 MB划分为 8 x 8象素 大小的两类子块的第三排列模式图; FIG. 10 is a third arrangement pattern diagram of dividing each MB into two types of sub-blocks of 8 × 8 pixels in the video encoding and decoding processing method of the present invention; FIG.
图 11是一个宏块行为一个 SLICE时, 其编码预测的原理示意图; 图 12是两个宏块行为一个 SLICE时, 其编码预测的原理示意图。
具体实施方式 本发明视频编解码处理方法的基本设计思想是: 将宏块 MB中的子块 分类, 使得各类子块编码信息分别属于不同的宏块组 SLICE, 或者属于同 一宏块组 SLICE的子宏块组 SLICE,从而在图像恢复还原过程中,对由于 传输信道质量问题所引起的误码子块组, 可以根据每个出错子块周围的其 他相邻子块的图像信息, 对相应每个出错子块进行插入逼近图像信息的处 理, 从而使插入的逼近图像信息精确度更高, 错误掩盖效果更好, 以在视 觉角度上减 d、视频图像的失真度。相应于 POLYCOM公司提出的灵活宏块 排序 (FMO, Flexible Macroblock Ordering )视频编码处理方式, 这里将 FIG. 11 is a schematic diagram of the coding prediction principle when a macro block acts as a SLICE; FIG. 12 is a schematic diagram of the coding prediction principle when two macro blocks act as a SLICE. DETAILED DESCRIPTION The basic design idea of the video encoding and decoding processing method of the present invention is: classify the sub-blocks in the macro block MB, so that the encoding information of each type of sub-block belongs to different macro block groups SLICE, or belongs to the same macro block group SLICE The sub-macro block group SLICE, so that during the image restoration and restoration process, the error code sub-block group caused by the transmission channel quality problem can be based on the image information of other adjacent sub-blocks around each of the error sub-blocks. The error sub-block performs the process of inserting the approximate image information, so that the inserted approximate image information has higher accuracy and better error concealment effect, so as to reduce d and the distortion degree of the video image in the visual angle. Corresponding to Flexible Macroblock Ordering (FMO) video coding processing method proposed by POLYCOM company, here will be
Ordering )编解码处理方法, 即 FBO编解码处理方法。 Ordering) encoding and decoding processing method, that is, FBO encoding and decoding processing method.
请参照图 4, 该图是本发明视频编解码处理方法的实现过程流程图; 本发明 FBO编解码处理过程包括如下步骤: Please refer to FIG. 4, which is a flowchart of an implementation process of the video codec processing method of the present invention; the FBO codec processing process of the present invention includes the following steps:
步骤 S10, 分别对视频图像经过变换宏块处理后的每个宏块组(即每 个图像片段 SLICE ) 中包含的每个宏块进行变换子块划分处理, 以将每个 宏块组中的每个宏块 MB划分为多个子块; 如原来每个 MB的大小为 16 X 16象素, 则可以根据具体情况需要对每个 MB进行变换子块处理, 以得 到 4 X 4象素或 8 x 8象素大小的子块; Step S10: Transform subblock division processing is performed on each macroblock included in each macroblock group (that is, each image segment SLICE) after the video image is subjected to transform macroblock processing, so as to divide the Each macroblock MB is divided into multiple sub-blocks. If the size of each MB is 16 × 16 pixels, you can perform transform sub-block processing on each MB to obtain 4 X 4 pixels or 8 pixels. x 8 pixel-sized sub-blocks;
步骤 S20, 分别对步骤 S10中每个宏块组变换处理后得到的所有子块 进行分类处理, 以使每个宏块组中变换处理后的所有子块按照类型归类到 一起, 以组成不同类型的子块组, 如可以按照每个变换得到的子块的变换 系数对每个宏块组变换子块处理后得到的所有子块进行分类处理; 果, 可以将各类子块组分别归属于独立的宏块组(即形成独立的视频片段 SLICE ), 以将原相应每个宏块组分裂为多个独立的宏块组,如原来某个宏 块组 SLICE0中每个宏块经过变换子块处理后, 再将相应得到的所有子块 划分为 3组, 则每一组子块就定义为一个宏块组, 即将原宏块组 SLICE0 分裂成了 3个独立的宏块组, 为 SLICE1、 SLICE2和 SLICE3。
处理的结果, 将各类子块组分别归属于相应宏块组的子宏块组, 以使原相 应每个宏块组由多个子宏块组。 如原来某个宏块组 SLICE0中每个宏块经 过变换子块处理后, 再将相应得到的所有子块划分为 3组, 则每一組子块 就定义为原宏块组 SLICE0下的一个子宏块组,即原宏块组 SLICE0就由 3 个子宏块组组成, 为 SLICE0_1、 SLICE0— 2和 SLICE0_3。 Step S20: Classify all the subblocks obtained after the transformation processing of each macroblock group in step S10, so that all the subblocks after the transformation processing in each macroblock group are grouped together according to type to form different Type sub-block groups, for example, all sub-blocks obtained after transform sub-block processing of each macro block group can be classified and processed according to the transform coefficients of each sub-block obtained from the transformation; as a result, various types of sub-block groups can be separately assigned To separate macroblock groups (that is, to form independent video segments SLICE), to split each corresponding macroblock group into multiple independent macroblock groups, such as each macroblock in a certain macroblock group SLICE0 is transformed After the sub-blocks are processed, all the corresponding sub-blocks are divided into 3 groups. Each group of sub-blocks is defined as a macro block group. That is, the original macro block group SLICE0 is split into 3 independent macro block groups. SLICE1, SLICE2, and SLICE3. As a result of the processing, each type of sub-block group is respectively assigned to the sub-macro block group of the corresponding macro-block group, so that each corresponding macro-block group originally has multiple sub-macro-block groups. For example, after each macroblock in a certain macroblock group SLICE0 is subjected to transform subblock processing, and then all the corresponding subblocks are divided into three groups, each group of subblocks is defined as one of the original macroblock group SLICE0. The sub-macroblock group, that is, the original macroblock group SLICE0 is composed of three sub-macroblock groups, which are SLICE0_1, SLICE0-2, and SLICE0_3.
步骤 S30, 视频图像发送方对视频图像划分出的每个宏块组头信息、 宏块头信息和每类子块的变换系数分别进行编码处理后发送; 如上例, 本 发明视频编解码处理方法在视频图像编码过程中, 如对宏块组 SLICE0编 码发送时, 需要对宏块组 SLICE0的头信息、 及 SLICE0中包含的宏块头 信息和 SLICE0中包含的每类子块的变换系数分别进行编码处理。 In step S30, the video image sender encodes and sends each macroblock group header information, macroblock header information, and transform coefficients of each type of subblocks after the video image is divided. As shown in the above example, the video codec processing method of the present invention is In the video image encoding process, if the macro block group SLICE0 is encoded and sent, the header information of the macro block group SLICE0, the macro block header information contained in SLICE0, and the transform coefficients of each type of subblock included in SLICE0 are separately coded. .
步骤 S40, 视频图像接收方分别对正确接收的码流进行解码处理, 以 恢复还原出相应子块的图像信息; 由于在某些情况下, 如传输信道质量恶 劣, 常常会造成视频图像的编码码流不能完全正确的到达接收方, 而会丢 失一些码流数据包或造成某些码流数据包出现误码信息, 所以在接收方, 这些出现错误的数据包所在的相应子块就会出现错误图像信息或失真程 度较大的图像信息; In step S40, the video image receiver performs decoding processing on the correctly received code stream to recover and restore the image information of the corresponding subblock. In some cases, such as the poor quality of the transmission channel, the video image encoding code is often caused. The stream cannot reach the receiver completely correctly, but some stream data packets will be lost or some bit stream data packets will have error information, so on the receiver side, the corresponding sub-blocks where these error packets are located will have errors. Image information or image information with a high degree of distortion;
步骤 S50, 在解码恢复视频图像后, 针对产生了误码的子块, 则根据 该产生了误码的子块周围的其他相邻子块的图像信息, 在该产生了误码的 子块中进行插入逼近图像信息处理, 以对产生了误码的子块进行错误掩盖 处理。 Step S50: After decoding and restoring the video image, for the sub-blocks that generate errors, according to the image information of other neighboring sub-blocks around the sub-blocks that generate errors, in the sub-blocks that generate errors Information processing for inserting and approximating images is performed to perform error concealment processing on a sub-block in which an error occurs.
下面以将视频图像中每个宏块组(SLICE ) 变换处理后得到的所有子 块划分为两种类型为例进行说明本发明 FBO编解码的处理中每个宏块中 两种类型的子块组的排列模式。 The following description is based on dividing all the subblocks obtained after each macroblock group (SLICE) transformation process in a video image into two types to illustrate the two types of subblocks in each macroblock in the FBO encoding and decoding process of the present invention. Arrange pattern of groups.
其中对每个 MB进行变换子块处理后得到的每个子块大小可以为 4 X 4象素: The size of each sub-block obtained after transforming sub-blocks for each MB can be 4 X 4 pixels:
如图 5所示, 该图是本发明视频编解码处理方法中, 将每个 MB划分 为 4 x 4象素大小的两类子块的第一排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别
属于不同类型的两个子块是呈对角状态分布的; 这样, 有五角星标识的子 块组可以单独属于一个 SLICE, 没有五角星标识的子块组则单独属于另一 个 SLICE; 或者有五角星标识的子块组和没有五角星标识的子块组分别组 成原宏块组 SLICE下的子 SLICE, 这样原来一个 SLICE就相当于划分为 了两个 SLICE。 As shown in FIG. 5, this figure is a first arrangement pattern diagram of dividing each MB into two types of sub-blocks of 4 x 4 pixel size in the video encoding and decoding processing method of the present invention; among them, sub-blocks identified by a five-pointed star Forms a type of sub-block group, and the sub-blocks without a pentagram mark form another type of sub-block group, that is, respectively Two sub-blocks belonging to different types are distributed diagonally; in this way, a group of sub-blocks identified by a pentagram can belong to one SLICE alone, and a group of sub-blocks not identified by a pentagram can belong to another SLICE alone; or there are five-pointed stars The identified sub-block group and the sub-block group without the pentagram mark respectively form the sub-SLICEs under the original macro-block group SLICE, so that one SLICE is equivalent to being divided into two SLICEs.
请参照图 6, 该图是本发明视频编解码处理方法中, 将每个 MB划分 为 4 x 4象素大小的两类子块的第二排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别 属于不同类型的子块呈隔行状态分布; Please refer to FIG. 6. This figure is a second arrangement pattern diagram of dividing each MB into two types of sub-blocks of 4 x 4 pixel size in the video encoding and decoding processing method of the present invention; One type of sub-block group, and the sub-blocks without a five-pointed star mark constitute another type of sub-block group, that is, sub-blocks belonging to different types are distributed in an interlaced state;
请参照图 7, 该图是本发明视频编解码处理方法中, 将每个 MB划分 为 4 x 4象素大小的两类子块的第三排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别 属于不同类型的子块呈隔列状态分布。 Please refer to FIG. 7, which is a third arrangement pattern diagram of dividing each MB into two types of sub-blocks with a size of 4 x 4 pixels in the video encoding and decoding processing method of the present invention; One type of sub-block group, and the sub-blocks without a five-pointed star mark form another type of sub-block group, that is, the sub-blocks belonging to different types are distributed in an alternate column state.
另外对每个 MB进行变换子块处理后得到的每个子块大小也可以为 8 X 8象素: In addition, the size of each sub-block obtained after transforming sub-blocks for each MB can also be 8 X 8 pixels:
请参照图 8, 该图是本发明视频编解码处理方法中, 将每个 MB划分 为 8 x 8 象素大小的两类子块的第一排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别 属于不同类型的两个子块是呈对角状态分布的。 Please refer to FIG. 8. This figure is a first arrangement pattern diagram of dividing each MB into two types of sub-blocks of 8 x 8 pixel size in the video encoding and decoding processing method of the present invention; One type of sub-block group, and the sub-blocks without a five-pointed star mark constitute another type of sub-block group, that is, two sub-blocks respectively belonging to different types are distributed diagonally.
请参照图 9, 该图是本发明视频编解码处理方法中, 将每个 MB划分 为 8 x 8 象素大小的两类子块的第二排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别 属于不同类型的子块呈行状态分布。 Please refer to FIG. 9, which is a second arrangement pattern diagram of dividing each MB into two types of sub-blocks of 8 x 8 pixel size in the video encoding and decoding processing method of the present invention; One type of sub-block group, and the sub-blocks without a five-pointed star mark form another type of sub-block group, that is, the sub-blocks belonging to different types are distributed in a row state.
请参照图 10,该图是本发明视频编解码处理方法中,将每个 MB划分 为 8 x 8 象素大小的两类子块的第三排列模式图; 其中有五角星标识的子 块组成一类子块组, 而没有五角星标识的子块组成另一类子块组, 即分别 属于不同类型的子块呈列状态分布。 Please refer to FIG. 10, which is a third arrangement pattern diagram of dividing each MB into two types of sub-blocks of the size of 8 x 8 pixels in the video encoding and decoding processing method of the present invention; One type of sub-block group, and the sub-blocks without a five-pointed star mark form another type of sub-block group, that is, sub-blocks belonging to different types are distributed in a column state.
当然, 无论是将每个 MB划分为 4 x 4象素大小的子块还是划分为 8 x 8象素大小的子块, 其不同类型的子块排列模式都不局限于上述所举的
实施例情况, 根据具体情况, 还可以有其他的排列模式, 则其他不同类型 的子块排列模式也都在本发明所要保护的范围内。 Of course, regardless of whether each MB is divided into 4 x 4 pixel-sized subblocks or 8 x 8 pixel-sized subblocks, the different types of subblock arrangement modes are not limited to those mentioned above. In the embodiment, according to specific situations, there may be other arrangement modes, and other different types of sub-block arrangement modes are also within the scope of the present invention.
而且上述所举的不同类型的子块排列模式是基于视频图像格式 4:2:0 中宏块 MB中亮度块丫、 色度块 Cr、 色度块 Cb结构的情况进行说明的, 而针对视频图象格式为 4: 2: 2和 4: 4: 4的两种情况, 其原理同 4: 2: 0, 这里不再过多赘述。 Moreover, the above-mentioned different types of subblock arrangement modes are described based on the structure of the luminance block y, chrominance block Cr, and chrominance block Cb in the macro block MB in the video image format 4: 2: 0. The image format is two cases of 4: 2: 2 and 4: 4: 4, the principle is the same as 4: 2: 0, so I won't go into details here.
由于本发明视频编解码处理方式是基于 FBO的, 即以宏块 MB中的 子块属于不同的宏块组 SLICE方式进行视频编解码操作,所以会引起编解 码过程中一些标准过程的相应变化, 如: Since the video encoding and decoding processing method of the present invention is based on FBO, that is, the video encoding and decoding operations are performed in the SLICE manner in which subblocks in a macroblock MB belong to different macroblock groups, it will cause corresponding changes in some standard processes in the encoding and decoding process. Such as:
1 )视频图像发送方和视频图像接收方要对属于同一类型的相邻子块 进行去方块处理(De - blocking ), 如对图 6、 图 7、 图 8、 图 9和图 10所 示的有五角星标识的相邻 4X4象素子块进行 De - blocking处理, 及对没 有五角星标识的相邻 4X4象素子块进行 De - blocking处理; 而对于图 5 所示的排列模式的 4X4象素子块就无需进行 De - blocking处理, 因为在 这种排列模式中, 没有同一类型的子块存在相邻的情况出现。 1) The video image sender and the video image receiver need to perform de-blocking on adjacent sub-blocks of the same type, as shown in FIG. 6, FIG. 7, FIG. 8, FIG. 9, and FIG. 10 De-blocking processing is performed on adjacent 4X4 pixel sub-blocks with a five-pointed star mark, and De-blocking processing is performed on adjacent 4X4 pixel sub-blocks without a five-pointed star mark; and the 4X4 pixel sub-blocks in the arrangement pattern shown in FIG. There is no need to perform De-blocking, because in this arrangement mode, no sub-blocks of the same type exist adjacently.
2 )视频图像发送方和视频图像接收方应该对属于同一类型的子块中 的图像信息进行帧内预测。 2) The video image sender and the video image receiver should perform intra prediction on the image information in the subblocks belonging to the same type.
3 )视频图像发送方和视频图像接收方的编解码运动估计和运动补偿 基本不受影响, 只是在进行运动矢量预测时需要参考同一类型的子块的运 动矢量。 3) The codec motion estimation and motion compensation of the video image sender and the video image receiver are basically unaffected, but it is necessary to refer to the motion vectors of the same type of subblocks when performing motion vector prediction.
4 )视频图像发送方和视频图像接收方在进行子块变换系数编码上下 文考虑时, 只能使用同一类型子块部分的信息。 4) The video image sender and the video image receiver can only use the information of the same type of sub-block part when considering the sub-block transform coefficient encoding and context consideration.
5 ) 冗余参数信息问题, 如在上述将原来一个 SLICE划分为两个独立 的 SLICE的例子中, 由于对于每一个独立的 SLICE部分来说, 只编码了 原来 SLICE部分的一半图像信息, 因此相对而言, 采用 FBO视频编解码 方法后, 一个 SLICE所包含的宏块数目可以比常规下的宏块数增加近一 倍, 但宏块头信息显然是重复了一些公共部分, 而属于不同类型的子块头 信息可以分别在相应的独立 SLICE部分中分别编码, 如块运动矢量信息 等。
6 )编码效率问题, 可以从两个方面考虑: 5) The problem of redundant parameter information, as in the above example of dividing an original SLICE into two independent SLICEs, because for each independent SLICE part, only half of the image information of the original SLICE part is encoded, so the relative In terms of the FBO video codec method, the number of macroblocks included in a SLICE can be nearly doubled compared to the regular macroblocks. However, the header information of the macroblock obviously repeats some common parts and belongs to different types of sub-blocks. The block header information can be separately encoded in the corresponding independent SLICE part, such as block motion vector information. 6) The encoding efficiency problem can be considered from two aspects:
一方面, 由于本发明 FBO编解码处理方式将每一宏块 MB的不同类 型的子块归属于不同的宏块组 SLICE , 显然从一定程度上降低了原来同一 个 SLICE部分中相邻块的相关性, 其编码效率肯定会有一定程度的降低。 On the one hand, because the FBO codec processing method of the present invention assigns different types of subblocks of each macroblock MB to different macroblock groups SLICE, it is obvious that the correlation of adjacent blocks in the same SLICE part is reduced to a certain extent. The coding efficiency will definitely be reduced to some extent.
另一方面, 参照图 11, 该图是一个宏块行为一个 SLICE时, 其编码 预测的原理示意图; 在该情况下, 对一个宏块行进行逐一编码处理时, 一 般只能参照左面宏块信息进行预测; 参照图 12,该图是两个宏块行为一个 SLICE时,其编码预测的原理示意图;对于两个宏块行为一个 SLICE的情 形, 则下面宏块行的宏块可以参照该宏块左上角宏块的信息、 正上方宏块 的信息、 右上角宏块的信息和左面宏块的信息进行编码预测, 可见由于编 码预测性能的提高, 可以从一定程度上提高编码效率(上述理由是因为为 了提高 H.264帧内编码的效率, 在给定帧中充分利用相邻宏块的空间相关 性, 相邻的宏块通常含有相似的属性。 因此, 在对一给定宏块编码时, 首 先可以才艮据周围的宏块预测, 典型的是^^据左上角的宏块, 因为此宏块已 经被编码处理, 然后对预测值与实际值的差值进行编码, 这样, 相对于直 接对该帧编码而言, 可以大大减小码率)。 On the other hand, referring to FIG. 11, this figure is a schematic diagram of the principle of encoding prediction when a macroblock acts as a SLICE. In this case, when one macroblock line is encoded one by one, generally only the left macroblock information can be referred to. Make predictions; refer to FIG. 12, which is a schematic diagram of the principle of encoding prediction when two macroblocks act as one SLICE; for the case where two macroblocks act as one SLICE, the macroblocks in the following macroblock row can refer to the macroblock The top left corner macroblock information, the top right macroblock information, the top right corner macroblock information, and the left side macroblock information are used to perform coding prediction. It can be seen that due to the improvement of the coding prediction performance, the coding efficiency can be improved to a certain extent (the above reasons are Because in order to improve the efficiency of H.264 intra-frame coding, the spatial correlation of adjacent macroblocks is fully utilized in a given frame. Adjacent macroblocks usually contain similar attributes. Therefore, when encoding a given macroblock, First of all, it can be predicted based on the surrounding macroblocks. Typically, it is based on the macroblock in the upper left corner, because this macroblock has been encoded and processed. The difference between the measured value and the actual value is encoded, so that, with respect to the direct coding for this frame, can be greatly reduced rate).
当然, 综合上述两方面的影响, FBO编解码方式总的看来其编码效率 还会有一定程度的降低。 Of course, considering the impact of the two aspects above, it seems that the encoding efficiency of FBO encoding and decoding methods will still be reduced to a certain extent.
在具体实际应用中, 根据具体应用情况, 可以有选择的使用 FMO和 FBO编解码处理方式,同时对不同类型子块的排列模式也可以有选择的进 行使用, 如针对前述的将多个子块划分为两种类型的情况, 可以进行如下 设置: ' In specific practical applications, according to specific applications, FMO and FBO codec processing methods can be selectively used, and at the same time, the arrangement modes of different types of sub-blocks can be selectively used, such as dividing multiple sub-blocks according to the foregoing For two types of cases, the following settings can be made: '
1、 设立一个 FBO标识 "FBO— label" 1. Establish an FBO logo "FBO— label"
当 FBO— label = 0, 选取现有技术的 FMO编解码处理方式, 主要应用 于误码率较低的传输信道; When FBO_ label = 0, the FMO codec processing method of the prior art is selected, which is mainly applied to transmission channels with lower bit error rates;
当 FBO_label = 1, 选取本发明 FBO编解码处理方式, 主要应用在误 码率较差的传输信道; When FBO_label = 1, the FBO encoding and decoding processing method of the present invention is selected, which is mainly applied to a transmission channel with a poor bit error rate;
2、 对于视频图像中某个 SLICE, 为其设立一个主标识 Sub— SLICE, 分别取值 0和 1分别对应于其被分裂开的两个子 SLICE, 也可以将其分裂
后的两个独立 SLICE部分分别都取值为 Sub_SLICE; 2. For a SLICE in a video image, set up a main logo Sub- SLICE, with values of 0 and 1 respectively corresponding to the two sub-SLICEs that are split, or they can be split. The last two independent SLICE parts are respectively set to Sub_SLICE;
3、 设立一个 FBO模式标识 "FBO— mode" , FBO— mode取值 0〜5分别 对应于图 5至图 10所示的六种 FBO排列模式; 3. Establish an FBO mode identifier "FBO_mode". The value of FBO_mode 0 ~ 5 corresponds to the six FBO arrangement modes shown in Figure 5 to Figure 10.
4、 六种 FBO排列模式的选择, 可以根据系统编码性能要求和抗误码 性能要求综合考虑进行选取; 当然,考虑到具体处理过程中的计算复杂度, 也可以仅取六种 FBO排列模式中的某几种; 例如, 可以仅取图 5和图 8 所示的 FBO排列模式。 4. The selection of six FBO arrangement modes can be selected according to the comprehensive consideration of system coding performance requirements and error resistance performance requirements. Of course, considering the calculation complexity in the specific processing process, only six FBO arrangement modes can be selected. For example, only the FBO arrangement patterns shown in FIG. 5 and FIG. 8 may be taken.
可见, 在本发明 FBO视频编解码处理方式中, 图像信息恢复还原时, 对于由于误码而出现的失真图像, 可以以 4 x 4象素或 8 x 8象素大小的块 信息作为参考信息对产生了错误的块进行插入逼近信息的处理, 比起现有 技术中以 16 x 16 象素大小的宏块进行信息逼近处理, 其插入的逼近信息 精确度会更高一些, 图像还原效果会更好一些。 It can be seen that, in the FBO video codec processing method of the present invention, when the image information is restored and restored, for a distorted image due to bit errors, block information of the size of 4 x 4 pixels or 8 x 8 pixels can be used as the reference information pair. The processing of inserting the approximation information into the wrong block is performed. Compared with the prior art, which uses a 16 x 16 pixel size macroblock to perform the information approximation processing, the accuracy of the inserted approximation information will be higher, and the image restoration effect will be more accurate. a little better.
上述主要针对将对同一宏块组进行变换子块处理后得到的所有子块 划分为两种类型的例子进行了说明, 其中对于对同一宏块组进行变换子块 处理后得到的所有子块划分为多种类型的实施情况和划分为两种类型的 情况相似, 这里不再过多赘述, 但是其对所有子块划分为多种类型的实施 情况也包括在本发明所要保护的范围内。 The above mainly describes the example of dividing all sub-blocks obtained after transforming sub-block processing on the same macro block group into two types, where all sub-blocks obtained after transforming sub-block processing on the same macro block group are divided The implementation for multiple types is similar to the case where it is divided into two types, which is not described in detail here, but its implementation for dividing all sub-blocks into multiple types is also included in the scope of the present invention.
本发明 FBO编解码处理方式是在 H.264参考程序 JM76的基础上进行 的修改, 针对一些测试序列, 对本发明所提出的 FBO编解码处理方法进 行了测试, 结果显示, 能够得到较好的错误掩盖效果, 减小了由于传输信 道质量问题而引起的图象失真, 较好的提高了视频通信的效果。 The FBO codec processing method of the present invention is a modification based on the H.264 reference program JM76. For some test sequences, the FBO codec processing method proposed by the present invention is tested, and the results show that better errors can be obtained. The masking effect reduces the image distortion caused by the transmission channel quality problem, and improves the video communication effect better.
以上所述仅是本发明的优选实施方式, 应当指出, 对于本技术领域的 普通技术人员来说, 在不脱离本发明技术原理的前提下, 还可以作出若干 改进和润饰, 这些改进和润饰也应视为本发明的保护范围。
The above is only a preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principles of the present invention, several improvements and retouches can be made. These improvements and retouches also It should be regarded as the protection scope of the present invention.