CN1581972A - Error-corcealed video decoding method - Google Patents

Abstract

The invention is a video decoding method for concealing errors. The present invention utilizes the by-product information obtained during decoding by a video decoder, including motion vectors and direct current (DC) images of I frames, to perform real-time shot segmentation, and to judge whether the current frame in which an error occurs is a shot boundary. If the current frame is a shot boundary, the block-based segmentation-match error concealment method in spatial domain is used for error concealment; if not, the forward-backward block matching error concealment method in temporal domain is used for error concealment. The method has the advantages of fast speed, small system overhead, good hiding effect, good robustness, etc., and is very suitable for various types of video players.

Description

A Video Decoding Method with Error Concealment

technical field

The invention belongs to the field of video technology, and in particular relates to a video decoding method for concealing errors. It is mainly used to improve the image quality of the video decoder. Its principle is to hide (that is, try to make people invisible) the mosaic phenomenon in the decoded image caused by the error of the video code stream, and reduce the sensitivity of the human visual system to the deterioration of image quality. .

With the continuous development and rapid popularization of technologies such as digital TV, conference TV, and computer networks, how to transmit video streams on the Internet has become more and more important. Since the Internet does not reserve special channels for real-time streaming media such as video and audio, the transmission of audio and video is easily affected by network congestion and delay. Some basic compression techniques of video coding itself, such as time domain prediction and variable word length coding in MPEG-1/2/4 standards, are particularly vulnerable to network congestion and packet loss, even if a bit is wrong or lost, It will also cause a large number of errors in the video image, resulting in a serious decline in subjective visual quality.

Data loss may occur at any time during the stream transmission over the network. In MPEG video data, most of the bits are Huffman codes (Huffman) and discrete cosine transform (DCT) coefficients for encoding motion vectors. As we all know, Huffman codes are a variable word length code. Once an error occurs, It is difficult to find out immediately. Usually, it is only when the error spreads that it cannot be decoded that it is known that there is an error in the code stream at an unknown position in front. The characteristics of variable-length coding make it impossible for the decoding algorithm to go back and locate the wrong place. It can only search the start code again and continue decoding from the next synchronization point. The decoding of a macroblock between the time an error is found (the actual error is often before) and the next synchronization point is undefined. This error is exacerbated by temporal prediction with high compression efficiency. If this error occurs in an earlier reference frame in a group of pictures, the decoding of the macroblock at the same position in all subsequent decoded frames in the group of pictures will cause distortion, which usually lasts for half a second, and the consequences are very serious.

A large number of studies have shown that this error diffusion problem can be overcome in many ways, such as data classification, adding forward error correction (FEC) codes, sender flow control, etc., adding error concealment measures in video decoders has also been proved to be a It is a very effective technical means, and its cost is only to increase the complexity of some decoding algorithms.

Commonly used error concealment methods include: forward error concealment, error concealment through post-processing, and error concealment between decoder and encoder interaction. (1) Forward error concealment: The encoder plays the main role. By adding a certain amount of redundant information at the information source encoder end or the transport layer encoder end, when data errors occur on the network, the receiver may be transmitted from Recover lost information from redundant information. Usually the increased amount of redundant information data cannot be ignored. When congestion occurs in the network, the redundant information will aggravate the congestion and cause a more serious decline in video quality. (2) In the error concealment method where the decoder and the encoder interact, the encoder adjusts its encoding strategy to obtain better performance by obtaining feedback information from the decoder, which requires the encoder and the decoder to have certain Interactivity, so the encoder and decoder are usually the same developer. The resulting video stream from such an encoder lacks versatility. In view of some essential flaws in the above two error concealment methods, error concealment via post-processing has been widely used.

Error concealment methods through post-processing usually include the following:

(1) Time-domain error concealment based on motion compensation: a simple method to utilize the time-domain correlation characteristics of the video signal, that is, to use the macroblock with the same spatial position as the error macroblock in the previous frame of the frame where the error macroblock is located. Replaces erroneous macroblocks in the current frame. But this method will have a great impact on the visual effect when there are more violent motions in the video. When the corresponding replacement macroblock is found through motion compensation, the concealment effect is greatly improved.

(2) Maximum smoothness recovery: Utilize the spatial smoothness characteristics of most images and video signals, and realize it through a stepwise energy minimization method.

(3) Error concealment in the space domain or frequency domain: the error concealment method in the space domain or frequency domain is to use the image around the data loss block in a frame of image, and use the interpolation method to recover the loss block. Because a sudden error often causes the loss of adjacent blocks of data, in this case, the calculation of interpolation will be more complicated and the effect will be poor.

Contents of the invention

The purpose of the present invention is to propose a video decoding method with relatively simple calculation and better effect for concealing errors.

The error-concealing video decoding method proposed by the present invention is an error-concealing method embedded in a video decoder, which combines the compressed-domain fast shot segmentation method with the spatial-domain error concealment and time-domain error concealment methods to obtain Good error hiding effect. Its steps are: first locate the error in the video, specifically two methods can be used to check the error: (1) calculate the pixel value of the macroblock boundary and the difference between the pixel value of the macroblock boundary around it; (2) check the MPEG code stream syntax to detect errors in the codestream. Then, for the frame with error, adopt the method based on motion vector (P and B frame) and direct current (DC) image (I frame) to detect whether the frame with error is on the boundary of shot (shot). If it is not at the boundary of the shot, use the forward-backward block matching error concealment method in the time domain for error concealment; if it is at the boundary of the shot, the difference between the front and rear adjacent frames is very large, and the error concealment in the time domain cannot be performed. In this case, the block-based segmentation-matching error concealment method is used for error concealment.

Description of drawings

Fig. 1 is the illustration of 4 steps of segmentation-matching process of the present invention, wherein (a) is the initial matching illustration, (b) is the first segmentation illustration, (c) is the secondary segmentation illustration, (d) is not Divide and enlarge the search area icon.

Figure 2 is an illustration of error concealment in the time domain. Wherein (A) is to select the macroblock above the wrong macroblock, (B) is to select the macroblock below the wrong macroblock, and (C) is to select both the macroblocks above and below the wrong macroblock.

Detailed ways

The error concealment method based on shot segmentation of the present invention can be summarized into the following basic steps:

1. Decode the nth frame.

2. Check whether an error occurs in the nth frame, if an error occurs, go to 3, otherwise go to 5.

3. Detect whether the nth frame is the boundary of a shot.

4. If the nth frame is not the boundary of a shot, perform temporal error concealment. If it is a shot boundary, perform airspace error concealment.

5. If the nth frame is the last frame, exit, otherwise n=n+1, go to 1.

In the following, the error detection mechanism and the adopted methods of spatial error concealment and temporal error concealment are introduced in detail, and then a fast shot segmentation method in compressed domain is given. Finally, the shot segmentation method is combined with spatial error concealment and temporal error concealment. When discussing the specific implementation method, the video stream of the MPEG-2 standard is used as an example to introduce, but the method of the present invention is not limited to the MPEG-2 standard, and is also applicable to the MPEG-1/4 standard or other video coding standards.

1. Error discovery mechanism

In order to do a good job of error concealment, the most important thing is to be able to correctly locate the location where the error occurs. If the location of the error cannot be accurately located, error concealment cannot be carried out. The present invention mistake has mainly used two kinds of methods:

(1) Check the difference between the pixel value of the macroblock boundary and the pixel value of its surrounding macroblock boundary

When an error occurs in the video stream and a "mosaic" phenomenon occurs during playback, it can be intuitively felt that there is an obvious difference between the macroblock where the error occurred and the surrounding macroblocks. Here, the position of the error macroblock is discovered by using the difference in chrominance and brightness between the error macroblock and the surrounding macroblocks. According to the encoding method of MPEG-2, VLC encoding is adopted for macroblocks. When an error occurs in a certain macroblock, it will often affect subsequent macroblocks in the slice until the arrival of the next slice. And because a slice does not exceed one row, the error caused by this macroblock will only affect the macroblock of this row at most.

Through the above analysis, it can be known that among the macroblocks around the error macroblock, the left and right macroblocks are often affected, but the upper and lower macroblocks are generally not affected, so here only the upper boundary of the macroblock and the upper macroblock are calculated The difference between the lower boundary and the difference between the lower boundary of the macroblock and the upper boundary of the macroblock below. Through the calculation and comparison of many frames, it is found that among the three components of color YUV (wherein, the Y component represents brightness, U represents the color difference between blue and green, and the V component represents the color difference between red and green), the difference in U component value is the most important. Significantly, so in this method, the difference of the U component is selected as the judgment of the difference between the macroblock and the surrounding macroblocks. A small number of pixel values are selected to participate in the calculation in order to reduce the complexity of the algorithm.

In this method, choose to calculate the difference between the pixel value of the upper boundary of the current macroblock and the pixel value of the lower boundary of the upper macroblock on the U component, and the pixel value of the lower boundary of the macroblock and the pixel value of the upper boundary of the lower macroblock in the U component. The difference above is used as the judgment condition at the same time. When the two differences are greater than the threshold (in this method, the threshold range is 25-35), it can be determined that the macroblock has an error.

(2) When the detected code value is not in the code table, it means that an error has occurred in the code stream

Since the VLC code table used in the MPEG standard is not complete, when an error occurs in the code stream, it is likely that the motion vector of a macroblock or the variable word length code word (VLC) of the discrete cosine transform (DCT) coefficient is not in the VLC. In the code table, when this situation occurs, it can be concluded that an error has occurred in the code stream. However, this method often cannot accurately locate the wrong position, because when a codeword does not appear in the VLC code table, it may not be that the codeword has an error, but that the previous codeword in this slice has an error. , is just the error propagation caused by a codeword becoming another codeword due to a bit error and changing the length of the codeword, resulting in the error being discovered after a period of time. However, once this happens, it can be concluded that a transmission error has occurred in the slice, so in the error concealment implemented by this method, we will implement error concealment for the entire slice when this happens.

2. Error hiding in the airspace

The error concealment in the airspace adopts the block-based segmentation-matching error concealment method in the airspace. The following Algorithm 1 is a method for error concealment of a macroblock:

Algorithm 1, the specific steps are as follows:

(1) locate the position of the macroblock in error;

(2) Judging whether "the adjacent macroblock directly above the error macroblock and the adjacent macroblock directly below" can be used; if not available, find the nearest macroblock that can be used for error concealment;

(3) The similarity calculation of the area in the error macroblock and the area in the available macroblock;

(4) Complete the segmentation-matching process;

(5) whether the two regions used in the division-matching process are similar regions in the fourth step of calculation; if not, turn to (3), if yes, turn to (6);

(6) complete error hiding;

(7) Judging that each region of the current erroneous macroblock has carried out error concealment; if not, turn to (3); if yes, exit.

The technical details involved in Algorithm 1 will be explained below.

Regional similarity estimation:

In this method, the mean of the absolute error between neighboring blocks _BT and _BB of size bx×by is estimated by:

$\mathrm{<span\; class="notranslate">\; MAD</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; bx</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; by</span>}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; bx</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; by</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$

When the MAD value is less than a preset threshold, or when a minimum MAD is found in the specified search area, the two matching blocks at this time are used for error concealment.

Split-Match Process

The 4 steps of the segmentation-matching process are shown in Figure 1. Among them, MBc represents the macroblock to be error concealed, MB _T , MB _TL , MB _TR , MB _B , MB _BL , and MB _BR represent the top, top left, top right, bottom, bottom left, and bottom right of the damaged macroblock, respectively available macroblocks.

In the first 2 steps, larger adjacent blocks are defined for matching, while in the last 2 steps, blocks of minimum size 4×4 are used for matching, but different search regions are defined. Adopting this method guarantees the minimum processing time and the best possible hiding effect. Matching with larger blocks at the beginning of the process ensures that regions with greater consistency can be reconstructed directly, avoiding more computation and delay. On the other hand, areas with more detail require smaller blocks to match. The 4 steps are detailed below:

(1) Initial match - largest block size

The initial matching tries to match the largest vertically adjacent blocks b ₁ and t ₁ , which have a size of 16×8 pixels. As shown in Figure 1a. If the two regions are considered similar, error concealment of the entire macroblock can be achieved by copying. Otherwise, the algorithm continues to execute step (2).

(2) First split - only in the vertical direction

Divide the initial block in step (1) into two smaller blocks, namely b ₁ , b ₂ and t ₁ , t ₂ , with a size of 8×8. Matching is done for each pair only in the vertical direction. Such as b ₁ and t ₁ and b ₂ and t ₂ (see Figure 1(b)). Concealment with smaller textured regions is thus performed directly.

(3) Secondary segmentation - define the initial search area - a set of less directions

In step (2), the block is further divided into the smallest allowed size 4×4, denoted by b _i and t _i . In this step, as shown in Fig. 1(c), an initial search area in the horizontal direction is defined, and the smallest blocks (b _i and t _i ) to be matched slide in the defined search area. All their combinations use the similarity estimation algorithm introduced earlier to decide whether it is the best match. This step is performed iteratively until the next best block combination no longer satisfies similarity matching or error concealment of the entire macroblock has been achieved. The concept of sliding ensures that borders or lines in any direction can be reconstructed.

(4) No segmentation - increase the search area - a collection of more directions

The search area defined in step (3) is further enlarged as shown in Fig. 1(d). The direction of reconstruction is allowed to extend to adjacent upper left, upper right or lower left, lower right macroblocks, thus expanding the direction of reconstruction. In this step, the size of the block used for matching remains unchanged, as shown in Fig. 1(d). Matching is performed between b _k and t _k . This step is also performed iteratively until the entire macroblock has been error concealed.

error hiding

The method of copying is adopted in the error concealment method of the method. As shown in Figure 1, the image content of the upper and lower adjacent best matching blocks are copied to the upper half and lower half of the lost area respectively according to the matching direction. (For example, in Figure 1a, if b ₁ and t ₁ are considered to match in the first step, the content of b ₁ is copied to an equal-sized area in the upper half of the error macroblock, i.e. light area; t ₁ is copied to an equal-sized area in the lower half of the error macroblock, which is the darker area in the figure). In the first two steps of the segmentation-matching process, error concealment is performed directly without an iterative process, while in the last two steps an iterative process is required to perform error concealment on regions that have not been error concealed in the previous step. Therefore, a flag is set for each pixel in the macroblock in the algorithm, which is used to indicate whether the pixel has been error concealed. If a pixel has been error concealed, it does not need to be error concealed when it appears in the direction of the next best match.

3. Error concealment in time domain

Compared with the error concealment in the spatial domain, the error concealment in the temporal domain can achieve nearly perfect results in regions with little or no motion. When the motion vectors of the macroblocks around the erroneous macroblock are inconsistent or their motion information cannot be obtained because the surrounding macroblocks are intra-frame coded, the simple block matching method cannot obtain satisfactory results, while in this method The time-domain forward-backward block matching error concealment method can reduce these disadvantages. The method selects a neighborhood for an erroneous macroblock in which the best possible concealment of the erroneous macroblock can be carried out on the basis of block matching.

Forward-backward block matching error concealment in the temporal domain exploits the redundancy in the temporal domain of the video sequence. It is assumed here that at least some neighboring blocks have smooth, consistent motion vectors. The algorithm is described in detail below.

MB _C is the macroblock where an error occurs currently, MB _T is the macroblock above the error macroblock, and MB _B is the macroblock below the error macroblock. In the reference frame, with the macroblock at the same position as the error macroblock as the center, an area of N×M size is pre-defined as the search area, and the surrounding macroblocks of the error macroblock are selected as candidates. Here we can only select the macroblocks above the error macroblock, as shown in Figure 2A; we can also select only the macroblocks below the error macroblock, as shown in Figure 2(B); we can also select both above and below the error macroblock The macroblocks of are used as candidates, as shown in Fig. 2. In the last case, the vertical distance between the upper and lower macroblocks is limited to 16 pixels, while the motion of MB _T and MB _B in the search area is consistent. Block matching in the temporal domain searches all candidates for the block that results in the smallest MAD in the selected region as an estimate of the erroneous macroblock content. Error concealment copies the content of the appropriate surrounding block of the found best matching block to the position of the macroblock where the error occurred. For example, if we select the candidate in the way shown in Figure 2 (A), then the macro The block is copied to the location of the bad macroblock.

4. Fast Shot Segmentation Method in the Compressed Domain

In the traditional error concealment strategy, generally speaking, temporal domain error concealment has the characteristics of low computational cost and good concealment effect, and it is a method that is tended to be used in error concealment. But when the content changes between adjacent frames in the video are particularly obvious (such obvious changes are generally caused by camera cuts), the effect of temporal error concealment is poor. With this in mind, our proposed new error concealment method combines real-time shot segmentation and traditional error concealment methods.

A shot is a video clip captured during the time between when the camera is turned on and when it is turned off. A video program can be regarded as a combination of some shots semantically, and shot segmentation is to find the boundaries of these semantics. There are many algorithms for shot segmentation. Considering that in our application, the accuracy of shot segmentation is not very important, but the speed of shot segmentation is very critical. In the present invention, a real-time shot segmentation method based on motion vectors is adopted. The basic idea is as follows:

In the MPEG-2 stream, there are three types of frames: I, P, and B. There is no motion vector in the I frame, there is a forward motion vector in the P frame, and there may be forward, backward, and bidirectional in the B frame. Motion vector. However, no matter how complicated the situation is, when the MPEG-1/2 encoder is working, it will always choose a reasonable encoding mode according to a certain basis. Therefore, the encoder still follows some reasonable strategy when dealing with changing shots. For example, when encountering a jumping shot, the inter-frame prediction will inevitably fail. At this time, it is more inclined to choose the intra-frame coding mode, and on the front and rear sides of the shot boundary, the selection of forward, backward and bidirectional prediction modes must be in line with some reasonable pattern.

Let R _P denote the ratio of macroblocks in a P frame without motion vectors to macroblocks with forward motion vectors in the frame:

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; MV</span>}}_{\mathrm{<span\; class="notranslate">\; non</span>}}}{{\mathrm{<span\; class="notranslate">\; MV</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}$

MV _F indicates the number of macroblocks with forward motion vectors in the frame, and MV _non indicates the total number of macroblocks without motion vectors. Under normal circumstances, most of the macroblocks in a P frame will be coded with reference to its previous reference frame to reduce coding bits, so R _P is very small (generally 1<R _P ); when a camera switch occurs at a P frame , since the content of the current P frame is very different from the previous reference frame, more bits will be used for encoding referring to the previous reference frame, and most macroblocks in the P frame will use intra-frame coding (no motion vector), so _RP is huge. The value of R _P can be used to determine whether a lens switch occurs at the P frame.

Similarly, let _RB represent the ratio of macroblocks with backward motion vectors and macroblocks with forward motion vectors in the B frame

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; MV</span>}}_{\mathrm{<span\; class="notranslate">\; B</span>}}}{{\mathrm{<span\; class="notranslate">\; MV</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}}$

Under normal circumstances, most of the macroblocks in a B frame will be coded with reference to its previous reference frame and the following reference frame to reduce the coding bit. R _B is a value that fluctuates around 1; when a shot occurs at a B frame When switching, the content of the current B frame is very different from the previous reference frame. Referring to the previous reference frame encoding will use more bits, and most of the macroblocks in the B frame will use the subsequent reference frame for prediction instead of referring to the previous reference frame for prediction, so the _RB is very large. The value of R _B can be used to determine whether a lens switch occurs at the B frame.

Using a sliding window, it is possible to judge the possible shot switching between the P frame and the B frame. In this method, separate sliding windows are established for R _P and _RB , here only the sliding window of R _P is taken as an example.

Suppose W is a sliding window of length l, w _i (i∈[0, l)) is an element in the window (i.e. the value of R _P ), and the expectation of the window is

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; l</span>}}$

FrameNo represents the frame number of the P frame currently entering the sliding window, and ShotCount represents the current shot counter. The following algorithm can be used to find the switch that may occur at the P frame.

The sliding window algorithm, the specific steps are as follows:

(1) FrameNo=0, ShotCount=0;

(2) For each element in the window, let w _i =InitialEle, (i∈[0,l)), i=0, calculate E(W);

(3) If R _P (FrameNo)>TH_P×E(W), go to (4), otherwise go to (5);

(4) Shot[ShotCount]=FrameNo, ShotCount=ShotCount+1, FrameNo=FrameNo+1, go to (3), otherwise, go to (5);

(5) W(imodl)=R _P (FrameNo), recalculate E(W), i=i+1, FrameNo=FrameNo+1, if FrameNo＞Max_X_Frame, exit, otherwise go to (3);

In the present invention, for P frames (that is, X is P), InitialEle=1000, sliding window width l∈[8,10], TH_P=4. For B frame (X is B), InitialEle=1000, sliding window width l∈[18,22], TH_B=6.

For camera cuts that may occur at I-frames, the following methods are used to detect:

Perform DCT transformation on each block (8×8 pixels) of the macroblock to obtain 64 DCT coefficients, wherein the first coefficient is a DC coefficient, ie, a DC coefficient. Let c(0,0) be the DC coefficient of each block, then

$\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 0,0</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 7</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; 64</span>}}$

That is, the DC coefficient is the average value of the entire block. The DC image of the original image is formed by all the DC coefficients in one frame. The resolution of the DC image is 1/64 of the original image, but most of the information of the original image is retained.

To further reduce the computation, we only utilize the DC image of the luminance component of the I frame to detect the dissolve. The DC image of the I frame can be directly extracted from the compressed domain. Let I _i and I _i+1 be two adjacent I frames, and the difference between the DC images between them is defined as:

$\mathrm{<span\; class="notranslate">\; Diff</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; \_</span>\mathrm{<span\; class="notranslate">\; DC</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; Total\; Block</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; DC</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; DC</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; |</span>$

Diff_I_DC(I _i , I _i+1 ) will have an obvious peak when the lens is blurred and switched. Similarly, the sliding window method can also be used for Diff_I_DC (I _i , I _i+1 ), in the above sliding window algorithm, X is I,

$\mathrm{InitialEle}=\frac{\mathrm{width}\&times;\mathrm{height}\&times;8}{256},$ Sliding window width l=4, TH_I=3.

Claims (9)

1, a kind of video decoding method of concealing errors is characterized in that: at first mistake in the video flowing is positioned: the pixel value of inspection macroblock boundaries and it is the difference of macroblock boundaries pixel value on every side, perhaps realizes error checking from the grammer of MPEG code stream; To wrong frame, adopt then, judge whether it appears on the shot boundary based on motion vector or direct current (DC) image method; If not on shot boundary, just with time domain go forward to-back carry out error concealing to piece matching error concealment method; If on shot boundary, just hide with the error hidding method of block-based cutting apart on the spatial domain-mate.

2, method according to claim 1, it is characterized in that the pixel value of described inspection macroblock boundaries and the difference of macroblock boundaries pixel value around it, be calculate the lower boundary of the difference of U component of the coboundary of current macro and top macro block lower boundary and this macro block and macro block coboundary, below the difference of U component; If these two differences conclude then that all greater than a given threshold value TH_1 current macro is wrong, the U component is green poor here.

3, method according to claim 1, it is characterized in that described from the grammer of MPEG code stream retrieval error, be in the mutilation long code word (VLC) of motion vector that detects a macro block or discrete cosine transform (DCT) coefficient is not on the permanent staff code table the time, conclude that then the code stream of current macro is wrong.

4, method according to claim 1 is characterized in that the error hidding method of on the spatial domain block-based cutting apart-mate, and concrete steps are as follows:

(1) position of the macro block of alignment error;

(2) judge " error macro block directly over adjacent macroblocks and under adjacent macroblocks " whether can use; If unavailable, find the nearest macro block that can be used for error concealing;

(3) similarity in the zone in zone and the available macroblock is calculated in the error macro block;

(4) finish and cut apart-matching process;

(5) calculate in the 4th step cut apart-whether two used zones of matching process are similar areas; If not, change (3), if change (6);

(6) finish error concealing;

(7) error concealing has been carried out in each zone of judging current error macro block; If not, change (3); If withdraw from.

5, method according to claim 4 is characterized in that the similarity of described two macro blocks is weighed with the absolute error average, and size is the image block B of bx * by _TAnd B _BBetween the average of absolute error calculate by following formula:

$\mathrm{MAD}=\frac{1}{\mathrm{bx}\&times;\mathrm{by}}\underset{x=0}{\overset{\mathrm{bx}}{\mathrm{\&Sigma;}}}\underset{y=0}{\overset{\mathrm{by}}{\mathrm{\&Sigma;}}}|B_T(x,y)-B_B(x,y)|$

6, method according to claim 1 is characterized in that camera lens cuts apart behind the real-time camera lens dividing method of employing based on motion vector hidden to the piece matching error.

Whether 7, method according to claim 6 is characterized in that when carrying out real-time camera lens when cutting apart, to the P frame, detect the macro block that does not have motion vector and have the ratio of macro block of forward motion vector unusual; Whether to the B frame, it is unusual with the ratio of the macro block that forward motion vector is arranged to detect the macro block that backward motion vector is arranged; To the I frame, whether the difference that detects I frame DC image occurs unusually.

8, method according to claim 7 is characterized in that detecting the unusual of P frame, B frame, J frame, adopts sliding window algorithm, and concrete steps are as follows:

In the above-mentioned steps, FrameNo is a frame number, and ShotCount is current camera lens counter, and W is that a length is 1 sliding window, w _iBe the element in the window, E (W) is the expectation of window; X represents a kind of of P frame, B frame, I frame type.

9, method according to claim 8 is characterized in that the frame for P, and X is taken as P, at this moment, and InitialEle=1000, l ∈ [8,10], TH_P=4; For the B frame, X is taken as B, at this moment, and InitialEle=1000, l ∈ [18,22], TH_B=6; For the I frame, X is taken as I, $\mathrm{InitialEle}=\frac{\mathrm{width}\&times;\mathrm{height}\&times;8}{256},$ l＝4，TH_I＝3。