JP3884003B2 - Image decoding device - Google Patents

Description

  The present invention relates to an image decoding apparatus for detecting an error partial image area affected by a bit error generated in an encoded bit stream in various image compression / decompression schemes.

  As a method for efficiently encoding an image signal, for example, as adopted in MPEG-4 (Moving Picture Experts Group Phase-4), which is being standardized by ISO / IEC JTC11 / SC29 / WG11 For each block, a discrete cosine transform (DCT: Discrete Cosine Transform) is performed, and after quantization, the obtained quantized transform coefficient is variable-length encoded.

  The bit stream encoded in this manner is subjected to variable length decoding for each block data on the decoding side, and then subjected to inverse quantization and inverse DCT to be restored as an image signal. At this time, for example, when a bit error occurs in the variable length codeword of this block data area, if the variable length codeword in which the error has occurred is a value defined in a predetermined variable length decoding table, an error Is decoded without being detected, so that an erroneous image signal is restored.

  Also, since each block data has a variable length, the block data delimiters cannot be determined until decoding. When a variable-length codeword is decoded in error, the block data delimiter is often erroneously determined, and the data after the next block is also decoded incorrectly. In this case, image quality degradation due to an error propagates after the next block.

Japanese Patent Laid-Open No. 7-154783 (paragraphs 0033, 0042-0045, FIG. 5)

  Since the conventional image decoding apparatus is configured as described above, for example, when a bit error occurs in the block data area, the decoding is continued until a value not defined in the variable length decoding table appears. Since the image decoded erroneously from the position where the error has occurred until the error is detected is significantly deteriorated, and when the image is decoded in the subsequent frames, the image is used as a predicted image. There was a problem that the decoded image of the subsequent frames also deteriorated.

  As a prior art related to the above problem, there is one disclosed in Patent Document 1. This is to detect an error by decoding the block of a predetermined size and calculating the amount of change between the peripheral pixel of the block and the peripheral pixel of the adjacent block and comparing it with a predetermined critical value. Yes. However, there are cases where it is difficult to reliably detect an error only with the amount of change between the peripheral pixels of the block and the peripheral pixels of the adjacent block.

  The present invention has been made to solve the above-described problems. Due to the statistical properties of image signals, blocks in which errors have occurred are reliably detected, and image quality degradation and error It is an object to obtain an image decoding apparatus that minimizes propagation.

An image decoding apparatus according to the present invention decodes an encoded bitstream generated by compressing and encoding an input image signal in units of partial image areas, and decodes the encoded bitstream in units of partial image areas. A partial image area data decoding means to be obtained, and an average value or a first representative value of the pixel level by dispersion in the decoded partial image area decoded by the partial image area data decoding means, and the decoding Obtaining an absolute difference between the second representative value of the pixel level due to the average value or variance of the pixel levels in the partial image region in the vicinity of the partial image region and the first representative value, for a plurality of neighboring partial image regions; When the sum of absolute differences for each of the neighboring partial image areas is larger than a predetermined threshold, the decoded partial image area is encoded bitstream. An error partial image region detecting means for detecting that an error partial image area affected by bit errors that have occurred, based on the decoded quantization step size than the encoded bit stream, the error partial image region The predetermined threshold when the detection means detects the error partial image area is set to be small when the quantization step size is large, and to the predetermined threshold when the quantization step size is small. Threshold control means for controlling the threshold value to increase is provided.

In the image decoding device according to the present invention, when the error partial image area detecting means has a non-uniform correlation between the decoded partial image area and each of the neighboring partial image areas, the absolute difference between the neighboring partial image areas having a strong correlation is obtained. preferentially have line weights the one in which the sum of the absolute differences for each partial image region of the neighborhood.

The present invention obtains a first representative value of a pixel level based on an average value or variance of pixel levels in a decoded partial image region, and determines a pixel level average value or variance of a pixel level in a partial image region near the decoded partial image region. When the absolute difference between the second representative value and the first representative value is obtained for a plurality of neighboring partial image areas, and the sum of the absolute differences for each neighboring partial image area is larger than a predetermined threshold value, Detecting that the decoded partial image area is an error partial image area, and controlling a predetermined threshold for detecting the error partial image area based on the quantization step size decoded from the encoded bitstream avoid allows early detection of occurrence of an error, it is possible to minimize the image degradation due to propagation or incorrect error, the erroneous determination of the generated macroblock error There is an effect that it is Rukoto.

  This invention weights each absolute difference based on the correlation between the decoded partial image area and each neighboring partial image area, and more reliably detects the occurrence of an error by considering the correlation with the neighboring block. There is an effect that can be done.

An embodiment of the present invention will be described below.
Embodiment 1 FIG.
In Embodiment 1 of the present invention, when an error occurs in a block data area of an encoded bit stream, a block decoded without detecting an error is determined based on continuity of image signals near the block boundary. It is to detect.

  FIG. 1 is a diagram showing the structure of a texture data area in an MPEG-4 simple profile encoded bitstream. In MPEG-4, texture data of an object to be encoded is encoded in units of blocks including a luminance component of 16 × 16 pixel size called “macroblock” and a color difference component of 8 × 8 pixel size. The luminance component of 16 × 16 pixel size is further divided into blocks of 8 × 8 pixel size, and DCT and quantization are performed in units of blocks of 8 × 8 pixel size. Here, an 8 × 8 pixel size block in the “macroblock” is hereinafter specifically referred to as a “block”.

  As shown in FIG. 1, the texture data area is composed of a plurality of macroblock data areas, and each macroblock data area includes macroblock data header information including macroblock coding type and motion vector information, It is composed of four luminance block data (Y) and two color difference block data (Cb / Cr). In this embodiment, when an error occurs in the macroblock data area, the macroblock in which the error has occurred is detected early.

  FIG. 2 is a block diagram showing the configuration of the image decoding apparatus according to the first embodiment. In FIG. 2, reference numeral 1 denotes a variable length decoding unit that inputs an encoded bitstream 101 and outputs decoded block data 102, a motion vector 103, and additional information 104. A motion compensation unit 2 receives the motion vector 103 and outputs a predicted image 107 using the reference image 106 stored in the memory 3. Reference numeral 4 denotes a switching unit that outputs a predicted image 107 or a 0 signal based on the additional information 104.

  In FIG. 2, reference numeral 5 denotes an inverse quantization unit which receives the decoded block data 102 and performs inverse quantization. Reference numeral 6 denotes an inverse DCT unit which receives the inversely quantized data 108, performs inverse DCT processing, and outputs a prediction error signal 109. Reference numeral 7 denotes an adder that adds the predicted image 107 or 0 signal from the switching unit 4 to the prediction error signal 109 and outputs a decoded image signal 110.

  Further, in FIG. 2, 8 is an error macroblock detection unit (error partial image region detection means), which receives the decoded image signal 110, determines whether an error has occurred, and displays an error macroblock determination result 111. Output. Reference numeral 9 denotes a switching unit that switches the output destination of the decoded image signal 110 based on the error macroblock determination result 111. An error macroblock correction unit 10 corrects the decoded image signal 110 in which an error has occurred, and outputs a corrected image signal 112. Reference numeral 11 denotes a partial image region data decoding means, which includes a variable length decoding unit 1, a motion compensation unit 2, a memory 3, a switching unit 4, an inverse quantization unit 5, an inverse DCT unit 6, and an addition unit 7. .

Next, the operation will be described.
The variable length decoding unit 1 extracts macroblock data from the encoded bit stream 101 and performs variable length decoding. The motion compensation unit 2 extracts the predicted image 107 with reference to the reference image 106 in the memory 3 based on the decoded motion vector 103. The inverse quantization unit 5 inversely quantizes the decoded decoded block data 102, and the inverse DCT unit 6 receives the inversely quantized data 108, performs inverse DCT processing, and outputs a prediction error signal 109.

  When the macro block type included in the additional information 104 decoded by the variable length decoding unit 1 indicates intra coding, the switching unit 4 selects the 0 signal and outputs it to the adding unit 7. When encoding other than intra encoding is selected, the predicted image 107 extracted by the motion compensation unit 2 is selected and output to the adding unit 7. The adding unit 7 adds the 0 signal or the predicted image 107 from the switching unit 4 to the prediction error signal 109 from the inverse DCT unit 6 and outputs a decoded image signal 110. Here, the decoded image signal 110 means a decoded macroblock, and is composed of four luminance blocks and two color difference blocks.

  The error macroblock detection unit (error partial image region detection means) 8 receives the decoded image signal 110, determines whether the error is a macroblock image signal in which an error has occurred, and outputs an error macroblock determination result 111. .

  The switching unit 9 switches the output of the decoded image signal 110 based on the error macroblock determination result 111 output from the error macroblock detection unit 8. If the error macroblock determination result 111 indicates that the decoded image signal 110 is an error macroblock, the decoded image signal 110 is output to the error macroblock correction unit 10. On the other hand, when the error macroblock determination result 111 indicates that the decoded image signal 110 is not an error macroblock, the decoded image signal 110 is output as it is and as a reference image 106 in the subsequent decoding processing. It is stored in the memory 3 for use.

  The error macroblock correction unit 10 performs a process of making the error macroblock inconspicuous and outputs a corrected image signal 112. As the simplest method, there is a method in which a macroblock located at the same position as a macroblock determined to have an error is extracted from the reference image 106 in the memory 3 and replaced with a macroblock in which an error has occurred. The corrected image signal 112 corrected by the error macroblock correction unit 10 is output as a decoded image and stored in the memory 3 for use as the reference image 106 in the subsequent decoding processing.

  FIG. 3 is a block diagram showing an internal configuration of the error macroblock detection unit (error partial image region detection means) 8 which is a feature of the first embodiment. In FIG. 3, reference numeral 21 denotes a horizontal direction difference calculation unit, which receives a decoded image signal 110 and a horizontal block signal 121, and changes in pixel level between adjacent pixels across the boundary between adjacent blocks in the horizontal direction. Then, the amount of change in the pixel level between the pixels in the block adjacent in the horizontal direction is obtained and output as the horizontal pixel difference 123.

  In FIG. 3, reference numeral 22 denotes a vertical direction difference calculation unit that inputs a decoded image signal 110 and a vertical block signal 122, and sets the pixel level between adjacent pixels across the boundary between adjacent blocks in the vertical direction. The change amount and the change amount of the pixel level between the pixels in the block adjacent in the vertical direction are obtained and output as the vertical pixel difference 124.

  Further, in FIG. 3, reference numeral 23 denotes a horizontal direction continuity determination unit which inputs a horizontal inter-pixel difference 123, determines continuity with a block in the horizontal direction, and outputs a horizontal direction continuity 125. Reference numeral 24 denotes a vertical direction continuity determination unit which inputs the vertical direction inter-pixel difference 124 to determine continuity with a block in the vertical direction and outputs a vertical direction continuity 126.

  Further, in FIG. 3, reference numeral 25 denotes a block continuity determination unit that inputs horizontal continuity 125 and vertical continuity 126, determines continuity with adjacent blocks, and outputs adjacent block continuity 127. . Reference numeral 26 denotes an error macroblock determination unit that inputs adjacent block continuity 127, determines whether it is an error macroblock, and outputs an error macroblock determination result 111.

Next, the operation of the error macroblock detection unit 8 will be described.
FIG. 4 is a flowchart showing the operation of the error macroblock detection unit 8. The error macroblock detection unit 8 determines whether the decoded image signal 110 is an image signal of a macroblock in which an error has occurred based on the continuity of the image signal. Usually, an image signal has a large correlation with neighboring pixels, and pixel values continuously change within the same object. However, the image signal of a block in which an error has occurred often cannot maintain continuity with the image signal of an adjacent block. In this embodiment, a macroblock in which an error has occurred is detected using continuity with adjacent blocks.

FIG. 5 is a diagram for explaining processing of the horizontal direction difference calculation unit 21 and the vertical direction difference calculation unit 22. In step ST1 of FIG. 4, the horizontal direction difference calculation unit 21 receives the decoded image signal 110 from the determination target block (m, n) in FIG. 5 and the horizontal block (m, n−1) in FIG. The horizontal block signal 121 is input, and for each of the color difference block and the luminance block of the decoded image signal 110, three columns (P _{m, n} ) near the block boundary in the block (m, n−1) in FIG. ₋₁ (i, 5) to P _{m, n−1} (i, 7)) and one column (P _{m, n} (i, 0)) near the block boundary of the determination target block (m, n) , The amount of change (difference value) between adjacent pixels is calculated and output as a horizontal pixel difference 123.

The amount of change (difference value) between adjacent pixels is obtained by the following equation (1).
DP _{m, n-1} (i, j + 1) = P _{m, n-1} (i, j) -P _{m, n-1} (i, j + 1)
(I = 0, 1,..., 7, j = 5, 6)
DP _{m, n} (i, 0) = P _{m, n−1} (i, 7) −P _{m, n} (i, 0)
(I = 0, 1, ..., 7) (1)

In step ST2, the vertical direction difference calculation unit 22 decodes the decoded image signal 110 from the determination target block (m, n) and the vertical block signal 122 from the vertical block (m-1, n) in FIG. For each of the color difference block and the luminance block of the decoded image signal 110, three rows of pixels (P _{m−1, n} (5, j) near the block boundary in the block (m−1, n) in FIG. 5). ) To P _{m-1, n} (7, j)) and one row of pixels (P _{m, n} (0, j)) near the block boundary of the determination target block (m, n) A difference value is calculated and output as a vertical pixel difference 124.

  In step ST3 of FIG. 4, the horizontal continuity determination unit 23 changes the amount of pixel change (difference value) near the block boundary calculated by the horizontal direction difference calculation unit 21, that is, the horizontal direction represented by the above equation (1). Based on the inter-pixel difference 123, it is determined whether or not the decoded block (m, n) is correlated with the left adjacent block (m, n-1), and the horizontal continuity 125 is output. . If there is no correlation with the left adjacent block, the pixels near the boundary with the left adjacent block are discontinuous.

When the pixel at the block boundary becomes discontinuous, it is compared with the change amount (difference value) of the adjacent pixel levels of P _{m, n-1} (i, 5) to P _{m, n-1} (i, 7). Thus, the amount of change (difference value) in pixel level from P _{m, n-1} (i, 7) to P _{m, n} (i, 0) increases. Therefore, the discontinuity is determined based on the two types of pixel level changes (difference values). First, according to the following equation (2), a change amount (difference value) DP _{m, n-1} between adjacent pixels of P _{m, n-1} (i, 5) and P _{m, n-1} (i, 6). (I, 6) and the change amount (difference value) DP _{m, n-1} (i, 6) between adjacent pixels of P _{m, n-1} (i, 6) and P _{m, n-1} (i, 7). The amount of change (difference value) ΔP _{m, n-1} (i, 7) from 7) is obtained.
ΔP _{m, n−1} (i, 7) = DP _{m, n−1} (i, 6) −DP _{m, n−1} (i, 7)
(I = 0, 1, ..., 7) (2)

Similarly, the amount of change (difference value) DP _{m, n−} between adjacent pixels of P _{m, n−1} (i, 6) and P _{m, n−1} (i, 7) is _expressed by the following equation (3). ₁ (i, 7), P _{m, n-1} (i, 7) and P _{m, n} (i, 0) change amounts (differential values) DP _{m, n} (i, 0) between adjacent pixels Change amount (difference value) ΔP _{m, n} (i, 0) is obtained.
ΔP _{m, n} (i, 0) = DP _{m, n−1} (i, 7) −DP _{m, n} (i, 0)
(I = 0, 1, ..., 7) (3)

Next, (2) change amount obtained by equation (difference _{value) ΔP m, n-1 (} i, 7) and (3) the amount of change obtained by the equation (difference value) [Delta] P _{m, n} (i, 0) is determined based on a predetermined threshold value TH1 to determine discontinuity. That is, when the following equation (4) is satisfied with respect to a predetermined threshold TH1, a pixel in the i-th row is determined to be discontinuous.
│ΔP _{m, n} (i, 0) │-│ΔP _{m, n-1} (i, 7) │> TH1 (4)
When it is determined that all pixels on the block boundary are discontinuous, the block is discontinuous with the horizontal block.

  In step ST4 of FIG. 4, the vertical continuity determining unit 24, like the horizontal continuity determining unit 23, changes in pixels (difference values) in the vicinity of the block boundary calculated by the vertical direction difference calculating unit 22, That is, based on the vertical inter-pixel difference 124, it is determined whether or not the decoded block has a correlation with the upper block, and the vertical continuity 126 is output.

  In step ST5, the block continuity determination unit 25 determines whether the decoded block is continuous with the adjacent block on the left based on the horizontal continuity 125 that is the determination result of the horizontal direction continuity determination unit 23. As a result of the determination, if the decoded block is continuous with the adjacent block on the left, the block continuity determination unit 25 determines that it is continuous with the adjacent block in step ST8.

  If the result of determination in step ST5 is that the decoded block is not continuous with the adjacent block on the left, in step ST6, the block continuity determination unit 25 determines the vertical continuity that is the determination result of the vertical continuity determination unit 24. Based on the property 126, it is determined whether the decoded block is continuous with the upper block. As a result of the determination, if the decoded block is continuous with the upper block, in step ST8, the block continuity determination unit 25 determines that it is continuous with the adjacent block.

  If it is determined in step ST6 that the decoded block is not continuous with the upper block, in step ST7, the block continuity determining unit 25 determines that the block is discontinuous with the adjacent block. As described above, when both the horizontal direction and the vertical direction are discontinuous, it is determined that the adjacent block is discontinuous.

  In step ST9, when the determination of continuity with adjacent blocks is completed for all luminance blocks and color difference blocks, the block continuity determination unit 25 outputs the adjacent block continuity 127 as a determination result. In step ST <b> 10, the error macroblock determination unit 26 determines whether it is an error macroblock based on the adjacent block continuity 127.

  The error macroblock determination unit 26 determines whether or not the decoded macroblock is an error macroblock based on continuity with the neighboring blocks of the luminance block and the color difference block. For example, when discontinuity is detected in any one of two color difference blocks or four luminance blocks, a macroblock including that block is determined as a macroblock in which an error has occurred. It can also be determined by a combination of discontinuity determination of two color difference blocks and four luminance blocks.

In the first embodiment, three columns of pixels (P _{m, n-1} (i, 5) to P _{m, n-1} ( _Pn , n-1) (Pm, n-1) near the block boundary of the adjacent horizontal block (m, n-1) are used. i, 7)) and a difference value between adjacent pixels in a pixel column (P _{m, n} (i, 0)) in the vicinity of the block boundary of the determination target block, the error block determination is performed. The number of columns and the number of pixel columns is not particularly limited, and it is also possible to check the amount of change (difference value) between pixels in a wider range and use it for error block determination. Further, in this embodiment, the amount of change (difference value) between adjacent pixels is used in FIG. 5, but it is also possible to take the amount of change (difference value) every other pixel. These can also be applied to adjacent vertical blocks (m−1, n).

Furthermore, in this embodiment, in step ST3, the horizontal direction continuity determination unit 23 calculates the change amount ΔP _{m, n−1} (i, 7) obtained by the above equation (2) and the above equation (3). The obtained amount ΔP _{m, n} (i, 0) is used to determine the continuity in the horizontal direction by comparing with the predetermined threshold value TH1 by the above equation (4). The continuity in the horizontal direction may be determined by comparing the amount of change ΔP _{m, n} (i, 0) obtained by the above equation (3) with another predetermined threshold value TH1a without using the equation. This can also be applied to the vertical direction continuity determination unit 24.

  As described above, according to the first embodiment, an error is detected at an early stage by detecting a macroblock in which an error has occurred by utilizing the continuity of image signals with adjacent blocks. Image quality degradation due to propagation and errors can be minimized, and not only the amount of change between the block to be decoded and the neighboring pixels of the adjacent block, but also the amount of change in the pixels in the adjacent block As a result, it is possible to reliably detect a macroblock in which an error has occurred.

Embodiment 2. FIG.
The block diagram showing the configuration of the image decoding apparatus in the second embodiment of the present invention is the same as that in FIG. 2 in the first embodiment. In this embodiment, the threshold values in the horizontal continuity determination unit 23 and the vertical continuity determination unit 24 of the error macroblock detection unit 8 described in the first embodiment are set to the quantization step size of each macroblock. Based on this, it is switched adaptively.

  As described in Embodiment 1, when quantization is performed after orthogonal transformation such as DCT in units of blocks, noise specific to orthogonal transformation coding called block distortion occurs. Even with this block distortion, the amount of change (difference value) between pixels at the block boundary may increase. Therefore, the error macroblock detection unit 8 in the second embodiment is provided with a threshold control unit that adaptively controls the threshold in consideration of the influence of block distortion.

  FIG. 6 is a block diagram showing an internal configuration of the error macroblock detection unit (error partial image region detection means) 8 according to the second embodiment. In the figure, reference numeral 27 denotes a threshold control unit, which controls the threshold 128 based on the quantization step size of the macroblock included in the additional information 104 decoded by the variable length decoding unit 1 of FIG. To do. The operations of the horizontal direction difference calculating unit 21, the horizontal direction continuity determining unit 23, the vertical direction difference calculating unit 22, the vertical direction continuity determining unit 24, the block continuity determining unit 25, and the error macroblock determining unit 26 are described in the first embodiment. It is the same.

Next, the operation will be described.
FIG. 7 is a flowchart showing the operation of the threshold control unit 27 according to the second embodiment. The threshold control unit 27 controls the threshold 128 based on the quantization step size of the macroblock included in the additional information 104. In step ST11, the threshold control unit 27 confirms whether the quantization step size of the macroblock included in the additional information 104 is larger than a predetermined value Q1, and if it is larger than Q1, near the block boundary due to block distortion. Since the inter-pixel change amount (difference value) may become large, in step ST12, the threshold value control unit 27 also sets the discontinuity determination threshold value to a large value THa accordingly.

  In step ST11, when the quantization step size is smaller than the predetermined value Q1, in step ST13, the threshold value control unit 27 confirms whether the quantization step size is larger than the predetermined value Q2 (Q2 <Q1). If larger than Q2, in step ST14, the threshold value control unit 27 sets the discontinuity determination threshold value to a value THb smaller than THa accordingly.

  Hereinafter, similarly, the processes of steps ST15, ST16, and ST17 are performed, and when the quantization step size is larger than a predetermined value Q3 (Q3 <Q2 <Q1), the threshold value is set to THc, and is smaller. Since the influence of block distortion is small, the threshold value is set to a small value THd (THd <THc <THb <THa) accordingly.

  In the process of FIG. 7, the range of the quantization step size is divided into four stages, and the threshold value is set according to each value. However, the threshold value is not limited to four stages, but adaptively according to the quantization characteristics. Can be set.

  In FIG. 6, the threshold value 128 set by the threshold value control unit 27 is output to the horizontal direction continuity determining unit 23 and the vertical direction continuity determining unit 24, and the horizontal block is the same as in the first embodiment. This is used as a threshold value for determining continuity between the signal 121 and the block signal 122 in the vertical direction.

  As described above, according to the second embodiment, the threshold value used when determining the continuity with adjacent blocks is controlled by the quantization step size, so that the discontinuity at the block boundary due to the block distortion. And the discontinuity at the block boundary due to errors can be distinguished from each other, and it is possible to avoid erroneously discriminating a macroblock in which block distortion has occurred as a macroblock in which an error has occurred.

Embodiment 3 FIG.
The block diagram showing the configuration of the image decoding apparatus in the third embodiment of the present invention is the same as that in FIG. 2 in the first embodiment. In the first embodiment, the discontinuity between blocks is determined based on the inter-pixel difference signal near the block boundary and the error macroblock is detected. In this embodiment, the representative pixel level calculated for each block is used. An error macroblock is detected based on the value.

  FIG. 8 is a block diagram showing an internal configuration of the error macroblock detection unit (error partial image region detection means) 8 which is a feature of this embodiment. In FIG. 8, reference numeral 31 denotes an intra-block pixel average value calculation unit that receives the decoded image signal 110, obtains an intra-block pixel average value as a representative value of the pixel level for each block, and outputs it as a pixel level average value 131. Reference numeral 32 denotes a pixel average difference calculation unit that inputs a pixel level average value 131 and a neighboring block pixel level average value 132 that is a representative value of a pixel level of a neighboring block stored in a predetermined memory (not shown). The difference is taken and the pixel average difference 133 is output. Reference numeral 33 denotes an error macroblock determination unit, which inputs the pixel average difference 133, determines whether it is an error macroblock, and outputs an error macroblock determination result 111.

Next, the operation will be described.
FIG. 9 is a flowchart showing the operation of the error macroblock detection unit 8 according to the third embodiment. The error macroblock detection unit 8 performs the following processing for each of the color difference block and the luminance block of the decoded image signal 110. In step ST21, the pixel average value calculation unit 31 in the block receives the decoded image signal 110 and calculates the average value of the pixel level in the block according to the following equation (5).

In step ST22, the pixel average difference calculation unit 32 calculates the average value 131 (AveP _{m, n} ) of the decoded block pixel level calculated by the intra-block pixel average value calculation unit 31 according to the following equation (6) and the vicinity thereof. The pixel average difference 133 (ΔAveP _{m, n, s, t} ) with the average value 132 (AveP _{s, t} ) of the pixel level in the block is calculated.
ΔAveP _{m, n, s, t} = AveP _{s, t} −AveP _{m, n} (6)
FIG. 10 is a diagram showing neighboring blocks that are targets of difference calculation. For example, there are four blocks as shown in the figure. In the above equation (6), s and t are the neighboring blocks (m, n−1), (m−1, n−1), (m−1, n), (m−1, n + 1) in FIG. Is shown.

  The error macroblock determination unit 33 determines an error macroblock by using the pixel average difference 133 with the neighboring block calculated by the pixel average difference calculation unit 32 in the processing after step ST23, and the error macroblock determination result 111 is output.

As an error macroblock determination method, in step ST23, the error macroblock determination unit 33 calculates, for example, the absolute difference of the pixel average of each of the chrominance block and the luminance block with the neighboring block by the following equation (7). Then, a value C _{m, n} is obtained by adding all absolute differences of pixel averages with neighboring blocks.
C _{m, n} = │ΔAveP _{m, n, m-1, n-1} │ + │ΔAveP _{m, n, m-1, n} │
+ │ΔAveP _{m, n, m-1, n + 1} │ + │ΔAveP _{m, n, m, n-1} │ (7)

Next, in step ST24, the error macroblock determination unit 33 compares the value C _{m, n} obtained by the equation (7) with a predetermined threshold value TH2 based on the following equation (8). Determine the error macroblock. That is, when the following equation (8) holds for a predetermined threshold TH2, the block is determined as an error block.
C _{m, n} > TH2 (8)

  In step ST25, the error macroblock determination unit 33 determines that the macroblock including the error block is an error macroblock.

  If the above equation (8) is not satisfied in step ST24, it is checked in step ST26 whether or not error determination has been performed for all blocks. If all blocks are determined to be not error blocks, In step ST27, the error macroblock determination unit 33 determines that the decoded image signal 110 is a normal macroblock.

Further, when calculating the sum of absolute differences of pixel averages with neighboring blocks, each absolute difference value can be weighted and added according to the following equation (9).
C _{m, n} = w ₁ │ΔAveP _{m, n, m-1, n-1} │ + w ₂ │ΔAveP _{m, n, m-1, n} │
+ W ₃ │ΔAveP _{m, n, m-1, n + 1} │ + w ₄ │ΔAveP _{m, n, m, n-1} │ (9)
Thereby, when the correlation with the neighboring block is not uniform, the difference with the block that seems to have a strong correlation can be preferentially handled.

  In this embodiment, the average value of the pixel level is used as the representative value of the pixel level of the block, but dispersion of the pixel level may be used.

  As described above, according to the third embodiment, since a representative value is calculated for each decoded image block and compared with a representative value of a neighboring block, an erroneous macroblock can be detected. Can be detected at an early stage to minimize the propagation of errors and image quality degradation due to errors.

Embodiment 4 FIG.
The block diagram showing the configuration of the image decoding apparatus in the fourth embodiment of the present invention is the same as that in FIG. 2 in the first embodiment. In this embodiment, the threshold value in the error macroblock determination unit 33 of the error macroblock detection unit 8 described in the third embodiment is adaptively controlled based on the quantization step size of each macroblock. is there.

  As described in the first embodiment, when quantization is performed after orthogonal transformation such as DCT in units of blocks, when the quantization step size is large, the amplitude of the image signal is smoothed to reduce the amplitude. Therefore, the average value for each block may be small when the quantization step size is large. Therefore, the error macroblock detection unit 8 in this embodiment is provided with a threshold value control unit that controls the threshold value in consideration of the influence of the quantization step size.

  FIG. 11 is a block diagram showing the internal configuration of the error macroblock detection unit (error partial image region detection means) 8 in the fourth embodiment. In FIG. 11, reference numeral 34 denotes a threshold control unit, which sets the threshold 134 based on the quantization step size of the macroblock included in the additional information 104 decoded by the variable length decoding unit 1 of FIG. Set. The operations of the pixel average value calculation unit 31, the pixel average difference calculation unit 32, and the error macroblock determination unit 33 are the same as those in the third embodiment.

Next, the operation will be described.
The threshold control unit 34 controls the threshold 134 based on the quantization step size of the macroblock included in the additional information 104. For example, when the quantization step size is large, the amplitude of the image signal is small, and the average value may be small. Therefore, the threshold value is set to a small value. When the quantization step size is small, the amplitude of the image signal is maintained and the average value does not become small, so the threshold value is set to a large value.

  The threshold value 134 set by the threshold value control unit 34 is output to the error macroblock determination unit 33 and is used as a threshold value when determining an error macroblock, as in the third embodiment.

  As described above, according to the fourth embodiment, by controlling the threshold value used when determining an error macroblock based on the quantization step size, erroneous determination on a macroblock in which an error has occurred is avoided. The effect of being able to be obtained.

Embodiment 5 FIG.
The block diagram showing the configuration of the image decoding apparatus in the fifth embodiment of the present invention is the same as that in FIG. 2 in the first embodiment. In this embodiment, an error macroblock is determined by combining the error macroblock determination methods described in the first to fourth embodiments.

  FIG. 12 is a block diagram showing an internal configuration of the error macroblock detection unit (error partial image region detection means) 8 according to the fifth embodiment. In FIG. 12, reference numeral 41 denotes a threshold value control unit, which is based on the quantization step size of the macroblock included in the additional information 104 decoded by the variable length decoding unit 1 in FIG. 141 and a predetermined threshold value 142 are set. An error macroblock determination unit 42 determines an error macroblock based on the adjacent block continuity 127 and the pixel average difference 133.

  In FIG. 12, the horizontal direction difference calculating unit 21, the horizontal direction continuity determining unit 23, the vertical direction difference calculating unit 22, the vertical direction continuity determining unit 24, the block continuity determining unit 25, the intra-block pixel average value calculating unit 31, The pixel average difference calculation unit 32 is the same as that in the first and third embodiments.

Next, the operation will be described.
The operation of threshold control unit 41 is the same as in the second and fourth embodiments. FIG. 13 is a flowchart showing the operation of the error macroblock determination unit 42 according to the fifth embodiment. The error macroblock determination unit 42 determines an error macroblock based on the adjacent block continuity 127 for each color difference block and luminance block and the pixel average difference 133. In step ST31, the error macroblock determination unit 42 determines the continuity with adjacent blocks based on the adjacent block continuity 127 in order from the color difference block to the luminance block.

  If the result of determination in step ST31 is that it is continuous with adjacent blocks, in step ST35, the error macroblock determination unit 42 checks whether error determination has been performed for all blocks, and determines for all blocks. If not, the process returns to step ST31 to check the continuity with the adjacent block for the next block.

  On the other hand, if the result of determination in step ST31 is discontinuous with adjacent blocks, in step ST32, the error macroblock determination unit 42 is based on the pixel average difference 133 as described in the third embodiment. Then, it is determined whether it is an error block.

  Here, if it is determined in step ST33 that the block is an error block, in step ST34, the error macroblock determination unit 42 determines that the macroblock including the block is an error macroblock, and sets the error macroblock determination result 111 as the error macroblock determination result 111. It outputs to the switch 9 shown in FIG.

  If the error block is not determined in step ST33, the above determination is performed for all the blocks in step ST35. If no error block is determined for any block, the error block is determined in step ST36. The error macroblock determination unit 42 determines that the block is a normal macroblock and outputs an error macroblock determination result 111 to the switch 9.

  In step ST36, it is determined that the macro block is a normal macro block when it is determined in step ST31 that there is continuity with an adjacent block, or in step ST33, it is not determined as an error block by the pixel average difference. If it is determined in step ST31 that there is continuity with adjacent blocks, and if it is not determined as an error block based on the pixel average difference in step ST33, it may be determined as a normal macroblock in step ST36.

  In this embodiment, the threshold value control unit 41 controls the predetermined threshold value 141 and the predetermined threshold value 142 based on the quantization step size of the macroblock. As in the third embodiment, the control may not be performed based on the quantization step size.

  As described above, according to the fifth embodiment, since an error macroblock is determined based on a plurality of determination methods, a macroblock in which an error has occurred can be determined more reliably, and erroneous determination can be avoided. The effect that it can be obtained.

  Also, the determination process is hierarchically performed from the color difference block to the luminance block, and when a certain block is determined to be an error block, the macro block including the block is determined to be an error macro block, and the subsequent block determination process is performed. Therefore, there is an effect that the calculation amount for the determination process can be reduced.

Embodiment 6 FIG.
In the first to fifth embodiments, the image decoding apparatus using the MPEG-4 encoding method for moving images has been described. However, the present invention can also be applied to still images. In addition, the coding method for performing DCT and quantization in units of blocks has been described. However, the present invention is not limited to this, and is applicable to all methods of performing coding in units of blocks, such as when orthogonal transforms other than DCT are used. It is possible.

  Furthermore, the continuity of neighboring blocks was examined in units of 8 × 8 pixels. However, the present invention is not limited to this. For example, continuity with neighboring macroblocks in units of 16 × 16 pixels is determined. You can also investigate.

  Furthermore, when coding not only in block units but also in arbitrary-shaped area units, the continuity with neighboring areas can be examined in the same way, and is applicable to all methods of encoding in arbitrary-shaped area units. Is possible.

Embodiment 7 FIG.
The block diagram showing the configuration of the image decoding apparatus in the seventh embodiment of the present invention is the same as that in FIG. 2 in the first embodiment. In Embodiments 1 to 6, the macroblock in which a bit error has occurred is detected using the continuity of the image signal in the spatial direction, but this embodiment uses the correlation of the image signal in the time direction. Thus, a macro block in which a bit error has occurred is detected.

  When an image is encoded, if there is little motion, inter-frame predictive encoding using the correlation between frames is performed. A typical method is motion compensation prediction. A block having the smallest difference signal is searched from a past frame in a predetermined range in a block unit of a predetermined image size, and a block and a code as a result of the search are searched. A position shift from the encoding target block is used as a motion vector, and the motion vector and a difference signal (prediction error signal) are encoded and multiplexed into an encoded bit stream.

  If there are similar blocks in the past frames, the variance of the prediction error signal becomes small, so that encoding can be performed efficiently. On the other hand, when there is no similar block in the past frame, since the variance of the prediction error signal becomes large, encoding (intra coding) is performed only with information in the frame. On the decoding device side, as described in the first embodiment, the motion vector 103 and the prediction error signal 109 are decoded from the encoded bitstream 101, the prediction image 107 is extracted based on the decoded motion vector 103, and the decoding is performed. The decoded image signal 110 is restored by adding to the prediction error signal 109 that has been converted.

  In this embodiment, when the variance of the prediction error signal 109 increases on the encoding device side, the prediction error signal 109 decoded on the decoding device side is used by utilizing the fact that intra coding is selected. The macro block in which a bit error has occurred is detected by thresholding the variance of the.

  FIG. 14 is a block diagram showing an internal configuration of the error macroblock detection unit (error partial image region detection means) 8 according to the seventh embodiment. In the figure, 51 is a prediction error signal variance calculation unit, which obtains the variance of the prediction error signal 109 and outputs a prediction error signal variance 151. A decoded image signal variance calculation unit 52 obtains the variance of the decoded image signal 110 and outputs a decoded image signal variance 152. 53 is an error macroblock determination unit that determines whether an error macroblock is based on the prediction error signal variance 151 and the decoded image signal variance 152 and outputs an error macroblock determination result 111.

Next, the operation will be described.
FIG. 15 is a flowchart showing the operation of the error macroblock detection unit 8. The error macroblock detection unit 8 in this embodiment is applied only to inter macroblocks, and is not applied to intra macroblocks.

  In step ST41, it is determined whether it is an inter macro block or an intra macro block based on the additional information 104 decoded by the variable length decoding unit 1 of FIG. In the case of an intra macroblock, any of the methods in the first to fifth embodiments can be used. In the case of an inter macro block, an error macro block is determined by subsequent processing.

In step ST42, the prediction error signal variance calculation unit 51 obtains the variance of the decoded prediction error signal 109 by the following equation (10) and outputs the prediction error signal variance 151. Here, σ ² (P) is a prediction error signal variance, p _{i, j} is a pixel value of i row and j column of a prediction error block of N × N pixel size, and m _p is a pixel value of N × N pixel size. The average is shown.

In step ST43, the decoded image signal variance calculation unit 52 obtains the variance of the decoded image signal 110 by the following equation (11) and outputs the decoded image signal variance 152. Here, σ ² (D) is a decoded image signal variance, d _{i, j} is a pixel value of i row and j column of a decoding block of N × N pixel size, and m _d is a pixel value of N × N pixel size. The average is shown.

In step ST44, the error macroblock determination unit 53 determines the prediction error signal variance 151 obtained by the prediction error signal variance calculation unit 51, the decoded image signal variance 152 obtained by the decoded image signal variance calculation unit 52, and a predetermined threshold. Comparison is made by the following equation (12) using the value TH3.
σ ² (P)> σ ² (D) + TH3 (12)

In step ST44, if the prediction error signal variance σ ² (P) is larger than the decoded image signal variance σ ² (D) plus a predetermined threshold TH3, in step ST45, an error macroblock determination unit 53 determines that the macro block is an error macro block.

If the prediction error signal variance σ ² (P) is not larger than the value obtained by adding the predetermined threshold value TH3 to the decoded image signal variance σ ² (D) in step ST44, an error macroblock determination is made in step ST46. The unit 53 determines that the macro block is a normal macro block.

In this embodiment, the prediction error signal variance σ ² (P) and the decoded image signal variance σ ² (D) are the representative values of the prediction error signal 109 and the decoded image signal 110, respectively. It is good as a value. Note that the error macroblock can also be determined by combining the method described in this embodiment and the method described in the first to fifth embodiments.

  As described above, according to the seventh embodiment, by comparing the prediction error signal variance 151 and the decoded image signal variance 152, the error macroblock is detected using the correlation of the image signal in the time direction. Thus, it is possible to detect errors early and to minimize the propagation of errors and image quality degradation due to errors.

Embodiment 8 FIG.
Embodiment 8 of the present invention is an image decoding apparatus that receives an encoded bit stream from a transmission path or a recording medium, decodes and displays an image, detects the occurrence frequency of bit errors, and generates the occurrence frequency. If the error macroblock detection unit is determined to be high, the error macroblock detection unit is activated to reduce malfunctions of the error macroblock detection unit and to stably suppress image quality degradation due to decoding errors.

  The error macroblock detection unit may detect an image signal before detecting a fatal error for image data in which decoding is impossible in the process of syntax analysis or illegal data is decoded. Since the error is estimated and detected based on the stationary nature of the error, there is a possibility of false detection. Therefore, in this embodiment, only in the case of a reception situation in which bit errors frequently occur, the error macroblock detection unit is activated to increase tolerance against errors and perform a total and stable decoding operation. .

  FIG. 16 is a block diagram showing a configuration of an image decoding apparatus according to the eighth embodiment. In the figure, 61 is an error monitoring unit, which receives a media packet sequence 201 and outputs a bit error count value 202. 62 is an error detection activation control unit, which controls the activation of the error macroblock detection unit 8 by inputting the bit error count value 202 and outputting the error detection activation instruction flag 203. Reference numeral 63 denotes a switching unit which causes the error macroblock detection unit 8 to input the decoded image signal 110 from the addition unit 7 by the error detection activation instruction flag 203. Reference numeral 71 denotes a monitoring unit, which includes an error monitoring unit 61 and an error detection activation control unit 62. The other configuration is the same as the configuration of FIG. 2 in the first embodiment.

  FIG. 17 is a diagram showing the configuration of the media packet sequence 201. A media packet is a digital data unit that packs various media such as video data, audio data that is decoded and presented in synchronization with it, text, graphics, and still images according to a predetermined rule. It is defined as For example, MPEG-2 (ISO / IEC 13818-1) Packetized Elementary Stream (PES), ITU-TH. 223 AL-PDU etc. are mentioned.

  In FIG. 17, a packet header is various header information describing packet attributes, media types, etc., media data is unique encoded data such as video, audio, data, etc., and an error detection code is: It is an additional bit for error detection such as CRC (Cyclic Redundancy Check). It is transmitted in the form of a unified media packet as shown in FIG. 17, and only the encoded image data (encoded bit stream 101) is supplied to the variable length decoding unit 1 in FIG.

Next, the operation will be described.
The operations after the partial image area data decoding unit 11 and the error macroblock detection unit 8 are the same as those in the first embodiment. The error monitoring unit 61 inputs the media packet sequence 201, monitors the bit error occurrence frequency by the error detection code included therein, counts the number of detected bits, and sets the bit error count value 202 as the bit error count value 202. Output.

  The error detection activation control unit 62 checks the bit error count value 202 that is the error detection result from the error monitoring unit 61 at a predetermined unit time interval, and the bit error count value 202 at the predetermined unit time interval When the threshold value is exceeded, it is determined that the reception status is poor (the bit error rate of the transmission path is large, reading from the storage medium is unstable, etc.), and the error detection activation instruction flag 203 is output. On the other hand, if the bit error count value 202 in a predetermined unit time interval does not exceed a predetermined threshold, it is determined that the reception status is good, and the error detection activation instruction flag 203 is not output.

  When the error detection activation instruction flag 203 from the error detection activation control unit 62 is input, the switching unit 63 inputs the decoded image signal 110 from the addition unit 7 to the error macroblock detection unit 8, and the error macroblock detection unit 8 Start up. On the other hand, when the error detection activation instruction flag 203 from the error detection activation control unit 62 is not input, the switching unit 63 does not activate the error macroblock detection unit 8 and outputs the decoded image signal 110 from the addition unit 7 as it is. .

  The error macroblock detection unit 8 detects a macroblock in which an error has occurred in various configurations from the first embodiment to the seventh embodiment, and the error macroblock correction unit 10 corrects the lost image area.

  FIG. 18 is a block diagram showing the internal configuration of the error monitoring unit 61. The bit error detection unit 81 receives the media packet sequence 201, detects a bit error in the packet using an error detection code added to each packet, and outputs a bit error count value 211 for each packet. The bit error counter 82 accumulates the bit error count value 211 in units of packets from the bit error detection unit 81 for each media packet, and outputs the accumulated bit error count value 202 to the error detection activation control unit 62. As described above, the error monitoring unit 61 monitors the transition of the bit error count value 202, thereby generating a bit error occurrence frequency during transmission on the transmission line and a bit error occurrence frequency during reading from the storage medium. To monitor.

  As a monitoring method of the error monitoring unit 61, in addition to the method based on the above error detection code, the received electric field strength of the radio wave carrying the media packet sequence and the phase information after the delay detection of the radio wave may be monitored. In addition, the status of bit errors in other media packets such as audio may be monitored, and the monitoring result may be taken into consideration.

  Regarding the control in the error detection activation control unit 62, various control methods are conceivable in addition to the on / off control described above. For example, in the second embodiment and the fourth embodiment, the threshold value is controlled based on the quantization step size of the macroblock included in the additional information 104, but the bit error count value 202 from the error monitoring unit 61 is controlled. The threshold value may be controlled based on the above.

  That is, when the bit error count value 202 is large, the reception situation is poor and the probability of an error occurring is high, so the threshold value is set small. On the other hand, when the bit error count value 202 is small, the reception situation is stable and the probability that an error will occur is low, so the threshold value is set large. Thus, by setting a threshold value according to the output result of the error monitoring unit 61, the processing of the error macroblock detection unit 8 is controlled, and when the reception status is good, the error macroblock detection unit 8 False detection can be avoided.

  As another control method of the error detection activation control unit 62, the continuity evaluation range can be changed. In the first embodiment, continuity is determined in units of 8 × 8 pixel size for each luminance block and chrominance block, and when at least one block is determined to be discontinuous, a macro including the block is included. Although the block is determined to be an error macroblock, if it is determined that the reception status is stable based on the output result of the error monitoring unit 61, the error occurrence probability is low. In the case of a luminance block, discontinuity may be determined in units of 16 × 16 pixel macroblocks. In addition, the number of pixels to be evaluated for the inter-pixel difference used in the determination may be increased.

  As described above, according to the eighth embodiment, the error macroblock detection unit 8 is adaptively operated according to the line quality of the transmission path, the reliability of reading from the storage medium, and the like. Is good, it is possible to avoid erroneous detection by the error macroblock detection unit 8 and to obtain a total and stable decoding operation.

It is a figure which shows the structure of the texture data area | region of the encoding bit stream of an MPEG-4 simple profile. It is a block diagram which shows the structure of the image decoding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the internal structure of the error macroblock detection part by Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the error macroblock detection part by Embodiment 1 of this invention. It is a figure explaining the process of the horizontal direction difference calculation part and vertical direction difference calculation part by Embodiment 1 of this invention. It is a block diagram which shows the internal structure of the error macroblock detection part by Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the threshold value control part by Embodiment 2 of this invention. It is a block diagram which shows the internal structure of the error macroblock detection part by Embodiment 3 of this invention. It is a flowchart which shows operation | movement of the error macroblock detection part by Embodiment 3 of this invention. It is a figure which shows the block of the vicinity used as the object of the difference calculation by Embodiment 3 of this invention. It is a figure which shows the internal structure of the error macroblock detection part in Embodiment 4 of this invention. It is a figure which shows the internal structure of the error macroblock detection part by Embodiment 5 of this invention. It is a flowchart which shows operation | movement of the error macroblock determination part by Embodiment 5 of this invention. It is a figure which shows the internal structure of the error macroblock detection part by Embodiment 7 of this invention. It is a flowchart which shows operation | movement of the error macroblock detection part by Embodiment 7 of this invention. It is a block diagram which shows the structure of the image decoding apparatus by Embodiment 8 of this invention. It is a figure which shows the structure of a media packet sequence. It is a block diagram which shows the structure of the error monitoring part by Embodiment 8 of this invention.

Explanation of symbols

  8 Error macroblock detection unit (error partial image region detection unit), 11 Partial image region data decoding unit, 27, 34, 41 Threshold control unit (threshold control unit), 71 Monitoring unit

Claims (2)

In an image decoding apparatus for decoding an encoded bitstream generated by compressing and encoding an input image signal in units of partial image areas,
Partial image area data decoding means for decoding the encoded bitstream in units of partial image areas;
A pixel level first average value or a pixel level first representative value in the decoded partial image area decoded by the partial image area data decoding means is obtained, and pixels in the partial image area in the vicinity of the decoded partial image area The absolute difference between the second representative value of the pixel level due to the average value or variance of the level and the first representative value is obtained for a plurality of neighboring partial image areas, and the absolute difference of each of the neighboring partial image areas is calculated. Error partial image region detection means for detecting that the decoded partial image region is an error partial image region affected by a bit error generated in the encoded bit stream when the sum is larger than a predetermined threshold; ,
Based on the quantization step size decoded from the encoded bitstream, a predetermined threshold value when the error partial image region detecting unit detects the error partial image region is set when the quantization step size is large. An image decoding apparatus comprising: threshold control means for controlling the predetermined threshold to be small and to increase the predetermined threshold when the quantization step size is small . Error partial image area detecting means, when the correlation of the decoded partial image region and each partial image region in the vicinity is not uniform, the line physician preferentially weighted absolute differences between the correlation is strong near the partial image area 2. The image decoding apparatus according to claim 1, wherein a sum of absolute differences for each of the neighboring partial image areas is obtained .

Images