JP2007306538A - Image decoding apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image decoding device that reduces the amount of calculation by a filter of a motion compensation process in a decoding process while taking image quality deterioration into consideration, and speeds up the motion compensation process.
A filter unit 110 of a motion compensation unit 100 includes a 2-tap first filter 111 having low-pass characteristics and a 4-tap second filter 112 having edge enhancement characteristics. In accordance with the motion vector, the reference image data is subjected to filter processing by alternately switching the first filter and the second filter every frame to generate motion compensated motion compensated image data. As a result, the required calculation amount for the filter processing can be reduced to 55% of the normal 6-tap filter. In addition, image quality deterioration can be significantly suppressed by the effect of storing the image quality of the decoded image in the frame memory 26, compared to the case where the first filter or the second filter is used alone. As a result, it is possible to realize an image decoding apparatus that significantly reduces the amount of processing in decoding while suppressing image quality deterioration.
[Selection] Figure 1

Description

  The present invention relates to an image decoding device, and more particularly to a technique for speeding up a motion compensation process used when decoding image data that has been inter-coded.

  ITU-T (International Telecommunication Union Telecommunication Standardization Sector) recommended by H.264 as a standard for encoding moving image data and decoding encoded moving image data. H.264.

  FIG. 1 is a block diagram of a conventional image decoding device 1 disclosed in H.264. The image decoding device 1 includes a variable length decoding unit 2, an inverse quantization unit 3, an inverse orthogonal transform unit 4, a motion compensation unit 5, an addition unit 6, an in-loop filter 7, and a frame memory 8.

  When the variable-length encoded signal (bit stream) input to the input terminal 1a is intra-screen encoded, the variable-length decoding unit 2 performs variable-length decoding on the input variable-length encoded signal and performs quantization. Output coefficients. The output quantized coefficients are inversely quantized to DCT coefficients by the inverse quantizing unit 3 and inversely orthogonal transformed to decoded image data by the inverse orthogonal transforming unit 4. The decoded image data is filtered by the in-loop filter 7, output as a decoded image to the output terminal 1 b, and stored as reference image data in the frame memory 8.

  When the input variable length encoded signal is inter-coded, the variable length decoding unit 2 performs variable length decoding on the input variable length encoded signal and outputs a quantization coefficient and a motion vector. The output quantized coefficients are inversely quantized to DCT coefficients by the inverse quantizing unit 3 and inversely orthogonally transformed to difference image data by the inverse orthogonal transforming unit 4. The motion compensation unit 5 performs motion compensation on the reference image data of the reference frame that has already been decoded and stored in the frame memory 8 according to the motion vector output from the variable length decoding unit 2 to obtain the motion compensated image data. Generate. The adding unit 6 adds the difference image data and the motion compensation image data to generate reconstructed image data. The reconstructed image data is filtered by the in-loop filter 7, output as a decoded image to the output terminal 1 b, and stored as reference image data in the frame memory 8.

  In the above-described decoding of variable length encoded moving image data, processing with a large amount of calculation includes filtering processing for motion compensation processing in the motion compensation unit 5 and filtering processing for reducing inter-block distortion in the in-loop filter 7. (Deblocking filter processing).

  In a portable information terminal (for example, a mobile phone) that handles moving image data, decoding of moving image data with a large amount of calculation presents an important problem from the viewpoint of processing capability and power consumption required for processing. That is, in a portable information terminal having a low processing capability, the received moving image data cannot be decoded in time, and problems such as display screen distortion and dropped frames occur. Also, in a portable information terminal that is normally driven by a battery, there is a problem that decoding of received moving image data increases power consumption and accelerates battery consumption. In general, if a part of the process of decoding moving image data is omitted or simplified, the image quality of the decoded moving image deteriorates. Therefore, in the decoding of moving image data in a portable information terminal, it is important how to balance reduction in the amount of calculation required for decoding and ensuring the image quality of the decoded moving image.

  Several techniques have been disclosed for reducing the amount of calculation in decoding of moving image data and preventing degradation of the image quality of the decoded image.

  Patent Document 1 predicts the appearance of block distortion and mosquito using quantization information at the time of encoding, and appropriately switches the in-loop filter 7 to a low-pass filter and an edge enhancement filter to improve the quality of a decoded image. Disclose. The technique disclosed in Patent Document 1 reduces block distortion that occurs when a large quantization step is used to reduce the amount of code for an image with large motion, and prevents the image from being blurred. However, it does not mention reduction in the amount of calculation required for decoding. In addition, the filtering process of the motion compensation unit 5 that requires the largest amount of computation for decoding is not described at all.

  Patent Document 2 describes encoding / decoding of high-quality images with emphasis on image quality and encoding / decoding of images with low image quality / low throughput with emphasis on processing amount according to various situations. Disclosed is a technique related to an image encoding device and a decoding device that can select the image. The technique disclosed in Patent Document 2 switches an inter-pixel filter (corresponding to the in-loop filter 7 in FIG. 19) according to the image quality of an image to be encoded / decoded. However, the technique disclosed in Patent Document 2 is based on the assumption that decoding is performed using the same interpixel filter as the interpixel filter used at the time of encoding. Therefore, when it is desired to reduce the amount of calculation only at the time of decoding, it is difficult to apply. In addition, the filter processing of the motion compensation unit 5 that requires the largest amount of calculation for decoding is not described at all.

  According to Patent Document 3, the processing in the inverse orthogonal transform unit 4 and the motion compensation unit 5 is set to a normal processing mode or a processing mode with a small amount of calculation depending on whether or not there is room in the processing of the CPU related to decoding. Disclosed is a technique that can be selected. In the technique disclosed in Patent Document 3, the inverse orthogonal transform unit 4 has 4 × 8 DCT coefficients or 4 × 4 DCT coefficients with respect to a normal 8 × 8 DCT coefficient in a processing mode with a small amount of calculation. The amount of calculation is reduced by inversely changing only the DCT coefficient. As a result, the image quality naturally deteriorates. Further, the motion compensation unit 5 limits the normal motion compensation processing performed with reference to the bidirectional image to the unidirectional motion compensation processing performed with reference to only the unidirectional image in the processing mode with a small amount of calculation. . By doing so, the amount of calculation when there is no margin in the processing of the CPU related to decoding is reduced, and occurrence of frame dropping due to delay of decoding processing is suppressed. However, in the technique disclosed in Patent Document 3, the filtering process in the motion compensation unit 5 having the largest calculation amount in the decoding process is the same in both the normal processing mode and the processing mode having the small calculation amount. It must be said that the effect of reducing the amount is poor.

As described above, in the conventional technique, the filter processing of the motion compensation unit 5 that requires the largest amount of calculation for decoding is hardly studied. In addition, the omission of part of the decoding process for reducing the amount of computation necessarily involves degradation of the image quality of the decoded image, but no consideration has been given from the viewpoint of a trade-off between the amount of computation reduction and image quality degradation. .
JP 2003-18600 A JP 2003-179933 A JP 2004-289745 A

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image decoding apparatus that reduces the amount of calculation by a filter of a motion compensation process in a decoding process while taking image quality degradation into account, and speeds up the motion compensation process.

  An image decoding device according to a first invention is an image decoding device that decodes compressed image data with reference to reference image data that has already been decoded and stored in a frame memory, and decodes the image data, Variable length decoding means for outputting decoded data and motion vectors, and motion compensation for generating predicted image data based on motion vectors generated by the variable length decoding means from reference image data that has already been decoded and stored in the frame memory And the motion compensation means includes a first filter, a second filter, and a control means for controlling the first filter and the second filter. The motion compensation means performs motion compensation processing at the time of encoding. Regardless of the filter used, the first filter and the second filter are switched.

  In addition to the first invention, the image decoding apparatus according to the second invention comprises a first filter as a low-pass filter and a second filter as an edge enhancement filter.

  According to these configurations, by performing the motion compensation process by alternately switching the first filter and the second filter having a smaller calculation amount, it is possible to suppress the image quality deterioration and reduce the calculation amount.

  An image decoding apparatus according to a third aspect of the present invention is an image decoding apparatus for decoding variable-length encoded image data, wherein the variable-length decoding unit decodes the image data to generate a quantization coefficient and a motion vector. An inverse quantization unit that inversely quantizes the quantized coefficient to generate a DCT coefficient, an inverse orthogonal transform unit that performs inverse orthogonal transform on the DCT coefficient to generate difference image data, and a motion vector and reference image data A motion compensation unit that generates motion compensated image data, an adder that adds difference image data and motion compensated image data to generate reconstructed image data, and deblocks the reconstructed image data. An intra-loop filter that generates decoded image data by filtering, and a frame memory that stores the decoded image data as reference image data. Made to filter the characteristics, and generates motion compensation image data.

  According to this configuration, by performing filter processing with different characteristics for each frame in the motion compensation unit, it is possible to suppress deterioration in image quality even when performing filter processing with a smaller amount of calculation. As a result, according to this configuration, it is possible to maintain the balance between the reduction effect of the calculation amount and the deterioration of the image quality.

  In addition to the third invention, the image decoding device according to the fourth invention has a plurality of filters having different filter characteristics, and switches the plurality of filters for each predetermined number of frames to obtain a motion compensated image. Generate data.

  According to this configuration, the motion compensation unit can switch a plurality of filters having different filter characteristics for each predetermined number of frames to reduce the amount of calculation by the filter while suppressing image quality deterioration.

  In addition to the third invention, the image decoding device according to the fifth invention includes a first filter having a low-pass characteristic and a second filter having an edge enhancement characteristic. Two filters are switched every predetermined number of frames to generate motion compensated image data.

  According to this configuration, in the motion compensation unit, by switching the low pass filter and the edge enhancement filter for each frame and performing the filter processing, it is possible to reduce the amount of calculation by the filter while suppressing image quality deterioration.

  In addition to the fifth invention, the image decoding apparatus according to the sixth invention is configured by a tap filter having a tap number of less than 6 taps, and the first filter has a tap number of taps of the second filter. It consists of tap filters that are less than a number.

  In addition to the fifth invention, the image decoding device according to the seventh invention is constituted by a 2-tap filter, and the second filter is constituted by a 4-tap filter.

  According to these configurations, the first filter is configured by a 2-tap low-pass filter, the second filter is configured by a 4-tap edge enhancement filter, and these are switched for each frame, thereby suppressing image quality degradation. However, the amount of calculation by the filter can be reduced.

  In addition to the fifth invention, the image decoding apparatus according to the eighth invention includes the first filter and the second filter configured by a circuit that can dynamically change the circuit configuration.

  According to this configuration, a motion compensation unit that can change the circuit configuration of the filter depending on the situation can be realized.

  In addition to the third invention, the image decoding device according to the ninth invention includes load information indicating a load amount of a processor that performs at least a part of the processing in decoding of image data and processing other than the decoding processing of image data. Based on the determination result of the load determination unit, the motion compensation unit performs a filter process with different characteristics for each frame on the reference image data based on the determination result of the load determination unit, Motion compensated image data is generated.

  According to this configuration, since the motion compensation unit having a filter with a different amount of computation can be used according to the load state of the processor involved in the decoding of the image data, even when the load amount of the processor is large, a predetermined time The motion compensation process can be completed within this period, and the occurrence of dropped frames can be suppressed.

  In addition to the ninth invention, the image decoding apparatus according to the tenth invention includes a normal motion compensation unit that performs normal filter processing and a filter that has a smaller amount of computation than the filter processing performed by the normal motion compensation unit. A high-speed motion compensation unit that performs processing, and when the load determination unit determines that the load amount of the processor is equal to or less than a predetermined value, the motion compensation unit performs filter processing by the normal motion compensation unit, When the determination unit determines that the load amount of the processor exceeds a predetermined value, the motion compensation unit performs filter processing by the high-speed motion compensation unit.

  According to this configuration, when there is a margin in the processing of the processor involved in the decoding of the image data, the motion compensation unit performs normal filtering, and only when there is no margin in the processing of the processor It is possible to perform high-speed filter processing with a reduced amount of noise. As a result, the image quality degradation of the decoded image can be suppressed as much as possible.

  In addition to the tenth invention, the image decoding device according to the eleventh invention includes a high-speed motion compensation unit having a 2-tap first filter having a low-pass characteristic and a 4-tap second filter having an edge enhancement characteristic. Then, the first filter and the second filter are switched every predetermined number of frames to generate motion compensated image data, and the normal motion compensator has tap values (1, -5, 20, 20, -5, 1). And a third filter having 6 taps, and generating motion compensated image data.

  According to this configuration, the H.264 Using a 6-tap filter standardized by H.246, normal motion compensation processing is performed, and a 2-tap filter having a low-pass characteristic and a 4-tap filter having an edge enhancement characteristic are alternately switched to reduce the amount of calculation. High-speed motion compensation processing can be executed. As a result, the image quality degradation of the decoded image can be suppressed as much as possible.

  In addition to the eleventh invention, the image decoding apparatus according to the twelfth invention includes a first filter, a second filter, and a third filter that are circuits that can dynamically change the circuit configuration.

  According to this configuration, the circuit configuration of the filter of the motion compensation unit can be suitably changed according to the situation.

  According to the present invention, it is possible to provide an image decoding apparatus that reduces the amount of computation by a filter of a motion compensation process in a decoding process and takes the speed of the motion compensation process into consideration while taking image quality degradation into consideration.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram of an image decoding apparatus according to Embodiment 1 of the present invention. The image decoding apparatus 10 of this embodiment includes a variable length decoding unit 21, an inverse quantization unit 22, an inverse orthogonal transform unit 23, an addition unit 24, an in-loop filter 25, a frame memory 26, and a motion compensation unit 100. .

  The operation of the image decoding apparatus 10 according to the present embodiment will be described below when the variable length encoded signal (bit stream) input to the input terminal 91 is inter-screen encoded.

  The variable length decoding unit 21 performs variable length decoding on the input variable length encoded signal and outputs a quantization coefficient and a motion vector. The output quantized coefficients are inversely quantized into DCT coefficients by the inverse quantizing unit 22 and inversely orthogonal transformed by the inverse orthogonal transforming unit 23 into difference image data. The motion compensation unit 100 performs motion compensation on the reference image data of the reference frame that has already been decoded and stored in the frame memory 26 in accordance with the motion vector output from the variable length decoding unit 21 to obtain the motion compensated image data. Generate. The adding unit 24 adds the difference image data and the motion compensation image data to generate reconstructed image data. The reconstructed image data is filtered by the in-loop filter 25, output as a decoded image to the output terminal 99, and stored as reference image data in the frame memory 26.

  Except for the operation in the motion compensation unit 100, the operation of the other components of the present embodiment is the same as the operation of the corresponding components of the image decoding device 1 in FIG. Therefore, in the following description, the description will be made mainly focusing on the motion compensation unit 100.

  The motion compensation unit 100 of the present embodiment includes a filter unit 110 having a first filter 111 and a second filter 112, and a control unit 120 that controls the filter unit 110. The first filter 111 is a 2-tap filter having a low-pass characteristic. The second filter 112 is a 4-tap filter having edge enhancement characteristics.

  The variable length decoding unit corresponds to the variable length decoding unit 21, the motion compensation unit corresponds to the motion compensation unit 100, and the control unit corresponds to the control unit 120.

  The motion compensation unit 100 according to the present embodiment uses the first filter 111 or the second filter 112 on the reference image data of the previous frame stored in the frame memory 26 according to the motion vector output from the variable length decoding unit 21. Filter processing is performed, and motion compensated motion compensated image data is generated. At this time, the filter unit 110 switches the first filter 111 and the second filter 112 for each frame under the control of the control unit 120. By doing so, it is possible to considerably suppress the degradation of the image quality of the decoded image even though a filter with a small amount of calculation is used.

  Before carrying out the detailed description of the present invention, although the present inventor switches the first filter 111 and the second filter 112 for each frame, the required amount of calculation is greatly reduced. First, the background to the idea that the image quality degradation of the decoded image is effective will be described.

  H. According to H.264, when obtaining motion compensated image data with ½ pixel accuracy according to a motion vector, the motion compensation unit (for example, the motion compensation unit 100 in FIG. 1) applies a tap value (1) to the reference image data. , -5, 20, 20, -5, 1) is performed using a 6-tap filter.

  FIG. 8 is an arrangement diagram of integer precision pixels and 1/2 precision pixels. In FIG. 8, the pixel value “a” of the ½ precision pixel is calculated as follows using the pixel values A, B, C, D, E, and F of six integer precision pixels in the vicinity thereof.

a = (A-5B + 20C + 20D-5E + F + 16) >> 5
Here, each coefficient in parentheses is a tap value of the above-described 6-tap filter, a constant “16” is a constant for rounding operation, and a sign and a numerical value “>> 5” are for standardization. Is a 5-bit shift operation corresponding to the division by the value “32”.

The filtering process using the 6-tap filter described above has a large amount of calculation. Therefore, when decoding moving image data with a portable information terminal or the like, the amount of computation of the 6-tap filter becomes a problem because of limited calculation resources. Therefore, the present inventor considered substituting a 6-tap filter with a 2-tap filter and a 4-tap filter that have fewer taps than a 6-tap filter in order to reduce the amount of calculation. (Note that the 2-tap filter is a linear filter, but is sometimes referred to as a bilinear filter. In this specification, a 2-tap filter is used as a name.)
FIG. 9 shows the frequency characteristics of the 2-tap filter, 4-tap filter, and 6-tap filter. The 2-tap filter obtained from the frequency characteristics shown in FIG. 9 has a tap value (1, 1), and the 4-tap filter has a tap value (−4, 20, 20, −4).

  As shown in FIG. 9, it can be seen that the frequency characteristic of the 2-tap filter (curve A) has a low cutoff frequency and a low-pass characteristic compared to the frequency characteristic of the 6-tap filter (curve C). For this reason, the decoded image decoded using the 2-tap filter in the motion compensation process becomes an image with a blurred boundary. Also, since this decoded image is used as a reference image for the next frame, blurring of the image accumulates. For this reason, there arises a problem that image quality deterioration of the decoded image becomes severe as the inter-frame encoded frame continues.

  Next, the frequency characteristic of the 4-tap filter (curve B) has a higher passband gain than the frequency characteristic of the 6-tap filter (curve C). Therefore, the decoded image obtained by performing the motion compensation process using the 4-tap filter is an image in which the specific frequency component is large and the boundary is emphasized. This image distortion accumulates as inter-frame encoded frames continue, resulting in further image quality degradation of the decoded image.

  As described above, the motion compensation processing using the 2-tap filter and the 4-tap filter significantly reduces the amount of calculation compared with the motion compensation processing using the 6-tap filter, but the image quality of the decoded image is significantly deteriorated. However, considering the state of image quality degradation in detail, a decoded image using the 2-tap filter is an image with a blurred boundary, and a decoded image using the 4-tap filter is an image with an enhanced boundary. . This means that if the 2-tap filter and 4-tap filter are used alternately, the effect of accumulating the image quality due to the continued inter-frame coding frame acts, and the blurring and enhancement of the image boundary are offset, resulting in image quality degradation. It can be suppressed.

  The inventor has obtained a hint from this phenomenon and has conceived that the filter processing in the motion compensation processing is executed by switching between the 2-tap filter and the 4-tap filter for each frame. The effect is described below.

  FIG. 10 is an explanatory diagram illustrating a process of generating half-accuracy pixels by a 6-tap filter. Frame P2 (pixels b1 to b8) is a frame in which interpolation of half-precision pixels is performed using frame P1 (pixels a1 to a11) as a reference image. Frame P3 (pixels c1 to c8) is a frame in which interpolation of half-precision pixels is performed using frame P2 as a reference image. A frame P2 is generated by multiplying the frame P1 by a 6-tap filter (tap values (1, -5, 20, 20, -5, 1), and a frame P3 is generated by applying the same 6-tap filter to the frame P2. The filter characteristic at this time is obtained as shown in (Equation 1).

Here, in (Equation 1), the right side is divided by a constant 32 for normalization. From (Equation 1), applying the 6-tap filter twice means that tap values (1, −10, 65, −160, 190, 852, 190, −160, 65, −10, 1) (normalization constant = It can be seen that this is equivalent to generating the frame P3 by multiplying the frame P1 by the 1024) 11-tap filter.

  Next, the frequency response of the above 11 tap filter is obtained. After z-transforming (Equation 1), the exponential function is transformed into a trigonometric function to obtain (Equation 2).

  The frequency response of the 11-tap filter described above is given by the amplitude term (absolute value of H (ω)) of (Equation 2) and is given by (Equation 3).

  FIG. 11 is an explanatory diagram illustrating a process of generating a ½ precision pixel by a 2-tap filter. In FIG. 11, a frame P2 is created by applying a 2-tap filter with a tap value (1, 1) (normalization constant = 2) to the frame P1, and then applying the same 2-tap filter to the frame P2. Create P3. The filter characteristic at this time is obtained as shown in (Expression 4).

  From (Equation 4), multiplying the 2-tap filter twice is equivalent to multiplying the frame P1 by the 3-tap filter having the tap value (1, 2, 1) (normalization constant 4) to generate the frame P3. It turns out that it is.

  Next, the frequency response of this 3-tap filter is obtained. (Equation 5) is obtained by the same calculation as the derivation of (Equation 2).

As a result, the frequency response of the 3-tap filter is given by the amplitude term (absolute value of H (ω)) of (Equation 5) and is given by (Equation 6).

  FIG. 12 is an explanatory diagram illustrating a process of generating a ½ precision pixel by a 4-tap filter.

  In FIG. 12, a frame P2 is created by applying a 4-tap filter of tap values (−4, 20, 20, −4) (normalization constant = 32) to the frame P1, and the same 4-tap is further applied to the frame P2. A filter is applied to create a frame P3. The filter characteristic at this time is obtained as shown in (Expression 7).

  From (Equation 7), applying the 4-tap filter twice means that the 7-tap filter having the tap value (1, -10, 15, 52, 15, -10, 1) (normalization constant 64) is applied to the frame P1. This is equivalent to generating the frame P3.

  Next, the frequency response of this 7-tap filter is obtained. (Equation 8) is obtained by the same calculation as the derivation of (Equation 2).

As a result, the frequency response of the 7-tap filter is given by the amplitude term (absolute value of H (ω)) of (Equation 8), as shown in (Equation 9).

  FIG. 13 is an explanatory diagram illustrating a process of generating a ½ precision pixel using a 2-tap filter and a 4-tap filter.

  In FIG. 13, a frame P2 is created by applying a 2-tap filter with a tap value (1, 1) (normalization constant = 2) to the frame P1, and further, the tap value (−4, 20, 20, 4) (normalization constant = 32) is applied to create a frame P3. The filter characteristic at this time is obtained as shown in (Equation 10).

  From (Equation 10), applying a 2-tap filter and a 4-tap filter applies a 5-tap filter of tap values (−1, 4, 10, 4, −1) (normalization constant 16) to the frame P1, It can be seen that this is equivalent to generating the frame P3.

  Next, the frequency response of this 5-tap filter is obtained. (Equation 11) is obtained by the same calculation as the derivation of (Equation 2).

As a result, the frequency response of the 7-tap filter is given by the amplitude term (absolute value of H (ω)) of (Equation 11), and is given by (Equation 12).

  FIG. 14 is a diagram of the frequency response when the tap filter is applied twice. In the figure, a curve D is obtained by continuously applying a 2-tap filter (Equation 6), a curve E is obtained by alternately applying a 2-tap filter and a 4-tap filter (Equation 12), and a curve F is represented by 4 When the tap filter is continuously applied (Equation 9), the curve G indicates the frequency response when the 6-tap filter is continuously applied (Equation 3).

  As shown in FIG. 14, the frequency response when the 2-tap filter and the 4-tap filter are alternately applied has a higher cutoff frequency than that when the 2-tap filter is applied twice in succession, and the image quality blur is improved. I understand that. The frequency response when the 2-tap filter and the 4-tap filter are alternately applied has a lower cutoff frequency than when the 4-tap filter is applied twice in succession, but the frequency response in the passband Is flat, it can be seen that image degradation due to frequency response distortion in the passband is reduced.

  In order to clarify the difference in image quality due to the difference in frequency response shown in FIG. 14, the following subjective evaluation experiment was performed.

  Using a decoded image using a 6-tap filter for motion compensation processing as a reference image, a decoded image using a 2-tap filter, a decoded image using a 4-tap filter, and using a 2-tap filter and a 4-tap filter for each frame Decoded images were prepared, and their image quality was evaluated in five stages shown in Table 1.

  Subjective evaluation experiments were performed under the conditions shown in Table 2.

  In the subjective evaluation experiment, a reference image (using a 6-tap filter) was first presented, and thereafter, three images to be evaluated were presented in order with the number of taps of the filter used being reduced. Table 3 shows the results of the subjective evaluation experiment.

  Table 3 shows the average score of 10 subjective evaluation points, with 5 subjective evaluation points of the reference image (using a 6-tap filter) for three evaluated images. The figure also shows the relative required amount of computation when each filter is used for motion compensation processing.

  According to this subjective evaluation experiment, the decoded image by the motion compensation process using the 2-tap filter or the 4-tap filter has a subjective evaluation point of 1.7 or 1.6, and the image quality deterioration is so visually disturbing. It is remarkable. However, the decoded image by the motion compensation process using the 2-tap filter and the 4-tap filter alternately for each frame has a subjective evaluation point of 2.9, which is extremely difficult to evaluate because the evaluation criterion is “I am worried about degradation but it does not get in the way”. It turned out to be close. Moreover, the amount of computation required for motion compensation processing using a 2-tap filter and a 4-tap filter alternately for each frame can be reduced to 55% of the amount of computation required for motion compensation processing using a 6-tap filter.

  That is, according to the motion compensation processing using the 2-tap filter and the 4-tap filter alternately for each frame of the present invention, it is clear that subjective image quality deterioration is considerably suppressed, and the amount of computation required for motion compensation processing can be greatly reduced. became. As a result, in the motion compensation process, switching the first filter 111 and the second filter 112 for each frame suppresses the deterioration of the image quality of the decoded image, although the required calculation amount is greatly reduced. The validity of the inventor's idea that this is effective is proved.

  Next, the operation of the image decoding apparatus 10 according to the present embodiment shown in FIG. 1 will be described in more detail.

  FIG. 15 is a flowchart of the image decoding apparatus 10 according to Embodiment 1 of the present invention.

  In step S0, the decoding process is started.

  In step S1, it is determined which of the first filter 111 and the second filter 112 is used for the motion compensation processing of the frame, and a filter flag indicating the determined filter is stored. Details of this filter control step will be described later.

  Subsequently, a decoding process is performed for each macroblock of the frame.

  In step S2, the variable length decoding unit 21 performs variable length decoding on the first macroblock of the input variable length encoded signal, and outputs a quantization coefficient and a motion vector.

  In step S3, the inverse quantization unit 22 performs an inverse quantization process on the quantization coefficient output in step S2, and outputs a DCT coefficient.

  In step S4, the inverse orthogonal transform unit 23 performs inverse orthogonal transform on the DCT coefficient output in step S3, and outputs difference image data.

  In step S5, the control unit 120 of the motion compensation unit 100 determines the filter flag stored in step S1. When the filter flag indicates the first filter 111, the control proceeds to step S6, and the filter unit 110 selects the first filter 111 and performs motion compensation processing. When the filter flag indicates the second filter 112, the control proceeds to step S7, and the filter unit 110 selects the second filter 112 and performs motion compensation processing. At this time, the filter unit 110 performs motion compensation processing on the reference image data of the reference frame that has already been decoded and stored in the frame memory 26 in accordance with the motion vector output from the variable length decoding unit 21 in step S2. To generate motion compensated image data.

  In step S8, the adding unit 24 adds the difference image data output in step S4 and the motion compensation image data generated in step S6 or step S7 to generate reconstructed image data. Further, the reconstructed image data is filtered by the in-loop filter 25, output as a decoded image to the output terminal 99, and stored as reference image data in the frame memory 26.

  In step S9, it is determined whether or not the macroblock is the last macroblock of the frame. As a result of the determination, if the macroblock is not the last macroblock of the frame, the control returns to step S2 to perform the decoding process for the next macroblock. As a result of the determination, if the macroblock is the last macroblock of the frame, the control proceeds to step S10.

  In step S10, it is determined whether or not the frame is a stream end frame. As a result of the determination, if the frame is not a stream end frame, the control returns to step S1 to perform a decoding process for the next frame. If it is determined that the frame is a stream end frame, the control proceeds to step S11, and the decoding process is terminated.

  FIG. 16 is a flowchart of the image decoding device 10 according to the first embodiment of the present invention, and shows details of the filter control step of step S1 of FIG.

  When the filter control is started in step S20, the filter flag used in the decoding of the previous frame is read in step S21. If the frame to be decoded is the first frame, the filter flag is set to the first filter flag. (Alternatively, the filter flag may be set to the second filter flag.) The first filter flag indicates that the first filter 111 is used in the filter unit 110, and the second filter flag is the filter unit 110. 2 shows that the second filter 112 is used.

  In step S22, it is determined whether the filter flag is the first filter flag or the second filter flag. If the result of determination is that the filter flag is the second filter flag, the filter flag is set to the first filter flag in step S23. If the result of determination is that the filter flag is the first filter flag, the filter flag is set to the second filter flag in step S24.

  In step S25, the newly set filter flag is saved, and the process ends in step S26.

  With the filter control described above, the filter used by the filter unit 110 for motion compensation processing can be switched alternately for each frame.

  FIG. 17 is a diagram showing the relationship between the decoded image and the filter flag according to Embodiment 1 of the present invention. Since the frame I0 with the frame number “0” is an intra-frame encoded frame, the motion compensation process is not performed.

  From the next frame P1, an inter-frame encoded frame is obtained. In the motion compensation process for the frame P1, the filter flag is the first filter flag, and the first filter 111 is used.

  In the motion compensation process for the next frame P2, the filter flag is the second filter flag, and the second filter 112 is used. Hereinafter, the first filter 111 and the second filter 112 are alternately used for each frame.

  As described above, the image decoding apparatus 10 according to the present embodiment alternately uses the 2-tap filter and the 4-tap filter for each frame in the motion compensation process, and the required calculation amount is 55% of the required calculation amount of the normal six-tap filter. Has been reduced. In addition, by alternately using a 2-tap filter and a 4-tap filter for each frame, the effect of accumulating the image quality of the decoded image in the frame memory 26, despite the use of a low-order filter, is subjective. Image quality degradation is greatly suppressed.

  Therefore, the image decoding apparatus 10 according to the present embodiment can significantly reduce the amount of computation in decoding while significantly suppressing subjective image quality degradation, and thus is particularly effective in portable information terminals and the like that have a large demand for reducing the amount of computation. is there.

(Embodiment 2)
FIG. 2 is a block diagram of an image decoding apparatus according to Embodiment 2 of the present invention. 2, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  2 further includes a load determination unit 27 in addition to the image decoding device 10 according to Embodiment 1 of the present invention, and the motion compensation unit 200 includes a high-speed motion compensation unit 210 and a normal motion. And a compensation unit 220.

  The load determination unit 27 includes a part of the processing of the image decoding device 11 according to the present embodiment (for example, the motion vector decoding processing performed by the variable length decoding unit 21) and the other processing (for example, audio signal processing, communication processing, etc.). Load information indicating the load amount of the CPU that performs () is input from the input terminal 92. Then, based on the load information, the load determination unit 27 compares the CPU load amount with a predetermined set value, and when it is determined that the CPU load amount is sufficient, the motion compensation unit 200 performs normal motion compensation. When the CPU 220 is instructed to select the CPU 220 and it is determined that the CPU load is not sufficient, the motion compensator 200 is instructed to select the high-speed motion compensator 210. The normal motion compensation unit 220 has a normal calculation amount, and the high-speed motion compensation unit 210 has a calculation amount smaller than the calculation amount performed by the normal motion compensation unit 220.

  The high-speed motion compensation unit 210 of the motion compensation unit 200 is the same as the motion compensation unit 100 according to Embodiment 1 of the present invention. That is, the motion compensation unit 210 of this embodiment includes a filter unit 110 having a first filter 111 and a second filter 112 and a control unit 120 that controls the filter unit 110. The first filter 111 is a 2-tap filter having a low-pass characteristic. The second filter 112 is a 4-tap filter having edge enhancement characteristics. Therefore, the high-speed motion compensation unit 210 can perform high-speed motion compensation processing by filter processing with a reduced amount of calculation.

  The normal motion compensation unit 220 includes a filter unit 230 having a third filter 233 and a control unit 240 that controls the filter unit 230. The third filter 233 is a 6-tap filter having tap values (1, -5, 20, 20, -5, 1), and converts the pixel value of a target ½ precision pixel into six neighboring pixels. Generated using pixel values of integer precision pixels. Therefore, the normal motion compensation unit 220 has a normal calculation amount and performs a normal motion compensation process. The decoded image obtained there is ensured normal image quality.

  The first motion compensation unit corresponds to the normal motion compensation unit 220, the second motion compensation unit corresponds to the high-speed motion compensation unit 210, the second control unit corresponds to the control unit 120, and the first control The means is included in the load determination unit 27.

  FIG. 18 is a flowchart of the image decoding apparatus according to Embodiment 2 of the present invention. The operation of the image decoding apparatus 11 according to the present embodiment shown in FIG. 2 will be described with reference to FIG.

  In step S30, the decoding process is started.

  In step S31, it is determined which of the first filter 111 and the second filter 112 is used for the motion compensation processing of the frame, and a filter flag indicating the determined filter is stored. The details of this filter control step are the same as those in FIG. 16 described in the first embodiment of the present invention, and therefore the description thereof is omitted.

  Subsequently, a decoding process is performed for each macroblock of the frame.

  In step S32, the variable length decoding unit 21 performs variable length decoding on the first macroblock of the input variable length encoded signal, and outputs a quantization coefficient and a motion vector.

  In step S33, the inverse quantization unit 22 performs an inverse quantization process on the quantization coefficient output in step S32, and outputs a DCT coefficient.

  In step S34, the inverse orthogonal transform unit 23 performs inverse orthogonal transform on the DCT coefficient output in step S33, and outputs difference image data.

  In step S <b> 35, the load determination unit 27 compares the CPU load amount with a predetermined value to determine whether there is room in the CPU processing. As a result of the determination, if the load amount of the CPU exceeds a predetermined value (“Yes”), the control proceeds to step S36. As a result of the determination, if the load amount of the CPU is equal to or less than a predetermined value (“No”), the control proceeds to step S37.

  In step S36, the motion compensation unit 200 selects the high speed motion compensation unit 210.

  In step S38, the control unit 120 of the high-speed motion compensation unit 210 determines the filter flag stored in step S31. When the filter flag indicates the first filter 111, the control proceeds to step S39, and the filter unit 110 selects the first filter 111 and performs motion compensation processing. When the filter flag indicates the second filter 112, the control proceeds to step S40, and the filter unit 110 selects the second filter 112 and performs motion compensation processing. At this time, the filter unit 110 performs motion compensation processing on the reference image data of the reference frame that has already been decoded and stored in the frame memory 26 according to the motion vector output from the variable length decoding unit 21 in step S32. To generate motion compensated image data.

  In step S37, the motion compensation unit 200 selects the normal motion compensation unit 220.

  In step S <b> 41, the filter unit 230 of the normal motion compensation unit 220 performs a normal motion compensation process using the third filter 233. At this time, the filter unit 230 performs normal motion on the reference image data of the reference frame that has been decoded and stored in the frame memory 26 according to the motion vector output from the variable length decoding unit 21 in step S32. Compensation processing is performed to generate motion compensated image data.

  In step S42, the adding unit 24 adds the difference image data output in step S34 and the motion compensation image data generated in step S36 or step S37 to generate reconstructed image data. Further, the reconstructed image data is filtered by the in-loop filter 25, output as a decoded image to the output terminal 99, and stored as reference image data in the frame memory 26.

  In step S43, it is determined whether or not the macroblock is the last macroblock of the frame. As a result of the determination, if the macroblock is not the last macroblock of the frame, the control returns to step S32, and the next macroblock is decoded. If the result of determination is that the macroblock is the last macroblock of the frame, control proceeds to step S44.

  In step S44, it is determined whether or not the frame is a stream end frame. If it is determined that the frame is not a stream end frame, the control returns to step S31 to decode the next frame. As a result of the determination, if the frame is a stream end frame, the control proceeds to step S45, and the process ends.

  In this way, even when there is no margin in the processing capacity of the CPU, by switching between the normal motion compensation processing with the normal computation amount and the high-speed motion compensation processing with reduced computation amount according to the load amount of the CPU, The decoding of the stream can continue. In addition, since high-speed motion compensation processing with a reduced amount of computation is performed by switching between a 2-tap filter and a 4-tap filter for each frame, image quality degradation due to a reduction in the amount of computation can be reduced.

(Embodiment 3)
FIG. 3 is a block diagram of an image decoding apparatus according to Embodiment 3 of the present invention. In FIG. 3, the same components as those of FIG.

  3 further includes a load determination unit 27 in addition to the image decoding device 10 according to Embodiment 1 of the present invention, and the motion compensation unit 300 includes a filter unit 310, a control unit 320, and the like. including. The filter unit 310 includes a first filter 111, a second filter 112, a third filter 233, and a selector 340 that selects them.

  The motion compensation unit 300 according to the present embodiment has a function equivalent to that of the motion compensation unit 200 according to the second embodiment of the present invention.

  The load determination unit 27 compares the CPU load amount with a predetermined set value based on the load information indicating the CPU load amount, and if it is determined that the CPU load amount has a margin, the motion compensation unit 300. Is instructed to select normal motion compensation processing, and when it is determined that there is no room in the CPU load, the motion compensation unit 300 selects high-speed motion compensation processing with a smaller amount of computation than usual. To instruct.

  In response to the instruction from the load determination unit 27, the control unit 320 controls the selector 340 and performs filter switching.

  The first filter 111 is a 2-tap filter having a low-pass characteristic. The second filter 112 is a 4-tap filter having edge enhancement characteristics. These two filters are alternately selected for each frame when the load determination unit 27 determines that there is no margin in the CPU load, and the functions of the high-speed motion compensation unit 210 according to the second embodiment of the present invention are selected. It performs the same function.

  The third filter 233 is a 6-tap filter having tap values (1, -5, 20, 20, -5, 1), and when the load determination unit 27 determines that there is a margin in the CPU load. When selected, the same function as that of the normal motion compensation unit 220 according to the second embodiment of the present invention is achieved.

  As described above, the motion compensation unit 300 according to the present embodiment can perform normal motion compensation processing and high-speed motion compensation processing with a reduced amount of computation in accordance with the load amount of the CPU.

(Embodiment 4)
FIG. 4 is a block diagram of an image decoding apparatus according to Embodiment 4 of the present invention. 4, the same components as those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  The image decoding device 13 according to the present embodiment shown in FIG. 4 further includes a first memory 28 and a second memory 29 in addition to the image decoding device 10 according to the first embodiment of the present invention. A configuration filter 410, a selector 420, and a control unit 430 are included.

  The dynamic reconfiguration filter 410 is configured by a circuit using a so-called dynamic reconfiguration (dynamic reconfigurable) technique capable of switching the circuit configuration in a short time as necessary. A circuit using the dynamic reconfiguration technique can easily switch a circuit in a short time only by exchanging configuration information, so that a plurality of functions can be switched while the circuit is operating.

  There are two types of circuits using the dynamic reconfiguration technology: one that reconfigures the architecture that is a basic circuit element by a selector and a register, and the other that reconfigures a circuit such as an FPGA. The dynamic reconfiguration filter 410 of this embodiment may be configured using any method.

  The first memory 28 stores first reconfiguration information for making the dynamic reconfiguration filter 410 a circuit configuration having the same function as the first filter 111 of the first embodiment of the present invention.

  The second memory 29 stores second reconfiguration information for making the dynamic reconfiguration filter 410 a circuit configuration having the same function as the second filter 112 of the first embodiment of the present invention.

  The control unit 430 controls the selector 420 to alternately select the first reconstruction information and the second reconstruction information for each frame, and outputs the first reconstruction information and the second reconstruction information to the dynamic reconstruction filter 410.

  The dynamic reconfiguration filter 410 includes a circuit having the same function as the first filter 111 based on the first reconfiguration information and a second filter based on the second reconfiguration information for each frame under the control of the control unit 430. A circuit having the same function as 112 is configured. As a result, the motion compensation unit 400 according to the present embodiment can execute high-speed motion compensation processing with a reduced amount of computation.

  Therefore, the image decoding apparatus 13 according to the present embodiment can realize the functions of a plurality of filters by using one dynamic reconfiguration filter 410 by reconfiguring the filter functions in units of frames. The number of circuit blocks having a filter function can be reduced as compared with the image decoding apparatus 10 in FIG. As a result, the image decoding apparatus 13 of this embodiment can reduce the circuit scale, and is effective in reducing the cost and power consumption when the LSI is implemented.

  In the image decoding device 13 of the present embodiment, the case where the dynamic reconstruction filter 410 configures two filters has been described. However, the image decoding device 13 of the present embodiment is not limited to this. For example, the dynamic reconfiguration filter 410 may constitute three or more filters, and each of the filters may be switched for each frame to perform finer motion correction processing.

  In the image decoding apparatus 10 according to the first embodiment of the present invention, since the filters are mounted in different circuit blocks, the circuit scale increases as the number of filters increases. Compared to this, in the image decoding device 13 of the present embodiment, the function of a plurality of filters is realized by a single dynamic reconstruction filter 410, so the circuit scale related to the filter is a circuit obtained by adding the circuit scales of all the filters. Instead of the scale, the circuit scale of the filter having the largest circuit scale among the plurality of filters is sufficient. As described above, in the image decoding device 13 according to the present embodiment, the effect of suppressing the circuit scale increases as the number of filters realized by the dynamic reconfiguration filter 410 increases.

(Embodiment 5)
FIG. 5 is a block diagram of an image decoding apparatus according to Embodiment 5 of the present invention. 5, the same components as those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  The image decoding apparatus 14 according to the present embodiment shown in FIG. 5 is configured by using a dynamic reconfiguration circuit for the motion compensation section 500 having the same function as the motion compensation section 300 of the image decoding apparatus 12 according to the third embodiment of the present invention. ing.

  That is, the motion compensation unit 500 includes a dynamic reconfiguration filter 510, a reconfiguration information selection unit 520, a third memory 530, and a control unit 540.

  In the third memory 530, a circuit having the same function as the first filter 111, the second filter 112, and the third filter 233 of the filter unit 310 according to the third embodiment of the present invention is added to the dynamic reconfiguration filter 510. First reconfiguration information, second reconfiguration information, and third reconfiguration information for configuration are stored.

  The reconfiguration information selection unit 520 selects one of the first reconfiguration information, the second reconfiguration information, and the third reconfiguration information stored in the third memory 530 under the control of the control unit 540. Select and output to dynamic reconfiguration filter 510.

  The dynamic reconfiguration filter 510 is the same as the first filter 111, the second filter 112, or the third filter 233 using the reconfiguration information output from the dynamic reconfiguration filter 510 under the control of the control unit 540. A circuit having a function is configured.

  When the load determination unit 27 has a margin in the CPU load and is instructed to select normal motion compensation processing, the control unit 540 selects the third reconfiguration information from the reconfiguration information selection unit 520. The dynamic reconfiguration filter 510 is configured to have a circuit having the same function as the third filter 233. As a result, when there is a margin in the load amount of the CPU, the motion compensation unit 500 can execute a normal motion compensation process.

  When the load determination unit 27 instructs that the CPU load amount is not sufficient and a high-speed motion compensation process with a reduced calculation amount is selected, the control unit 540 instructs the reconfiguration information selection unit 520 to The first reconstruction information and the second reconstruction information are instructed to be alternately selected every time the frame changes, and the dynamic reconstruction filter 510 has the same function as the first filter 111 or the second filter 112. A circuit is configured for each frame. As a result, when there is no margin in the load amount, the motion compensation unit 500 can execute a high-speed motion compensation process with a reduced amount of computation.

  As described above, in the image decoding apparatus 14 according to the present embodiment, the motion compensation unit 500 can realize the functions of a plurality of filters with one dynamic reconfiguration filter 510 by reconfiguring the function of the filter in units of frames. Therefore, normal motion compensation processing and high-speed motion compensation processing with a reduced amount of computation can be performed in accordance with the CPU load. Furthermore, the image decoding device 14 according to the present embodiment can reduce the number of circuit blocks constituting the filter function as compared with the image decoding device 11 according to the second embodiment of the present invention. As a result, the image decoding apparatus 14 of the present embodiment can reduce the circuit scale, and is effective in reducing the cost and power consumption when the LSI is implemented.

  In the image decoding device 14 of the present embodiment, the case where the dynamic reconstruction filter 510 configures three filters has been described. However, the image decoding device 14 of the present embodiment is not limited to this. For example, the dynamic reconfiguration filter 510 may constitute four or more filters, and each of the filters may be switched for each frame or according to the load amount of the CPU to perform finer motion correction processing. .

  In the realization method according to the second embodiment or the third embodiment of the present invention, since the filters are mounted in different circuit blocks, the circuit scale increases as the number of filters increases. In contrast, in this embodiment, since the function of a plurality of filters is realized by a single dynamic reconfiguration filter 510, the circuit scale related to the filter is not a total circuit scale of all filters, The circuit scale of the filter having the largest circuit scale among the filters is sufficient. As described above, the image decoding device 14 according to the present embodiment increases the effect of suppressing the circuit scale as the number of filters realized by the dynamic reconfiguration filter 510 increases.

(Embodiment 6)
FIG. 6 is a block diagram of a semiconductor integrated circuit according to the sixth embodiment of the present invention. The semiconductor integrated circuit 600 of this embodiment includes an image decoding unit 610, an image encoding unit 620, an audio signal processing unit 630, and a ROM 640, and is connected to an external CPU 650 and an external memory 660.

  The image decoding unit 610 according to the present embodiment is configured by any of the image decoding devices 10 to 14 according to the first to fifth embodiments of the present invention, and decodes an input variable-length encoded signal (bit stream). .

  The image encoding unit 620 performs variable length encoding on the input image data to generate a variable length encoded signal (bit stream).

  The audio signal processing unit 630 performs encoding and decoding of an audio signal and signal processing.

  The ROM 640 stores, for example, reconfiguration information for constructing a dynamic reconfiguration filter.

  The CPU 650 performs at least a part of the decoding process of the variable-length encoded signal and other processes.

  The external memory 660 functions as a substitute for the frame memory 26 used in the image decoding devices 10 to 14 according to the first to fifth embodiments of the present invention.

  The semiconductor integrated circuit 600 configured as described above can perform normal motion compensation processing and high-speed motion compensation processing with a reduced amount of computation in accordance with the load amount of the CPU 650.

  Therefore, the semiconductor integrated circuit 600 of this embodiment can be suitably used for, for example, a portable information terminal.

  As shown in FIG. 6, the external memory 660 may be configured independently of the semiconductor integrated circuit 600, or may be included in the semiconductor integrated circuit 600.

(Embodiment 7)
FIG. 7 is a block diagram of an electronic device according to Embodiment 7 of the present invention. The electronic apparatus 700 of this embodiment includes a semiconductor integrated circuit 710, a CPU 720, a communication circuit unit 730, an external memory 740, an input / output interface 750, a display 760, and a keyboard 770.

  The semiconductor integrated circuit 710 according to the present embodiment is the semiconductor integrated circuit 600 according to the sixth embodiment of the present invention, and can perform encoding / decoding of image data, encoding / decoding of an audio signal, signal processing, and the like.

  The CPU 720 performs at least a part of processing such as encoding / decoding of image data of the semiconductor integrated circuit 710, encoding / decoding of an audio signal, and signal processing, and communication processing of the communication circuit unit 730 described below. Processing and overall control of the electronic apparatus 700 of this embodiment are performed. The external memory 740 corresponds to the external memory 660 externally connected to the semiconductor integrated circuit 600 according to the sixth embodiment of the present invention.

  The communication circuit unit 730 processes communication of image data / audio data between the electronic device 700 of the present embodiment and an external electronic device.

  The display 760 displays image data.

  The keyboard 770 is used for information input and control input from the outside.

  The input / output interface 750 interfaces other components with the display 760, the keyboard 770, and the like.

  The electronic device 700 configured as described above can perform normal motion compensation processing and high-speed motion compensation processing with a reduced amount of computation in accordance with the load amount of the CPU 720.

  Therefore, the electronic device 700 of this embodiment can be suitably used for, for example, a personal digital assistant (PDA), a mobile phone, a car navigation system, an in-vehicle television, a notebook personal computer, an HDD recorder, a DVD recorder, and the like.

  In the second, third, and fifth embodiments of the present invention described above, the filter that performs normal motion compensation processing is H.264. The H.264 6-tap filter has been described as an example, but a filter that performs normal motion compensation processing is not limited to this. If necessary, pixel interpolation may be performed using a tap filter other than 6 taps. The 6-tap filter may have a tap value different from the tap value described in the second, third, and fifth embodiments of the present invention.

  In the first to fifth embodiments of the present invention, the first filter 111 with a small amount of calculation is described as a 2-tap filter, and the second filter 112 with a small amount of calculation is described as a 4-tap filter. However, the first filter 111 and the second filter 112 are not limited to this, the first filter 111 has a low-pass characteristic, the second filter 112 has an edge enhancement characteristic, As long as the tap filter has a small amount of calculation, a filter having an arbitrary number of taps may be used. Further, the 2-tap filter and the 4-tap filter may have tap values different from the tap values described in the first to fifth embodiments of the present invention.

  In Embodiments 1 to 5 of the present invention, the case where the first filter 111 and the second filter 112 are switched for each frame has been described. However, the number of frames for switching the first filter 111 and the second filter 112 is determined by decoding. What is necessary is just to determine arbitrarily by trade-off with suppression of the image quality degradation of an image, and reduction of the amount of calculations. For example, the first filter 111 has a low-pass characteristic comparable to that of the first filter 111 according to the first embodiment of the present invention, and the second filter 1112 is more than the second filter 112 according to the first embodiment of the present invention. If the first filter 111 and the second filter 112 are switched at a 2-to-1 frame number when the edge enhancement characteristic is further steep, the amount of calculation of the filter processing is further reduced, and the image quality of the decoded image is also deteriorated. It can suppress suitably.

  In general, as the number of filter taps used for pixel interpolation increases, the amount of calculation for pixel interpolation increases and the need for reduction in the amount of calculation increases. Therefore, the motion correction method of the present invention, which performs pixel interpolation by switching a filter with a small number of taps for each frame and obtains a result close to pixel interpolation by a filter with a large number of taps, is extremely effective.

  As described above, the gist of the present invention is to provide an image decoding device that reduces the amount of calculation by the filter of the motion compensation process in the decoding process and speeds up the motion compensation process in consideration of image quality degradation. Therefore, various modifications can be made without departing from the spirit of the present invention.

  The image decoding apparatus according to the present invention can be used, for example, in an electronic device such as a portable information terminal in which it is important to reduce the processing amount of image data and its application field.

Block diagram of an image decoding apparatus according to Embodiment 1 of the present invention Block diagram of an image decoding apparatus according to Embodiment 2 of the present invention The block diagram of the image decoding apparatus in Embodiment 3 of this invention The block diagram of the image decoding apparatus in Embodiment 4 of this invention Block diagram of an image decoding apparatus in Embodiment 5 of the present invention Block diagram of a semiconductor integrated circuit according to a sixth embodiment of the present invention Block diagram of electronic device in Embodiment 7 of the present invention Arrangement diagram of integer precision pixel and 1/2 precision pixel Frequency characteristics of 2-tap filter, 4-tap filter, and 6-tap filter Explanatory drawing which shows the production | generation process of the 1/2 precision pixel by 6 tap filter Explanatory drawing which shows the production | generation process of the 1/2 precision pixel by 2 tap filter Explanatory drawing which shows the production | generation process of the 1/2 precision pixel by 4 tap filter Explanatory drawing which shows the production | generation process of the 1/2 precision pixel by 2 tap filter and 4 tap filter Figure of frequency response when tap filter is applied twice Flowchart of image decoding apparatus in Embodiment 1 of the present invention Flowchart of image decoding apparatus in Embodiment 1 of the present invention Relationship diagram between decoded image and filter flag in Embodiment 1 of the present invention Flowchart of image decoding apparatus in Embodiment 2 of the present invention Block diagram of a conventional image decoding apparatus 1

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image decoding apparatus 2, 21 Variable length decoding part 3, 22 Inverse quantization part 4, 23 Inverse orthogonal transformation part 5, 100, 200, 300, 400, 500 Motion compensation part 6, 24 Adder part 7, 25 In-loop filter 8, 26 Frame memory 1a, 91, 92 Input terminal 1b, 99 Output terminal 10, 11, 12, 13, 14 Image decoding device 27 Load determination unit 28 First memory 29 Second memory 110, 230, 310 Filter unit 111 First 1 filter 112 2nd filter 120, 240, 320, 430, 540 Control unit 210 High-speed motion compensation unit 220 Normal motion compensation unit 233 Third filter 340, 420 Selector 410, 510 Dynamic reconfiguration filter 520 Reconfiguration information selection unit 530 Third memory 600 Semiconductor integrated circuit 620 Image encoding unit 640 ROM
610 Image decoding unit 630 Audio signal processing unit 650, 720 CPU
660 External memory 700 Electronic device 710 Semiconductor integrated circuit 740 External memory 730 Communication circuit unit 750 Input / output interface 760 Display 770 Keyboard

Claims (22)

An image decoding apparatus for decoding compressed image data with reference to reference image data that has already been decoded and stored in a frame memory,
Variable length decoding means for decoding the image data and outputting decoded data and motion vectors;
Motion compensation means for generating predicted image data based on a motion vector generated by the variable length decoding means from reference image data that has already been decoded and stored in a frame memory;
The motion compensation means includes
A first filter;
A second filter;
Control means for controlling the first filter and the second filter;
The image decoding apparatus according to claim 1, wherein the motion compensation means switches between the first filter and the second filter regardless of a filter used in motion compensation at the time of encoding. 2. The image decoding apparatus according to claim 1, wherein the first filter is constituted by a low-pass filter, and the second filter is constituted by an edge enhancement filter. The motion compensation means includes
A first motion compensation unit having a normal calculation amount;
A second motion compensation means having a smaller amount of calculation than usual;
First control means for controlling switching between the first motion compensation means and the second motion compensation means,
The first motion compensation means includes a normal calculation amount filter,
The second motion compensation means includes
A first filter having a smaller amount of computation than normal;
A second filter having a smaller amount of computation than usual;
Including second control means for controlling the first filter and the second filter;
The image decoding device according to claim 1, wherein the first control unit performs control based on a load of the arithmetic device. An image decoding device for decoding variable length encoded image data,
A variable length decoding unit that performs variable length decoding of the image data and generates a quantization coefficient and a motion vector;
An inverse quantization unit that inversely quantizes the quantization coefficient to generate a DCT coefficient;
An inverse orthogonal transform unit that performs inverse orthogonal transform on the DCT coefficient to generate difference image data;
A motion compensation unit that generates motion compensation image data using the motion vector and the reference image data;
An adder for adding the difference image data and the motion compensation image data to generate reconstructed image data;
An in-loop filter for generating decoded image data by deblocking the reconstructed image data;
A frame memory for storing the decoded image data as the reference image data,
The motion compensation unit is an image decoding device that performs filtering processing with different characteristics for each frame on the reference image data to generate the motion compensated image data. The image decoding device according to claim 4, wherein the motion compensation unit includes a plurality of filters having different filter characteristics, and generates the motion compensated image data by switching the plurality of filters for each predetermined number of frames. The motion compensation unit includes a first filter having a low-pass characteristic and a second filter having an edge enhancement characteristic, and the motion compensation unit switches the first filter and the second filter every predetermined number of frames. The image decoding device according to claim 4, which generates image data. The second filter is constituted by a tap filter having a tap number of less than 6 taps,
The image decoding apparatus according to claim 6, wherein the first filter is configured by a tap filter having a tap number less than that of the second filter. The first filter is composed of a 2-tap filter,
The image decoding device according to claim 6, wherein the second filter includes a 4-tap filter. The image decoding device according to claim 6, wherein the first filter and the second filter are configured by a circuit capable of dynamically changing a circuit configuration. A load determination unit that determines a load amount of the processor based on load information indicating a load amount of a processor that performs at least a part of the processing in decoding of the image data and processing other than the decoding processing of the image data; Prepared,
5. The image according to claim 4, wherein the motion compensation unit generates the motion compensation image data by performing filtering processing with different characteristics for each frame on the reference image data based on a determination result of the load determination unit. Decoding device. The motion compensation unit
A normal motion compensator for performing normal filter processing;
A high-speed motion compensation unit that performs filter processing with a smaller amount of computation than the filter processing performed by the normal motion compensation unit,
When the load determination unit determines that the load amount of the processor is equal to or less than a predetermined value, the motion compensation unit performs filter processing by the normal motion compensation unit,
The image decoding device according to claim 10, wherein when the load determination unit determines that the load amount of the processor exceeds the predetermined value, the motion compensation unit performs a filtering process by the high-speed motion compensation unit. . The high-speed motion compensation unit includes a 2-tap first filter having a low-pass characteristic and a 4-tap second filter having an edge enhancement characteristic, and the first filter and the second filter are arranged for each predetermined number of frames. To generate the motion compensated image data,
The said normal motion compensation part has a 6-tap 3rd filter which has a tap value (1, -5, 20, 20, -5, 1), and produces | generates the said motion compensation image data. Image decoding device. The image decoding device according to claim 12, wherein the first filter, the second filter, and the third filter are configured by a circuit that can dynamically change a circuit configuration. An image decoding method for decoding variable length encoded image data,
A variable length decoding step for variable length decoding the image data to generate a quantization coefficient and a motion vector;
An inverse quantization step of inversely quantizing the quantization coefficient to generate a DCT coefficient;
An inverse orthogonal transform step for generating difference image data by performing an inverse orthogonal transform on the DCT coefficients;
A motion compensation step of generating motion compensated image data using the motion vector and the reference image data;
An addition step of adding the difference image data and the motion compensation image data to generate reconstructed image data;
A filtering step for deblocking the reconstructed image data to generate decoded image data;
Storing the decoded image data as the reference image data,
The motion compensation step is an image decoding method in which the reference image data is subjected to filter processing with different characteristics for each frame to generate the motion compensated image data. 15. The motion compensation step generates the motion compensation image data by switching and executing a first filter process having a low-pass characteristic and a second filter process having an edge enhancement characteristic every predetermined number of frames. Image decoding method. A load determination step of determining a load amount of the processor based on load information indicating a load amount of a processor that performs at least a part of the process in decoding the image data and a process other than the decoding process of the image data; Including
15. The image according to claim 14, wherein the motion compensation step generates the motion compensation image data by performing filtering processing with different characteristics for each frame on the reference image data based on a determination result in the load determination step. Decryption method. In the load determination step, when it is determined that the load amount of the processor is equal to or less than a predetermined value, the motion compensation step performs a normal motion compensation process of a normal calculation amount,
In the load determination step, when it is determined that the load amount of the processor exceeds the predetermined value, the motion compensation step performs a high-speed motion compensation process with a smaller calculation amount than the normal motion compensation process. The image decoding method according to claim 16. The high-speed motion compensation process is executed by switching between a first filter process using a 2-tap filter having a low-pass characteristic and a second filter process using a 4-tap filter having an edge enhancement characteristic for each predetermined number of frames. Generate image data,
The normal motion compensation process performs a third filter process using a 6-tap filter having tap values (1, -5, 20, 20, -5, 1) to generate the motion compensated image data. The image decoding method as described. A variable-length decoding unit that performs variable-length decoding on the variable-length encoded image data and generates a quantization coefficient and a motion vector;
An inverse quantization unit that inversely quantizes the quantization coefficient to generate a DCT coefficient;
An inverse orthogonal transform unit that performs inverse orthogonal transform on the DCT coefficient to generate difference image data;
A motion compensation unit that generates motion compensation image data using the motion vector and the reference image data;
An adder for adding the difference image data and the motion compensation image data to generate reconstructed image data;
An in-loop filter for generating decoded image data by deblocking the reconstructed image data;
A frame memory for storing the decoded image data as the reference image data,
The motion compensation unit is a semiconductor integrated circuit that performs filtering processing with different characteristics for each frame on the reference image data to generate the motion compensated image data. The motion compensation unit includes a first filter having a low-pass characteristic and a second filter having an edge enhancement characteristic, and the motion compensation unit switches the first filter and the second filter every predetermined number of frames. 20. The semiconductor integrated circuit according to claim 19, which generates image data. A load determination unit that determines a load amount of the processor based on load information indicating a load amount of a processor that performs a part of the decoding process of the image data and a process other than the decoding process of the image data;
The motion compensation unit
A normal motion compensator for performing normal filter processing;
A high-speed motion compensation unit that performs filter processing with a smaller amount of computation than the filter processing performed by the normal motion compensation unit,
When the load determination unit determines that the load amount of the processor is equal to or less than a predetermined value, the motion compensation unit performs filter processing by the normal motion compensation unit,
20. The semiconductor integrated circuit according to claim 19, wherein when the load determination unit determines that the load amount of the processor exceeds the predetermined value, the motion compensation unit performs a filtering process by the high-speed motion compensation unit. . The high-speed motion compensation unit includes a 2-tap first filter having a low-pass characteristic and a 4-tap second filter having an edge enhancement characteristic, and the first filter and the second filter are arranged for each predetermined number of frames. To generate the motion compensated image data,
The said normal motion compensation part has a 6-tap 3rd filter which has a tap value (1, -5, 20, 20, -5, 1), and produces | generates the said motion compensation image data. Semiconductor integrated circuit.

Images