JP6837110B2 - Video decoding method - Google Patents

Description

The present invention relates to a moving image decoding technique for decoding a moving image.

The MPEG (Moving Picture Experts Group) method and other coding methods have been established as methods for recording and transmitting a large amount of moving image information as digital data. These standards predict the image to be encoded in block units using the image information for which the coding process has been completed, and encode the difference (predicted difference) from the original image to provide the redundancy of the moving image. The amount of code is reduced except for.

In particular, interscreen prediction that refers to an image different from the target image enables highly accurate prediction by searching the reference image for a block that has a high correlation with the coded target block. Further, in the coding of the predicted difference, frequency conversion, for example, Discrete Cosine Transform (DCT) is performed once in order to increase the degree of integration of numerical values, and the coefficient value after conversion is quantized. Since the predicted difference also has a strong correlation with the local region, the frequency conversion is also performed in block units in which the image is finely divided.

However, since these methods set a fixed size block (macroblock) as the basic unit of coding processing, it is possible to set a block with a size larger than the macroblock or a block that spans multiple macroblocks. However, this hindered the improvement of compression efficiency.

On the other hand, for example, in Patent Document 1, as described in paragraphs 0003 to 0005, "improvement of coding efficiency" is provided for "video material having a large region regarded as the same amount of movement such as a high-definition moving image". In order to "aim", "in a moving image coding device that predicts motion and encodes a moving image, the upper limit of the macroblock size of the picture to be encoded is set to the picture immediately before the picture and / or the picture. A technique is disclosed in which a means for optimally determining a macroblock based on a feature amount is provided, and the upper limit of the macroblock size when predicting motion can be arbitrarily selected in units of pictures or macroblocks. "

JP 2006-339774

The technique disclosed in Patent Document 1 has a problem that the prediction accuracy is lowered because the block for making a prediction is expanded. If the prediction accuracy is lowered, it causes noise that is noticeable to human eyes, and the subjective image quality is lowered.

The present invention has been made in view of the above problems, and an object of the present invention is to reduce the amount of code and improve the subjective image quality.

The processing method in the system including the moving image coding device and the moving image decoding device according to one aspect of the present invention performs the following processing. In the moving image coding device, a coded stream generation step of encoding a moving image to generate a coded stream, an input step of inputting the coded stream in the moving image decoding device, and the moving image decoding device. In the variable length decoding step of performing the variable length decoding process on the coded stream input in the input step, and in the moving image decoding apparatus, the variable length decoding process was performed in the variable length decoding step. In the reverse quantization / reverse frequency conversion step of performing the reverse quantization processing and the reverse frequency conversion processing in the first block unit or the second block unit to generate the predicted difference, and in the moving image decoding apparatus, the data is described. The decoded image is generated based on the prediction step in which the prediction processing is performed in units of the second block, the prediction difference generated in the inverse quantization / inverse frequency conversion step in the moving image decoding device, and the prediction processing result in the prediction step. The first block unit used in the inverse quantization / inverse frequency conversion step in the moving image decoding apparatus includes a plurality of decoding image generation steps to be generated, and a plurality of the first block units in the second block unit used in the prediction step. It is a block unit larger than the second block unit in which a block group consisting of blocks is integrated, and in the moving image decoding device, the inverse quantization / inverse frequency conversion step includes an in-screen prediction block in the block group. In this case, in the block group, the inverse quantization process and the inverse frequency conversion process are performed in the second block unit in the order of the upper left block, the upper right, the lower left, and the lower right, and the moving image decoding device is used. In the reverse quantization / reverse frequency conversion step, when the in-screen prediction block is not included in the block group, the reverse quantization processing and the reverse frequency conversion are performed in the first block unit in the block group. After processing, the size of the block in the first block unit is determined by the information of the flag included in the coded stream in the moving image decoding device.

According to the present invention, it is possible to more preferably reduce the amount of code and improve the subjective image quality.

It is a block diagram of the image coding apparatus used in this embodiment. It is a block diagram of the image decoding apparatus used in this embodiment. H. It is a conceptual explanatory drawing about the coding method of 264 / AVC. H. It is a conceptual explanatory drawing about the coding method of 264 / AVC. H. It is a conceptual explanatory drawing about the coding method of 264 / AVC. H. It is a conceptual explanatory drawing about the coding method of 264 / AVC. H. It is a conceptual explanatory drawing about the coding method of 264 / AVC. It is a conceptual explanatory drawing about the coding method of this embodiment. It is a conceptual explanatory drawing about the coding method of this embodiment. This is an example of a variable length code table used in this embodiment. It is a conceptual explanatory drawing about the coding method of this embodiment. It is a conceptual explanatory drawing about the coding method of this embodiment. It is a conceptual explanatory drawing about the coding method of this embodiment. This is an example of a coded stream generated by the present embodiment. It is a flowchart of the image coding apparatus used in this embodiment. It is a flowchart of the image decoding apparatus used in this embodiment. It is a conceptual explanatory drawing about the coding method of this embodiment. This is an example of a variable length code table used in this embodiment. This is an example of a coded stream generated by the present embodiment. It is a flowchart of the image coding apparatus used in this embodiment. It is a flowchart of the image decoding apparatus used in this embodiment.

Embodiment 1.
Hereinafter, with respect to Embodiment 1 of the present invention, H. This will be described in comparison with the processing in 264 / AVC. First, H. The 264 / AVC predicts the image to be coded using the image information for which the coding process has been completed, and encodes the predicted difference from the original image to reduce the redundancy of the moving image and reduce the code amount. It is reducing. Here, in order to utilize the local properties of the moving image, the prediction is performed in block units in which the image is finely divided.

As shown in FIG. 3, the coding process is executed for the target image 305 in units of macroblocks 302 composed of 16 × 16 pixels according to the raster scan order (arrows) 301. In FIG. 3, the target image 305 is composed of a coded region 306 and an uncoded region 307. Predictions are roughly divided into in-screen predictions and inter-screen predictions.

FIG. 4 shows H. The operation of the inter-screen prediction processing by 264 / AVC is conceptually shown. When performing inter-screen prediction, the decoded image of the coded image included in the same image 401 as the coded target image 403 is used as the reference image 402, and a block having a high correlation with the target block 404 in the target image (predicted image). ) 405 is searched for in the reference image 402.

At this time, in addition to the predicted difference calculated as the difference between the two blocks, the motion vector 406 represented as the difference between the coordinate values of the two blocks is encoded as the side information required for the prediction. On the other hand, in the case of decoding, the reverse procedure described above may be performed, and the decoded image can be obtained by adding the decoded predicted difference to the block (predicted image) 405 in the reference image.

In addition, H. The 264 / AVC can make the above prediction by dividing the macroblock into smaller blocks. FIG. 5 shows a macroblock division pattern that is allowed when performing inter-screen prediction. That is, H. 264 / AVC is optimal in predicting each macroblock 502 in the target image 501 from among the predefined division patterns (macroblock division patterns) 503 from 4 × 4 pixel size to 16 × 16 pixel size. You can choose one. Information indicating which division pattern was used for each macroblock is encoded in macroblock units.

On the other hand, the prediction difference generated by the prediction processing is decomposed into frequency components by DCT (Discrete Cosine Transformation), which is one of the frequency conversion methods, and the coefficient value thereof is encoded. FIG. 6 conceptually shows how the predicted difference is decomposed into frequency components by the DCT. DCT is a method of frequency conversion in which an input signal is expressed by a weighted sum of a basis signal 603 and its coefficient value. By applying the DCT to the predicted difference 601 the coefficient value 602 is often biased toward the low frequency component, so that variable length coding can be performed efficiently.

In addition, H. In 264 / AVC, the macroblock can be divided into smaller blocks and the DCT can be applied to the predicted difference, but the block size when the DCT is performed is fixed, for example, H.I. The 264 / AVC Baseline profile stipulates that the size be 4 x 4 pixels, as shown in FIG. In FIG. 7, the macroblock 702 with the predicted difference 701 is divided into 4 × 4 small blocks (pixels) (703 in FIG. 7).

As described above, H. The 264 / AVC achieves high performance by adaptively dividing an image into small blocks and coding the image. However, H. Since the 264 / AVC uses a macroblock as a basic unit for coding processing, it has not been possible to handle a block having a size larger than the macroblock or a block that straddles a plurality of macroblocks. These restrictions on the block shape were one of the factors hindering the improvement of compression efficiency.

In general, when a block having a small size is used, fine processing is possible, so that the accuracy of prediction and DCT is improved and the image quality is improved. However, on the other hand, there is a problem that the amount of code increases when a small block is used. This is due to the increase in the number of blocks in the image. For example, when performing inter-screen prediction, it is necessary to encode the motion vector required for the prediction process for each block. Therefore, as the number of blocks increases, the number of motion vectors also increases and the code amount increases.

Further, when DCT is performed, as the number of blocks increases, the number of significant low-frequency components in the DCT coefficient increases, so that the efficiency of VLC (Variable Length Coding) decreases and the code amount increases. To do. Therefore, it is necessary to consider such a trade-off between image quality and code amount when determining an appropriate block size.

On the other hand, in recent years, there has been an increasing demand for high-definition images exceeding high-definition such as digital cinema and super HD, and it is desired to introduce a method for efficiently encoding these high-definition images. In general, it is known that high-definition video with high resolution has a high correlation in the screen, so that the deterioration of image quality is small even if a large-sized block is used.

Therefore, if the coding target is narrowed down to a high-resolution video, the compression rate can be dramatically improved by increasing the size of the block that is the coding processing unit. For example, the technique of Patent Document 1 makes it possible to change the size of a macroblock, and adaptively changes the upper limit value according to the feature amount of the coded region. According to this method, the macroblock can be enlarged according to the nature of the image, and the compression efficiency of the high-definition video can be particularly improved.

However, this method has a problem that the prediction accuracy is lowered because the block for making a prediction is expanded. If the prediction accuracy is lowered, it causes noise that is noticeable to human eyes, and the subjective image quality is lowered.

The present embodiment improves the above-mentioned problems and further reduces the amount of code while maintaining the subjective image quality more preferably. Specifically, in the present embodiment, the inter-screen prediction processing is performed finely in small block units, for example, macroblock units, while the applied size of frequency conversion (in this embodiment, DCT is used as an example) for the prediction difference is determined. Make it expandable.

For example, FIG. 8 shows the predicted difference 801. As shown in FIG. 8, even when the target image has a complicated texture composed of a plurality of objects, if the prediction accuracy is high, the prediction difference has a gentle distribution with many low frequency components, and the DCT is in large block units. Even if this is applied, the deterioration of image quality is reduced. Therefore, the code amount of the DCT coefficient can be significantly reduced by integrating the prediction differences of a plurality of adjacent blocks for the region 802 with high prediction accuracy, forming a large block, and applying the DCT.

Further, for the region 803 where the prediction accuracy is low, such as a part of an object with complicated movement, the distribution of the prediction difference becomes complicated and the high frequency component increases, so that the image quality deterioration is conspicuous when DCT is applied in large block units. .. Therefore, for such areas where the prediction accuracy is low, the blocks are not integrated, and the image quality is maintained by applying DCT in block units that are the same as or smaller than the blocks when the prediction was performed. Can be done. As described above, by integrating the predicted differences of a plurality of blocks and applying DCT, the compression rate can be increased and the code amount can be reduced.

Hereinafter, the details of this embodiment will be described. The process described in this embodiment will be described as being applied to a frame (P slice or B slice in H.264 / AVC) capable of performing interscreen coding. The processing described in the following embodiments may or may not be applied to the frame (I slice in H.264 / AVC) that encodes the entire area in the screen. good.

In the first embodiment, when all areas are coded between screens in a frame (P slice or B slice in H.264 / AVC) capable of performing interscreen coding, that is, an intermacro in the screen. An example will be described in which only a block (a macro block that performs inter-screen coding) exists and an intra-macro block (a macro block that performs in-screen coding) does not exist.

FIG. 9 shows an example of the block size used for the DCT in this embodiment. Here, the predicted difference 901 is generated by, for example, the following method. This method is described in, for example, H. Block division is performed in macroblock units of 16 × 16 pixel size by the same means as 264 / AVC (FIG. 5), inter-screen prediction of each macroblock is performed, and the prediction difference is integrated for one screen. In the present embodiment, when DCT is applied to this predicted difference, for example, a block group 902 (64 × 64 pixels) in which 16 adjacent macroblocks are integrated is formed, and block division is performed for each block group.

However, the size of the block group 902 is not limited to the size of 64 × 64 pixels, and any size such as 32 × 32 or 128 × 128 may be used as long as a plurality of macroblocks 903 are integrated. One preferable method is to prepare many types such as 8 × 8 pixels, 16 × 16 pixels, 32 × 32 pixels, and 64 × 64 pixels as the division pattern 903 of the block group 902 in advance, and the optimum method is selected from among them. Select a pattern and apply DCT.

At that time, for example, using a code table as shown in FIG. 10, information indicating which pattern is selected is encoded for each block group. Here, the total code amount can be reduced by assigning a short code length to the frequently selected pattern. Further, it is effective to select the block pattern by using, for example, the cost function shown in Equation 1 and determining that the division pattern that minimizes the cost function is optimal.

However, in Equation 1, Dist represents the sum of errors between the original image and the decoded image, Rate represents the sum of the code amount of the DCT coefficient and the code amount of the block division pattern, and Weight represents the weight coefficient. Here, the trade-off between the image quality and the code amount can be controlled by adjusting the Weight value. For example, if it is desired to significantly reduce the code amount even if the image quality is slightly deteriorated, the Weight value may be set high so that the contribution rate of the code amount to the cost value becomes large.

FIG. 11 shows a procedure for coding the predicted difference for each block. In this coding, first, DCT is applied to the predicted difference 1101 of the target block, and the DCT coefficient 1102 is acquired. Subsequently, the DCT coefficient 1102 is quantized to reduce the number of elements to be encoded. At this time, if DCT is applied with a large block size as in the present embodiment, a large DCT coefficient is generated in the high frequency component and the code amount increases. Therefore, for example, a large quantization step is applied to the high frequency component of the DCT coefficient. By setting the weight 1103 of the quantization step as described above, high frequency components can be significantly reduced and coding can be performed efficiently. However, in this figure, the reference quantization step is represented as Q.

Subsequently, the quantized DCT coefficient 1104 is subjected to one-dimensional expansion by scanning in a two-dimensional zigzag direction from a low frequency component to a high frequency component (1105), and VLC is performed to generate a code word (1105). 1106). The above process is repeated for all the blocks in which the block group is divided.

The processing order for each block group may be any, and an example thereof is shown in FIG. Here, an example of processing a block group according to the order of raster scan is shown. First, the block group 1201 located at the upper left corner of the screen is processed, and then the block group 1202 adjacent to the right side of the block group 1201 is processed. After that, the processing is further advanced for the block group 1203 and the block group 1204 adjacent to the right side, and when the processing reaches the right end of the screen, the block group 1205 adjacent to the lower side of the block group 1201 is processed. Perform the above processing until the lower right corner of the screen is reached. At this time, the processing order of the macroblocks included in the same block group may be any, but it is effective to process them along the zigzag direction 1210, for example.

Further, in the present embodiment, in order to integrate the predicted differences by a plurality of macroblocks and perform DCT, a memory for temporarily storing the predicted differences is required. The area stored in the memory at one time is called an "access group". At this time, the prediction and the DCT are performed for each access group. In this coding method, for example, when the entire screen is set as one access group, all macroblocks in the screen are predicted and sequentially stored in the memory.

Subsequently, the predicted difference for one screen stored in the memory is divided into blocks for each block group, and DCT is applied. The access group may be set in any range, but for example, if one access group is composed of a block group for one line as shown in FIG. 13, coding can be performed efficiently.

In this case, first, the access group 1311 composed of the block group 1301 to the block group 1304 located on the top line of the screen is predicted and DCT is performed, and then the block group 1305 to the block group located on the next line is performed. Prediction and DCT are performed on the access group 1312 composed of 1308. If this is continued until the bottom line of the screen is reached, the coding process for one frame is completed.

FIG. 14 shows a configuration example (for one block group) of the coded stream in the present embodiment. Here, a case where 16 macroblocks, which are the basic units of prediction processing, exist in the corresponding block group will be described. Here, first, as a combination of a prediction method (forward screen-to-screen prediction, reverse screen-to-screen prediction, bidirectional screen-to-screen prediction, in-screen prediction, etc.) and its block division pattern for the first macro block (macro block 1). The represented prediction mode 1401 is encoded, and then the motion vector in each block is encoded as the side information 1402 required for the prediction.

Subsequently, the prediction mode 1403 for the second macro block 2 and the motion vector 1404 in each block obtained by dividing the macro block are encoded. This is repeated for all macroblocks included in the corresponding block group. Subsequently, the block division pattern 1405 when DCT is applied to the predicted difference of the corresponding block group and the DCT coefficient 1406 of each block are encoded. At this time, the block size for performing DCT may be set to a fixed value such as 64 × 64, and in this case, the coding of the block group division pattern 1405 is unnecessary.

FIG. 1 shows an example of a moving image coding device according to the present embodiment. The motion image coding device performs in-screen prediction in block units, the input image memory 102 that holds the input original image 101, the block division unit 103 that divides the image in the input image memory 102 into blocks, and the block unit. An in-screen prediction unit 104, an inter-screen prediction unit 106 that makes inter-screen predictions based on motion vectors detected by the motion search unit 105, and a prediction method and a prediction method that determines a block shape that matches the properties of an image. It has a block determination unit 107.

The moving image coding device further includes a subtraction unit 108 for generating a predicted difference, a DCT unit 110 that performs frequency conversion on the predicted difference, and a frequency that determines a block shape of frequency conversion that matches the nature of the predicted difference. The conversion block determination unit 116, the quantization processing unit 111 that performs quantization on the coefficient value after frequency conversion, and the variable length coding processing unit 112 that performs coding according to the occurrence probability of the symbol, are coded once. Holds an inverse quantization processing unit 113 and an inverse DCT unit 114 for decoding the decoded predicted difference, an addition unit 115 for generating a decoded image using the decoded predicted difference, and a decoded image. It has a reference image memory 117 for later prediction.

The input image memory 102 holds one image from the original image 101 as an image to be encoded. The block division unit 103 divides the image data into blocks of an appropriate size and sends the image data to the in-screen prediction unit 104, the motion search unit 105, the inter-screen prediction unit 106, and the subtraction unit 108. The motion search unit 105 calculates the amount of motion of the corresponding block using the decoded image stored in the reference image memory 117, and sends the motion vector to the inter-screen prediction unit 106. The in-screen prediction unit 104 and the inter-screen prediction unit 106 execute the in-screen prediction processing and the inter-screen prediction processing in block units of several types of shapes. The prediction method / block determination unit 107 selects the optimum prediction method and block shape (macroblock division pattern).

Subsequently, the subtraction unit 108 generates a prediction difference by the optimum prediction coding means using the original image and the prediction result, and sends the prediction difference to the prediction difference memory 109. The predicted difference memory 109 sends the predicted difference to the DCT unit 110 when the predicted difference for one access group is stored. The DCT unit 110 and the quantization processing unit 111 are divided into blocks having several types of shapes in block group units to perform frequency conversion and quantization processing such as DCT, respectively, and the variable length coding processing unit 112 and the inverse quantization processing unit 112. Send to 113. The inverse quantization processing unit 113 and the inverse DCT unit 114 perform inverse quantization and inverse frequency conversion (for example, IDCT (Inverse DCT)) on the frequency conversion coefficient after quantization, respectively, and acquire a predicted difference. And send it to the addition unit 115.

Subsequently, the addition unit 115 generates a decoded image. The frequency conversion block determination unit 116 and the reference image memory 117 store the decoded image. The frequency conversion block determination unit 116 determines the optimum block shape (block group division pattern) when performing frequency conversion, and sends the information to the variable length coding processing unit 112. Further, the variable-length coding processing unit 112 provides optimum block shape information (macroblock, division pattern of block group) when performing prediction / frequency conversion, frequency conversion coefficient (prediction difference information) based on the optimum block shape, and Side information required for prediction processing at the time of decoding (for example, prediction direction when performing in-screen prediction and motion vector when performing inter-screen prediction) is variable-length coded based on the occurrence probability of a symbol and encoded stream. To generate.

FIG. 2 shows an example of the moving image decoding apparatus according to the present embodiment. The moving image decoding device is, for example, a variable length decoding device for decoding various information by performing the reverse procedure of variable length coding on the coded stream 201 generated by the moving image coding device shown in FIG. The conversion unit 202, the inverse quantization processing unit 203 and the inverse DCT unit 204 for decoding the prediction difference information, the prediction difference memory 205 for storing the prediction difference for one access group, and the screen-to-screen prediction are performed. An inter-screen prediction unit 206, an in-screen prediction unit 207 for performing in-screen prediction, an addition unit 208 for acquiring a decoded image, and a reference image memory 209 for temporarily storing the decoded image. Have.

The variable-length decoding unit 202 performs variable-length decoding of the coded stream 201, and acquires block shape information when performing prediction and frequency conversion, prediction difference information, and side information necessary for prediction processing at the time of decoding. Of these, the block shape information (block group division pattern) and prediction difference information for frequency conversion are sent to the inverse quantization processing unit 203, and the block shape information (macroblock division pattern) for prediction is performed. The side information required for the prediction process at the time of decoding is sent to the inter-screen prediction unit 206 or the in-screen prediction unit 207.

Subsequently, the inverse quantization processing unit 203 and the inverse DCT unit 204 have the block shape (block group division pattern) specified for each block group, respectively, for inverse quantization and inverse frequency conversion such as inverse DCT for the predicted difference information. Is performed to perform decoding, and the data is sent to the predicted difference memory 205. Subsequently, the inter-screen prediction unit 206 or the in-screen prediction unit 207 has a block shape (macroblock division pattern) designated with reference to the reference image memory 209 based on the information sent from the variable length decoding unit 202. Execute the prediction process at. The addition unit 208 generates a decoded image from the result of the prediction process and the prediction difference for one access group stored in the prediction difference memory 205, and stores the decoded image in the reference image memory 209.

FIG. 15 shows a one-frame coding processing procedure in the present embodiment. First, the following processing is performed on all the regions existing in the frame to be encoded (1501). That is, for all macroblocks in the access group (1502), all available prediction methods (forward screen-to-screen prediction, backward screen-to-screen prediction, two-way screen-to-screen prediction, in-screen prediction, etc.) and Prediction is executed with the block shape (macro block division pattern) (1503), and the prediction difference is calculated.

Then, the optimum combination is selected from the results of prediction by all the prediction methods and block shapes (1504), the information of the combination is encoded, and the prediction difference is stored in the memory. The term "optimal" as used herein means a case where both the predicted difference and the code amount are small. For selecting the combination of the prediction method and the block shape, for example, it is effective to calculate the coding cost (Cost) represented by the equation 2 for all the combinations and select the combination having the smallest value.

Here, SAD (Square Absolute Difference) represents the sum of the absolute values of the predicted differences, and R (Rate) represents the estimated value of the code amount when the predicted differences are encoded. In addition, λ is a constant for weighting, and the optimum value differs depending on the prediction method (in-screen prediction / inter-screen prediction) and parameters at the time of quantization, so it is effective to use different values according to these. Is the target. It is desirable to calculate the estimated value of the code amount in consideration of not only the predicted difference information but also the code amount such as block shape information and motion vector.

When the above processing is completed for all macroblocks in the access group, all the predicted differences for the corresponding access group stored in the memory (1505) are subsequently available for each block group. DCT (1506), quantization (1507), and variable length coding (1508) are performed on the block shape (division pattern of the block group). Then, the DCT coefficient after quantization is subjected to inverse quantization (1509) and inverse DCT (1510) to decode the predicted difference information, and further, the optimum block shape (division of the block group) is performed by using Equation 1. Pattern) is selected (1511) and its shape information is encoded.

Further, for the selection of the shape information, in addition to the mathematical expression 1, for example, an RD-Optimization method for determining the optimum coding mode from the relationship between the image quality distortion and the code amount can be used. The RD-Optimization method is a widely known technique, and detailed description thereof will be omitted here. For details, see, for example, Reference 1 (Reference 1: G. Sullivan and T. Wiegand: “Rate-Distortion Optimization for Video Compression”, IEEE Signal Processing Magazine, vol.15, no.6, pp. .74-90, 1998.).

Subsequently, the decoded image is acquired by adding the decoded predicted difference and the predicted image (1512), and stored in the reference image memory. When the above processing is completed for all the access groups, the coding for one frame of the image is completed (1513).

FIG. 16 shows a one-frame decoding processing procedure in the present embodiment. First, the following processing is performed for all access groups in one frame (1601). That is, all the block groups in the access group are subjected to the variable length decoding process (1603), and the inverse quantization process (1604) is performed with the specified block shape (block group division pattern). Inverse DCT (1605) is applied to decode the predicted difference and store it in the memory.

When the above processing is completed for all the block groups in the access group, subsequently, for the same access group (1606), the variable length decoded prediction method and the block shape for prediction (macroblock). A decoded image is acquired (1608) by performing prediction (1607) based on the division pattern) and adding it to the prediction difference stored in the memory. When the above processing is completed for all the access groups in the frame, the decoding for one frame of the image is completed (1609).

In this embodiment, DCT is given as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT (Karhunen) Any orthogonal transform used for inter-pixel correlation removal, such as -Loeve Transformation, may be used, and the predicted difference itself may be encoded without any frequency transformation. .. Further, variable length coding does not have to be performed. Moreover, this embodiment may be used in combination with another method.

According to the moving image coding device and the coding method, the moving image decoding device and the decoding method according to the first embodiment described above, the coding and decoding of the frame composed of the intermacroblock can be predicted. A moving image coding device and a coding method, a moving image decoding device and a decoding method that further reduce the amount of coding while maintaining the subjective image quality more favorably by performing frequency conversion in block units larger than the macro block used. Can be realized.

Embodiment 2.
The first embodiment is a case where all areas are coded between screens in a frame (P slice or B slice in H.264 / AVC) capable of performing interscreen coding, that is, an intermacro in the screen. An example has been described in which only a block (a macro block that performs inter-screen coding) exists and an intra-macro block (a macro block that performs in-screen coding) does not exist.

On the other hand, in the second embodiment, when in-screen coding can be applied in a frame (P slice or B slice in H.264 / AVC) capable of performing interscreen coding, that is, an intermacroblock ( A case where it is possible to perform coding by mixing an intra-screen macro block (a macro block that performs inter-screen coding) and an intra-macro block (a macro block that performs in-screen coding) will be described.

In the intra macroblock, prediction is performed using the decoded image of the block that has already been encoded. Therefore, when predicting the target block, it is necessary that the DCT processing in the adjacent block is completed. Therefore, it is not possible to make a batch prediction for one block group of macroblocks and integrate the prediction differences to perform DCT.

Therefore, for a block group including at least one intra macroblock, prediction and DCT are performed in macroblock units. At this time, the DCT process is performed in block units in which the macro block is divided, and the size is smaller than or equal to the macro block. Therefore, 32 × 32, 64 × 64, etc. in FIG. 9 cannot be used as the block size (macroblock division pattern for DCT) in the DCT process. At this time, for example, the division pattern is coded using the code table shown in FIG.

FIG. 17 conceptually shows an example of a coding method for each block group. This example shows the case where the access group matches the block group. The coding process first predicts all the macroblocks included in the block group 1701 located at the upper left corner of the image, and if all the macroblocks are inter-macroblocks, the same means as in the first embodiment (FIG. In 9), the division pattern of the block group is determined and DCT is applied. Subsequently, the same processing is performed for the block group 1702 adjacent to the right side of the block group 1 as long as the intra macroblock is not included.

Further, the same processing is performed for the block group 1703 and the block group 1704, and when the right edge of the screen is reached, the block group 1705 adjacent to the lower side of the block group 1701 is processed. Here, for example, if one or more macroblocks included in the block group 1706 are intra macroblocks, prediction and DCT are performed for this block group in macroblock units. As described above, in the present embodiment, the unit for performing DCT is switched between the macroblock and the macroblock group depending on whether or not the corresponding macroblock group includes the intra macroblock. When the process reaches the lower right corner of the screen, the coding for the position screen ends.

FIG. 19 shows a configuration example of a coded stream for a block group in which one or more intra macroblocks exist in the present embodiment. Here, a case where 16 macroblocks exist in the corresponding block group will be described. First, the prediction method (forward screen-to-screen prediction, reverse screen-to-screen prediction, bidirectional screen-to-screen prediction, in-screen prediction, etc.) and its division pattern when making a prediction for the first macro block (macro block 1). The prediction mode 1901 represented as a combination is encoded. Subsequently, as the side information 1902 required for the prediction, the motion vector is encoded in the case of the intermacroblock, and the information regarding the prediction direction is encoded in the case of the intramacroblock.

Subsequently, the macroblock block division pattern 1903 when DCT is applied to the same macroblock and the DCT coefficient 1904 of each block are encoded. The above processing is performed for all macroblocks included in one block group. Further, the block size for performing DCT may be set to a fixed value such as 8 × 8, and in this case, coding of the division pattern in macroblock units is unnecessary. The configuration of the coded stream for the block group in which no intra macroblock does not exist is the same as that in the first embodiment (FIG. 14).

FIG. 20 shows a coding processing procedure in a block group in which one or more intra macroblocks exist in the present embodiment. This process performs all available prediction methods (forward screen-to-screen prediction, backward screen-to-screen prediction, two-way screen-to-screen prediction, in-screen prediction) for all macro blocks included in the block group (2001). Etc.) and the block shape (macro block division pattern), the prediction is executed (2002), and the prediction difference is calculated.

Then, a suitable combination is selected from the results of prediction by all the prediction methods and block shapes (2003), and the information of the combination is encoded. The term “preferable” here means that both the predicted difference and the code amount are small, and it is effective to use the cost function represented by Equation 2 or another equation for the evaluation.

Subsequently, for the predicted difference of the same macroblock, DCT (2004), quantization (2005), and variable length coding (2006) are performed with available block shapes (macroblock division patterns to which DCT is applied). Do. Then, the DCT coefficient after quantization is subjected to inverse quantization (2007) and inverse DCT (2008) to decode the predicted difference information, and further, the optimum block shape (for DCT) is used by using Equation 1. Macroblock division pattern) is selected (2009) and its shape information is encoded. For the selection of the shape information, another method or another method of Reference 1 described above may be used.

Subsequently, the decoded image is acquired by adding the decoded predicted difference and the predicted image (2010), and is stored in the reference image memory. When the above processing is completed for all the macroblocks included in the corresponding block group, the coding of the corresponding block group is completed. The coding processing procedure in the block group in which the intra macroblock does not exist is the same as in the first embodiment (FIG. 15).

FIG. 21 shows a decoding processing procedure in a block group in which one or more intra macroblocks exist in the present embodiment. In this process, all macroblocks included in the block group are subjected to variable length decoding process (2102), and dequantization process (2103) and inverse DCT (2104) are performed in the specified block shape. ) Is applied to decode the predicted difference and store it in the memory.

Subsequently, prediction (2105) is performed based on the variable-length decoded prediction method and the block shape (macroblock division pattern for DCT) at the time of prediction, and the prediction difference is added to the prediction difference stored in the memory. By doing so, the decoded image is acquired (2106). When the above processing is completed for all the macroblocks included in the corresponding block group, the decoding of the corresponding block group is completed. The decoding processing procedure in the block group in which the intra macroblock does not exist is the same as in the first embodiment (FIG. 16).

The moving image coding device and the moving image decoding device that perform the processing of the second embodiment are each component of the configuration of the moving image coding device and the moving image decoding device of the first embodiment shown in FIGS. 1 and 2. Since the operation of the above operation may be changed to the above-mentioned operation, the description of the configuration itself will be omitted.

In this embodiment, DCT is given as an example of frequency transformation, but DST (Discrete Sine Transformation), WT (Wavelet Transformation), DFT (Discrete Fourier Transformation), KLT (Karhunen) -Loeve Transformation: Any orthogonal transform used for inter-pixel correlation removal, such as Carunen-Reve transformation, may be used, and the predicted difference itself may be encoded without any particular frequency transformation. .. Further, variable length coding does not have to be performed. Moreover, this embodiment may be used in combination with another method.

According to the moving image coding device and coding method, moving image decoding device and decoding method according to the second embodiment described above, coding and decoding of a frame having a configuration in which inter-macro blocks and intra-macro blocks coexist. In the case of conversion, a moving image coding device, a coding method, and a moving image decoding that further reduce the amount of coding while maintaining the subjective image quality more favorably by performing frequency conversion in block units larger than the macro block used for prediction. A device and a decoding method can be realized.

Although the present invention has been described in detail with reference to the accompanying drawings, the present invention is not limited to such a specific configuration, and various changes and equivalents within the scope of the attached claims. It includes the composition of.

The following processing can be mentioned as a typical method from the viewpoint of the present invention other than those described in the claims.

(1) Input the coded stream. A variable length decoding process is performed on the input coded stream. The data subjected to the variable length decoding process is subjected to an inverse quantization process and an inverse frequency conversion process in units of the first block or the second block to generate a predicted difference. At this time, when the block group includes an in-screen prediction block, in the block group, the inverse quantization process and the inverse frequency conversion process are performed in the second block unit, and the screen is displayed on the block group. When the internal prediction block is not included, the inverse quantization process and the inverse frequency conversion process are performed in the first block unit in the block group. Prediction processing is performed in units of the second block. A decoded image is generated based on the generated prediction difference and the prediction processing result. One block in the first block unit is one block in which a block group composed of a plurality of blocks in the second unit is integrated.

(2) Input the coded stream. A variable length decoding process is performed on the input coded stream. The data subjected to the variable length decoding process is subjected to an inverse quantization process and an inverse frequency conversion process in units of the first block or the second block to generate a predicted difference. Prediction processing is performed in units of the second block. A decoded image is generated based on the generated prediction difference and the prediction processing result. One block in the first block unit is one block in which a block group composed of a plurality of blocks in the second unit is integrated. When the in-screen prediction block is included in the block group, the inverse quantization process and the inverse frequency conversion process are performed in the second block unit in the block group. When the in-screen prediction block is not included in the block group, the inverse quantization process and the inverse frequency conversion process are performed in the first block unit in the block group. In the coded stream to be input, the stream configuration of the block group including the in-screen prediction block includes the information of the prediction mode of the second block unit and the frequency conversion coefficient of the second block unit. The stream configuration of the block group that does not include the in-screen prediction block includes the information of the prediction mode of the second block unit and the frequency conversion coefficient of the first block unit.

(3) Input The image is input. Prediction processing is performed on the input image in units of the first block to generate a prediction difference. Quantization data is generated by performing frequency conversion processing and quantization processing on the generated predicted difference. At this time, when the in-screen prediction block is included in the block group composed of a plurality of blocks, the frequency conversion process and the quantization process are performed in the first block unit in the block group, and the block group is subjected to the frequency conversion process and the quantization process. When the in-screen prediction block is not included, the frequency conversion process and the quantization process are performed in the second block unit having a size in which a plurality of blocks of the first block unit are integrated in the block group. Do. Variable-length coding is performed on the generated quantization data to generate a coded stream.

(4) Input The image is input. Prediction processing is performed on the input image in units of the first block to generate a prediction difference. Quantization data is generated by performing frequency conversion processing and quantization processing on the generated predicted difference. Variable-length coding is performed on the generated quantization data to generate a coded stream. When an in-screen prediction block is included in a block group composed of a plurality of blocks, the frequency conversion process and the quantization process are performed in the first block unit in the block group, and the in-screen prediction block is added to the block group. When is not included, the frequency conversion process and the quantization process are performed in the second block unit having a size in which a plurality of blocks of the first block unit are integrated in the block group. In the stream configuration of the block group consisting of the plurality of blocks of the generated coded stream, when the block group includes an in-screen prediction block, the stream configuration of the block group includes the first block. When the unit prediction mode information and the frequency conversion coefficient of the first block unit are included and the in-screen prediction block is not included in the block group, the information of the prediction mode of the first block unit is included. The frequency conversion coefficient of the second block unit is included.

As described above, the present invention can be applied to coding / decoding of moving images, and in particular, can be applied to coding / decoding in block units.

Claims (1)

An input step of inputting a mark Goka stream,
A variable length decoding step for performing variable length decoding on the coded stream inputted by the entering force step,
The data subjected to the variable length decoding process by pre-Symbol variable length decoding step, inverse of generating a prediction difference by performing an inverse quantization process and an inverse frequency conversion processing in the first block or the second block Quantization / inverse frequency conversion steps and
A prediction step of performing prediction processing before Symbol second blocks,
And a decoded image generation step of generating a decoded image based on the predicted process result in said prediction step and the predicted differential generated in the previous Kigyaku quantization and inverse frequency conversion step,
Before the first block used in Kigyaku quantization and inverse frequency conversion step, the than the second block unit with integrated block group including a plurality of blocks in said second blocks used in the prediction step It is a large block unit,
Before Kigyaku quantization and inverse frequency conversion step, wherein, when included in the group of blocks intra prediction block in the block group, the upper right from the upper left block, lower left, in the order of lower right, the second Inverse quantization processing and inverse frequency conversion processing are performed in block units of
Before Kigyaku quantization and inverse frequency conversion step, when said block group does not include intra prediction block in the block group, said first inverse quantization process in units of blocks and inverse frequency transform And
Processing in units of the first block that completes the inverse quantization processing and the inverse frequency conversion processing for the block group and proceeds to the inverse quantization processing and the inverse frequency conversion processing of the next block group having the size of the first block. The order follows the order of raster scans,
Before SL size of the block of the first block unit is determined by the information of the flag included in the coded stream, the moving picture decoding method.

Images