JP2006005438A - Image processing apparatus and method - Google Patents

Abstract

An image processing apparatus capable of appropriately determining a prediction mode of intra prediction of each of a plurality of blocks constituting a two-dimensional image from the viewpoint of coding efficiency.
In an intra prediction circuit, when a predetermined intra prediction mode around block data to be processed by an MPM selection circuit 52 has not been determined, a prediction mode is determined based on the already determined intra prediction mode. The intra prediction mode used for generating the designated data PREV and REM is specified in a pseudo manner.
[Selection] Figure 8

Description

  The present invention relates to an image processing apparatus and method for performing intra prediction of image data.

  In recent years, image data has been handled as digital data. At that time, MPEG (compressed by orthogonal transformation such as discrete cosine transformation and motion compensation is used for the purpose of efficient transmission and storage of information, and using redundancy unique to image information. Devices conforming to the system such as Moving Picture Experts Group (2, 4) are becoming widespread in both information distribution at broadcasting stations and information reception in general households.

Following MPEG, MPEG-AVC (Advanced Video
A coding method called “Coding” has been proposed.
In MPEG-AVC, intra prediction is performed as in MPEG, but a plurality of intra prediction modes are defined, and an intra prediction mode that minimizes the prediction coding amount of each block data is determined.
In the MPEG-AVC intra prediction, for example, prediction mode designating data indicating the determined intra prediction mode is added to encoded data as header data.
The prediction mode designating data is generated based on the intra prediction mode determined for the block data around the block data to be processed, and the prediction code amount depends on the intra prediction mode determined for the surrounding block data. .
By the way, as the intra prediction mode of the block data, a block data that minimizes the prediction code amount of the block data is selected and determined. However, the prediction code amount corresponds to the prediction code amount of the prediction mode designation data of the block data. Dependent.

By the way, in the above-described MPEG-AVC intra prediction, the intra prediction modes of a plurality of block data constituting two-dimensional image data are determined by pipeline processing in a predetermined order.
For this reason, the intra prediction mode of the surrounding block data of the block data may not be determined at the timing of determining the intra prediction mode of the block data to be processed. In this case, the prediction mode designation data for the block data to be processed cannot be generated. As a result, the prediction code amount cannot be obtained, and the intra prediction mode may not be determined.

  In order to solve the above-described problems of the prior art, the present invention provides an image processing apparatus and an image that can appropriately determine the prediction mode of intra prediction of each of a plurality of blocks constituting a two-dimensional image from the viewpoint of coding efficiency. An object is to provide a processing method.

  In order to solve the above-described problems of the prior art and achieve the above-described object, the image processing apparatus of the first invention selects each of a plurality of blocks constituting a two-dimensional image in a predetermined order, and performs the selection. The intra prediction is performed based on pixel data of blocks other than the block defined by the prediction mode, and the prediction mode in which the encoding efficiency satisfies a predetermined criterion is selected from the plurality of prediction modes. An image processing apparatus for determining a first prediction mode of a block, wherein the determination unit determines whether or not the first prediction mode of a predetermined block other than the selected block has already been determined. And when the determining means determines that the first prediction mode of the predetermined block has already been determined, The second prediction mode used to generate prediction mode designation data for designating the first prediction mode determined for the selected block based on the first prediction mode of the block And when the determination means determines that the first prediction mode of the predetermined block has not been determined, the previously determined blocks other than the predetermined block have been determined. Based on one prediction mode, prediction mode specifying means for specifying the second prediction mode, and for each of the plurality of prediction modes, the prediction mode is determined as the first prediction mode of the selected block. A code amount determining unit that determines a prediction code amount of the prediction mode designating data when the prediction mode is specified based on the second prediction mode specified by the prediction mode specifying unit A prediction mode for determining, for the selected block, the first prediction mode in which the encoding efficiency satisfies a predetermined criterion among the plurality of prediction modes based on the prediction code amount determined by the code amount determination unit Determination means.

The operation of the image processing apparatus of the first invention is as follows.
A determination means determines whether the first prediction mode of a predetermined block other than the selected block has already been determined.
Next, when the determination unit determines that the first prediction mode of the predetermined block has already been determined, the prediction mode specifying unit determines the first of the predetermined block. Based on the prediction mode, the second prediction mode used to generate prediction mode specifying data for specifying the first prediction mode determined for the selected block is specified.
On the other hand, when the determination unit determines that the first prediction mode of the predetermined block is undecided, the prediction mode specifying unit has already detected blocks other than the predetermined block. The second prediction mode is specified based on the determined first prediction mode.
Next, for each of the plurality of prediction modes, a code amount determination unit determines a prediction code amount of the prediction mode designation data when the prediction mode is determined as the first prediction mode of the selected block. The determination is made based on the second prediction mode specified by the prediction mode specifying means.
Next, for the selected block, the prediction code determined by the code amount determination unit for the selected block is the first prediction mode in which the encoding efficiency satisfies a predetermined criterion among the plurality of prediction modes. Determine based on amount.

  In the image processing method of the second invention, each of a plurality of blocks constituting a two-dimensional image is selected in a predetermined order, and pixel data of blocks other than the block defined by the prediction mode is selected for the selected block. An image processing method for performing intra prediction based on the prediction mode, and determining the prediction mode satisfying a predetermined standard of coding efficiency among a plurality of the prediction modes as the first prediction mode of the selected block, A first step of determining whether the first prediction mode of a predetermined block other than the determined block has already been determined, and the first prediction mode of the predetermined block has already been determined The selected block based on the first prediction mode of the block determined in advance when it is determined in the first step that Identifying the second prediction mode used to generate prediction mode designating data for designating the first prediction mode determined for the first prediction of the predetermined block When it is determined in the first step that a mode has not yet been determined, the second prediction mode is based on the first prediction mode already determined for a block other than the predetermined block. A prediction code amount of the prediction mode designation data when the prediction mode is determined as the first prediction mode of the selected block for each of the plurality of prediction modes. A third step of determining based on the second prediction mode identified in the second step, and encoding efficiency among the plurality of prediction modes for the selected block. And a fourth step of determining the first prediction mode which satisfies a predetermined standard, based on the predicted code amount determined in the third step.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus and image processing method which can determine appropriately the prediction mode of each intra prediction of the some block which comprises a two-dimensional image from a viewpoint of encoding efficiency can be provided.

Hereinafter, an image processing apparatus and an image processing method of the present invention will be described.
<First Embodiment>
First, the correspondence between the components of the present embodiment and the components of the present invention will be described.
The 4 × 4 block (block data) of this embodiment corresponds to the block (block data) of the present invention.
Further, the macroblock of this embodiment corresponds to the macroblock of the present invention.
Further, the intra prediction mode of the present embodiment corresponds to the prediction mode of the present invention.
Further, the intra prediction mode S56 of the present embodiment corresponds to the first prediction mode of the present invention.
Further, the intra prediction mode indicated by the MPM data S52 of the present embodiment corresponds to the second prediction mode of the present invention.
Further, the prediction mode designation data PREV and REM of the present embodiment correspond to the prediction mode designation data of the present invention.
Also, the prediction mode determination presence / absence determination circuit 51 of the intra prediction circuit 43 shown in FIG. 8 corresponds to the determination means of the first invention, and the MPM selection circuit 52 corresponds to the prediction mode specifying means of the first invention, and the prediction mode The designated data / code amount generation (estimation) circuit 53 corresponds to the code amount determination (estimation) means of the first invention, and the prediction mode determination circuit 56 corresponds to the prediction mode determination means of the first invention.
A lossless encoding circuit 27 shown in FIG. 2 corresponds to the encoding means of the first invention.

Hereinafter, a communication system to which the image processing apparatus and the image processing method of the present embodiment are applied will be described.
FIG. 1 is a conceptual diagram of a communication system 1 according to the present embodiment.
As shown in FIG. 1, the communication system 1 includes an encoding device 2 provided on the transmission side and a decoding device 3 provided on the reception side.
In the communication system 1, the encoding device 2 on the transmission side generates frame image data (bit stream) compressed by orthogonal transformation such as discrete cosine transformation and Karhunen-Labe transformation and motion compensation, and modulates the frame image data. Later, it is transmitted via a transmission medium such as a satellite broadcast wave, a cable TV network, a telephone line network, or a mobile phone line network.
On the receiving side, after demodulating the received image signal, frame image data expanded by inverse transformation of orthogonal transformation and motion compensation at the time of modulation is generated and used.
The transmission medium may be a recording medium such as an optical disk, a magnetic disk, and a semiconductor memory.
The decoding device 3 illustrated in FIG. 1 performs decoding corresponding to the encoding of the encoding device 2.

Hereinafter, the encoding device 2 shown in FIG. 1 will be described.
FIG. 2 is an overall configuration diagram of the encoding device 2 shown in FIG.
As shown in FIG. 2, the encoding device 2 includes, for example, an A / D conversion circuit 22, a screen rearrangement circuit 23, an arithmetic circuit 24, an orthogonal transformation circuit 25, a quantization circuit 26, a lossless encoding circuit 27, and a buffer 28. , An inverse quantization circuit 29, an inverse orthogonal transform circuit 30, a frame memory 31, a rate control circuit 32, an adder circuit 33, a deblock filter 34, a motion prediction / compensation circuit 41, an intra prediction circuit 43, and a selection circuit 44.
The present embodiment is particularly characterized in the processing of the intra prediction circuit 43.

Hereinafter, components of the encoding device 2 will be described.
The A / D conversion circuit 22 converts an original image signal composed of the input analog luminance signal Y and color difference signals Pb and Pr into a digital image signal, and outputs this to the screen rearrangement circuit 23.
The screen rearrangement circuit 23 encodes the frame image signal in the original image signal input from the A / D conversion circuit 22 according to the GOP (Group Of Pictures) structure composed of the picture types I, P, and B. The original image data (frame image data) S23 rearranged in order is output to the arithmetic circuit 24 and the motion prediction / compensation circuit 41.

The arithmetic circuit 24 generates image data S24 indicating a difference between the original image data S23 and the predicted image data PI input from the selection circuit 44, and outputs this to the orthogonal transform circuit 25.
The orthogonal transform circuit 25 subjects the image data S24 to orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform to generate image data (for example, DCT coefficient signal) S25, and outputs this to the quantization circuit 26.
The quantization circuit 26 quantizes the image data S25 with the quantization scale input from the rate control circuit 32 to generate image data S26, and outputs this to the lossless encoding circuit 27 and the inverse quantization circuit 29.

The lossless encoding circuit 27 stores, in the buffer 28, image data obtained by variable-length encoding or arithmetic encoding of the image data S26.
At this time, the lossless encoding circuit 27 encodes the motion vector MV input from the motion prediction / compensation circuit 41 or a difference thereof, and stores it in the header data.
The lossless encoding circuit 27 encodes the prediction mode designation data PREV and REM input from the intra prediction circuit 43 and stores them in the header data.

The image data stored in the buffer 28 is transmitted after being modulated or the like.
The inverse quantization circuit 29 generates a signal obtained by inversely quantizing the image data S26 and outputs the signal to the inverse orthogonal transform circuit 30.
The inverse orthogonal transform circuit 30 outputs the image data generated by performing the inverse transform of the orthogonal transform in the orthogonal transform circuit 25 to the image data input from the inverse quantization circuit 29 to the adder circuit 33.
The adder circuit 33 adds the image data input from the inverse orthogonal transform circuit 30 and the predicted image data PI input from the selection circuit 44 to generate reconstructed image data, and outputs this to the deblock filter 34.
The deblocking filter 34 writes the image data from which the block distortion of the reconstructed image data input from the addition circuit 33 is removed to the frame memory 31 as reference image data.
The rate control circuit 32 generates a quantization scale based on the image data read from the buffer 23, and outputs this to the quantization circuit 26.

The motion prediction / compensation circuit 41 searches the reference image data REF stored in the frame memory 31 to generate a motion vector MV of the block data to be encoded in the original image data S23.
In the search, the motion prediction / compensation circuit 41 generates predicted image data PIm and index data COSTm. The index data COSTm is data that serves as an index for determining the code amount when the difference between the block data in the original image data S23 and the predicted image data PIm is encoded.
When the motion prediction / compensation circuit 41 receives the selection signal S44 indicating that motion prediction / compensation has been selected from the selection circuit 44, the motion prediction / compensation circuit 41 outputs the motion vector MV to the lossless encoding circuit 27.
In addition, the motion prediction / compensation circuit 41 outputs the predicted image data PIm and the index data COSTm to the selection circuit 44.

[Intra prediction circuit 43]
The intra prediction circuit 43 performs an intra prediction process on a 16 × 16 macroblock MB using block data of a plurality of sizes such as 4 × 4 and 16 × 16 defined in advance as a unit.
In this embodiment, a case where intra prediction is performed in units of block data having a size of 4 × 4 and 16 × 16 is illustrated.
In particular, in the present embodiment, an intra prediction mode for 4 × 4 block data will be mainly described.
A plurality of intra prediction modes having different prediction methods are defined for each intra prediction in units of block data of 4 × 4 and 16 × 16 sizes.
Hereinafter, intra prediction using 4 × 4 block data as a unit will be described.
The intra prediction circuit 43 performs a process corresponding to each of the plurality of intra prediction modes on the 4 × 4 block data to be encoded in the original image data S23 input from the screen rearrangement circuit 23, and each intra prediction mode. Predicted image data PIi and index data COSTi are generated.
Here, the intra prediction circuit 43 performs a square sum of the difference between the pixel data of the 4 × 4 block data to be encoded in the original image data S23 and the predicted image data PIi or an adamall transform on the square sum. The index data COSTi is generated by adding the value and the predicted value of the code amount of the header data.
The intra prediction circuit 43 determines an intra prediction mode that minimizes the index data COSTi among the plurality of intra prediction modes (“encoding efficiency satisfies a predetermined criterion” in the present invention).

In the present embodiment, the following intra prediction mode is used as the intra prediction for the luminance signal Y having a size of 4 × 4.
As shown in FIG. 3, the intra prediction circuit 43 is arranged in a 4 × 4 matrix, and generates a prediction image in units of blocks BLOCK each including 16 pixel data of 4 × 4.
Here, the values “0” to “15” given to the respective blocks BLOCK shown in FIG. 3 indicate the order in which the pipeline processing is performed.
The 4 × 4 intra prediction mode includes nine modes “0” to “8” having different prediction directions as shown in FIGS. 4 and 5.

Hereinafter, each mode shown in FIG. 4 will be described.
FIG. 6 is a diagram for explaining the positional relationship between pixel data a to p belonging to a 4 × 4 block BLOCK to be processed for intra prediction and pixel data A to M of blocks located around the block BLOCK. is there.
The pixel data A to M are determined to be “unavailable” when they belong to a different picture or a different slice from the block BLOCK to be processed.

Mode 0:
Mode 0 is vertical (vertical) prediction, and is applied when all of the pixel data A, B, C, and D shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 generates the prediction values of the pixel data a to p of the block BLOCK using the pixel data A, B, C, and D as shown in FIG. 7 and (1) below.

Mode 1:
Mode 1 is horizontal (horizontal) prediction, and is applied when all of the pixel data I, J, K, and L shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 generates the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. 7 and (2) below using the pixel data I, J, K, and L.

Mode 2:
Mode 2 is DC prediction, and when all the pixel data A, B, C, D, I, J, K, and L shown in FIG. 6 are “available”, the intra prediction circuit 43 The predicted values of the BLOCK pixel data a to p are generated as shown in FIG. 7 and (3) below using the pixel data A, B, C, D, I, J, K, and L.

  In addition, when all of the pixel data A, B, C, and D shown in FIG. 6 are not “available”, the intra prediction circuit 43 uses the predicted values of the pixel data a to p of the block BLOCK as the pixel data A, Using B, C, and D, they are generated as shown in FIG. 7 and (4) below.

  In addition, when all of the pixel data I, J, K, and L shown in FIG. 6 are not “available”, the intra prediction circuit 43 calculates the predicted values of the pixel data a to p of the block BLOCK as the pixel data I, Using J, K, and L, they are generated as shown in FIG. 7 and (5) below.

  In addition, when all of the pixel data A, B, C, D, I, J, K, and L shown in FIG. 6 are not “available”, the intra prediction circuit 43 stores the pixel data a to p of the block BLOCK. The predicted value “128” is used.

Mode 3:
Mode 3 is a diagonal_down_left prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”. In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. 6).

Mode 4:
Mode 4 is a Diagonal_Down_Right prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. 7).

Mode 5:
Mode 5 is a diagonal_vertical_right prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. It is generated as in 8).

Mode 6:
Mode 6 is Horizontal_Down prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. It is generated as in 9).

Mode 7:
Mode 7 is Vertical_Left prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. 10).

Mode 8:
Mode 8 is Horizontal_Up prediction, and is applied when all of the pixel data A, B, C, D, I, J, K, L, and M shown in FIG. 6 are “available”.
In this case, the intra prediction circuit 43 uses the pixel data A, B, C, D, I, J, K, L, and M to calculate the predicted values of the pixel data a to p of the block BLOCK as shown in FIG. 11).

Hereinafter, the configuration of the intra prediction circuit 43 will be described.
FIG. 8 is a block diagram of the intra prediction circuit 43 shown in FIG.
As illustrated in FIG. 8, the intra prediction circuit 43 includes, for example, a prediction mode determination presence / absence determination circuit 51, an MPM selection circuit 52, a prediction mode designation data / code amount generation circuit 53, a 4 × 4 COST calculation circuit 54, a 16 × 16 COST calculation circuit 55, and a prediction. A mode determination circuit 56 and a prediction mode designation data regeneration circuit 57 are provided.
In the present embodiment, all or some of the components shown in FIG. 8 are realized by pipeline processing in units of 4 × 4 block data.
Hereinafter, each component shown in FIG. 8 will be described.

Prediction mode determination presence / absence determination circuit 51:
The prediction mode determination presence / absence determination circuit 51 determines that the block data to be processed is based on the address of the processing target 4 × 4 block data (block BLOCK) in the processing target macroblock MB in the original image data S23 as shown in FIG. A first group having “0”, “2”, “4”, “6”, “8”, “10”, “12”, “14” shown in FIG. 9B and any of the second groups having “1”, “3”, “5”, “7”, “9”, “11”, “13”, “15” as elements. It is judged whether it belongs to.
The prediction mode determination presence / absence determination circuit 51 outputs a determination result signal S51 indicating the result of the determination to the MPM selection circuit 52.
The purpose of defining the first group and the second group is as follows.
That is, in the MPM selection circuit 52, for example, if the block data C shown in FIG. 10 is a processing target, the block data A that is adjacent to the block data C in the horizontal direction of the two-dimensional image and is previously selected as the processing target; The block data B adjacent to the block data C in the vertical direction of the two-dimensional image and previously selected as the processing target is used as the surrounding block data determined in advance, and the intra prediction mode determined for these is used as the basis. MPM (Most
Probable Mode) data is specified.
Here, the order in which 4 × 4 block data is selected as a processing target in the macroblock MB is shown in FIG.

Here, the predetermined block data corresponding to the block data belonging to the first group shown in FIG. 9A is subjected to pipeline processing at least two stages before the block data. Therefore, at the timing when processing of block data belonging to the first group is started, the predetermined intra prediction mode of the block data corresponding thereto is already determined.
For example, the predetermined block data corresponding to the “6” th block data selected in FIG. 9A is selected as the “3” and “4” th, and is selected as the “6” th. Pipeline processing is performed two and three stages before block data.
On the other hand, some of the predetermined block data corresponding to the block data belonging to the second group are subjected to pipeline processing immediately before the block data. Accordingly, there is a case where the predetermined intra prediction mode of block data corresponding to the block data belonging to the second group is not yet determined at the timing of starting processing.
For example, the predetermined block data corresponding to the block data selected “3” shown in FIG. 9B is selected “1” and “2”, and is selected “3”. Pipeline processing is performed one and two stages before the block data to be processed.
Accordingly, there is a case where the intra prediction mode of the block data selected as the “2” is not yet determined at the timing of determining the MPM data of the block data selected as the “3”.

MPM selection circuit 52:
The MPM selection circuit 52 selects the 4 × 4 block selected as the processing target when the determination result signal S51 input from the prediction mode determination presence / absence determination circuit 51 indicates that “the block data to be processed belongs to the first group”. A first block (FIG. 10) that is adjacent to the data (“C” in FIG. 10) in the vertical direction of the two-dimensional image and in which the intra prediction mode is determined by the prediction mode determination circuit 56 before the selected block data. 10 “B”), the intra prediction mode is determined in the prediction mode determination circuit 56 before the selected block data that is adjacent to the selected block data in the horizontal direction of the two-dimensional image. The MPM data S52 is generated based on the intra prediction mode already determined for the second block (“A” in FIG. 10).
Specifically, the MPM selection circuit 52 generates MPM data indicating an intra prediction mode with a small mode number among the intra prediction modes of the first block and the second block.
For example, when the 4 × 4 block data C shown in FIG. 10 belongs to the first group and is the processing target block data, the intra prediction mode MPM (C) indicated by the MPM data S52 is expressed by the following equation (12). Is generated.

[Equation 12]
MPM (C) = min (intra prediction mode (A), intra prediction mode (B))
(12)

When the determination result signal S51 input from the prediction mode determination presence / absence determination circuit 51 indicates that “the block data to be processed belongs to the second group”, the MPM selection circuit 52 outputs 2 to the second block. A third block that is adjacent in the horizontal direction of the two-dimensional image and whose intra prediction mode is determined by the prediction mode determination circuit 56 before the second block is used as a predetermined block instead of the second block. Thus, the MPM data S52 of the selected block data to be processed is generated.
For example, when the 4 × 4 block data C shown in FIG. 11 belongs to the second group and is the block data to be processed, the MPM data S52 uses the intra prediction mode of the third block A ′ as described below. It is generated as in equation (13).

[Equation 13]
MPM (C) = min (intra prediction mode (A ′), intra prediction mode (B))
... (13)

In addition, when the determination result signal S51 input from the prediction mode determination presence / absence determination circuit 51 indicates that “the block data to be processed belongs to the second group”, the MPM selection circuit 52 outputs the second block to the second block. The first block adjacent to the first block in the horizontal direction of the two-dimensional image is adjacent to the first block in the horizontal direction of the two-dimensional image and the prediction mode is determined before the second block. The intra prediction mode is selected based on the fourth block in which the intra prediction mode is determined by the prediction mode determination circuit 56 before the block of the previous block, and the selected intra prediction mode and the first block are determined. The MPM data S52 of the selected block may be generated based on the intra prediction mode.
For example, when the 4 × 4 block data C shown in FIG. 11 belongs to the second group and is the processing target block data, the MPM data S52 is generated as shown in the following equation (14).
In the following formula (14), A ′ corresponds to the third block, and B ′ corresponds to the fourth block.

[Formula 14]
MPM (C) = min (MPM (intra prediction mode (A ′), intra prediction mode (B ′)), intra prediction mode (B))
... (14)

Prediction mode designation data / code amount generation circuit 53:
Based on the MPM data S52 input from the MPM selection circuit 52, the prediction mode designation data / code amount generation circuit 53 determines the header data of the block data to be processed and the code amount when the header data is encoded. Index data SATD0 serving as an index is generated.
The header data includes prediction mode designation data PREV and REM for designating the finally determined intra prediction mode.

FIG. 12 is a flowchart for explaining the processing of the prediction mode designation data / code amount generation circuit 53.
Step ST1:
The prediction mode designation data / code amount generation circuit 53 selects an unprocessed intra prediction mode among the nine intra prediction modes of modes 0 to 8 described with reference to FIGS.
Step ST2:
The prediction mode designation data / code amount generation circuit 53 determines whether or not the intra prediction mode indicated by the MPM data S52 input from the MPM selection circuit 52 matches the intra prediction mode selected in step ST1.
The prediction mode designation data / code amount generation circuit 53 proceeds to step ST3 when determining that they match in the above determination, and proceeds to step ST4 otherwise.

Step ST3:
The prediction mode designation data / code amount generation circuit 53 sets a first logical value (for example, a logical value “1”) in the prediction mode designation data PREV.
In this case, the prediction mode designation data / code amount generation circuit 53 sets a 1-bit default value in the prediction mode designation data REM.
Further, the prediction mode designation data / code amount generation circuit 53 may not use the prediction mode designation data REM. That is, the data amount may be “0”.

Step ST4:
The prediction mode designation data / code amount generation circuit 53 sets a second logical value (for example, a logical value “0”) in the prediction mode designation data PREV.
Step ST5:
The prediction mode designation data / code amount generation circuit 53 determines whether or not the mode number of the intra prediction mode selected in step ST1 is smaller than the mode number of the intra prediction mode indicated by the MPM data S52 input from the MPM selection circuit 52. If it is determined that the value is small, the process proceeds to step ST6. If not, the process proceeds to step ST7.

Step ST6:
The prediction mode designation data / code amount generation circuit 53 sets the mode number of the intra prediction mode selected in step ST1 in the prediction mode designation data REM.
Step ST7:
The prediction mode designation data / code amount generation circuit 53 sets a number obtained by subtracting “1” from the mode number of the intra prediction mode selected in step ST1 in the prediction mode designation data REM.
Step ST8:
The prediction mode designation data / code amount generation circuit 53 determines whether or not the selection of step ST1 has been performed for all the intra prediction modes. If it is determined that the selection has been performed, the process proceeds to step ST9. Otherwise, the process returns to step ST1.
Step ST9:
The prediction mode designation data / code amount generation circuit 53 generates prediction code amounts of all the prediction mode designation data PREV and REM obtained through the above-described processing, and outputs this to the 4 × 4 COST calculation circuit 54.

  The 4 × 4 COST calculation circuit 54 uses the index data serving as an index for predicting the code amount based on the following formula (15) for each of the nine intra prediction modes 0 to 8 described with reference to FIGS. COSTi is produced.

[Equation 15]
COSTi = SATD + λ (QP) × SATD0
... (15)

In the above equation (4), the SATD is a Hadamard transform into the difference between the pixel data of the 4 × 4 predicted block data composed of the predicted values described with reference to FIGS. 6 and 7 and the 4 × 4 block data to be processed. It is the cumulative value of the values that have been subjected to.
λ (QP) is a coefficient of the prediction code amount of header data determined according to the quantization parameter QP corresponding to the quantization scale input from the rate control circuit 32.
SATD0 indicates the prediction code amount of the prediction mode specification data PREV and REM input from the prediction mode specification data / code amount generation circuit 53.

  The 4 × 4 COST calculation circuit 54 outputs the index data COSTi generated in each of the nine intra prediction modes 0 to 8 described with reference to FIGS. 4 to 7 to the prediction mode determination circuit 56.

The 16 × 16 COST calculation circuit 55 generates index data COSTi that is an index of the prediction code amount based on each of a plurality of intra prediction modes defined in advance for block data in units of 16 × 16 pixels in the original image data S23. This is output to the prediction mode determination circuit 56.
In the present embodiment, for example, four modes of vertical prediction, horizontal prediction, DC prediction, and plain prediction are defined as 16 × 16 intra prediction modes.

The prediction mode determination circuit 56 specifies the smallest index data COSTi (which is predicted to have the highest encoding efficiency) among the index data COSTi input from the 4 × 4 COST calculation circuit 54 and the 16 × 16 COST calculation circuit 55 and corresponds to the intra prediction. The mode is determined as the intra prediction mode of the block data to be processed.
When the 4 × 4 intra prediction mode is determined as the intra prediction mode of the block data to be processed, the prediction mode determination circuit 56 predicts the prediction mode specification data generated by the prediction mode specification data / code amount generation circuit 53 correspondingly. PREV and REM are output to the prediction mode designation data regeneration circuit 57.
Further, the prediction mode determination circuit 56 outputs the index data COSTi corresponding to the determined 4 × 4 intra prediction mode and the prediction image data PIi thereof to the selection circuit 44 shown in FIG.
Further, when the 16 × 16 intra prediction mode is determined as the intra prediction mode of the block data to be processed, the prediction mode determination circuit 56 receives the selection signal S44 indicating that the intra prediction is selected from the selection circuit 44. Then, the IPM indicating the determined intra prediction mode is output to the lossless encoding circuit 27 shown in FIG.
Further, the prediction mode determination circuit 56 outputs the index data COSTi corresponding to the determined 16 × 16 intra prediction mode and the prediction image data PIi thereof to the selection circuit 44 shown in FIG.

The prediction mode designating data regeneration circuit 57 has already determined the intra prediction mode of the other block data defined in advance for the block data to be processed at the timing of starting the processing in the prediction mode determination circuit 56. Based on these, processing similar to the processing of the MPM selection circuit 52 described above is performed to regenerate the prediction mode designation data PREV and REM.
When receiving the selection signal S44 indicating that intra prediction is selected from the selection circuit 44, the prediction mode designation data regeneration circuit 57 outputs the prediction mode designation data PREV and REM to the lossless encoding circuit 27 shown in FIG. .

Hereinafter, an operation example of the intra prediction circuit 43 illustrated in FIG. 8 will be described.
First, the prediction mode determination presence / absence judgment circuit 51 determines that the block data is based on the address of the processing target 4 × 4 block data (block BLOCK) in the processing target macroblock MB in the original image data S23 (FIG. 9A). ) And (B), it is determined whether it belongs to the first group or the second group, and a determination result signal S51 indicating the determination result is output to the MPM selection circuit 52.

Next, the MPM selection circuit 52 generates the MPM data S52 as described above based on the determination result signal S51, and outputs this to the prediction mode designation data / code amount generation circuit 53.
Next, the prediction mode designation data / code amount generation circuit 53 generates the prediction mode designation data PREV, REM and the prediction code amount of the 4 × 4 block data to be processed, and generates the prediction mode designation data / code amount generation circuit 53. Output to.
Next, the 4 × 4 COST calculation circuit 54 generates the index data COSTi of the 4 × 4 block data to be processed, and outputs this to the prediction mode determination circuit 56.
Further, the 16 × 16 COST calculation circuit 55 generates index data COSTi of 16 × 16 block data to be processed, and outputs this to the prediction mode determination circuit 56.
Next, the prediction mode determination circuit 56 determines the intra prediction mode of the block data to be processed.
At this time, when the 4 × 4 intra prediction mode is determined, the prediction mode designation data PREV and REM are regenerated by the prediction mode designation data regeneration circuit 57.

Hereinafter, the selection circuit 44 will be described.
The selection circuit 44 compares the index data COSTi input from the intra prediction circuit 43 with the index data COSTm input from the motion prediction / compensation circuit 41.
If the selection circuit 44 determines that the index data COSTi input from the intra prediction circuit 43 is smaller by the comparison, the selection circuit 44 selects the prediction image data PIi input from the intra prediction circuit 43 and calculates the prediction image data PI. 24. Further, the selection circuit 44 outputs a selection signal S44 indicating that intra prediction has been selected to the intra prediction circuit 43.
When the selection circuit 44 determines that the index data COSTm input from the motion prediction / compensation circuit 41 is smaller by the comparison, the selection circuit 44 selects the predicted image data PIm input from the motion prediction / compensation circuit 41 as the predicted image data PI. And output to the arithmetic circuit 24. Further, the selection circuit 44 outputs a selection signal S44 indicating that the motion prediction / compensation has been selected to the motion prediction / compensation circuit 41.

Hereinafter, the overall operation of the encoding apparatus 2 shown in FIG. 2 will be described.
The input image signal is first converted into a digital signal by the A / D conversion circuit 22. Next, the frame image data is rearranged in the screen rearrangement circuit 23 according to the GOP structure of the image compression information to be output, and the original image data S23 obtained thereby is converted into the arithmetic circuit 24, the intra prediction circuit 43, and This is output to the motion prediction / compensation circuit 41.
Next, the arithmetic circuit 24 detects a difference between the original image data S23 from the screen rearrangement circuit 23 and the predicted image data PI from the selection circuit 44, and outputs image data S24 indicating the difference to the orthogonal transformation circuit 25. To do.

Next, the orthogonal transformation circuit 25 performs orthogonal transformation such as discrete cosine transformation and Karhunen-Labe transformation on the image data S24 to generate image data S25, which is output to the quantization circuit 26.
Next, the quantization circuit 26 quantizes the image data S25 and outputs the quantized transform coefficient S26 to the lossless encoding circuit 27 and the inverse quantization circuit 29.
Next, the inverse quantization circuit 29 inversely quantizes the transform coefficient S <b> 26 input from the quantization circuit 26 and outputs image data indicating the inversely quantized transform coefficient to the inverse orthogonal transform circuit 30.
Next, the inverse orthogonal transform circuit 30 outputs the image data generated by performing the inverse transform of the orthogonal transform in the orthogonal transform circuit 25 to the image data input from the inverse quantization circuit 29 to the adder circuit 33.
Next, the adder circuit 33 adds the image data input from the inverse orthogonal transform circuit 30 and the predicted image data PI input from the selection circuit 44 to generate reconstructed image data, which is supplied to the deblock filter 34. Output.
The deblocking filter 34 writes the image data from which the block distortion of the reconstructed image data input from the addition circuit 33 is removed to the frame memory 31 as reference image data.

  Then, the motion prediction / compensation circuit 41 generates a motion vector MV of the block data to be processed based on the reference image data REF stored in the frame memory 31, and selects the index data COSTm and the predicted image data PIm. Output to the circuit 44.

  Further, as described with reference to FIG. 8, the intra prediction circuit 43 performs intra prediction, and outputs index data COSTi and predicted image data PIi obtained thereby to the selection circuit 44.

Next, the selection circuit 44 compares the index data COSTi input from the intra prediction circuit 43 with the index data COSTm input from the motion prediction / compensation circuit 41.
Then, when the selection circuit 44 determines that the index data COSTi input from the intra prediction circuit 43 is smaller by the comparison, the selection circuit 44 selects the prediction image data PIi input from the intra prediction circuit 43 and sets it as the prediction image data PI. 24. Further, the selection circuit 44 outputs a selection signal S44 indicating that intra prediction has been selected to the intra prediction circuit 43.
When the selection circuit 44 determines that the index data COSTm input from the motion prediction / compensation circuit 41 is smaller by the comparison, the selection circuit 44 selects the predicted image data PIm input from the motion prediction / compensation circuit 41 as the predicted image data PI. And output to the arithmetic circuit 24. Further, the selection circuit 44 outputs a selection signal S44 indicating that the motion prediction / compensation has been selected to the motion prediction / compensation circuit 41.
As described above, in the encoding apparatus 2, as described with reference to FIG. 8, the MPM selection circuit 52 performs intra prediction by the prediction mode determination circuit 56 on predetermined block data around the block data to be processed. Even when the prediction mode is not yet determined, the MPM data S52 is generated in a pseudo manner based on the intra prediction mode that has already been determined for other block data around the prediction mode. Therefore, the prediction mode designation data / code amount generation circuit 53 can generate the prediction code amounts of the prediction mode designation data PREV and REM, and the pseudo data COSTi in the 4 × 4 COST calculation circuit 54 can be generated.
The MPM data S52 is highly reliable because it uses an intra prediction mode of block data adjacent to the predetermined block data.
Thereby, in the prediction mode determination circuit 56, the intra prediction mode with high encoding efficiency can be determined correctly.

Second Embodiment
In the above-described embodiment, the case where pipeline processing is performed in units of 4 × 4 block data in the 4 × 4 intra prediction mode is illustrated. However, in the present embodiment, all or one shown in FIG. 8 is performed in units of 16 × 16 macroblocks. The processing of the constituent elements is performed by pipeline processing.
In this case, the MPM selection circuit 52 shown in FIG. 8 has the predetermined 4 × 4 block data described in the first embodiment belonging to a different macroblock from the 4 × 4 block data to be processed, and the macroblock When the block data belonging to the macroblock for which the determination of the intra prediction mode by the prediction mode determination circuit 56 has not been determined is used as the predetermined block data at the timing when the MPM selection circuit 52 generates the MPM data S52. Instead of the predetermined block data, an intra prediction mode of another block data whose intra prediction mode has already been determined by the prediction mode determination circuit 56 is used.
For example, when the block data “0”, “2”, “8”, “10” shown in FIG. 9A is set as the processing target block data C shown in FIG. MPM data S52 indicating the intra prediction mode of block data B shown in FIG.
According to this embodiment, the same effect as that of the first embodiment can be obtained.

The present invention is not limited to the embodiment described above.
For example, although the case where the present invention is provided for 4 × 4 intra prediction is illustrated in the above-described embodiment, the present invention may be provided for intra prediction of other sizes.

  The present invention can be applied to a system for encoding image data.

FIG. 1 is a configuration diagram of a communication system according to an embodiment of the present invention. FIG. 2 is a functional block diagram of the encoding apparatus shown in FIG. FIG. 3 is a diagram for explaining the order of intra prediction of 4 × 4 block data in a macroblock. FIG. 4 is a diagram for explaining the 4 × 4 intra prediction mode according to the embodiment of the present invention. FIG. 5 is a diagram for explaining the 4 × 4 intra prediction mode according to the embodiment of the present invention. FIG. 6 is a diagram for explaining the 4 × 4 intra prediction mode according to the embodiment of the present invention. FIG. 7 is a diagram for explaining the 4 × 4 intra prediction mode according to the embodiment of the present invention. FIG. 8 is a block diagram of the intra prediction circuit shown in FIG. FIG. 9 is a diagram for explaining processing of the MPM selection circuit shown in FIG. FIG. 10 is a diagram for explaining processing of the MPM selection circuit shown in FIG. FIG. 11 is a diagram for explaining processing of the MPM selection circuit shown in FIG. FIG. 12 is a flowchart for explaining processing of the prediction mode designation data / code amount generation circuit shown in FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Communication system, 2 ... Encoding apparatus, 3 ... Decoding apparatus, 22 ... A / D conversion circuit, 23 ... Screen rearrangement circuit, 24 ... Operation circuit, 25 ... Orthogonal transformation circuit, 26 ... Quantization circuit, 27 ... Lossless encoding circuit, 28 ... buffer, 29 ... inverse quantization circuit, 30 ... inverse orthogonal transformation circuit, 31 ... frame memory, 32 ... rate control circuit, 33 ... addition circuit, 34 ... deblocking filter, 41 ... motion prediction / Compensation circuit 43 ... Intra prediction circuit 44 ... Selection circuit 51 ... Prediction mode determination presence / absence determination circuit 52 ... MPM selection circuit 53 ... Prediction mode designation data / code amount generation circuit 54 ... 4x4 COST calculation circuit 55 ... 16x16 COST Calculation circuit 56 ... Prediction mode determination circuit 57 ... Prediction mode designation data regeneration circuit

Claims (11)

Each of a plurality of blocks constituting the two-dimensional image is selected in a predetermined order, and the selected block is subjected to intra prediction based on pixel data of a block other than the block defined by the prediction mode, and An image processing apparatus that determines, as the first prediction mode of the selected block, the prediction mode in which the coding efficiency satisfies a predetermined criterion among prediction modes,
Determining means for determining whether the first prediction mode of a predetermined block other than the selected block has already been determined;
When the determination unit determines that the first prediction mode of the predetermined block has already been determined, the selection is made based on the first prediction mode of the predetermined block. Identifying a second prediction mode used to generate prediction mode designating data for designating the first prediction mode determined for a block, and determining the first of the predetermined block When the determination unit determines that the prediction mode is not yet determined, the second prediction mode is determined based on the already determined first prediction mode of a block other than the predetermined block. A prediction mode identification means to identify;
For each of the plurality of prediction modes, the prediction mode specifying unit specifies the prediction code amount of the prediction mode designation data when the prediction mode is determined as the first prediction mode of the selected block. Code amount determining means for determining based on the second prediction mode;
Prediction mode determination for determining the first prediction mode for which the coding efficiency among the plurality of prediction modes satisfies a predetermined criterion for the selected block based on the prediction code amount determined by the code amount determination unit And an image processing apparatus. The code amount determination unit includes, for each of the plurality of prediction modes, information indicating a match when the prediction mode matches the second prediction mode specified by the prediction mode specifying unit, and includes information indicating the match If the prediction mode designation data that does not include the mode identification value is generated and does not match, the difference between the identification value of the prediction mode and the identification value of the second prediction mode, or the identification value of the prediction mode The image processing apparatus according to claim 1, wherein the prediction mode designation data including a value obtained by subtracting a predetermined value is generated. The prediction mode determination means is further based on a prediction code amount of a difference between prediction image data obtained based on pixel data of a block other than the block defined by the prediction mode and block data of the selected block The image processing apparatus according to claim 1, wherein the first prediction mode is determined. The processing of the determination unit, the prediction mode selection unit, the code amount determination unit, and the prediction mode determination unit for the plurality of blocks is executed by pipeline processing in units of the blocks,
The image processing apparatus according to claim 1, wherein the predetermined block is a block in which the first prediction mode is determined by the prediction mode determination unit before the selected block. The prediction mode specifying means, when the determination means determines that the first prediction mode of the predetermined block has already been determined, for the selected block, A first block that is adjacent in the vertical direction and whose first prediction mode is determined before the selected block is adjacent to the selected block in the horizontal direction of the two-dimensional image and the selected block. The image processing apparatus according to claim 4, wherein a second block in which the first prediction mode is determined before the determined block is used as the predetermined block. The prediction mode specifying means, when the determination means determines that the first prediction mode of the predetermined block has not yet been determined, The third block that is adjacent in the horizontal direction and in which the first prediction mode is determined before the second block is used as the predetermined block instead of the second block. The image processing apparatus described. When the determination means determines that the prediction mode of the predetermined block has not yet been determined,
The prediction mode specifying means includes a third block that is adjacent to the second block in the horizontal direction of the two-dimensional image and in which the first prediction mode is determined before the second block; Selecting a prediction mode based on a fourth block adjacent to the first block in the horizontal direction of the two-dimensional image and determining the first prediction mode before the first block; The image processing apparatus according to claim 5, wherein the second prediction mode of the selected block is specified based on the selected prediction mode and the prediction mode of the first block. The determination unit, the prediction mode selection unit, the code amount determination unit, and the prediction mode determination unit for the plurality of blocks are processed by a plurality of macros that constitute the two-dimensional image and are each composed of the plurality of blocks. The image processing apparatus according to claim 1, wherein the image processing apparatus is executed by pipeline processing in units of blocks. The predetermined block belongs to the macroblock different from the selected block, and the prediction mode determination is performed at a timing when the prediction mode specifying unit specifies the second prediction mode for the macroblock. When the block belonging to the macro block that has not been determined by the means becomes the predetermined block, the first prediction mode has already been replaced with the predetermined block. The image processing apparatus according to claim 8, wherein the first prediction mode of another block for which is determined is used. The code amount determination means generates the prediction mode designation data,
The image processing apparatus includes:
Prediction image data obtained based on pixel data of a block other than the block defined by the first prediction mode determined by the prediction mode determination means for the selected block, and block data of the selected block; The image processing apparatus according to claim 1, further comprising: an encoding unit that encodes the difference between the two and the prediction mode designation data generated by the code amount determination unit. Each of a plurality of blocks constituting the two-dimensional image is selected in a predetermined order, and the selected block is subjected to intra prediction based on pixel data of a block other than the block defined by the prediction mode, and An image processing method for determining, as a first prediction mode of the selected block, the prediction mode in which an encoding efficiency among prediction modes satisfies a predetermined criterion,
A first step of determining whether or not the first prediction mode of a predetermined block other than the selected block has already been determined;
When it is determined in the first step that the first prediction mode of the predetermined block has already been determined, based on the first prediction mode of the predetermined block, Identifying a second prediction mode used to generate prediction mode designating data for designating the first prediction mode determined for the selected block, and the predetermined block of the block When it is determined in the first step that the first prediction mode has not yet been determined, the first prediction mode is determined based on the already determined first prediction mode of a block other than the predetermined block. A second step of identifying two prediction modes;
For each of the plurality of prediction modes, the prediction code amount of the prediction mode designation data when the prediction mode is determined as the first prediction mode of the selected block is specified in the second step. A third step of determining based on the second prediction mode;
For the selected block, a first prediction mode whose coding efficiency satisfies a predetermined criterion among the plurality of prediction modes is determined based on the prediction code amount determined in the third step. An image processing method comprising the steps of:

Images