JP2008271127A - Coding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a coding apparatus capable of performing a high-speed process by means of a pipeline process and generating coded data according to specifications. <P>SOLUTION: The coding apparatus divides input data into a plurality of blocks to perform a process in each block. Therefore, the coding apparatus includes a block processing part for generating reconfiguration data on the basis of the input data and an intra prediction signal generated by performing intra prediction of one block in accordance with a prediction mode, and a block selecting part 51 for selecting a block so as to make the block selecting part perform a pipeline processing, wherein the block selecting part 51 selects the next block after timing when the process of a preceding block ends according to a process sequence determined on the basis of an inter-block dependence relation showing a relation between one block and a block to be referred to when processing the one block. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to H.264. The present invention relates to an encoding device suitable for H.264 (MPEG (Moving Picture Experts Group) -4 AVC) system.

  In video encoding or decoding, H.264 provided for low bit rate transmission. According to the H.264 or MPEG-4 AVC standard, there are nine intra prediction modes. The nine prediction modes are vertical mode, horizontal mode, DC (Direct Current) mode, diagonal left down mode, diagonal right down mode, vertical right mode, horizontal down mode, vertical left mode and horizontal up mode. (See Patent Document 1).

  Conventionally, Patent Document 1 discloses an intra prediction method and apparatus aimed at improving the intra prediction speed by reducing the intra prediction processing steps in units of sub-blocks in such video encoding or decoding. .

  FIG. 16 is a block diagram showing an intra prediction apparatus at the time of video encoding described in Patent Document 1. The intra prediction apparatus 100 is applied to a video signal obtained by dividing a 16 × 16 macroblock into 4 × 4 subblocks, and includes a counter 101, an intra prediction processing unit 110, an adder 120, and a reference pixel generation unit 130.

  The counter 101 sets an intra prediction processing operation with the 16 × 16 macroblock (hereinafter referred to as a macroblock). That is, the counter 101 updates the count value of the intra prediction processing operation in the macroblock every time the reference pixel generation unit 130 generates the pixel value of the reference sub-block. When the counter 101 is an up counter, the update is performed so that the count value is increased, and when the counter 101 is a down counter, the update is performed so that the count value is decreased.

  The count value of each intra prediction processing operation is defined as intra prediction processing operation information. Therefore, the counter 101 can be defined as an intra prediction processing operation information providing unit. The intra prediction processing operation information output from the counter 101 is output to the intra prediction processing unit 110.

  The intra prediction processing unit 110 determines at least one sub-block among the 4 × 4 sub-blocks included in the macro block as a sub-block on which intra prediction is performed, and determines the sub-block on which the determined intra-prediction is performed. A plurality of intra prediction modes are performed to output an optimal intra prediction mode and intra prediction samples.

  For this purpose, the intra prediction processing unit 110 includes a first SAE calculator 111, a second SAE calculator 112, and a cost comparator 113. The first SAE calculator 111 determines a sub-block on which intra prediction is performed based on the intra prediction processing operation information output from the counter 101. The second SAE calculator 112 determines a sub-block on which intra prediction is performed based on the intra prediction processing operation information output from the counter 101 like the first SAE calculator 111.

  If the first SAE calculator 111 and the second SAE calculator 112 determine a sub-block on which intra-prediction is performed based on intra-prediction processing operation information, nine intra-prediction modes are determined for the determined intra-prediction sub-block. Perform intra prediction for.

  Each of the first SAE calculator 111 and the second SAE calculator 112 uses an intra prediction sample obtained by performing intra prediction corresponding to nine prediction modes and an input pixel value (4 × 4 pixel value) of the subblock. Thus, each intra prediction mode-specific SAE is detected. Then, the SAE and intra prediction mode information detected for each intra prediction mode (mode A in the case of the first SAE calculator 111, mode B in the case of the second SAE calculator 112) and the intra prediction sample to the cost comparator 113. Output.

  The cost comparator 113 compares the received SAEs separately, outputs the intra prediction modes each having the minimum SAE as the optimal intra prediction mode information (mode A, mode B) of the sub-block performing the intra prediction, and The intra prediction sample detected in the intra prediction mode is output as the intra prediction sample of the sub-block that performs the intra prediction. In that case, the intra prediction mode information and intra prediction sample of the subblock which performs each intra prediction are output sequentially.

  Intra prediction mode information (mode A, mode B) is used at the time of video decoding. Intra prediction samples of each sub-block on which intra prediction is performed are output to the adder 120.

  The adder 120 adds the residual value of the sub-block on which the intra prediction is performed to the intra prediction sample. The residual value is a difference value between the predicted pixel value of the sub-block and the pixel value currently input.

  The reference pixel generation unit 130 reconstructs and stores the pixel value of the sub-block on which the intra prediction is performed based on the addition result output from the adder 120, and stores the reconstructed pixel value in the next intra prediction processing operation. To generate pixel values for the reference sub-block. The generated pixel value is provided to the first SAE calculator 111 and the second SAE calculator 112 of the intra prediction processing unit 110. The reference pixel generation unit 130 provides a signal that informs the counter 101 of the generation of the pixel value of the reference sub-block every time the pixel value of the reference sub-block is generated. Thereby, the counter 101 updates the count value and updates the intra prediction processing operation information.

  FIG. 17 is a diagram illustrating processing timing of blocks of the intra prediction apparatus 100. This intra prediction apparatus 100 divides a 16 × 16 macroblock into 16 subblocks of 4 × 4, and when these blocks are numbered 0 to 15 as shown in FIG. 4, as shown in FIG. In addition, two arithmetic units (SAE calculation unit in Patent Document 1) process the sub-blocks having the numbers shown in the figure one by one. The block number is H.264. Although the order is defined by the H.264 standard, the processing order of blocks is changed in order to reduce the overall processing time. That is, a plurality of arithmetic units that change the order according to the dependency relationship between blocks determined by a processing block and a block that is referred to for processing of the block, and simultaneously execute block processing that can be executed in parallel without dependency relationship, Patent Document 1 Then, it assigns to two arithmetic units. Thus, when one block is processed in one stage by one arithmetic unit, 16 block processes can be completed in 10 stages.

Further, Patent Document 2 includes an image information encoding device that includes a plurality of blocks in a single macroblock, performs intra prediction for each block, and performs high-speed intra prediction based on pipeline processing. Is disclosed. In this encoding apparatus, as in the JVT encoding method, one macro block is divided into a plurality of blocks, and image compression information for performing intra prediction, orthogonal transform, and quantization processing is output for each block as a unit. The encoding device sequentially processes each block in the order other than the raster scan. Specifically, if the position of the block is represented by (X, Y) {X, Y = 0, 1, 2, 3}, (0, 0) (1, 0) (0, 1) ( 2,0) (1,1) (3,0) (2,1) (0,2) (3,1) (1,2) (0,3) (2,2) (1,3) ( 3, 2) (2, 3) (3, 3) in this order.
JP 2005-130509 A JP 2004-140473 A

  However, in the method described in Patent Document 1, since block processing is performed in parallel, a plurality of arithmetic units are required, and the circuit scale increases. This is because the processing that can be executed in parallel due to the dependency relationship between the blocks is executed as it is by parallel processing, and thus a plurality of arithmetic units corresponding to the degree of parallelism are required. In addition, the processing speed cannot be further reduced. This is because block processing is divided by the intra prediction processing stage, so part of the processing cannot be overlapped between the processing stages, so it takes at least processing time required for each processing stage. Because it ends up.

  In the method described in Patent Document 2, H.P. Intra prediction is performed by a method that is out of the H.264 standard, and there is a problem that image quality deteriorates. 18 and 19 are diagrams illustrating the processing timing described in Patent Document 2. FIG. As shown in FIG. 18 or FIG. 19, when sequentially processing in the order of the designated blocks, in the method shown in FIG. 18, the subsequent block is started after the end of the two previous blocks. 19 starts before the end of the previous block, and in the method shown in FIG. 19, the subsequent block starts after the end of the block 4 previous to the preceding block, but before the end of the block 1 to 3 before To begin. In such pipeline processing, blocks cannot be processed as specified. That is, since processing is performed even though processing of a block to be referred to in one block processing is not completed, for example, A, B, C, and D are not fixed as shown in FIG. In this case, reference is made to a, b, c, and d, and reference is made to i, j, k, and l when I, J, K, and L are not fixed. Furthermore, the prediction method does not refer to the pixels (E to H) in the upper right block. Therefore, there is a problem that encoded data according to the standard cannot be obtained.

  H. In H.264, as shown in FIG. 5, in the 4 × 4 pixel intra prediction, if there is an encoding target block, the encoded pixels (A to M) of the left, upper left, upper, and upper right blocks are used. Intra prediction of the code target block is performed. Therefore, when encoding the encoding target block, it is necessary to finish encoding the pixels (A to M) of the left, upper left, upper, and upper right blocks. On the other hand, in the method described in Patent Document 2, since the pipeline processing is performed at the timing as described above, the target block is encoded before the encoding of the left, upper, and upper right blocks is completed. As described above, it is necessary to refer to a pixel different from the standard.

  An encoding apparatus according to the present invention is an encoding apparatus that divides input data into a plurality of blocks and performs processing for each block, and performs intra prediction of one block according to the input data and a prediction mode. A block processing unit that generates reconstructed data based on the generated intra prediction signal, and a block selection unit that selects a block to cause the block processing unit to perform pipeline processing. The next block is selected after the timing at which the processing of the previous block ends in accordance with the processing order determined based on the inter-block dependency indicating the relationship between one block and the block referred to when processing the one block. .

  In the present invention, in accordance with the processing order determined based on the inter-block dependency relationship, pipeline processing is performed while selecting the next block after the timing when the processing of the previous block ends, so that the block that has been processed can be referred to. In addition, the processing speed can be increased.

  According to the present invention, it is possible to provide an encoding device that can perform high-speed processing by pipeline processing and can generate encoded data in accordance with the standard.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. This embodiment is an embodiment of the present invention. This is an apparatus that generates encoded data that conforms to H.264, and is applied to an image encoding apparatus that realizes high-speed processing by pipeline processing. That is, in this embodiment, H.264 is used. In an image coding method having a dependency relationship between the processing orders between blocks such as H.264, the processing in the block is operated by pipeline processing including a plurality of stages, the order of block processing is appropriately changed, and pipeline processing is performed. By assigning to each stage, the processing speed is improved without increasing the hardware scale.

Embodiment 1 FIG.
FIG. 1 is a diagram illustrating an encoding apparatus according to a first embodiment of the present invention. As illustrated in FIG. 1, the encoding device 1 includes a residual value encoding unit 11, a variable length encoding unit 23, a reconstructed data generation unit 12, a deblock filter 13, a frame memory 14, and an inter prediction unit (inter-frame prediction). ) 15, an intra prediction unit 16, and a prediction control unit 17.

  The residual value encoding unit 11 includes an orthogonal transform unit 21, a quantization unit 22, an inverse quantization unit 24, and an inverse orthogonal transform unit 25. A difference (residual value) between the input image and the prediction image (intra prediction signal or inter prediction signal) is input to the orthogonal transformation unit 21 and orthogonally transformed. The quantization unit 22 quantizes the orthogonally transformed data based on the quantization parameter. The quantized data is input to the inverse quantization unit 11, is inversely quantized, and is input to the inverse orthogonal transform unit 25. The inverse orthogonal transform unit 25 performs inverse orthogonal transform on this and outputs the result to the reconstructed data generation unit 12.

  The variable length coding unit 23 performs variable length coding on the quantized data, the quantization parameter, and the coding parameter based on the coding parameter, and outputs the result as a compressed image signal.

  The reconstructed data generating unit 12 includes an adder 31 that adds the predicted image and the data input from the inverse orthogonal transform unit 25. The output of the adder 31 is input to the deblock filter 13 and the intra prediction unit 16.

  The deblocking filter 13 smoothes only the block boundary of integer conversion and suppresses the occurrence of block noise. The frame memory 14 stores reference frame data necessary for inter-frame prediction.

  The inter prediction unit 15 has a total of seven types in units of 1 macroblock (16 × 16 pixels), 16 × 8 pixels, 8 × 16 pixels, 8 × 8 pixels, 8 × 4 pixels, 4 × 8 pixels, and 4 × 4 pixels. Motion compensation is performed with a block size of 2 and a predicted image is generated using a plurality of reference frames. In inter prediction, it is possible to designate a previous frame as a reference frame in consideration of, for example, a scene change or a moving object.

  The intra prediction unit 16 includes an intra predictor 42 and a subtractor 41. The intra predictor 42 generates a prediction image by interpolation from adjacent pixels of a macroblock adjacent to the top or left of a macroblock that does not use interframe prediction. There are two types of block sizes, which are the generation units of predicted images, for the luminance (Y) component, 4 × 4 and 16 × 16 pixels. In addition, nine types of interpolation patterns can be used for predictive image generation in the case of 4 × 4 units of luminance components and four types of 16 × 16 units of luminance components. Furthermore, intra prediction in units of 8 × 8 pixels can be used for profiles higher than the high profile. Nine types of interpolation patterns similar to the case of 4 × 4 can be used. The subtracter 41 obtains the difference between the input image and the predicted image as a residual value.

  The prediction control unit 17 selects blocks to be input to the residual value encoding unit 11, the reconstructed data generation unit 12, the intra predictor 42, and the inter prediction unit 15 using a block selection unit described later. In this case, the next block is selected after the timing when the processing of the block ends in accordance with the processing order determined based on the inter-block dependency indicating the relationship between one block and the block referred to when processing the one block. Further, the selector 18 is controlled to select between a predicted image obtained from the inter prediction unit 15 and a predicted image obtained from the intra predictor 42.

  A residual value that is a difference between an intra prediction signal generated by performing intra prediction of one block according to a prediction mode by the residual value encoding unit 11, the reconstructed data generation unit 12, and the intra prediction unit 16, and the input data A block processing unit that encodes and decodes the residual value and adds the decoded signal and the intra prediction signal to generate reconstructed data is configured.

  FIG. 2 is a diagram illustrating the block processing unit. As shown in FIG. 2, the block processing unit is controlled by a block selection unit 51. In the present embodiment, the block selection unit is described as being included in the prediction control unit 17, but a block selection unit may be provided individually.

  As described above, the intra prediction unit 16 generates the prediction image by intra prediction according to the input prediction mode, and the difference between the input data and the prediction image generated by the intra predictor 42 is a residual value. As a subtractor. The residual value encoder 11 includes a residual value encoder 26 that receives the residual value and performs orthogonal transform, quantization, inverse quantization, and inverse orthogonal transform of the residual value. The reconstructed data generation unit 12 adds the decoded data from the residual value encoding unit 11 and the intra-predicted image from the intra predictor 42 to generate reconstructed data, and the reconstructed data that stores the reconstructed data. And a configuration data storage unit 32.

  Next, block processing will be described. In this embodiment, 4 × 4 pixel intra coding is pipelined to shorten the coding processing time, and other processing is the same as that of a known technique. Can do. Therefore, hereinafter, 4 × 4 pixel intra coding will be described. FIG. 3 is a diagram illustrating blocks. As shown in FIG. 3, one picture (frame) is a 16 × 16 pixel macroblock, and a 4 × 4 = 16 equal block is a 4 × 4 pixel block. In 4 × 4 intra coding (luminance), as shown in FIG. 4, normally, coding is performed in the order of 0 to 15 constituting 16 × 16 pixels. That is, the 4 × 4 blocks are arranged with encoded data in the order of the block numbers shown in the figure.

  As shown in FIG. 5, in the case of performing 4 × 4 block intra prediction drawn by a solid line, 13 pixels indicated by A to M at maximum are used. That is, the pixels in the upper left, right above, upper right, and left side blocks surrounded by a broken line are referred to. Since the pixels used for prediction need to be reconstructed pixels, in order to use these pixels, the encoding process of adjacent blocks including these must be completed. Specifically, for example, when the block 9 is encoded, the encoding of the blocks 2, 3, 6, and 8 needs to be completed. In this specification, this is referred to as block dependency.

  In intra prediction with 4 × 4 pixels, there are nine types of prediction modes from prediction mode 0 to prediction mode 8. In prediction mode 0, the upper block is referenced. In prediction mode 1, the left block is referred to. In the prediction mode 2, the left block and the upper block are referred to and their average values are used. In the prediction mode 3, the upper block and the upper right block are referred to. In the prediction mode 4, the left block, the upper left block, and the upper block are referred to. In the prediction mode 5, the upper left block and the upper block are referred to. In the prediction mode 6, the left block and the upper left block are referred to. In the prediction mode 7, the upper block and the upper right block are referred to. In prediction mode 8, the left block is referred to.

  FIG. 6 is a diagram illustrating a dependency relationship between blocks in one macroblock. The numbers in the figure are the block numbers shown in FIG. 4 and indicate that the processing of the block after the arrow cannot be performed unless the processing of the original block of the arrow is completed. All the arrows in the figure are directed to the right, and in this figure, the block processing can proceed from the left side to the right side. Blocks arranged in the same column can be processed in parallel. According to this, blocks 2 and 4, blocks 3 and 5, blocks 8 and 6, blocks 9 and 7, blocks 10 and 12, and blocks 11 and 13 can be encoded simultaneously. In Patent Document 1 described above, two block processing units are prepared and processed in parallel to improve the processing speed. However, as described above, the amount of hardware increases. On the other hand, as shown in FIG. 7, the processing time increases when the processing of each block is performed in series.

  Therefore, in the present embodiment, it is possible to execute the calculation according to the standard at high speed by executing each process of the blocks in the order shown in FIG. 8 and appropriately determining the processing timing of each block. FIG. 8 is a diagram illustrating a processing order of blocks in 4 × 4 intra prediction according to the present embodiment. In the present embodiment, intra prediction residual value calculation performed by the intra prediction unit 16, residual value encoding performed by the residual value encoding unit 11, and reconstruction data generation performed by the reconstruction data generation unit 12 in the present embodiment. These processes are assumed to be pipelined. In the present embodiment, these three processes are described as being pipelined, but the classification of processes is not limited to this. For example, the residual coding may be divided into orthogonal transformation and quantization processing and inverse quantization and inverse orthogonal transformation processing, and these may be processed in a total of four processes.

There are the following two types of block processing orders according to the present embodiment. That is, when the first order is displayed using the block numbers in FIG.
0 → 1 → (2 → 4) → (3 → 5) → (6 → 8) → (7 → 9) → (10 → 12) → (11 → 13) → 14 → 15.
Here, the block processing order in parentheses is in no particular order.

  When a block when a 16 × 16 macroblock is divided into 4 × 4 blocks for processing is represented by (X, Y) {X, Y = 0, 1, 2, 3}, the first processing order is ( 0,0) (1,0) (0,1) (2,0) (1,1) (3,0) (2,0) (0,2) (3,1) (1,2) ( 0,3) (2,2) (1,3) (3,2) (2,3) (3,3), (0,1) and (2,0), (1,1 ) And (3,0), (2,0) and (0,2), (3,1) and (1,2), (0,3) and (2,2), and (1,3) And the order of (3, 2) is in random order.

When the second order is displayed using the block numbers in FIG.
0 → 1 → 2 → 4 → 3 → 5 → 8 → 6 → 9 → 7 → 10 → 12 → 11 → 13 → 14 → 15
And when a 16 × 16 macroblock is divided into 4 × 4 blocks for processing, (0, 0) is represented by (X, Y) {X, Y = 0, 1, 2, 3}. (1, 0) (0, 1) (2, 0) (1, 1) (3, 0) (0, 2) (2, 1) (1, 2) (3, 1) (0, 3) The order is (2, 2) (1, 3) (3, 2) (2, 3) (3, 3).

  FIG. 9 is a diagram for explaining the operation of the block processing unit according to this embodiment, and is a flowchart of the encoding process of one macroblock in the intra 4 × 4 mode. As shown in FIG. 9, first, in the encoding mode determination step S1, the encoding mode of the macroblock is determined. This encoding mode determination step S1 determines how to encode the macroblock from among various encoding methods defined by the standard. The encoding mode includes intra prediction encoding and inter prediction encoding. Intra prediction is a method of predicting an image signal of the macroblock from the same frame, and inter prediction is a method of predicting from information of different frames. In the case of intra prediction, there are further intra 4 × 4 prediction, intra 16 × 16 prediction, and intra 8 × 8 prediction. Each prediction method has a plurality of prediction modes, and as described above, there are nine types of prediction modes for intra 4 × 4.

  Next, in intra prediction determination step S2, it is determined whether or not the encoding mode selected in encoding mode determination step S1 is intra prediction encoding. In this case, for example, for inter prediction or intra prediction, a prediction method with a higher prediction performance, for example, a smaller residual value is selected. In the case of inter prediction encoding, the process proceeds to an inter prediction encoding process (not shown). In the case of intra prediction encoding, it is further determined whether or not the mode is an intra 4 × 4 mode in a prediction block size determination step S3. In the case of Intra 16 × 16 or Intra 8 × 8, the process proceeds to each prediction method not shown. In the case of the intra 4 × 4 mode, the process proceeds to block processing in the macro block.

  In the block processing, intra prediction residual value calculation, residual value encoding, and generation of reconstructed data in FIG. 2 are executed at predetermined timings under the control of the block selection unit 51 in each circuit. FIG. 10 is a diagram illustrating allocation of block processing to a processing unit that performs three stage processing (intra prediction, prediction residual encoding, and reconstruction data generation) in a block. According to the inter-block dependency in FIG. In FIG. 10, processing is assigned to a plurality of blocks that can be processed in parallel. This corresponds to the first order described above. Blocks 2 and 4, blocks 3 and 5, blocks 6 and 8, blocks 7 and 9, blocks 10 and 12, and blocks 11 and 13 are pipeline processed. In the following description, the horizontal axis in FIG. 10 is referred to as a processing stage. For example, in the processing stage 7, the intra prediction residual value calculation of the block 4 is performed by the intra prediction unit 16, and the residual value encoding process of the block 2 is performed by the residual value encoding unit 11 at the same timing. As described above, these blocks are in no particular order, and may be block 2 → block 4 or block 4 → block 2. In the lower part of FIG. 10, such interchangeable process assignments are also shown. In the section divided by the broken line, either the upper part or the lower part can be selected to perform block processing.

  Also, as shown in FIG. 10, for example, block 1 depends on the processing result of block 0. Therefore, the processing of the first stage of block 1 is started after the processing of the last stage of block 0 is completed. On the other hand, in Patent Document 2 described above, pipeline processing is executed at the same timing for all blocks regardless of the dependency relationship between blocks as shown in FIGS. 18 and 19. There is a case where the processing is started before the calculation result is obtained, and there is a problem that processing according to the standard cannot be performed.

  As described above, in the present embodiment, the parallel processing is performed while selecting the next block after the timing when the processing of the previous block is finished in consideration of the dependency relationship between the blocks. As a result, the processing speed can be improved, and processing such as replacing the reference pixel with another pixel value as in Patent Document 2 described above is unnecessary, and encoded data that conforms to the standard can be obtained. .

  In the block processing, first, the number of processing stages in the processing stage is initialized (step S4). Then, blocks in the macroblock for which intra prediction is performed are sequentially selected one by one in accordance with the order determined in advance by the block selection unit 51, and the intra prediction unit 16, the residual value encoding unit 11, and the reconstructed data generation unit 12 are selected. Output (steps S5, S8, S10).

  Next, in the intra prediction unit 16, after the block selection, in the intra prediction step S6, the intra prediction unit 16 performs the intra prediction of the block according to the prediction mode selected in the encoding mode determination step S1, Outputs an intra prediction signal. In the predicted residual value calculation step S7, the subtracter 41 calculates the difference between the input data and the predicted value and outputs it as a predicted residual value.

  Next, after the residual value encoding block is selected, the residual encoding unit 11 encodes the predicted residual value in the residual value encoding step S9. The encoding process performs orthogonal transform, quantization, inverse quantization, and inverse orthogonal transform, and calculates a decoded signal of the residual value.

  Next, in the reconfiguration data generation unit 12, after a block for reconfiguration data generation is selected, in the reconfiguration data generation step S11, the adder 31 adds the decoded signal of the residual value and the intra prediction signal, Output reconstruction data. In the reconstruction data storage step S13, the reconstruction data storage unit 32 stores the reconstruction data necessary for prediction of adjacent blocks.

  When the processing in each processing unit is completed, the number of processing stages is updated (step S14). Then, it is determined whether or not the number of final processing stages is reached, and the process is repeated up to the final stage. When the process is completed up to the final stage, it is determined by block determination whether or not the encoding of all the blocks in the macroblock has been completed. If there is still a block to be subjected to intra prediction encoding, the next block is selected and intra prediction encoding from step S1 is performed.

  As described above, in the present embodiment, the screen is divided into a plurality of macroblocks, the macroblock is further divided into a plurality of blocks, and an image coding method having a dependency on the processing order between these blocks is used. In addition, the block encoding process includes a plurality of processing stages, includes a process of executing them in the pipeline, and each stage processing unit (intra prediction unit, reconstructed data generation unit, Residual value encoding unit) and a block selection unit that specifies a block to be processed for each stage processing unit. The block selection unit includes at least one block according to the dependency of the processing order of the blocks. The processing order of the block set is set in advance, and the blocks included in each stage processing device in the pipeline are set according to the set order It performs the processing at.

  Alternatively, the block selection unit selects a block in which there is no preceding block or all the last processing stages of the preceding block processing are finished, and pipes the block according to the dependency of the processing order of the blocks. Processing is performed by each stage processing apparatus of the line.

  Here, in the order of the block numbers, the degree of parallelism cannot be increased due to the dependency between the blocks, but there are blocks that can be processed in parallel by rearranging the processing order of the blocks in consideration of the dependency between the blocks. Furthermore, in multiple blocks that can be processed in parallel, intra prediction processing, prediction residual encoding processing, and reconstructed data generation processing executed in the block are combined and executed in a pipeline without increasing the same processing device. , Different stages of processing can be executed simultaneously. Then, processing is started immediately from the block in which the reference pixel used for prediction is generated without dividing the processing stage in units of combinations of blocks that can be parallelized.

  As a result, the operation order of the intra predictor, the prediction residual encoder, and the reconstructed data generator is improved by appropriately rearranging the processing order of the blocks and assigning them to the pipelined processing stages. The overall processing time can be shortened without increasing the wear. Thereby, compared with the case where the processing stage is configured by combining a plurality of blocks that can be executed in parallel, the overall processing time can be shortened rather than the sum of the processing times of the processing stages.

  Next, a modified example for shortening the entire processing time described above will be described with respect to the present embodiment. FIG. 11 and FIG. 12 are diagrams showing allocation of block processing to a processing unit that performs three stage processing in a block. As shown in FIG. 10, the processing can be further speeded up by performing the processing in the second order described above. That is, in the case of the process shown in FIG. 10, the process up to the process stage 35 is necessary, but in this example, the process of one macroblock is completed at the process stage 31. In this case as well, the next block is selected after the processing of the previous block is finished in consideration of the dependency between the blocks. However, if there are multiple blocks that can be processed in parallel, the subsequent blocks are processed as soon as possible. Select a block to begin. It is also possible to set the timing as shown in FIG.

Embodiment 2. FIG.
Next, a second embodiment according to the present invention will be described. The present embodiment is an encoding device that not only improves processing speed by pipeline processing but also reduces the amount of hardware. FIG. 13 is a diagram illustrating a block processing unit in the encoding device according to the present embodiment. The configuration other than the block processing unit is the same as that of the first embodiment. The same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the present embodiment, as shown in FIG. 13, the intra prediction unit and the reconfiguration data generation unit share one circuit. Specifically, the intra prediction / reconstruction unit 19 includes a selector 61, an adder / subtractor 62, a reconstruction data storage unit 63, and an intra predictor 64. The selector 61 selects and outputs the decoded residual value from the residual value encoder 26 or the input data. When performing intra prediction, a prediction image is generated by an intra predictor, input data is selected by a selector 61, and a difference from the input data is obtained by an adder / subtractor 62. When generating reconstructed data, the selector 61 selects the decoded residual value, and the adder / subtracter 62 adds the predicted image and the decoded residual value.

  With this configuration, the adder / subtracter can be shared. In addition, intra prediction values need not be temporarily stored. For intra prediction values used for reconstruction, the intra prediction values obtained during residual value calculation are temporarily stored and reused, or another intra predictor is used for reconstruction. In this embodiment, the intra prediction value may be generated by using the common intra predictor again, and it is not necessary to temporarily store the intra prediction value or use another intra predictor.

  FIG. 14 is a flowchart showing the encoding processing method according to the present embodiment, and FIG. 15 is a diagram showing allocation of block processing to processing units that perform three stage processing in the block. Steps S21 to S23 are the same as steps S1 to S3 described above. In the block process, first, in the process stage initialization step S24, the block selection unit 51 initializes the process stage to zero.

  Next, the process proceeds to intra prediction / reconstruction block selection step S25 and residual value coding block selection step S29. The block selection unit 51 selects a block to be processed (see FIG. 15).

  Next, in the intra prediction function determination step S26, it is determined whether the function executed by the intra prediction / reconstruction unit 19 is intra prediction or reconstruction. When executing the intra prediction function, the process proceeds to the intra prediction step S31. In the case of reconstruction, the process proceeds to reconstruction data generation step S27.

  In the intra prediction step S31, the intra prediction / reconstruction unit 19 performs intra prediction of the block according to the prediction mode selected in the encoding mode determination step, and outputs an intra prediction value (predicted image). In the predicted residual value calculating step S32, the selector 61 selects the input data, the adder / subtracter 62 calculates the difference between the input data and the predicted value, and outputs it as the predicted residual value.

  In the residual value encoding step S30, the residual value encoder 26 encodes the predicted residual value. The encoding process performs orthogonal transform, quantization, inverse quantization, and inverse orthogonal transform, and calculates a decoded signal of the residual value.

  In the reconstruction data generation step S27, the intra prediction / reconstruction unit 19 performs intra prediction of the block in accordance with the prediction mode selected in the encoding mode determination step, and outputs an intra prediction value. Then, the selector 61 selects the residual value decoded signal, and the adder / subtracter 62 adds the residual value decoded signal and the intra prediction value, and outputs reconstructed data. In the reconstruction data storage step 28, the reconstruction data storage unit 63 stores the reconstruction data necessary for prediction of adjacent blocks.

  When the processing of the block in each processing unit is completed, in the next processing step number update step S33, the block selecting unit 51 increments the processing step. Then, in the final processing stage determination step S34, the block selection unit 51 determines whether the processing stage has exceeded the last stage number. If the final processing stage number has not been exceeded, the process returns to block selection steps S25 and S29 and continues. If the number of processing stages is exceeded, the processing of the luminance block of the macro block is terminated, and the processing proceeds to the processing of the next color difference block, for example.

  In this embodiment, processing in a block is divided into a plurality of stages in the pipeline, blocks that can be processed in parallel are combined, and executed by only one set of pipeline arithmetic units, thereby increasing the amount of hardware. The processing speed is improved by pipeline processing. Furthermore, since the adder / subtracter can be shared and it is not necessary to temporarily store the intra prediction value or use another intra predictor, the amount of hardware can be further reduced.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a figure which shows the encoding apparatus concerning Embodiment 1 of this invention. It is a figure which shows the block process part in the encoding device. It is a figure explaining a block. It is a figure which shows the number of the block which comprises the macroblock in 4x4 intra coding. It is a figure which shows the pixel which the encoding object block of 4x4 pixel refers. It is a figure which shows the dependence relationship between the blocks in 1 macroblock. It is a figure which shows the case where each process of a block is performed in series. It is a figure which shows the process order of the block in 4 * 4 intra prediction concerning Embodiment 1 of this invention. It is a figure explaining operation | movement of the block process part concerning Embodiment 1 of this invention. It is the figure which showed allocation of the block process to the process part which performs three stage processes in the block in Embodiment 1 of this invention. It is the figure which showed allocation of the block process to the process part which performs three stage processes in the block in the modification of Embodiment 1 of this invention. Similarly, it is the figure which showed the allocation of the block process to the process part which performs three stage processes in the block in the modification of Embodiment 1 of this invention. It is a figure which shows the block process part in the encoding apparatus concerning Embodiment 2 of this invention. It is a flowchart which shows the encoding processing method concerning Embodiment 2 of this invention. It is a figure which shows allocation of the block process to the process part which performs three stage processes in the block in Embodiment 2 of this invention. FIG. 10 is a block diagram showing an intra prediction device at the time of video encoding described in Patent Literature 1. It is a figure which shows the process timing of the block of an intra prediction apparatus. It is a figure which shows the processing timing of patent document 2. FIG. Similarly, it is a figure which shows the process timing of patent document 2. FIG. It is a figure explaining the reference pixel in patent documents 2.

Explanation of symbols

1 Encoding Device 11 Residual Value Encoding Unit 12 Reconstructed Data Generation Unit 13 Deblocking Filter 14 Frame Memory 15 Inter Prediction Unit 16 Intra Prediction Unit 17 Prediction Control Units 18 and 61 Selector 19 Intra Prediction / Reconstruction Unit 21 Orthogonal Transform Unit 22 Quantization unit 23 Variable length encoding unit 24 Inverse quantization unit 25 Inverse orthogonal transform unit 26 Residual value encoder 31 Adder 32 Reconstructed data storage unit 41 Subtractor 42 Intra predictor 51 Block selection unit 62 Adder / subtractor 63 Reconstruction data storage unit 64 Intra predictor

Claims (9)

An encoding device that divides input data composed of image signals into a plurality of blocks and processes the blocks for each block,
A block processing unit that generates reconstruction data based on the input data and an intra prediction signal generated by performing intra prediction of one block according to a prediction mode;
A block selection unit that selects a block so that the block processing unit performs pipeline processing;
The block selection unit follows a processing order determined based on an inter-block dependency indicating a relationship between one block and a block referred to when the one block is processed, and the next block after the timing when the processing of the previous block ends. An encoding device for selecting. The encoding device according to claim 1, wherein the block processing unit divides a 16 × 16 macroblock into 4 × 4 blocks and processes each block. The block processing unit obtains a residual value that is a difference between an intra prediction signal generated by performing intra prediction of one block according to a prediction mode and input data, encodes and decodes the residual value, The decoded signal and the intra prediction signal are added to generate reconstructed data.
The encoding apparatus according to claim 1 or 2, characterized in that The block processing unit
An intra predictor for performing intra prediction of one block according to a prediction mode;
A residual value encoder for encoding a residual value obtained as a difference between input data and an intra prediction signal generated by the intra predictor;
A reconstructed data generating unit that generates reconstructed data by adding the decoded signal of the residual value and the intra prediction signal;
4. The encoding device according to claim 1, wherein the block selection unit selects a block to be input to the intra predictor, a residual value encoder, and a reconstructed data generation unit. 5. . The encoding device according to any one of claims 1 to 4, wherein the block selection unit selects a block so that pipeline processing is performed on blocks out of order in the processing order. When a block when a 16 × 16 macroblock is divided into 4 × 4 blocks for processing is represented by (X, Y) {X, Y = 0, 1, 2, 3}, the processing order is (0, 0) (1, 0) (0, 1) (2, 0) (1, 1) (3, 0) (0, 2) (2, 1) (1, 2) (3, 1) (0, The encoding according to any one of claims 1 to 5, wherein the order is 3) (2, 2) (1, 3) (3, 2) (2, 3) (3, 3). apparatus. (0,1) and (2,0), (1,1) and (3,0), (0,2) and (2,1), (1,2) and (3,1), (0 , 3) and (2, 2), and (1, 3) and (3, 2) are in random order. The encoding apparatus according to any one of claims 1 to 7, wherein the intra predictor and the reconstructed data generation unit are configured by the same hardware. H. The encoding apparatus according to any one of claims 1 to 8, wherein H.264 encoded data is generated.

Images