JP4257789B2 - Video encoding device - Google Patents

Description

  The present invention relates to a moving picture coding apparatus that performs coding with good coding efficiency and high efficiency, and particularly relates to a moving picture coding apparatus that includes both inter-frame prediction and intra-frame prediction coding means.

  Among the high-efficiency encoding methods that encode continuously input moving image signals with a smaller code amount, there is motion compensated prediction encoding as an encoding method that uses the correlation between image signals. In motion compensated predictive coding, a block having the highest correlation in a reference image is detected in an input image using a rectangular block called a block as a unit, and motion vector information representing the motion and the input image and reference image By encoding the prediction error information, the code amount can be greatly reduced.

  On the other hand, when the motion of an object on the screen is intense or when a scene is switched, there are cases where the accuracy of motion compensation prediction is not necessarily high. In such a case, for example, in MPEG-1, MPEG-2, etc., in-frame coding for directly coding pixel values is used instead of motion compensation prediction errors.

  As a method for further improving the coding efficiency of intraframe coding, there is intraframe predictive coding in which predictive coding is performed based on the characteristics of adjacent blocks. For example, H.M. In H.264, a prediction image is generated based on pixel values adjacent to a block to be encoded as shown in FIGS. 1A and 1B, and an error from the prediction image is encoded. Note that (a-1) to (a-9) in FIG. 1A are for 4 × 4 pixels, and (b-1) to (b-4) in FIG. 1B are 16 × 16 pixels. This case is shown.

  As a conventional moving image encoding apparatus, there is a method (hereinafter referred to as a first method) that performs both inter-frame prediction encoding and intra-frame encoding and selects an encoding method that minimizes encoding efficiency.

In addition, as another moving image encoding device, inter-frame prediction encoding is performed before intra-frame encoding, the variance of pixel value errors is obtained, and compared with the variance of pixel values before prediction encoding. There is a method of selecting whether to use the result of inter-frame prediction encoding as it is or whether to perform intra-frame encoding (hereinafter referred to as a second method, for example, Japanese Patent Laid-Open No. 10-322696).
Japanese Patent Laid-Open No. 10-322696

  In the first method, since the encoding efficiency is obtained for both encoding method modes, there is a problem that the calculation amount becomes enormous and the processing speed decreases.

  In the second method, since intra-frame coding processing is not performed when the prediction efficiency of inter-frame coding is high, a higher processing speed can be expected compared to the first method. Since intra-frame predictive coding such as H.264 is not assumed, inter-frame predictive coding is selected even for blocks that can be expected to have higher coding efficiency by selecting intra-frame predictive coding, There is a problem of reducing the encoding efficiency.

  An object of the present invention is to provide a moving image coding apparatus that solves the above-described problems of the prior art, performs coding of moving images with high coding efficiency, and high efficiency.

In order to solve the above-mentioned problems, the present invention provides a moving image comprising means for performing interframe predictive coding of continuously input image signals and means for performing intraframe predictive coding based on the characteristics of adjacent blocks. in the encoding apparatus, a first means for obtaining a prediction error characteristic amount between the frames obtained by aggregating the block prediction error values between frames for each pixel, each of the pixel value difference between adjacent pixels obtained for each pixel in the block Second means for obtaining an intra-frame adjacent error feature value obtained by aggregating the intra-block adjacent error value obtained as the minimum value in the pixel within the block, and the feature value obtained by the first and second means There is a first feature in that a means for determining whether to perform inter-frame predictive coding or intra-frame predictive coding based on the size of is provided.

According to the present invention, there is provided a third means for determining whether to perform an intra-frame detailed investigation or an inter-frame predictive coding based on the magnitude of the feature amount obtained by the first and second means, and the third means. If the detailed investigation is determined by the above, whether the inter-frame prediction encoding is performed based on the fourth means for obtaining the intra-frame prediction error, the inter-frame prediction error, and the intra-frame prediction error obtained by the fourth means. A second feature is that a means for determining whether to perform intraframe prediction encoding is provided.

Further, the present invention provides a first means for obtaining an inter-frame prediction error feature amount, and an intra-block adjacent error value that is a pixel value difference between adjacent pixels in the block with respect to at least one direction in the block. minimum and fifth means for obtaining the frame adjacent errors characteristic quantity is a value, whether the prediction encoding or inter-frame predictive-coded frame based on the feature value of the magnitude determined by the first and fifth means A third feature is that a means for determining is provided.

  As is apparent from the above description, according to the present invention, a feature amount representing the coding efficiency of inter-frame predictive coding is calculated, and the coding efficiency is estimated before performing intra-frame predictive coding. Are calculated, and whether to perform intra-frame predictive encoding is determined based on these feature values.

  Therefore, according to the present invention, it is possible to omit the intra-frame prediction process in many cases, so that the processing amount of the intra-frame prediction process is greatly reduced, and the moving picture encoding process is efficiently performed. Can be performed.

  In addition, in order to determine whether the intra-frame prediction is omitted after estimating in advance whether the coding efficiency by intra-frame prediction is higher than the inter-frame prediction, the code is executed while reducing the number of executions of intra-frame prediction. It is possible to maintain the conversion efficiency at substantially the same level as in the conventional case.

  Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 2 is a block diagram showing the configuration of one embodiment of the present invention. In the following description, H.264 is used as a moving image encoding method. H.264 / MPEG-4 AVC (Advanced Video Coding), using the method shown in FIG. 1 as an intra-frame predictive coding method, and a rectangle (hereinafter referred to as a block) fixed to one or more sizes. However, the present invention is not limited to this.

  In FIG. 2, an input image signal 1 is stored in an input image storage memory 2. Further, a reference image signal that is referred to when performing inter-frame prediction is stored in the reference image storage memory 4. The motion vector output unit 3 performs a motion search based on the image signals stored in the input image storage memory 2 and the reference image storage memory 4 and outputs the result as a motion vector. The position shifter 8 generates a predicted image in inter-frame prediction based on the motion vector input from the motion vector output unit 3 and the image signal input from the reference image storage memory 4.

  The inter-frame prediction error image signal generator 7 subtracts the inter-frame prediction image input from the position shifter 8 from the input image signal input from the input image storage memory 2 to generate an inter-frame prediction error image. The result is output to the predictive feature value output unit 6.

  The inter-frame prediction feature quantity output unit 6 calculates an inter-frame prediction feature quantity Cost (Inter) serving as an index representing the prediction efficiency in the inter-frame prediction based on the inter-frame prediction error image signal input from the error image signal generator 7. Output for each block. As an example of the inter-frame prediction feature quantity, an absolute value sum, a root mean square value, a variance value, and the like of pixel values of the inter-frame prediction error image signal can be used.

  On the other hand, the intra-frame prediction feature quantity output unit 5 outputs an intra-frame prediction feature quantity, for example, an intra-frame adjacent error feature quantity, based on the input image signal input from the input image storage memory 2.

  FIG. 3 is a diagram illustrating an example of a process for deriving an intra-frame prediction evaluation function. In general, intra-frame prediction has a feature of showing high prediction accuracy for an overall smooth image. Therefore, it is considered possible to estimate the accuracy of intraframe prediction based on the amount of change in pixel value between adjacent pixels in the entire image, and this value can be used as the intraframe prediction feature amount. In addition, 1 square in a figure shows a pixel.

In FIG. 3, P (x, y) represents a pixel value at coordinates (x, y) in the block. At this time, the pixel (x, y) 3 pixels adjacent to (x-1, y-1 ), (x, y-1), the difference of the pixel values of the (x-1, y) D1 = P (x , Y) -P (x-1, y-1), D2 = P (x, y) -P (x, y-1), D3 = P (x, y) -P (x-1, y) Among them, the one with the smallest absolute value is defined as the adjacent error D (x, y).

  Next, the minimum adjacent error D (x, y) is obtained for all the pixels in the block B, and any one of the absolute sum, the square sum, and the variance value is used as the intra-frame prediction feature value Cost (Intra). Can be used. Alternatively, the average value of D1, D2, and D3 can be used as the adjacent error D (x, y).

  FIG. 4 is a modification of FIG. Instead of obtaining the adjacent error D (x, y) for all the pixels in the block B as shown in FIG. (Intra) can be obtained.

  FIG. 5 is also a modification of FIG. Instead of determining the adjacent error D (x, y) for all the pixels in the block, the adjacent error D (x, y) is determined for only the uppermost column and the leftmost row in the block B. An intra prediction feature amount Cost (Intra) can be obtained. 4 and 5, the method for obtaining the intra-frame prediction feature value Cost (Intra) is the same as in the case of FIG. 3.

  The prediction mode determination unit 9 includes an inter-frame prediction feature amount Cost (Inter) input from the inter-frame prediction feature amount output unit 6 and an intra-frame prediction feature amount Cost (Intra) input from the intra-frame prediction feature amount output unit 5. ) To determine whether the block is to be encoded by the prediction encoding mode of inter-frame prediction or intra-frame prediction, and outputs it as a prediction mode control signal 19. As an example of a comparison method, a method of selecting an interframe prediction mode when the inequality Cost (Inter) <T × Cost (Intra) (T is a constant of 1 or more, for example) is satisfied, and an intraframe prediction mode when it is not satisfied. Is available.

  When the prediction mode control signal 19 specifies the intra-frame prediction mode, the intra-frame predictor 10 receives the input image signal input from the input image storage memory 2 and the adjacent block input from the decoded image signal generator 18. Based on the decoded image, an intra-frame prediction image is generated by one of the methods shown in FIGS. 1A and 1B, and a prediction image having the minimum data amount is selected and output. Note that this method is the same in other embodiments described below, and thus the description thereof is omitted below.

  When the prediction mode control signal 19 designates the inter-frame prediction mode, the prediction error image signal generator 11 designates the inter-frame prediction image signal output from the position shifter 8 and designates the intra-frame prediction mode. Subtracts the intra-frame prediction image signal output from the intra-frame predictor 10 from the input image signal input from the input image storage memory 2 to generate a prediction error image signal and outputs it to the orthogonal transformer 12. At this time, the prediction error image signal output from the prediction error image signal generator 11 is orthogonally transformed by the orthogonal transformer 12 and quantized by the quantizer 13 in order to obtain higher coding efficiency. The quantized transform coefficient, the quantized coefficient, and the motion vector output from the motion vector output unit 3 are entropy encoded by the variable length encoder 15 and output as image compression encoded data 20 for encoding. Processing is complete. The coding rate controller 14 determines a quantization coefficient for the transform coefficient of the next input block based on the encoding efficiency of the image compression encoded data output from the variable length encoder 15, Keep the bit rate constant.

  Based on the transform coefficient and the quantized coefficient output from the quantizer 13, the inverse quantizer 16 performs an inverse quantization process, and the error image signal is decoded and output by the inverse orthogonal transformer 17. The decoded image signal generator 18 adds the error image signal decoded with either the inter-frame prediction image signal or the intra-frame prediction image signal based on the prediction mode control signal 19 to generate a decoded image signal. The decoded image signal generated in this way is used for the intra-frame prediction process by the intra-frame predictor 10, is stored in the reference image storage memory 4, and is used for the inter-frame prediction process by the motion vector output unit 3. . The prediction error image signal generator 11, the orthogonal transformer 12, the quantizer 13, the coding rate controller 14, the variable length encoder 5, the inverse quantizer 16, the inverse orthogonal transformer 17, the decoded image signal. Since the operations of the generator 18 and the image compression encoded data 20 are the same in the following other embodiments, the description thereof is omitted below.

  FIG. 6 is a flowchart for explaining the schematic operation of the first embodiment. In steps S01 and S02, an inter-frame prediction feature value and an intra-frame prediction feature value are obtained, respectively. In step S03, it is determined whether or not to perform intraframe prediction based on both prediction feature quantities. If this determination is negative, the process proceeds to step S05 to perform interframe predictive coding.

  On the other hand, if this determination is affirmative, the process proceeds to step S04, and the intraframe predictor 10 calculates an intraframe prediction error for each prediction direction (see FIGS. 1A and 1B). In step S06, the intra-frame predictor 10 selects the smallest prediction direction among the intra-frame prediction errors, and generates and outputs an intra-frame prediction image by the method. In step S07, the intra-frame prediction image is sent to the prediction error image signal generator 11, and intra-frame prediction encoding is performed.

  According to this embodiment, since intra-frame coding processing is not performed when the prediction efficiency of inter-frame coding is high, a higher processing speed can be expected compared to the conventional first method. In addition, when performing intra-frame coding processing, H.264 is used. Since intra-frame predictive coding such as H.264 is selected, high coding efficiency can be expected.

  Next, FIG. 7 is a block diagram showing a second embodiment of the present invention. In this embodiment, when it is determined that the intra-frame prediction mode is more advantageous as a result of the comparison between the intra-frame prediction feature quantity and the inter-frame prediction feature quantity, encoding of each of the inter-frame prediction and the intra-frame prediction is performed. It is characterized in that the efficiency is investigated in detail, that is, the respective coding efficiencies are compared based on the results of inter-frame prediction and intra-frame prediction, and the final coding mode is determined.

  The intra-frame detailed investigation determination unit 21 includes an inter-frame prediction feature amount Cost (Inter) input from the inter-frame prediction feature amount output unit 6 and an intra-frame prediction feature amount Cost input from the intra-frame prediction feature amount output unit 5. By comparing (Intra), it is determined whether or not the intra-frame prediction error is actually obtained as the intra-frame detailed survey, and the determination result is output as the intra-frame detailed survey control signal 23. For example, when the inequality Cost (Inter) <T × Cost (Intra) (T is a constant of 1 or more, for example) is not established, the signal 23 indicating that the in-frame detailed investigation is to be performed is output.

  When the intra-frame detailed inspection control signal 23 indicates that the intra-frame detailed inspection is to be performed, the intra-frame predictor 10 is input from the input image signal input from the input image storage memory 2 and the decoded image generator 18. Based on the decoded image of the adjacent block, an intra-frame prediction image is generated and output by one of the methods shown in FIGS. The intra-frame prediction error image signal generator 22 subtracts the intra-frame prediction image output from the intra-frame prediction device 10 from the input image signal input from the input image storage memory 2, and finally obtains an intra-frame prediction error image as an intra-frame prediction error image. It outputs to the prediction mode determination device 24.

  The final prediction mode determination unit 24 generates a frame based on the intra-frame prediction error image output from the intra-frame prediction error image signal generator 22 and the inter-frame prediction error image output from the inter-frame prediction error image signal generator 7. The coding efficiency of the intra prediction coding and the inter-frame prediction coding is compared, and a prediction mode with high coding efficiency is selected and output as the prediction mode signal 19.

  When the intra-frame detailed inspection signal 23 indicates that the intra-frame detailed inspection is not to be performed, the final prediction mode determination unit 24 selects inter-frame prediction encoding as the prediction encoding mode as the prediction mode control signal 19. Output.

  FIG. 8 is a flowchart for explaining the schematic operation of the second embodiment. In FIG. 8, the same or equivalent processes as those in FIG. 6 are denoted by the same reference numerals, and description thereof is omitted.

  In step S03, the intra-frame detailed investigation determination unit 21 determines whether to perform intra-frame prediction based on the inter-frame prediction feature quantity and the intra-frame prediction feature quantity. If the determination is affirmative, the process proceeds to step S11 after the processing of steps S04 and S06, and the intra-frame prediction error image signal generator 22 generates an intra-frame prediction error. In step S12, the final prediction mode determination unit 24 compares the inter-frame prediction error with the intra-frame prediction error. In step S13, it is determined whether or not the intra-frame prediction mode is selected. The subsequent processing is the same as in FIG.

  According to this embodiment, compared with the said 1st Embodiment, the precision which selects the prediction mode with high encoding efficiency improves.

  FIG. 9 is a block diagram showing a third embodiment of the present invention. In this embodiment, in the prediction mode determination process, by simultaneously determining the prediction direction in the intra-frame prediction mode, the prediction direction in the intra-frame prediction process is limited to one, and the intra-frame prediction process is greatly reduced. There is a feature in the point.

  Based on the input image signal input from the input image storage memory 2, the intra-frame intra-frame prediction feature value output unit 25 performs the first to n-th directions with respect to the first to n-th (n is a natural number of 1 or more) prediction directions. An intra-frame prediction feature value is obtained and output to the intra-frame prediction direction determiner 26. As the first to n-th prediction directions, for example, top, diagonal, left, average of adjacent blocks, or the like can be used. At this time, the adjacent error D (x, y) is obtained based on the same prediction direction for all the pixels P (x, y) in the block, and any one of the absolute sum, the square sum, and the variance value is obtained. It can be used as the n-th intra-frame prediction feature value Cost (Intra (n)).

  The intra-frame prediction direction determiner 26 selects the smallest prediction direction from the first to n-th direction intra-frame prediction feature values input from the direction-specific intra-frame prediction feature value output unit 25 to select an intra-frame prediction direction control signal. 27, and outputs the intra-frame prediction feature quantity corresponding to the selected prediction direction to the prediction mode determination unit 9.

  The prediction mode determination unit 9 compares the intra-frame prediction feature amount corresponding to the prediction direction selected by the intra-frame prediction direction determiner 26 with the inter-frame prediction feature amount input from the inter-frame prediction feature amount output unit 6 to predict the prediction mode. Is output as the prediction mode control signal 19.

  The intra-frame predictor 10 generates and outputs an intra-frame prediction image signal according to the prediction direction indicated by the intra-frame prediction direction control signal 27 input from the intra-frame prediction direction determiner 26.

  FIG. 10 is a flowchart for explaining the schematic operation of the third embodiment. In FIG. 10, the same or equivalent processes as those in FIG. 6 are denoted by the same reference numerals, and the description thereof is omitted.

  In step S21, the intra-frame predicted feature value output unit 25 calculates the intra-frame predicted feature value for each prediction direction. Then, in step S06, the intra-frame prediction direction determiner 26 selects the direction in which the prediction feature quantity of each prediction direction is minimum as the intra-frame prediction direction. In step S03, the prediction mode determination unit 9 determines whether to perform intraframe prediction. When this determination is affirmative, the process proceeds to step S07, and intraframe prediction encoding is performed in the prediction direction selected in step S06.

  According to this embodiment, the processing of the intra-frame predictor 10 can be greatly reduced, and the processing speed when the prediction mode control signal 19 indicates intra-frame prediction can be greatly improved.

It is explanatory drawing of the prediction method in a flame | frame. It is a block diagram which shows the structure of 1st Embodiment of this invention. It is explanatory drawing of the 1st specific example in the case of calculating | requiring the adjacent pixel error in a block. It is explanatory drawing of the 2nd example in the case of calculating | requiring the adjacent pixel error in a block. It is explanatory drawing of the 3rd specific example in the case of calculating | requiring the adjacent pixel error in a block. It is a flowchart showing operation | movement of 1st Embodiment of this invention. It is a block diagram which shows the structure of 2nd Embodiment of this invention. It is a flowchart showing operation | movement of 2nd Embodiment of this invention. It is a block diagram which shows the structure of 3rd Embodiment of this invention. It is a flowchart showing operation | movement of 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 5 ... Intra-frame prediction feature-value output device, 6 ... Inter-frame prediction feature-value output device, 9 ... Prediction mode determination device, 10 ... In-frame prediction device, 11 ... Prediction error image signal Generator: 19 ... Prediction mode control signal, 21 ... Intra-frame detailed investigation / determination unit, 22 ... Intra-frame prediction error image signal generator, 23 ... In-frame detailed investigation control signal, 24 ... A final prediction mode determination unit, 25... Intra-frame prediction feature value output unit, 26... Intra-frame prediction direction determination unit, and 27.

Claims (13)

In a moving picture coding apparatus comprising: means for performing interframe predictive coding of continuously input image signals; and means for performing intraframe predictive coding based on features of adjacent blocks.
A first means for obtaining an inter-frame prediction error feature amount in which inter-frame prediction error values for each pixel are tabulated for a block;
Second determining a frame adjacent errors characteristic quantity blocks in adjacent error values obtained as the minimum value of each pixel is obtained by aggregation in the block for the pixel value difference between adjacent pixels obtained for each pixel in the block Means of
A moving picture coding apparatus comprising: means for determining whether to perform interframe predictive coding or intraframe predictive coding based on the magnitude of the feature amount obtained by the first and second means. . In a moving picture coding apparatus comprising means for performing interframe predictive coding of continuously input image signals and means for performing intraframe predictive coding based on the characteristics of adjacent blocks,
A first means for obtaining an inter-frame prediction error feature amount in which inter-frame prediction error values for each pixel are tabulated for a block;
Second determining a frame adjacent errors characteristic quantity blocks in adjacent error values obtained as the minimum value of each pixel is obtained by aggregation in the block for the pixel value difference between adjacent pixels obtained for each pixel in the block Means of
A third means for deciding whether to perform an intra-frame detailed investigation or an inter-frame prediction encoding based on the magnitude of the feature amount obtained by the first and second means;
A fourth means for determining an intra-frame prediction error when the third means determines that the detailed investigation is performed;
A moving picture code comprising means for determining inter-frame prediction coding or intra-frame prediction coding based on an inter-frame prediction error and an intra-frame prediction error obtained by the fourth means Device. The moving picture encoding apparatus according to claim 1 or 2,
The moving picture coding apparatus characterized in that an absolute sum of the intra-block adjacent error values is used as the intra-frame adjacent error feature quantity. The moving picture encoding apparatus according to claim 1 or 2,
A moving picture coding apparatus using the square sum of the adjacent error value in the block as the intra-frame adjacent error feature amount. The moving picture encoding apparatus according to claim 1 or 2,
A moving picture coding apparatus using variance of the adjacent error value in the block as the intra-frame adjacent error feature quantity. In the moving image encoder according to any one of claims 3 to 5,
A moving picture coding apparatus using a difference in pixel values between adjacent pixels in a specific direction as the adjacent error value in the block. In the moving image encoder according to any one of claims 3 to 5,
A moving picture encoding apparatus, wherein the adjacent error value in the block is selected and used for each pixel from among pixel value differences between adjacent pixels in several directions. The moving picture encoding apparatus according to claim 7,
At least one of top, diagonal, and left is used as an adjacent direction of adjacent pixels in the block. The moving image encoding device according to claim 1,
The moving picture coding apparatus characterized in that the means for determining whether the intra-frame prediction encoding or the inter-frame prediction encoding uses a linear inequality using each feature amount as a determination function. The moving image encoding device according to claim 2,
The moving picture coding apparatus characterized in that the means for determining whether the intra-frame detailed investigation or inter-frame prediction coding uses a linear inequality using each feature quantity as a decision function. In a moving picture coding apparatus comprising means for performing interframe predictive coding of continuously input image signals and means for performing intraframe predictive coding based on the characteristics of adjacent blocks,
A first means for obtaining an inter-frame prediction error feature amount in which inter-frame prediction error values for each pixel are tabulated for a block;
A fifth means for obtaining an in-frame adjacent error feature quantity that is a minimum value among the aggregated in-block adjacent error values that are pixel value differences between adjacent pixels in the block in at least one direction;
A moving picture coding apparatus comprising means for determining whether to perform intraframe predictive coding or interframe predictive coding based on the magnitude of the feature amount obtained by the first and fifth means. The moving image encoding device according to claim 11,
The means for determining whether to perform intra-frame prediction encoding or inter-frame prediction encoding determines whether intra-frame prediction or inter-frame prediction by using the feature amount in the direction with the smallest error as the intra-frame adjacent error feature amount. A moving picture coding apparatus characterized by the above. The moving image encoding device according to claim 11 or 12,
An intra-frame encoding apparatus that performs intra-frame prediction encoding in a direction with the smallest error when intra-frame prediction is selected.

Images