JP2013223149A - Image encoding device, image decoding device, image encoding program, and image decoding program - Google Patents

Abstract

An object of the present invention is to improve the prediction accuracy of intra prediction while suppressing an increase in calculation cost.
An image encoding device is an image encoding device that performs intra-frame prediction of an image, an acquisition unit that acquires a plurality of pixel values in an encoded block adjacent to a prediction target block, and a prediction target Filter processing for generating a prediction value of a prediction target pixel using a pixel value of an acquired encoded block and / or a predicted pixel value in a prediction target block within a predetermined range from the prediction target pixel in the block An intra-prediction unit that sequentially performs.
[Selection] Figure 3

Description

  The present invention relates to an image encoding device, an image decoding device, an image encoding program, and an image decoding program that perform intra prediction of an image.

  In recent years, with the advance of multimedia, there are many opportunities to handle moving images. Image data has a large amount of information, and a moving image, which is a set of images, has a data amount exceeding 1 Gbit / sec in high-definition (1920 × 1080) broadcasting, which is the mainstream of television broadcasting.

  Therefore, the moving image data is H.264. H.264 / AVC (Advanced Video Coding) and standardization work HEVC (High Efficiency Video Coding) and other encoding techniques represented by international standard encoding schemes are used to transmit and store data after compressing the amount of data. Yes.

  H. is an international standard encoding method. H.264 / AVC compresses moving image data to about 1/100 using tools such as orthogonal transform, inter prediction (inter prediction), intra prediction (intra prediction), arithmetic coding, and deblocking filter. Is realized.

  Intra prediction, which is one of the tools, performs intra prediction by determining an optimal prediction mode from a plurality of prediction modes using a pixel value adjacent to a processing target block as a reference pixel. In addition, in applications such as digital broadcasting, intra prediction is periodically performed to cope with transmission errors and channel switching. In intra-prediction coding, it is difficult to increase the compression rate compared to inter-prediction coding using data of other frames.

  Therefore, H.H. In H.264 / AVC, as shown in FIG. 1 and FIG. 2, intra prediction is performed using encoded pixels in the same frame to improve the efficiency of intra prediction encoding (Non-Patent Document 1). ). FIG. 1 is a diagram illustrating intra prediction in a 4 × 4 block. FIG. 2 is a diagram illustrating intra prediction in a 16 × 16 block.

  As a prediction method, there is an average value (DC: Direct Current) prediction in which an average value of a pixel value sequence around a prediction target block (typically upper or left) is used as a predicted value. Further, as prediction methods, there are plane prediction in which prediction values are generated by combining extrapolation and linear interpolation, and direction prediction in which zero-order extrapolation is performed in a specific direction.

  Also, predictive values are generated by repeatedly performing a convergence operation on a prediction target block using the surrounding pixels of the prediction target block as boundary conditions and a low frequency component of discrete cosine transform (DCT) as a regularization term. There is also a method (Patent Document 1).

JP 2011-188133 A

ISO / IEC 14496-10: Information technology-Coding of audio visual objects-Part 10: Advanced Video Coding (2010)

  However, H.C. In the intra prediction adopted in H.264 / AVC, only pixel data for one line of the upper left, upper, upper right, or left block of the encoding target block is used. Therefore, when the size of the prediction target block increases, the prediction efficiency increases. A drop will occur.

  In addition, in the method of performing extrapolation or interpolation using only encoded image data in the vicinity of the screen, the prediction accuracy may be deteriorated particularly when the pixel value waveform is non-planar.

  In addition, the method of repeatedly performing the convergence calculation is advantageous when the pixel value waveform is non-planar, but in addition to the encoded image data in the vicinity in the screen, the image data inside the target block being predicted is also included. It is necessary to consider. Therefore, this method has a problem that the calculation cost is high because the calculation is repeatedly performed.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an image processing device and a program capable of improving the prediction accuracy of intra prediction while suppressing an increase in calculation cost.

  An image encoding device according to an aspect of the present invention is an image encoding device that performs intra-screen prediction of an image, and obtains a plurality of pixel values in an encoded block adjacent to a prediction target block; Prediction of the prediction target pixel using the pixel value of the acquired encoded block and / or the predicted pixel value in the prediction target block that are within a predetermined range from the prediction target pixel in the prediction target block And an intra prediction unit that sequentially performs filter processing for generating values.

  Further, the image processing apparatus further includes a filter control unit that controls a filter that generates the prediction value, and the filter control unit includes a phase relationship between the encoded pixel value and / or the predicted pixel value in a horizontal direction and a vertical direction. A feature calculation unit for calculating a feature quantity of any one of number, variance value, pixel value difference with surrounding pixels, edge quantity, or texture feature quantity, and features in the horizontal and vertical directions calculated by the feature calculation unit A size determining unit that determines the size of the filter based on the amount.

And a filter control unit that controls a filter that generates the predicted value.
The filter control unit may include a plurality of different filters, the prediction unit may generate a predicted value using each filter, and a filter to be used may be determined based on a prediction error of each filter.

  Further, the intra prediction unit has a mode for generating a prediction value using the filter and each mode for generating a prediction value from an encoded pixel value using a plurality of prediction directions, a prediction error, The mode may be determined based on the generated code amount or the RD characteristic.

  An image decoding apparatus according to another aspect of the present invention is an image decoding apparatus that decodes a stream encoded by the image encoding apparatus, a decoding unit that performs entropy decoding on the stream, and prediction of intra prediction. Among the pixel values in the decoded block adjacent to the target block, the pixel value of the decoded block and / or the predicted pixel in the prediction target block within a predetermined range from the prediction target pixel in the prediction target block And an intra prediction unit that sequentially performs a filter process for generating a prediction value of the prediction target pixel using the value.

  An image encoding program according to another aspect of the present invention includes an acquisition step of acquiring a plurality of pixel values in an encoded block adjacent to a prediction target block, and a prediction target pixel in the prediction target block. Filter processing for generating a prediction value of the prediction target pixel sequentially using the pixel value of the acquired encoded block and / or the predicted pixel value in the prediction target block within a predetermined range from And performing an intra prediction step.

  An image decoding program according to another aspect of the present invention includes: a decoding step for entropy decoding an image-encoded stream; and a pixel value in a decoded block adjacent to a prediction target block for intra prediction. The prediction value of the prediction target pixel is generated using the pixel value of the decoded block and / or the predicted pixel value of the prediction target block within a predetermined range from the prediction target pixel in the prediction target block. And an intra prediction step for sequentially performing filtering processing.

  ADVANTAGE OF THE INVENTION According to this invention, the prediction precision of intra prediction can be improved, suppressing the increase in calculation cost.

The figure which shows the intra prediction in 4x4 block. The figure which shows the intra prediction in a 16x16 block. 1 is a block diagram illustrating an example of a schematic configuration of an image encoding device according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an example of a configuration of an intra prediction unit according to the first embodiment. The figure which shows the relationship between an in-plane prediction filter and a prediction object pixel. The figure which shows an example of the encoded pixel used for intra prediction, and the predicted pixel. The figure which shows the example (the 1) of the pixel used for a filter process. The figure which shows the example (the 2) of the pixel used for a filter process. FIG. 3 is a block diagram illustrating an example of a configuration of a filter control unit according to the first embodiment. The figure which shows the example of the shape of a filter. The figure which shows the example of a scanning order. 5 is a flowchart illustrating an example of intra prediction processing according to the first embodiment. FIG. 6 is a block diagram illustrating an example of a configuration of a filter control unit according to a second embodiment. 9 is a flowchart illustrating an example of intra prediction processing according to the second embodiment. FIG. 10 is a block diagram illustrating an example of a schematic configuration of an image decoding device according to a third embodiment. FIG. 10 is a diagram illustrating an example of a configuration of an image processing apparatus according to a fourth embodiment.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[Example 1]
<Configuration>
FIG. 3 is a block diagram illustrating an example of a schematic configuration of the image encoding device 10 according to the first embodiment. In the example illustrated in FIG. 3, the image encoding device 10, which is an example of an image processing device, includes a preprocessing unit 100, a prediction error signal generation unit 101, an orthogonal transformation unit 102, a quantization unit 103, an entropy encoding unit 104, and an inverse quantum. Conversion unit 105, inverse orthogonal transform unit 106, decoded image generation unit 107, filter control unit 109a, decoded image storage unit 110, intra prediction unit 111, inter prediction unit 112, motion vector calculation unit 113, and predicted image selection unit 115. Have. An outline of each part will be described below.

  In the example shown in FIG. A configuration according to the H.264 / AVC international standard encoding method will be described, but the present invention is not limited to this encoding method, and any technology can be applied as long as the encoding technology performs intra prediction.

  The preprocessing unit 100 rearranges the pictures in accordance with the picture type, and sequentially outputs the picture type and the frame image for each frame. The preprocessing unit 100 also performs block division and the like.

  The prediction error signal generation unit 101 acquires block data obtained by dividing an encoding target image of input moving image data into blocks such as 32 × 32, 16 × 16, and 8 × 8 pixels.

  The prediction error signal generation unit 101 generates a prediction error signal based on the block data and the block data of the prediction image output from the prediction image selection unit 115. The prediction error signal generation unit 101 outputs the generated prediction error signal to the orthogonal transformation unit 102.

  The orthogonal transform unit 102 performs orthogonal transform processing on the input prediction error signal. The orthogonal transform unit 102 outputs a signal separated into horizontal and vertical frequency components by the orthogonal transform process to the quantization unit 103.

  The quantization unit 103 quantizes the output signal from the orthogonal transform unit 102. The quantization unit 103 reduces the code amount of the output signal by quantization, and outputs this output signal to the entropy encoding unit 104 and the inverse quantization unit 105.

  The entropy encoding unit 104 performs entropy encoding on the output signal from the quantization unit 103, the motion vector information output from the motion vector calculation unit 113, and the like, and outputs the result. Entropy coding is a method of assigning variable-length codes according to the appearance frequency of symbols.

  The inverse quantization unit 105 dequantizes the output signal from the quantization unit 103 and then outputs the output signal to the inverse orthogonal transform unit 106. The inverse orthogonal transform unit 106 performs an inverse orthogonal transform process on the output signal from the inverse quantization unit 105 and then outputs the output signal to the decoded image generation unit 107. By performing decoding processing by the inverse quantization unit 105 and the inverse orthogonal transform unit 106, a signal having the same level as the prediction error signal before encoding is obtained.

  The decoded image generation unit 107 adds the block data of the image subjected to motion compensation by the inter prediction unit 112 and the prediction error signal decoded by the inverse quantization unit 105 and the inverse orthogonal transform unit 106. The decoded image generation unit 107 outputs the decoded image block data generated by the addition to the decoded image storage unit 110.

  The decoded image storage unit 110 stores the input block data of the decoded image as new reference image data, and outputs the data to the filter control unit 109a, the inter prediction unit 112, and the motion vector calculation unit 113.

  The filter control unit 109a selects a filter used for prediction based on the pixel value of the encoded block and / or the predicted pixel value in the prediction target block within a predetermined range from the prediction target pixel included in the prediction target block. decide. A filter used for prediction is also called an in-plane prediction filter. The prediction target block may be the same block as the encoding target block, or may be a block obtained by dividing the encoding target block.

  As the in-plane prediction filter, a low-pass linear filter, a median filter, a bilater filter known as an edge-preserving noise removal filter, or a nonlinear filter such as a sigma filter can be applied.

For details of each filter, refer to the following documents.
・ C. Tomai, R. Manduchi, "Bilateral Filtering for gray and color images," International Conference on Computer Vision, pp.839-846, 1998.
・ A. Buades, B. Coll, and J. Morel, "A non -local algorithm for image denoising," IEEE Conference on Computer Vision and Pattern Recognition, vol.2, p.60,2005
· RCBilcu, M.Vehvilainen, "A Modified Sigma Filter for Noise Reduction in Images," Proceedings of the 9 th WSEAS Circuits, Systems, Communications and Computers multiconference, WSEAS / CSCC2005, No.15, July 2005
As the in-plane prediction filter, there are a method of estimating the value of the target pixel using only the values of neighboring pixels and a method of estimating by inserting a temporary value into the target pixel. The provisional value is, for example, an intermediate value, a maximum value, a minimum value, an average value of pixels used for prediction in the in-plane prediction filter, or the like in a possible range as pixel values.

  The filter control unit 109a may adaptively change the filter size of the in-plane prediction filter and the scanning order so that the prediction accuracy is improved. Details of the filter control unit 109a will be described later.

  Note that if the filter used as the in-plane prediction filter, the filter size, and the scanning order are set in the intra prediction unit 111 in advance, the filter control unit 109a is not necessarily required.

  The intra prediction unit 111 generates a predicted image using the already-encoded reference pixels and predicted pixels for the prediction target block of the encoding target image. The intra prediction unit 111 generates a prediction value in the prediction target block by sequentially performing filter processing using the in-plane prediction filter controlled by the filter control unit 109a. As a result, intra-predicted block data is generated. Details of the intra prediction unit 111 will be described later. Moreover, the intra estimation part 111 may perform intra prediction based on a prediction direction which is a well-known technique.

  The inter prediction unit 112 performs motion compensation on the reference image data acquired from the decoded image storage unit 110 with the motion vector provided from the motion vector calculation unit 113. Thereby, block data as a motion-compensated reference image is generated.

  The motion vector calculation unit 113 obtains a motion vector using the block data in the encoding target image and the reference image acquired from the decoded image storage unit 110. The motion vector is a value indicating a spatial deviation in units of blocks obtained using a block matching technique for searching for a position most similar to the processing target block from the reference image in units of blocks.

  The motion vector calculation unit 113 outputs the obtained motion vector to the inter prediction unit 112, and outputs motion vector information including information indicating the motion vector and the reference image to the entropy coding unit 104.

  The block data output from the intra prediction unit 111 and the inter prediction unit 112 are input to the predicted image selection unit 115.

  The predicted image selection unit 115 selects one of the block data acquired from the intra prediction unit 111 and the inter prediction unit 112 as a predicted image. The selected prediction image is output to the prediction error signal generation unit 101.

<Intra prediction>
Next, intra prediction in the first embodiment will be described. In the first embodiment, a pixel value in a prediction target block is predicted by using an in-plane prediction filter using neighboring pixel values. Hereinafter, the predicted pixel value is also referred to as a predicted value. Generally, in a video signal, neighboring pixels have similar properties, and therefore, a predicted value in a prediction target block is generated using this property.

(Constitution)
FIG. 4 is a block diagram illustrating an example of the configuration of the intra prediction unit 111 according to the first embodiment. In the example illustrated in FIG. 4, the intra prediction unit 111 includes an acquisition unit 201, a prediction unit 202, and a buffer 203.

  The acquisition unit 201 acquires a plurality of pixel values in an encoded block adjacent to the prediction target block. The number of pixels to be acquired depends on the filter size used. For example, the acquisition unit 201 acquires an encoded pixel value within a predetermined range from the prediction target pixel. The predetermined range depends on the size of the in-plane prediction filter.

  The acquisition unit 201 may acquire the encoded pixel value used for the in-plane prediction filter, for example, from the filter control unit 109a or directly from the decoded image storage unit 110. The acquisition unit 201 outputs the acquired encoded pixel value to the prediction unit 202.

  The prediction unit 202 generates the prediction value of the prediction target pixel in the prediction target block using the pixel value of the encoded block acquired by the acquisition unit 201 and / or the predicted pixel value in the prediction target block. Perform filtering sequentially.

  For example, the prediction unit 202 generates a prediction value of the prediction target pixel using the pixel value of the encoded block and / or the predicted pixel value in the prediction target block included in the filter size including the prediction target pixel. The filtering process is performed sequentially. The prediction unit 202 only needs to be able to appropriately set the in-plane prediction filter to be used.

  The prediction unit 202 can sequentially predict the prediction values in the prediction target block by sequentially performing the filtering process in a predetermined scanning order. The prediction unit 202 outputs block data including the prediction value of the generated prediction target block to the prediction image selection unit 115, for example.

  Further, when the filter size and the scanning order of the in-plane prediction filter are determined by the filter control unit 109a, the prediction unit 202 acquires the filter size and the scanning order, and follows the filter size and the scanning order.

  The buffer 203 stores the prediction value in the prediction target block generated by the prediction unit 202. This predicted value represents a predicted pixel value. Predicted values stored in the buffer 203 are read out and processed by an amount necessary for the in-plane prediction filter processing by the prediction unit 202.

(Concrete example)
FIG. 5 is a diagram illustrating the relationship between the in-plane prediction filter and the prediction target pixel. In the example illustrated in FIG. 5, the in-plane prediction filter fi <b> 11 has a size of 7 × 7, for example, and includes a prediction target pixel P at the center thereof. In the example illustrated in FIG. 5, the pixel pi11 represents an encoded pixel, the pixel pi12 represents a predicted pixel in the prediction target block pb11, and the pixels included in the region ar11 are used in the filtering process. Represents a pixel.

  When the prediction target pixel P in the prediction target block pb11 is in the position illustrated in FIG. 5, the prediction unit 202 includes the encoded pixel value and the predicted pixel value included in the region ar11 of the in-plane prediction filter fi11. And the prediction value of the prediction target pixel P is generated. The filtering process may be performed using any of the above-described in-plane prediction filters.

  FIG. 6 is a diagram illustrating an example of encoded pixels and predicted pixels used for intra prediction. The pixels used in the intra prediction using the in-plane prediction filter are the pixels of the encoded block on the left above the prediction target block and / or the already predicted pixels in the prediction target block.

  As illustrated in FIG. 6, when the prediction target pixel is located at the position P21, the pixel processed by the in-plane prediction filter fi11 is an encoded pixel in the area ar21. When the prediction target pixel is located at the position P22, the pixels processed by the in-plane prediction filter fi11 are both encoded pixels and predicted pixels in the area ar22.

  When the prediction target pixel is at the position P23, the pixels processed by the in-plane prediction filter fi11 are both encoded pixels and predicted pixels in the area ar23. When the prediction target pixel is at the position P24, the pixel processed by the in-plane prediction filter fi11 is a predicted pixel in the area ar24.

  As illustrated in FIG. 6, when generating a prediction value of a prediction target pixel, the prediction unit 202 performs a filter process using an encoded pixel value and / or a predicted pixel value in an in-plane prediction filter. By performing the above, the pixel value of the prediction target pixel is predicted. Predicted values of pixels predicted in the past in the prediction target block are stored in the buffer 203.

  Since the pixels in the in-plane prediction filter are pixels close to the prediction target pixel, the prediction accuracy of the prediction value is improved. Further, since the prediction value is generated by performing the filtering process, for example, when the encoded pixel value includes the encoding distortion, the prediction value can be generated by removing the distortion.

  As illustrated in FIG. 6, the position and number of pixels used in the filter processing are determined by the relative position between the encoding target block and the in-plane prediction filter, or the position of the prediction target pixel.

  FIG. 7 is a diagram illustrating an example (part 1) of pixels used for the filter processing. In the example illustrated in FIG. 7, the pixels A <b> 11 to 43 in the area ar <b> 31 represent encoded or predicted pixels, and the prediction target pixel P is subjected to filter processing using these pixels A <b> 11 to 43. . The in-plane prediction filter using the pixels shown in FIG. 7 is, for example, a median filter or an average value filter.

  FIG. 8 is a diagram illustrating an example (part 2) of pixels used for the filter processing. In the example illustrated in FIG. 8, the pixels A <b> 11 to 43 in the area ar <b> 32 represent encoded or predicted pixels. In the example illustrated in FIG. 8, with respect to the prediction target pixel P, unprocessed prediction values are interpolated with these pixels A11 to 43 being symmetrical. In the example illustrated in FIG. 8, the filtering process is performed using the interpolated predicted value. The in-plane prediction filter using the pixels illustrated in FIG. 8 is, for example, a smoothing filter, an edge-preserving smoothing filter, or the like.

  As illustrated in FIGS. 7 and 8, the prediction unit 202 may determine the position and number of pixels used for the filter process according to the type of the in-plane prediction filter.

<Filter control>
Next, filter control in the first embodiment will be described. The filter control unit 109a described below changes the filter size and the scanning order (scanning order) according to the feature amount of the encoded pixel value and / or the predicted pixel value.

(Constitution)
FIG. 9 is a block diagram illustrating an example of the configuration of the filter control unit 109a according to the first embodiment. In the example illustrated in FIG. 9, the filter control unit 109 a includes a feature calculation unit 301, a size determination unit 302, and a scanning order determination unit 303.

  The feature calculation unit 301 acquires a pixel value within a predetermined range of an encoded block adjacent to the prediction target block. The predetermined range is, for example, a plurality of encoded pixels adjacent to the prediction target block and a plurality of encoded pixels adjacent to the left of the prediction target block.

  The feature calculation unit 301 calculates the feature amount of the acquired encoded pixel value and / or predicted pixel value. The feature amount is, for example, any one of a correlation coefficient in the horizontal direction and the vertical direction, a variance value, a difference between pixel values from neighboring pixels, an edge amount, a texture feature amount, and the like. Which is used as the feature amount may be set in advance. For example, the feature calculation unit 301 calculates the feature amount in the horizontal direction and the feature amount in the vertical direction of the encoded pixel value. The feature calculation unit 301 outputs the calculated horizontal feature amount and vertical feature amount to the size determination unit 302.

  The size determination unit 302 determines the size of the in-plane prediction filter based on the horizontal feature amount and the vertical feature amount acquired from the feature calculation unit 301. For example, the size determination unit 302 performs threshold determination for each feature amount in the horizontal direction and the vertical direction, and determines the size in each direction.

  For example, the size determination unit 302 determines a small size if the horizontal feature amount is smaller than the horizontal threshold, and determines a large size if the horizontal feature amount is equal to or greater than the horizontal threshold. For example, the size determining unit 302 determines a small size if the vertical feature amount is smaller than the vertical threshold, and determines a large size if the vertical feature amount is equal to or greater than the vertical threshold.

  Note that a plurality of horizontal threshold values and vertical threshold values may be set, and the size determination unit 302 may be able to finely set the horizontal and vertical sizes of the in-plane prediction filter. If the size in the horizontal direction and the size in the vertical direction are determined, the shape of the in-plane prediction filter is determined. The size determination unit 302 notifies the intra prediction unit 111 and the entropy encoding unit 104 of the determined horizontal size and vertical size of the in-plane prediction filter. In addition, when the scanning order determination unit 303 determines the scanning order based on the size of the in-plane prediction filter, the size determination unit 302 may notify the scanning order determination unit 303 of the horizontal size and the vertical size. . Further, the one or more horizontal threshold values and vertical threshold values are predetermined values set in advance, for example, and may be set to appropriate values through experiments or the like, or optimal values are set while learning for each image. Also good.

  The scanning order determination unit 303 determines the scanning order of the in-plane prediction filter based on the horizontal and vertical feature quantities, or the horizontal size and the vertical size acquired from the size determination unit 302. For example, if the horizontal feature amount is greater than the threshold value by a threshold value or more, the scanning order determining unit 303 determines the horizontal order priority scanning order. Also, the scanning order determination unit 303 determines the scanning order with priority in the vertical direction if the vertical feature quantity is larger than the horizontal feature quantity by a threshold value or more. This threshold value is, for example, a predetermined value set in advance, and an appropriate value may be set by an experiment or the like.

  The scanning order determination unit 303 holds a table in which the horizontal size and vertical size (or shape) of the in-plane prediction filter are associated with the scanning order, and the determined in-plane prediction filter The scanning order corresponding to the size may be determined. The scanning order determination unit 303 notifies the intra-prediction unit 111 and the entropy encoding unit 104 of the determined scanning order.

  Note that the scan order determination unit 303 is not necessarily required, and in which scan order the filter processing is performed may be set in the intra prediction unit 111 in advance. Further, the filter control unit 109a notifies the entropy encoding unit 104 of the filter type when one of the plurality of in-plane prediction filters is selected. The filter control unit 109a does not necessarily need to notify the entropy encoding unit 104 of filter information (filter type, filter size, scanning order, etc.).

(Concrete example)
FIG. 10 is a diagram illustrating an example of the shape of a filter. The shape shown in FIG. 10A is 3 (horizontal) × 9 (vertical), the shape shown in FIG. 10B is 3 × 3, and the shape shown in FIG. 3, the shape shown in FIG. 10 (D) is 5 × 11, the shape shown in FIG. 10 (E) is 5 × 5, and the shape shown in FIG. 10 (F) is 7 × 7. is there.

  In the case of the example illustrated in FIG. 10, the size determination unit 302 prepares a threshold th11 and a threshold th12 for the horizontal feature amount. The size determining unit 302 sets the size to “3” if the horizontal feature amount is less than the threshold th11, sets the size to “5” if the feature value is greater than or equal to the threshold th11 and less than the threshold th12, and sets the size to “ 7 ”.

  In addition, the size determination unit 302 prepares threshold values th21 to th24 for the feature amount in the vertical direction. The size determination unit 302 sets the size to “3” if the vertical feature amount is less than the threshold th21, sets the size to “5” if the feature amount is greater than or equal to the threshold th21 and less than the threshold th22, and if the feature amount is greater than or equal to the threshold th22 and less than the threshold th23. The size is set to “7”, the size is set to “9” if the threshold is th23 or more and less than the threshold th24, and the size is set to “11” if the threshold is th24 or more.

  Accordingly, the size determination unit 302 can determine the size of the in-plane prediction filter. As the threshold values th11 to 12 and th21 to th24, appropriate values obtained by experiments or the like may be set in advance, or optimal threshold values may be set while learning for each image. .

  FIG. 11 is a diagram illustrating an example of the scanning order. FIG. 11A shows the scanning order for scanning with priority in the horizontal direction. FIG. 11B shows the scanning order for scanning with priority in the vertical direction. FIG. 11C shows a scanning order in which the search is performed in order from the upper right to the lower left. FIG. 11D shows the scanning order for zigzag scanning.

  A block pb illustrated in FIG. 11 indicates a prediction target block, and a block fi indicates the size of the in-plane prediction filter. As shown in FIG. 11A, for example, in the case of a horizontally long in-plane prediction filter, the scanning order determination unit 303 determines the scanning order with priority in the horizontal direction, and as shown in FIG. In the case of an in-plane prediction filter having a vertically long shape, it is determined in the scanning order with priority in the vertical direction.

<Operation>
Next, the operation of the image encoding device 10 in the first embodiment will be described. FIG. 12 is a flowchart illustrating an example of intra prediction processing according to the first embodiment.

  In step S101, the feature calculation unit 301 acquires encoded pixel values around the prediction target block from the decoded image storage unit 110, and calculates the feature amount of the pixel. The feature amount of the pixel is, for example, a feature amount in the horizontal direction and a feature amount in the vertical direction.

  In step S102, the size determination unit 302 determines the size of the in-plane prediction filter based on the calculated horizontal feature amount and vertical feature amount.

  In step S103, the scanning order determination unit 303 determines the scanning order of the filters based on the horizontal and vertical feature quantities or the determined size or shape of the in-plane prediction filter.

  In step S104, the intra prediction unit 111 generates a prediction value of the prediction target pixel in the prediction target block using the set in-plane prediction filter according to the determined filter size and scanning order.

  In step S105, the intra prediction unit 111 determines whether all the pixels in the prediction target block have been processed. If all the pixels have been processed (step S105—YES), the process proceeds to step S106. If all the pixels have not been processed (step S105—NO), the process proceeds to step S104, and a predicted value of the next prediction target pixel is generated. .

  Accordingly, the prediction value is sequentially generated by performing the filtering process using the encoded pixel value and / or the predicted pixel value within the filter size.

  In step S106, the intra prediction unit 111 determines whether prediction processing has been performed for all prediction target blocks in the screen. If all the prediction target blocks have been processed (step S106-YES), the process ends. If all the prediction target blocks have not been processed (step S106-NO), the process returns to step S101, and the same applies to other prediction target blocks. Processing is performed.

  Note that steps S101 to S103 are not necessarily required processes. This is because the intra prediction unit 111 may set the filter size and the scanning order in advance.

  As described above, according to the first embodiment, since prediction values are sequentially generated using pixels in the vicinity of the prediction target pixel, it is possible to improve the prediction accuracy of intra prediction while suppressing an increase in calculation cost. Further, according to the first embodiment, the size and scanning order of the in-plane prediction filter can be determined based on the encoded pixel value around the prediction target block or the feature amount of the predicted pixel value. The prediction accuracy can be further improved in consideration of the nature of

  Further, according to the first embodiment, when the compression rate is higher than a predetermined threshold and the noise or distortion due to encoding is larger than the predetermined threshold, a smoothing filter or an in-plane prediction filter corresponding to the type of distortion is used. be able to. In the case of a fine texture with little pixel correlation, the size of the in-plane prediction filter is made smaller than a predetermined size. In addition, when the pixel correlation in the horizontal direction is higher than that in the vertical direction, it is effective to use a horizontally long in-plane prediction filter.

  Further, according to the first embodiment, even when the encoded pixel includes the encoding distortion, the pixel to be predicted is removed while removing the encoding distortion because the filtering process is performed using the peripheral pixel values. Prediction values can be generated.

  In addition, it is thought that the intra prediction method demonstrated in Example 1 acts suitably, when a prediction object block is comparatively large size. Therefore, for example, when the prediction target block is 16 × 16 or more, the intra prediction unit 111 performs intra prediction using the in-plane prediction filter described in the first embodiment, and when the prediction target block is less than 16 × 16. Existing intra prediction may be performed.

[Example 2]
Next, an image encoding apparatus as an example of the image processing apparatus according to the second embodiment will be described. In the second embodiment, the prediction accuracy is improved by having a combination of a plurality of in-plane prediction filters, performing intra prediction with each in-plane prediction filter, and determining an optimal combination of in-plane prediction filters.

  The combination of the in-plane prediction filters includes at least one of the types of the in-plane prediction filters, the filter size, and the scanning order as a combination element.

<Configuration>
The configuration of the image encoding device according to the second embodiment will be described using the same reference numerals for the same configuration as the schematic configuration shown in FIG. However, in the second embodiment, the filter control unit 109b having a function and configuration different from those in the first embodiment notifies the entropy coding unit 104 of information indicating the determined combination of the in-plane prediction filters. The entropy encoding unit 104 includes information indicating a combination of intra prediction filters in the bitstream.

<Filter control>
Next, filter control in the second embodiment will be described. The filter control unit 109b described below performs intra prediction with different combinations for a plurality of in-plane prediction filters, a plurality of filter sizes, and a plurality of scanning orders, and determines an optimal combination.

(Constitution)
FIG. 13 is a block diagram illustrating an example of the configuration of the filter control unit 109b according to the second embodiment. In the example illustrated in FIG. 9, the filter control unit 109b includes a selection unit 401, a prediction error generation unit 405, a result storage unit 406, and a determination unit 407.

  The selection unit 401 includes a filter selection unit 402, a size selection unit 403, and a scanning order selection unit 404 in order to select a combination. Also, the selection unit 401 acquires the encoded pixels used in the intra prediction unit 111 from the decoded image storage unit 110 and outputs the acquired encoded pixels to the intra prediction unit 111.

  The filter selection unit 402 selects one from a plurality of different in-plane prediction filters. The size selection unit 403 selects one from a plurality of filter sizes. The scanning order selection unit 404 selects one from a plurality of scanning orders.

  The selection unit 401 notifies the intra prediction unit 111 of the contents of the selected combination. When notified from the intra prediction unit 111 that the intra prediction by one combination has been completed, the selection unit 401 selects the next combination and notifies the intra prediction unit 111 of the content of the combination. Note that the selection unit 401 performs control so that intra prediction is performed with all combinations of types of in-plane prediction filters, filter sizes, and scanning orders.

  The prediction error generation unit 405 generates a prediction error from the block data including the prediction value generated from the intra prediction unit 111. The prediction error generation unit 405 writes the generated prediction error and information indicating the filter combination in the result storage unit 406.

  The result storage unit 406 stores information indicating a combination of filters and a prediction error. The result storage unit 406 finally stores prediction errors of all combinations for the in-plane prediction filter.

  The determination unit 407 determines the filter type, filter size, and scanning order of the combination having the smallest prediction error among the prediction errors stored in the result storage unit 406. The determination unit 407 notifies the determined information to the intra prediction unit 111 and the entropy encoding unit 104.

  The intra prediction unit 111 outputs block data generated by prediction based on the notified combination to the predicted image selection unit 115.

<Operation>
Next, the operation of the image coding apparatus according to the second embodiment will be described. FIG. 14 is a flowchart illustrating an example of intra prediction processing according to the second embodiment.

  In step S201, the filter selection unit 402 selects one from a plurality of in-plane prediction filters that are held in advance.

  In step S202, the size selection unit 403 selects one of a plurality of filter sizes held in advance.

  In step S203, the scanning order selection unit 404 selects one from a plurality of scanning orders held in advance.

  In step S204, the intra prediction unit 111 generates a prediction value of the prediction target pixel in the prediction target block according to the selected in-plane prediction filter, filter size, and scanning order.

  In step S205, the intra prediction unit 111 determines whether all the pixels in the prediction target block have been processed. If all the pixels have been processed (step S205—YES), the process proceeds to step S206. If all the pixels have not been processed (step S205—NO), the process proceeds to step S204, and a predicted value of the next prediction target pixel is generated. .

  As a result, by using the encoded pixel value and / or the predicted pixel value within the filter size and performing processing using the in-plane prediction filter in a predetermined combination, predicted values are sequentially generated. .

  In step S <b> 206, the selection unit 401 determines whether or not the processing has been performed in the entire scanning order. If the full scan order has been processed (step S206—YES), the process proceeds to step S207, and if the full scan order has not been processed (step S206—NO), the process proceeds to step S203.

  In step S207, the selection unit 401 determines whether processing has been performed with all filter sizes. If all filter sizes have been processed (step S207—YES), the process proceeds to step S208. If all filter sizes have not been processed (step S207—NO), the process returns to step S202.

  In step S208, the selection unit 401 determines whether or not processing has been performed for all filters. If all filters have been processed (step S208—YES), the process proceeds to step S209. If all filters have not been processed (step S208—NO), the process returns to step S201.

  In step S209, the determination unit 407 determines a combination of filters (filter type, filter size, scanning order) with the smallest prediction error based on the prediction error generated by the prediction error generation unit 405.

  As described above, according to the second embodiment, the prediction accuracy is improved by having a plurality of in-plane prediction filters, performing intra prediction with each in-plane prediction filter, and determining an optimal in-plane prediction filter. Can do.

  In order to suppress an increase in calculation cost, intra prediction performed to determine an in-plane prediction filter may be performed on only a part of the prediction target block.

  In addition, the existing intra prediction can also be added as a candidate with respect to Example 1 and Example 2, and the best intra prediction can also be selected. For example, the intra prediction unit 111 generates a prediction value using the in-plane prediction filter according to the first embodiment or the second embodiment, H.264 / AVC and other modes for generating a prediction value from pixel values that have been encoded using a plurality of prediction directions, and determining a mode based on a prediction error, a generated code amount, or an RD characteristic May be.

  Note that the combinations of the in-plane prediction filters are not necessarily combinations of the types of the in-plane prediction filters, the filter sizes, and the scan order, and may be any one or a combination of any two.

  Here, in the conventional intra prediction method, the prediction is performed using only pixel data for one line adjacent to the encoding target block (prediction target block). There wasn't.

  On the other hand, in the first embodiment and the second embodiment, not only the encoded pixel data in the vicinity of the prediction target pixel but also the predicted pixel data in the prediction target block are used together and sequentially filtered. Intra prediction can be performed with high accuracy based on the properties of neighboring pixels.

  Moreover, in Example 1 and Example 2, since the predicted pixel value is also considered, the prediction accuracy with respect to a non-planar pixel value waveform is improved. In addition, since the sequential filter configuration is used, the calculation cost can be reduced as compared with the repeated convergence calculation.

  In addition, by changing the type and characteristics of the in-plane prediction filter, it is possible to prevent a decrease in prediction efficiency due to noise and coding distortion included in pixels in the vicinity region. For example, the influence of noise and block distortion can be reduced by using a smoothing filter or a median filter.

  Further, by using a bilater filter, it is possible to reduce the influence of noise while preserving the edge component. Further, by using a sigma filter, it is possible to reduce the influence of mosquito noise due to encoding.

[Example 3]
Next, an image decoding apparatus as an example of an image processing apparatus according to the third embodiment will be described. In the third embodiment, the stream encoded in the first and second embodiments is decoded. In the third embodiment, the same in-plane prediction filter processing as that in the first and second embodiments is performed to generate the same prediction signal as that in the encoding device.

<Configuration>
FIG. 15 is a block diagram illustrating an example of a schematic configuration of the image decoding device 50 according to the third embodiment. As illustrated in FIG. 15, the image decoding device 50 includes an entropy decoding unit 501, an inverse quantization unit 502, an inverse orthogonal transform unit 503, an intra prediction unit 504, an inter prediction unit 506, a predicted image selection unit 507, and a decoded image generation unit. 508, a filter control unit 510, and a decoded image storage unit 511. An outline of each part will be described below.

  When the bit stream is input, the entropy decoding unit 501 performs entropy decoding corresponding to the entropy encoding of the image encoding device 10. The prediction error signal decoded by the entropy decoding unit 501 is output to the inverse quantization unit 502. When information indicating the combination of the in-plane prediction filters is included, information indicating the combination of the in-plane prediction filters is output to the filter control unit 510. A decoded motion vector or the like when inter prediction is performed is output to the inter prediction unit 506.

  The inverse quantization unit 502 performs inverse quantization processing on the output signal from the entropy decoding unit 501. The inversely quantized output signal is output to the inverse orthogonal transform unit 503.

  The inverse orthogonal transform unit 503 performs an inverse orthogonal transform process on the decoded block of the output signal from the inverse quantization unit 502 to generate a residual signal. The residual signal is output to the decoded image generation unit 508.

  The intra prediction unit 504 generates a predicted image from the already decoded peripheral pixels acquired from the decoded image storage unit 511 using a filter notified from the filter control unit 510 described later.

  The inter prediction unit 506 performs motion compensation on the reference image data acquired from the decoded image storage unit 511 using a motion vector. Thereby, block data as a motion-compensated reference image is generated.

  The predicted image selection unit 507 selects either one of the intra predicted image and the inter predicted image. The selected block data is output to the decoded image generation unit 508.

  The decoded image generation unit 508 adds the predicted image output from the predicted image selection unit 507 and the residual signal output from the inverse orthogonal transform unit 503 to generate a decoded image. The generated decoded image is output to the decoded image storage unit 511.

  The decoded image storage unit 511 stores a decoded image that serves as a reference image.

  When the stream encoded by the image encoding apparatus in the first embodiment is decoded, the filter control unit 510 controls the filter using the filter information decoded by the entropy decoding unit 501, and the filter information The intra prediction unit 504 is notified. The filter information includes at least one of filter type, filter size, and scanning order. Further, when the filter control unit 501 can determine the filter size and scanning order in the same manner as the processing of the filter control unit 109a described in the first embodiment, the intra prediction unit 504 determines the determined filter size and scanning order. May be notified.

  In addition, when the stream encoded by the image encoding apparatus in the second embodiment is decoded, the filter control unit 510 acquires information indicating a combination of the intra prediction filters, and the information of the intra prediction filter indicated by the information is acquired. The combination is notified to the intra prediction unit 504.

  As described above, according to the third embodiment, it is possible to appropriately decode the stream encoded in the first or second embodiment.

[Example 4]
Next, an image processing apparatus according to the fourth embodiment will be described. The image processing apparatus collectively refers to the image encoding apparatus and the image decoding apparatus as an image processing apparatus. In the fourth embodiment, a case where each process of the above-described embodiment is implemented by software will be described.

  FIG. 16 is a diagram illustrating an example of the configuration of the image processing device 60 according to the fourth embodiment. An image processing device 60 illustrated in FIG. 16 includes at least a control unit 601, a main storage unit 602, an auxiliary storage unit 603, a communication unit 604, and a recording medium I / F unit 605. Each unit is connected via a bus so that data can be transmitted / received to / from each other.

  The control unit 601 is a CPU (Central Processing Unit) that controls each device, calculates data, and processes in a computer. The control unit 601 is an arithmetic device that executes programs stored in the main storage unit 602 and the auxiliary storage unit 603. The control unit 601 receives data from the communication unit 604 and each storage unit, calculates, processes, and outputs the data. And output to each storage unit.

  In addition, the control unit 601 can execute the above-described processing by executing a program for performing any of the processing of the above-described embodiment stored in the auxiliary storage unit 603, for example.

  The main storage unit 602 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like, and a storage device that stores or temporarily stores programs and data such as an OS and application software that are basic software executed by the control unit 601. It is.

  The auxiliary storage unit 603 is an HDD (Hard Disk Drive) or the like, and is a storage device that stores data related to application software or the like. The auxiliary storage unit 603 may store a program acquired from the recording medium 606 or the like.

  The communication unit 604 performs wired or wireless communication. For example, the communication unit 604 may store a program transmitted from a server or the like in the auxiliary storage unit 603.

  A recording medium I / F (interface) unit 605 is an interface between the image processing apparatus 60 and a recording medium 606 (for example, a flash memory) connected via a data transmission path such as a USB (Universal Serial Bus).

  A predetermined program is stored in the recording medium 606, and the program stored in the recording medium 606 is installed in the image processing apparatus 60 via the recording medium I / F unit 605. The installed predetermined program can be executed by the image processing apparatus 60.

  For example, the decoded image storage unit 110 illustrated in FIG. 3 and the decoded image storage unit 511 illustrated in FIG. 15 can be realized by the main storage unit 602 or the auxiliary storage unit 603, for example. Each unit other than the storage units shown in FIGS. 3 and 15 can be realized by, for example, the control unit 601 and the main storage unit 602 as a working memory.

  It is also possible to record the program on the recording medium 606 and cause the computer or portable terminal to read the recording medium 606 on which the program is recorded to realize the above-described processing.

  The recording medium 606 is a recording medium that records information optically, electrically, or magnetically, such as a CD-ROM, flexible disk, magneto-optical disk, etc., and information is electrically stored, such as a ROM or flash memory. Various types of recording media such as a semiconductor memory for recording can be used. Further, the processes described in the above embodiments may be implemented in one or a plurality of integrated circuits.

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes other than the above-described modification are possible within the scope described in the claims. .

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 50 Image decoding apparatus 60 Image processing apparatus 109 Filter control part 111 Intra prediction part 201 Acquisition part 202 Prediction part 203 Buffer 301 Feature calculation part 302 Size determination part 303 Scan order determination part 401 Selection part 405 Prediction error generation part 406 Result storage unit 407 Determination unit 504 Intra prediction unit 510 Filter control unit 601 Control unit 602 Main storage unit

Claims (7)

An image encoding device that performs intra prediction of an image,
An acquisition unit that acquires a plurality of pixel values in an encoded block adjacent to the prediction target block;
Prediction of the prediction target pixel using the pixel value of the acquired encoded block and / or the predicted pixel value in the prediction target block that are within a predetermined range from the prediction target pixel in the prediction target block An intra prediction unit that sequentially performs filter processing to generate values;
An image encoding device comprising: A filter control unit for controlling a filter that generates the predicted value;
The filter control unit
One of the encoded pixel value and / or the predicted pixel value in the horizontal and vertical correlation coefficients, the variance value, the difference of the pixel value from the surrounding pixels, the edge amount, or the texture feature amount A feature calculation unit for calculating a feature amount;
The image encoding device according to claim 1, further comprising: a size determining unit that determines a size of the filter based on a feature amount in the horizontal direction and the vertical direction calculated by the feature calculating unit. A filter control unit for controlling a filter that generates the predicted value;
The filter control unit
The image coding apparatus according to claim 1, further comprising: a plurality of different filters, wherein a prediction value is generated by each filter in the prediction unit, and a filter to be used is determined based on a prediction error of each filter. The intra prediction unit
A mode in which a prediction value is generated using the filter and a mode in which a prediction value is generated from pixel values that have been encoded using a plurality of prediction directions, and a prediction error, a generated code amount, or RD The image coding apparatus according to any one of claims 1 to 3, wherein a mode is determined based on characteristics. An image decoding device for decoding a stream encoded by the image encoding device according to claim 1,
A decoding unit for entropy decoding the stream;
Among the pixel values in the decoded block adjacent to the prediction target block for intra prediction, the pixel value of the decoded block and / or in the prediction target block within a predetermined range from the prediction target pixel in the prediction target block An intra prediction unit that sequentially performs a filtering process for generating a predicted value of the prediction target pixel using the predicted pixel value of
An image decoding apparatus comprising: On the computer,
An obtaining step for obtaining a plurality of pixel values in an encoded block adjacent to the prediction target block;
Prediction of the prediction target pixel using the pixel value of the acquired encoded block and / or the predicted pixel value in the prediction target block that are within a predetermined range from the prediction target pixel in the prediction target block An intra-prediction step that sequentially performs filtering to generate values;
An image encoding program for executing On the computer,
A decoding step for entropy decoding the image encoded stream;
Among the pixel values in the decoded block adjacent to the prediction target block for intra prediction, the pixel value of the decoded block and / or in the prediction target block within a predetermined range from the prediction target pixel in the prediction target block An intra prediction step that sequentially performs a filter process for generating a predicted value of the prediction target pixel using the predicted pixel value of
An image decoding program for executing

Images