CN101227615A - image processing method - Google Patents

Abstract

An image processing method is provided, which extends the formula of the 4*4 block prediction mode in the image frame prediction mode to the prediction of each 4*4 block or 8*8 block in the 16*16 block, and determines the prediction mode of each 4*4 block or 8*8 block in the 16*16 block in one operation (Pass), or determines the prediction mode of each 8*8 block in the 16*16 block in three operations. In this way, the operation of determining the prediction mode of each 4*4 block can be omitted, and the processing time of the 4*4 block frame prediction can be effectively reduced, so as to achieve the purpose of increasing the image processing efficiency.

Description

image processing method

technical field

The present invention relates to an image processing method, and in particular relates to an image processing method capable of shortening the time for judging a prediction mode and accelerating the speed of image reconstruction.

Background technique

H.264 is developed by the Video Coding Expert Group (VCEG) of the International Telecommunication Union (International Telecommunication Union-Telecommunication Standardization Sector, ITU-T) and the motion picture expert group of the International Organization for Standardization (ISO). (Moving Pictures Experts Group, MPEG) is a new generation of video compression standard proposed by the Joint Video Group. This technology is called Advanced Video Decoding after it is included in Part 10 of MPEG-4 (ISO/IEC 14496-10). (Advanced Video Coding, AVC), or combined called H.264/AVC. Relevant research data show that compared with MPEG-2 and MPEG-4, H.264/AVC has greatly improved both in compression rate and video quality, so it is also widely used in video conferencing, video broadcasting or video applications such as video streaming services.

H.264 has better performance in spatial and temporal prediction than previous standards such as H.263+. It uses intra-frame (Intra Frame) and inter-frame (Inter Frame) encoded reconstructed pixels to predict internal code block. Among them, when using intra-frame prediction technology to predict pixels, in order to increase coding efficiency, H.264 uses the correlation with adjacent blocks in the spatial domain (Spatial domain) to make predictions, and only Record predicted patterns and actual errors. Adjacent blocks usually refer to the blocks to the left and above of the current coded block. The pixels of these blocks have been coded, so the recorded information can be reused.

Taking the prediction of a 4×4 block as an example, Figure 1 shows the configuration diagram of a known H.264 4×4 block, where a~p represent the pixels in the block, and A~H represent the area The edge pixel values of the block above the block, and I~L represent the edge pixel values of the left block. The prediction mode of H.264 uses these edge pixel values for prediction.

The intra-frame prediction technology can be divided into 4×4 luma (Luma) prediction mode, 16×16 luma (Luma) prediction mode and 8×8 chroma (Chroma) prediction mode according to the image complexity. Among them, the 4×4 luma prediction mode includes 9 different prediction modes according to the different prediction directions. FIG. 2 and FIG. 3 are respectively a schematic diagram and a formula list of nine prediction modes included in the 4×4 luma prediction mode of the known H.264. Please refer to Figure 2 and Figure 3 at the same time. These prediction modes include direct current (DC) mode and the other 8 directions, and the formula can be summarized as follows according to the position of the pixel:

Wherein i∈ block L, K, J, I, M, A, B, C, D, E, F, G, H, for example, if mode 0 (ie vertical mode) is selected, the following formula can be used to Predict the pixel value of the pixel (y, x) at row x, column y:

Pixels (0, 0), (1, 0), (2, 0), (3, 0) are predicted by A;

Pixels (0, 1), (1, 1), (2, 1), (3, 1) are predicted by B;

Pixels (0, 2), (1, 2), (2, 2), (3, 2) are predicted by C;

Pixels (0, 3), (1, 3), (2, 3), (3, 3) are predicted by D.

In addition, if mode 3 is selected (that is, the diagonal bottom left mode), the pixel value is predicted by the following formula:

Pixel (0, 0) is predicted by (A+2B+C+2)/4;

Pixels (0, 1), (1, 0) are predicted by (B+2C+D+2)/4;

Pixels (0, 2), (1, 1), (2, 0) are predicted by (C+2D+E+2)/4;

Pixels (0, 3), (1, 2), (2, 1), (3, 0) are predicted by (D+2E+F+2)/4;

Pixels (1, 3), (2, 2), (3, 1) are predicted by (E+2F+G+2)/4;

Pixels (2, 3), (3, 2) are predicted by (F+2G+H+2)/4;

Pixel (3,3) is predicted by (G+3H+2)/4.

The 4×4 luma prediction mode is based on the 4×4 sub-block (Sub-block), and uses the above nine prediction modes to find its reference object (Predictor), and subtracts it from the reference object to obtain Residual image. Finally, after converting the residual image and the used prediction mode, the image coding of the 4×4 sub-block can be obtained.

However, when H.264 performs image coding, it actually codes in units of 16×16 blocks, and a 16×16 block is subdivided into 4×4 sub-blocks for prediction. In this way, when decoding a 4×4 sub-block, it is necessary to refer to the reconstruction values of the left and upper blocks, so when decoding these 4×4 sub-blocks, one by one (by From top to bottom, from left to right) 4×4 sub-blocks, although this can obtain more accurate prediction results, but the occupied computing resources and computing time will be relatively increased, which cannot meet the current requirements. Emphasis on efficiency requirements.

Contents of the invention

In view of this, the present invention provides an image processing method, which can increase the efficiency of image processing by finding the prediction mode of each 4*4 blocks in the 16*16 blocks in one pass.

The present invention provides an image processing method, which can increase the efficiency of image processing by finding out the prediction mode of each 8*8 blocks in the 16*16 blocks in one operation.

The present invention provides an image processing method, which uses 8*8 blocks as a unit to successively find out the prediction modes of each 8*8 blocks in 16*16 blocks, so as to increase the efficiency of image processing.

The present invention proposes an image processing method, which is suitable for an image that can be divided into multiple prediction blocks. The method includes the following steps: a. Calculate the distance between the pixels of these prediction blocks and the corresponding image boundary pixels in multiple prediction modes b. According to the result of step a, determine the prediction mode used for pixel reconstruction of the prediction block; and c. Reconstruct the pixels of these prediction blocks according to the result of step b.

In an embodiment of the present invention, after step a, it further includes: a1. Using the result of step a, calculating the residual value of each prediction block pixel in each prediction mode; and a2. According to the result of step a and step a1, Determines the prediction mode used when predicting block pixel reconstruction.

In an embodiment of the present invention, the above image is a block composed of 16*16 pixels, and the prediction block is a 4*4 block.

In an embodiment of the present invention, the above method includes vertical prediction mode, horizontal prediction mode, DC prediction mode, diagonal lower left prediction mode, diagonal lower right prediction mode, vertical rightward prediction mode, and horizontal downward prediction mode , vertical left prediction mode, and horizontal upward prediction mode and other 9 prediction modes.

In an embodiment of the present invention, step a2 further includes: a2-1. According to the result of step a1, determine the bit transmission rate corresponding to the prediction block pixels in these prediction modes; a2-2. According to step a and The result of step a2-1, calculating the optimal bit rate-distortion value of the pixels in the prediction block; and a2-3. According to the result of step a2-2, determining the prediction mode used when the pixels of the prediction block are reconstructed.

In an embodiment of the present invention, the a2-1 step is to use the residual values of these prediction block pixels in these prediction modes after discrete cosine transform, quantization, and entropy operation to determine the prediction block pixels in The corresponding bit rates for these predictive modes.

In an embodiment of the present invention, the above-mentioned 16 preset prediction blocks are sequentially divided into prediction blocks at positions a to p from top to bottom, and from left to right, then the prediction blocks of each prediction block in step c The reconstruction sequence is: a→b, e→c, f, i→d, g, j, m→h, k, n→l, o→p.

The present invention proposes an image processing method, which is suitable for images that can be divided into multiple prediction blocks. The method includes the following steps: a. Divide these prediction blocks into multiple regions, wherein each region has at least two prediction block; b. first for one of these areas, calculate the sum of squares of the difference between each prediction block pixel in this area and the corresponding image boundary pixel in multiple prediction modes; c. according to the result of step b, determine The prediction mode used when reconstructing the pixels of the prediction block in this area; d. according to the result of step c, reconstruct the pixels of each prediction block in this area; and e. according to the result of step d, use the reconstructed pixels on the boundary of the area to reconstruct Pixels of predicted blocks in the region adjacent to this region.

In an embodiment of the present invention, after step b, it further includes: b1. Using the result of step b, calculate the residual value of each prediction block pixel in this area under these prediction modes; and b2. According to step b and step b1 As a result, the prediction mode used for pixel reconstruction of the prediction block in this region is determined.

In an embodiment of the present invention, the above-mentioned image is a block composed of 16*16 pixels, the prediction block is a 4*4 block, and the above-mentioned area is an 8*8 block.

In an embodiment of the present invention, step b2 further includes: b2-1. According to the result of step b1, determine the bit transmission rate corresponding to each prediction block pixel in this area under these prediction modes; b2-2. According to Step b and the result of step b2-1, calculate the optimal bit rate-distortion value of each prediction block pixel in this area; and b2-3. According to the result of step b2-2, determine the reconstruction of each prediction block pixel in this area The prediction mode used when .

In an embodiment of the present invention, the four preset prediction blocks are sequentially divided into prediction blocks at positions a, b, e and f from top to bottom and from left to right, then these prediction blocks in step e The reconstruction order of blocks is: a→b, e→f.

In an embodiment of the present invention, step e further includes: e1. Calculating the sum of squared differences between each prediction block pixel in at least one adjacent region and the reconstructed pixel of the corresponding region boundary under multiple prediction modes ; e2. Using the result of step e1, calculate the residual value of each prediction block pixel in the adjacent area under these prediction modes; e3. According to the results of step e1 and step e2, determine when the pixels of the prediction block in the adjacent area are reconstructed The prediction mode used; and e4. Reconstructing the pixels of each prediction block in the adjacent area according to the result of step e3. In addition, the above-mentioned steps e1-e4 are to reconstruct pixels in two adjacent regions.

The present invention proposes an image processing method, which is suitable for images that can be divided into multiple prediction blocks. The method includes the following steps: a. Divide these prediction blocks into multiple regions, wherein each region has at least two prediction block; b. calculate the sum of the squares of the difference between each prediction block pixel in each region and the corresponding image boundary pixel in multiple prediction modes; c. determine the prediction block pixel in each region according to the result of step b The prediction mode used during reconstruction; and d. Reconstructing the pixels of each prediction block in each region according to the result of step c.

In an embodiment of the present invention, after step b, it further includes: b1. Using the result of step b, calculate the residual value of each prediction block pixel in each region under these prediction modes; and b2. According to step b and b1 As a result of the step, the prediction mode used for pixel reconstruction of the prediction block in each region is determined.

In an embodiment of the present invention, the above-mentioned image is a block composed of 16*16 pixels, the prediction block is a 4*4 block, and the above-mentioned area is an 8*8 block.

In an embodiment of the present invention, step b2 further includes: b2-1. According to the result of step b-1, determine the corresponding bit transmission rate of each prediction block pixel in each region under these prediction modes; b2- 2. According to the results of step b and step b2-1, calculate the optimal bit rate-distortion value of each prediction block pixel in each region; and b2-3. According to the result of step b2-2, determine the optimal The prediction mode used when predicting block pixel reconstruction.

In an embodiment of the present invention, the above-mentioned four preset areas are sequentially divided into four areas of positions I, II, III and IV from top to bottom and from left to right, then each prediction block in step e The order of reconstruction is: I→II, III→IV.

In the present invention, the formula of the 4*4 block prediction mode is extended and applied to the prediction of each 4*4 block or 8*8 block in the 16*16 block, which saves the calculation and judgment of each 4*4 block The calculation time and computing resources consumed by the prediction mode can increase the efficiency of image processing.

In order to make the above and other objects, features and advantages of the present invention more comprehensible, preferred embodiments will be described in detail below together with the accompanying drawings.

Description of drawings

FIG. 1 is a configuration diagram of a known H.264 4×4 block.

FIG. 2 is a schematic diagram of nine prediction modes included in the known 4×4 luma prediction mode of H.264.

Fig. 3(a)-(b) is a formula list of 9 prediction modes included in the 4×4 luma prediction mode of known H.264.

FIG. 4 is a flowchart of an image processing method according to the first embodiment of the present invention.

FIG. 5 is a schematic diagram of prediction mode 3 of the H.264 intra-frame prediction algorithm according to the first embodiment of the present invention.

FIG. 6 is a flowchart of a method for calculating the sum of squares of differences according to the first embodiment of the present invention.

FIG. 7 shows the difference sum of squares in nine different prediction modes according to the first embodiment of the present invention.

8(a)-(b) are an example of an image processing method according to the first embodiment of the present invention.

9(a)-(f) are an example of reconstructed pixels according to the first embodiment of the present invention.

FIG. 10 is a flowchart of an image processing method according to a second embodiment of the present invention.

FIG. 11 is the difference sum of squares in 9 different prediction modes according to the second embodiment of the present invention.

12(a)-(b) are schematic diagrams of image processing according to the second embodiment of the present invention.

FIG. 13 is a flowchart of an image processing method according to a third embodiment of the present invention.

FIG. 14 is the difference sum of squares in 9 different prediction modes according to the third embodiment of the present invention.

15(a)-(f) are schematic diagrams of an image processing method according to a third embodiment of the present invention.

[Description of main component labels]

a～p, A1～A4, B1～B4, C1～C4, D1～D4, E1～E4, F1～F4, G1～G4, H1～H4, I1～I4, J1～J4, K1～K4, L1～ L4: Pixels

A～H, I～L, M, I, II, III, IV: blocks

Ra1～Ra16, Rb1～Rb16, Rc1～Rc16, Rd1～Rd16, a'～p': reconstruction value

S410-S430: each step of the image processing method in the first embodiment of the present invention

S411-S415: each step of the method for calculating the sum of squared differences in the first embodiment of the present invention

S1010-S1040: each step of the image processing method of the second embodiment of the present invention

S1310-S1328: each step of the image processing method of the third embodiment of the present invention

Detailed ways

In-frame prediction must refer to the edge pixel values of the block above and to the left of the current block in order to calculate the predicted value of the pixel and find out the most suitable prediction mode. This feature allows the image processing algorithm to predict only one time Process a block. Aiming at this point, the present invention expands the prediction formula of the original small block and applies it to the large block, and only uses one operation (pass) to find out the prediction mode applicable to each small block in the large block or an area including several small blocks . In this way, the edge pixel values of each block can be calculated early, and the prediction values of multiple blocks can be calculated simultaneously, thereby increasing the efficiency of image processing. In order to make the content of the present invention clearer, the following specific examples are given as examples in which the present invention can actually be implemented.

first embodiment

FIG. 4 is a flowchart of an image processing method according to the first embodiment of the present invention. Please refer to Fig. 4, this embodiment is applicable to an image that can be divided into multiple 4*4 blocks, a 16*16 block in this image includes 16 4*4 blocks, and each 4*4 block 16 pixels are included. Wherein, in this embodiment, the prediction formula of the original 4*4 block is expanded and applied to the 16*16 block, and the prediction mode of each 4*4 block in the 16*16 block is found out in one operation, and it is possible to Increase the efficiency of image processing.

In detail, the nine prediction modes of the original intra-frame prediction algorithm (as shown in FIG. 2 and FIG. 3 ) will be extended and applied to 16*16 blocks. For example, FIG. 5 is a schematic diagram of the prediction mode 3 (ie, the diagonal bottom left mode) of the image intra-frame prediction algorithm according to the first embodiment of the present invention. Please refer to FIG. 2 and FIG. 5 at the same time. The prediction mode 3 of this embodiment is similar to the known prediction mode 3. The direction of the reference pixel value is 45 degrees to the lower left, but this embodiment uses the original 4* The prediction of 4 blocks is further extended to the prediction of 16*16 blocks, and the corresponding prediction formula is also extended to the size of 16*16 blocks. For example, pixel (0,0) is predicted by (A1+2A2+A3+2)/4; pixel (0,1), (1,0) is predicted by (A2+2A3+A4+2)/ 4 to predict; pixels (0, 2), (1, 1), (2, 0) are predicted by (A3+2A4+B1+2)/4, and so on.

In this embodiment, the above method is used to extend the formulas of the nine prediction modes, and calculate the sum of squares of differences between the pixels of these 4*4 blocks in each prediction mode and the corresponding image boundary pixels (step S410 ). Wherein, the aforementioned image boundary pixel is, for example, a row/column matrix pixel. In addition, the above steps can be subdivided into a plurality of sub-steps. FIG. 6 is a flowchart of a method for calculating the sum of squares of differences according to the first embodiment of the present invention. Please refer to FIG. 6 , first use the formulas of the above prediction modes to calculate the prediction values of all pixels in the 16*16 block corresponding to the 9 prediction formulas (step S411 ).

Then, taking the 4*4 block as a unit, the predicted value of each 4*4 block pixel is subtracted from its original value to obtain the residual value (Residual) of these 4*4 block pixels (step S412 ). These residual values are transformed by discrete cosine transformation (Discrete Cosine Transformation, DCT), quantization (Quantization, Q), inverse quantization (Inverse Quantization, IQ) and inverse discrete cosine transformation (Inverse Discrete Cosine Transformation, IDCT), etc. After the step, it will be transformed into a converted value (step S413). These conversion values are added to the prediction value to obtain the mode prediction value (step S414). Finally, the difference between the model prediction value and the original value of each 4*4 block pixel is squared and added to obtain the sum of squares of the difference in 9 different prediction modes as shown in Figure 7 (step S415), the calculation formula of this difference sum of squares is as follows:

Wherein, x represents the number of rows of the block, and y represents the number of columns of the block. According to the sum of squared differences calculated by these prediction modes, the most suitable prediction mode for each 4*4 block can be found (step S420) . Specifically, in this embodiment, for each 4*4 block, the minimum value is selected from the 9 difference sums of squares calculated by using the above 9 prediction modes, and the prediction mode with the minimum value is selected to make The most suitable prediction mode for this 4*4 block. For example, FIG. 8 is an example of an image processing method according to the first embodiment of the present invention. As shown in FIG. 8( a ), the prediction mode selected by block a is the 8th mode, the prediction mode selected by block b is the 0th mode, and so on. The above steps are all completed in one operation (Pass), so the time spent in finding the prediction mode block by block can be saved.

It is worth mentioning that the above method of calculating the sum of squared differences to determine the prediction mode is a simple mode of using the intra-frame prediction algorithm. However, when complex mode is used, the bit rate must be taken into consideration. Among them, the bit transmission rate can be calculated by calculating the residual value of each 4*4 block pixel in each prediction mode (as shown in Figure 8(b)), and after discrete cosine transform, quantization, and entropy (entropy) operation , to obtain the bit transmission rate corresponding to each 4*4 block pixel in each prediction mode. Then, after adding the bit transmission rate to the original sum of squares of the differences, the optimal value of the bit transmission rate-distortion (Rate-distortion optimization, RDO) can be obtained. Accordingly, the prediction mode corresponding to the minimum value of RDO is found out as the prediction mode used for pixel reconstruction of each 4*4 block. The prediction model obtained here is more accurate than the previous prediction model obtained by using the difference to the sum of squares because of the factor of the bit rate.

After the prediction mode of each 4*4 block is determined, in another operation (Pass), the determined prediction mode can be used to reconstruct the pixels of these 4*4 blocks based on the image boundary pixels. (Step S430 ), and when the reconstruction values of the pixels in all 4*4 blocks are calculated, the reconstruction step of the pixels in the 16*16 block is completed. Wherein, in this embodiment, a 16*16 block is divided into 16 4*4 blocks during pixel reconstruction, and these 4*4 blocks are sequentially numbered as a~p from top to bottom and from left to right (As shown in Figure 5), and the reconstruction sequence of these 4*4 blocks is: a→b, e→c, f, i→d, g, j, m→h, k, n→l, o→ p.

In one embodiment, in order to speed up the above-mentioned steps of pixel reconstruction, this embodiment also includes that when reconstructing a 4*4 block, first calculate the edge pixels of the 4*4 block by using a prediction formula, and further provide the Its adjacent 4*4 blocks are used for prediction. With this method, while rebuilding a 4*4 block, the reconstruction of the pixels of the next adjacent 4*4 block can be carried out, and the pixels can be greatly reduced. The time required to rebuild.

In order to help understanding, the following examples are given to illustrate the detailed steps of the above-mentioned reconstruction pixels, and for the convenience of description, only four 4*4 blocks a, b, e, f (that is, the four blocks in the upper left corner of Figure 5) are reconstructed here. 4*4 blocks) as an example. FIG. 9 is an example of reconstructed pixels according to the first embodiment of the present invention. Please refer to FIG. 9. In this embodiment, first, the prediction formula of the prediction mode corresponding to the 4*4 block a is used to calculate the predicted values of the pixels in the 4*4 block a in FIG. 9(a), and these predicted values are added After adding the residual value originally calculated in step S412, the reconstructed value of the pixel in the 4*4 block a can be obtained.

It is worth mentioning that in the above steps, the predicted values of the 7 pixels including the right edge and the lower edge of the 4*4 block a can be calculated by using the prediction formula, and provided to the right and lower 4*4 areas first Blocks b and e start reconstruction work early, so that the time of waiting for the reconstruction of 4*4 block a can be saved, and the overall image reconstruction speed can be accelerated. For example, as shown in FIG. 9( b ), the prediction mode of the 4*4 block a is vertically downward, so the pixels at the bottom and right edges of the 4*4 block a can be predicted first.

After obtaining the predicted values of the right edge and lower edge pixels of the 4*4 block a, the predicted values of the pixels of the 4*4 block b and e can be further calculated, and after adding the difference value, the 4*4 Reconstructed values of pixels in blocks b and e. Among them, since the pixels at the lower edge of the 4*4 block b may need to be used for prediction when calculating the e pixels of the 4*4 block, so in the above steps, the prediction formula can be used to calculate the 4*4 block The prediction value of the 4 pixels at the lower edge of b is first provided to the 4*4 block e to start the reconstruction work early, so that the time of waiting for the reconstruction of the 4*4 block b can be saved, and the speed of the overall image reconstruction can be accelerated .

For example, as shown in Figure 9(c), the prediction mode of the 4*4 block b is horizontal to the right, so the pixels at the lower edge of the 4*4 block b can be predicted first, and the 4*4 block b is also predicted at the same time. *4 Calculation of reconstruction values Ra1-Ra16 of pixels in block a. After obtaining the pixels at the lower edge of the 4*4 block b, it can be used to calculate the predicted value of the 4*4 block e pixel. In order to provide the 4*4 block f for prediction as early as possible, the 4*4 block f can be calculated first. 4 Pixels on the right edge of block e. For example, as shown in Figure 9(d), the prediction mode of the 4*4 block e is the vertical bottom left mode, so the pixels at the lower edge of the 4*4 block b can be used to first predict the 4*4 block e The pixels on the right edge are used to obtain predicted values e4, e8, e12, and e16. Wherein, the calculation formulas of these predicted values are as shown in FIG. 3 , so they will not be repeated here. Of course, the reconstruction values Rb1-Rb16 of pixels in the 4*4 block b can also be calculated while predicting these predicted values.

After obtaining the predicted value of the lower edge of the 4*4 block b and the right edge pixel of the 4*4 block e, the predicted value of the f pixel of the 4*4 block can be further calculated, and after adding the difference value, 4 *4 Reconstructed value of pixel in block f. As shown in Figure 9(e), the prediction mode of the 4*4 block f is the horizontal right mode, so the predicted value of the right edge pixel of the 4*4 block e can be used to calculate the pixel value of the 4*4 block f Predictive value. When the prediction of the 4*4 block f is performed, the reconstruction values Re1˜Re16 of the pixels of the 4*4 block e can also be calculated at the same time. Finally, as shown in FIG. 9( f ), after obtaining the predicted value of the 4*4 block f and adding the residual value, the reconstructed values Rf1˜Rf16 of the 4*4 block f can be obtained.

In this embodiment, the prediction mode of each 4*4 block in the 16*16 block is determined in one operation, so that the calculation of 9 prediction modes for the 16 4*4 blocks is simplified to only need to Nine prediction modes are calculated for one 16*16 block, which also means that this embodiment can effectively reduce the processing time for intra-frame prediction of 4*4 blocks, thereby achieving the purpose of increasing image processing efficiency.

second embodiment

FIG. 10 is a flowchart of an image processing method according to a second embodiment of the present invention. Please refer to FIG. 10 , this embodiment is applicable to an image that can be divided into multiple 4*4 blocks. First, these 4*4 blocks are divided into multiple (for example, 4) areas, each of which has at least two 4*4 blocks (step S1010), and only four 4*4 blocks are included below The area (that is, 8*8 blocks) is taken as an example. That is to say, a 16*16 block includes four 8*8 blocks, each 8*8 block includes four 4*4 blocks, and each 4*4 block includes 16 pixels. Among them, in this embodiment, the prediction formula of the original 4*4 block is expanded and applied to the 16*16 block, and the prediction mode of each 8*8 block in the 16*16 block is found in one operation (Pass). , which can increase the efficiency of image processing.

In this embodiment, similar to the method of the first embodiment, first calculate the sum of squares of the differences between the pixels of each 4*4 block in each 8*8 block and the corresponding image boundary pixels in multiple prediction modes (step S1020 ). Wherein, the calculation formula of the difference sum of squares is similar to that of the aforementioned first embodiment, so the same method will not be repeated here. The only difference is that this embodiment uses 8*8 block As a unit, the difference between the mode prediction value and the original value of each 8*8 block pixel is squared and added to obtain the sum of the squares of the differences under 9 different prediction modes as shown in FIG. 11 .

According to the sum of squared differences calculated by these prediction modes, the most suitable prediction mode for each 8*8 block can be found (step S1030 ). Specifically, in this embodiment, for each 8*8 block, the minimum value is selected from the 9 difference sums of squares calculated by using the above 9 prediction modes, and the prediction mode with the minimum value is selected to make The most suitable prediction mode for this 8*8 block. For example, FIG. 12 is a schematic diagram of image processing according to the second embodiment of the present invention. As shown in FIG. 12( a ), the prediction mode selected by block I is the fourth mode, the prediction mode selected by block II is the second mode, and so on. The above steps are all completed in one operation (Pass), so the time spent in finding the prediction mode block by block can be saved.

Likewise, in this embodiment, the factor of the bit transmission rate can also be taken into consideration. Among them, the bit transmission rate can be obtained by calculating the residual value of each 8*8 block pixel in each prediction mode (as shown in Figure 12(b)), and after discrete cosine transform, quantization, and entropy operation, each 8*8 block pixels correspond to bit transfer rates in each prediction mode. Then, the bit rate-distortion optimal value (RDO) can be obtained by adding the bit rate to the original sum of squared differences. Accordingly, the prediction mode corresponding to the minimum value of RDO is found out as the prediction mode used for pixel reconstruction of each 8*8 block. The prediction model obtained here is more accurate than the previous prediction model obtained by using the difference to the sum of squares because of the factor of the bit rate.

After the prediction modes of each 8*8 blocks are determined, these determined prediction modes can be used to reconstruct the pixels of these 8*8 blocks based on the image boundary pixels (step S1040), and when all 8*8 blocks After all the reconstruction values of the pixels in the are calculated, the reconstruction step of the pixels in the 16*16 block is completed. Wherein, the present embodiment divides a 16*16 block into four 8*8 blocks when performing pixel reconstruction, and these 8*8 blocks are numbered as I and II from top to bottom and from left to right , III, and IV (as shown in FIG. 12( a )), and the reconstruction order of these 8*8 blocks is: I→II, III→IV.

In one embodiment, in order to speed up the above-mentioned steps of pixel reconstruction, this embodiment also includes that when reconstructing an 8*8 block, first calculate the edge pixels of the 8*8 block by using a prediction formula, and further provide the Its adjacent 8*8 blocks are used for prediction. By using this method, one 8*8 block can be reconstructed, and the pixels of the next adjacent 8*8 block can be reconstructed, which greatly reduces the number of pixels. The time required to rebuild. Wherein, the part performing prediction and reconstruction at the same time is the same or similar to that described in the first embodiment, so it will not be repeated here.

In this embodiment, the prediction mode of each 8*8 block in the 16*16 block is determined in one operation, so that the calculation of 9 prediction modes for the 4 8*8 blocks is simplified to only need to Nine prediction modes are calculated for one 16*16 block, which also means that this embodiment can effectively reduce the processing time for intra-frame prediction of 8*8 blocks, thereby achieving the purpose of increasing image processing efficiency.

third embodiment

FIG. 13 is a flowchart of an image processing method according to a third embodiment of the present invention. Please refer to FIG. 13 , this embodiment is applicable to an image that can be divided into multiple 4*4 blocks. First, the 4*4 blocks are divided into multiple (for example, 4) regions, each of which has at least two 4*4 blocks (step S1310 ). The following only takes an area including four 4*4 blocks (ie, 8*8 blocks) as an example. That is to say, a 16*16 block includes four 8*8 blocks such as upper left, upper right, lower left, and lower right, each 8*8 block includes four 4*4 blocks, and each 4*4 area A block consists of 16 pixels. Among them, in this embodiment, the prediction formula of the original 4*4 block is expanded and applied to the 16*16 block, and the prediction mode of each 8*8 block in the 16*16 block is found by using three operations (Pass). , which can increase the efficiency of image processing.

In this embodiment, similar to the method of the first embodiment, the nine prediction modes of the original intra-frame prediction algorithm are extended to 16*16 blocks. However, different from the foregoing embodiments, this embodiment only first calculates four 4*4 regions in the 8*8 block for one of these 8*8 blocks (for example, the 8*8 block in the upper left corner) The sum of squares of the difference between the block pixel and the corresponding image boundary pixel in multiple prediction modes (step S1312 ), the obtained sum of squares of the difference in 9 different prediction modes is shown in FIG. 14 . The method for calculating the sum of squared differences is the same as or similar to that of the foregoing embodiment, so it will not be repeated here.

According to the sum of squared differences calculated by these prediction modes, the most suitable prediction mode for each 4*4 block in the upper left 8*8 blocks can be found (step S1314 ). Specifically, in this embodiment, for each 4*4 block, the minimum value is selected from the 9 difference sums of squares calculated by using the above 9 prediction modes, and the prediction mode with the minimum value is selected to make The most suitable prediction mode for this 4*4 block. For example, FIG. 15 is a schematic diagram of an image processing method according to a third embodiment of the present invention. As shown in FIG. 15( a ), the prediction mode selected by block a is the second mode, the prediction mode selected by block b is the seventh mode, and so on.

After the prediction mode of each 4*4 block is determined, these determined prediction modes can be used to reconstruct the pixels of each 4*4 block in the upper left 8*8 block based on the image boundary pixels, and when all 4*4 After the reconstruction values of the block pixels are all calculated, the reconstruction step of the upper left 8*8 block pixels is completed (step S1316 ). The above-mentioned steps can be completed within one operation (pass), so the time and computing resources consumed by sequentially calculating the prediction mode and reconstruction value of the 4*4 blocks can be saved.

Please refer to FIG. 15(b), in which the reconstruction values of the 4*4 blocks a, b, c, and d have been calculated, and the reconstruction value matrices a', b', c', and d' are obtained. In one embodiment, the above step of calculating the reconstruction value can be further divided into two sub-steps, including firstly calculating the prediction value of the pixels in the 4*4 block by using the prediction formula of the prediction mode corresponding to each 4*4 block. These predicted values are then added to the residual values to obtain reconstructed values.

After obtaining the reconstructed values of the pixels in the upper left 8*8 block, it means that the pixels in the left block of the upper right 8*8 block and the pixels in the upper block of the lower left block are known. Therefore, the next operation of this embodiment is to use the formulas of the above nine prediction modes to calculate the distance between each 4*4 block pixel in the upper right and lower left 8*8 blocks and the corresponding area boundary in multiple prediction modes. Reconstruct the sum of squared differences between pixels (step S1318), and determine the prediction mode used when reconstructing the upper right and lower left 8*8 block pixels according to the calculated sum of squared differences in the prediction mode (step S1320). Finally, the upper right and lower left 8*8 block pixels are reconstructed based on the determined prediction mode and based on the boundary pixels of the image (step S1322 ). Please refer to FIG. 15(c), where the difference sum of squares and prediction mode of 4*4 blocks c, d, g, h and 4*4 blocks i, j, m, n have been determined, and FIG. 15( d) shows that the reconstruction values of the original 4*4 blocks g, h and 4*4 blocks j, n have been calculated, and the reconstruction value matrices g', h', j', and n' are obtained.

It is worth mentioning that after obtaining the predicted values of the pixels in the upper right 8*8 block and the lower left 8*8 block by the prediction formula, it represents the upper block and the left block of the lower right 8*8 block. pixels are known. Therefore, the next operation (Pass) of this embodiment is to use the formulas of the above nine prediction modes to calculate the distance between each 4*4 block pixel in the lower right 8*8 block and the corresponding area boundary in multiple prediction modes. The sum of squared differences between the reconstructed pixels (step S1324), and the sum of squared differences calculated according to the prediction mode determines the prediction mode used for reconstruction of the lower right 8*8 block pixels (step S1326) (as shown in Figure 15 (e)). Finally, reconstruct the lower right 8*8 block pixels based on the determined prediction mode and based on the boundary pixels of the image (step S1328 ). So far, the reconstructed values of the pixels of each 8*8 block in the 16*16 block (as shown in FIG. 15( f )) have been obtained, and the reconstruction of the pixels of the entire 16*16 block is completed.

In this embodiment, the prediction mode of each 8*8 block in the 16*16 block is determined in three operations (Pass), so that the calculation of 9 prediction modes for the 16 4*4 blocks is simplified to only Nine prediction modes need to be calculated for four 8*8 blocks, which also means that this embodiment can effectively reduce the processing time for intra-frame prediction of 4*4 blocks, thereby achieving the purpose of increasing image processing efficiency.

In summary, the image processing method of the present invention has at least the following advantages:

1. The prediction mode of each 4*4 block or 8*8 block in the 16*16 block is determined in one operation, and there is no need to predict each 4*4 block or 8*8 block one by one, so it can Save computing time and increase image processing efficiency.

2. The prediction modes of the upper left 8*8 block, the upper right and lower left 8*8 blocks, and the lower right 8*8 block of the 16*16 blocks are determined in three operations, instead of 4*4 blocks one by one Prediction is done in blocks, so it can save computing time and increase image processing efficiency.

3. After the prediction mode of a block is determined, the subsequent steps of discrete cosine transformation, quantization, and entropy calculation can be performed on the block while predicting other blocks, without waiting until all blocks are predicted Therefore, it can save computing time and increase image processing efficiency.

Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, this The scope of protection of the invention should be defined by the appended claims.

Claims (20)

1. A method of image processing, applicable to an image that can be distinguished into a plurality of prediction blocks, the method comprising: a. Calculate the sum of squares of the difference between the prediction block pixels and the corresponding boundary pixels of the image in multiple prediction modes; b. According to the result of step a, determine the prediction mode used for pixel reconstruction of the prediction block; as well as c. Reconstruct the pixels of the predicted blocks according to the result of step b. 2. image processing method according to claim 1, wherein after a step also comprises: a1. Using the results of step a, calculate the residual values of the prediction block pixels in the prediction modes; and a2. According to the results of step a and step a1, determine the prediction mode used for pixel reconstruction of the prediction block. 3. The image processing method according to claim 1, wherein the image is a block composed of 16*16 pixels, and the prediction block is a 4*4 block. 4. The image processing method according to claim 1, wherein the prediction modes include: vertical prediction mode, horizontal prediction mode, DC prediction mode, diagonal lower left prediction mode, diagonal lower right prediction mode, vertical rightward prediction mode, horizontal down prediction mode, vertical left prediction mode, and horizontal up prediction mode. 5. image processing method according to claim 2, wherein a2 step also comprises: a2-1. According to the result of step a1, determine the bit transmission rate corresponding to the prediction block pixels in the prediction modes; a2-2. According to the results of step a and step a2-1, calculate the optimal bit rate-distortion value of the predicted block pixels; and a2-3. According to the result of step a2-2, determine the prediction mode used for pixel reconstruction of the prediction blocks. 6. The image processing method according to claim 3, wherein the 16 predictive blocks are preset from top to bottom and from left to right to be divided into those predictive blocks of positions a to p, then in step c The reconstruction sequence of the predicted blocks is: a→b, e→c, f, i→d, g, j, m→h, k, n→l, o→p. 7. A method of image processing, applicable to an image that can be distinguished into a plurality of prediction blocks, the method comprising: a. dividing the predicted blocks into a plurality of regions, each of which has at least two predicted blocks; b. For one of the regions, calculate the sum of the squares of the differences between the prediction block pixels in the region and the corresponding boundary pixels of the image in multiple prediction modes; c. According to the result of step b, determine the prediction mode used when reconstructing the pixels of the prediction block in the region; d. Reconstruct the pixels of the prediction blocks in the area according to the result of step c; and e. According to the result of step d, using the reconstructed pixels on the boundary of the region, reconstruct the pixels of the prediction blocks in the region adjacent to the region. 8. The image processing method according to claim 7, further comprising after the b step: b1. Using the results of step b, calculate the residual values of the prediction block pixels in the region under the prediction modes; and b2. According to the result of step b and step b1, determine the prediction mode used when reconstructing the pixels of the prediction block in the region. 9. The image processing method according to claim 7, wherein the image is a block composed of 16*16 pixels, the prediction block is a 4*4 block, and the area is an 8*8 block. 10. The image processing method according to claim 7, wherein the prediction modes include: vertical prediction mode, horizontal prediction mode, DC prediction mode, diagonal lower left prediction mode, diagonal lower right prediction mode, vertical rightward prediction mode, horizontal down prediction mode, vertical left prediction mode, and horizontal up prediction mode. 11. image processing method according to claim 8, wherein b2 step also comprises: b2-1. According to the result of step b1, determine the bit transmission rate corresponding to the prediction block pixels in the region under the prediction modes; b2-2. According to the results of step b and step b2-1, calculate the optimal bit rate-distortion value of the predicted block pixels in the area; and b2-3. According to the result of step b2-2, determine the prediction mode used when reconstructing the pixels of the prediction blocks in the region. 12. The image processing method according to claim 9, wherein the four prediction blocks are preset to be sequentially divided into prediction blocks at positions a, b, e, and f from top to bottom and from left to right, Then the reconstruction sequence of the prediction blocks in step e is: a→b, e→f. 13. The image processing method according to claim 7, wherein the e step further comprises: e1. Calculating the sum of squared differences between the pixels of the predicted block in at least one adjacent region and the reconstructed pixels corresponding to the boundary of the region in multiple prediction modes; e2. Using the result of step e1, calculate the residual values of the pixels of the prediction block in the adjacent area under the prediction modes; e3. According to the results of step e1 and step e2, determine the prediction mode used for pixel reconstruction of the prediction block in the adjacent area; and e4. According to the result of step e3, reconstruct the pixels of the prediction blocks in the adjacent area. 14. The image processing method according to claim 7, further comprising: f1. Calculating the sum of the squares of the differences between the predicted block pixels in the region and the reconstructed pixels corresponding to the boundaries of the two adjacent regions under multiple prediction modes; f2. Using the results of the f1 step, calculate the residual values of the prediction block pixels in the region under the prediction modes; f3. According to the results of step f1 and step f2, determine the prediction mode used for pixel reconstruction of the prediction block in the region; and f4. According to the result of step f3, reconstruct the pixels of the prediction blocks in the area. 15. A method of image processing, applicable to an image distinguishable into a plurality of prediction blocks, the method comprising: a. dividing the predicted blocks into a plurality of regions, each of which has at least two predicted blocks; b. Calculate the sum of squares of the difference between the prediction block pixels in each of the regions and the corresponding boundary pixels of the image in multiple prediction modes; c. According to the result of step b, determine the prediction mode used when the pixels of the prediction block in each region are reconstructed; and d. Reconstruct the pixels of the prediction blocks in each region according to the result of step c. 16. The image processing method according to claim 15, further comprising after the b step: b1. Using the results of step b, calculate the residual values of the prediction block pixels in each of the regions under the prediction modes; and b2. According to the result of step b and step b1, determine the prediction mode used for pixel reconstruction of the prediction block in each region. 17. The image processing method according to claim 15, wherein the image is a block composed of 16*16 pixels, the prediction block is a 4*4 block, and the area is an 8*8 block. 18. The image processing method according to claim 15, wherein the prediction modes include: vertical prediction mode, horizontal prediction mode, DC prediction mode, diagonal lower left prediction mode, diagonal lower right prediction mode, vertical rightward prediction mode, horizontal down prediction mode, vertical left prediction mode, and horizontal up prediction mode. 19. The image processing method according to claim 16, wherein the b2 step further comprises: b2-1. According to the result of step b-1, determine the bit transmission rate corresponding to the pixels of the prediction block in each of the prediction modes in the region; b2-2. According to the results of step b and step b2-1, calculate the optimal bit rate-distortion value of the prediction block pixels in each region; and b2-3. According to the result of step b2-2, determine the prediction mode used when reconstructing the pixels of the prediction blocks in each region. 20. The image processing method according to claim 17, wherein the four regions are preset to be divided into the four regions of I, II, III and IV positions sequentially from top to bottom and from left to right, then in step e The reconstruction order of the predicted blocks is: I→II, III→IV.

Images