CN1700775A - A Method and Apparatus for Determining the Edge Trend of a Pixel of Interest - Google Patents

Abstract

A method and device for determining the edge trend of a pixel of interest are applied to a color filter image array interpolation system. The method first determines whether the surrounding pixels of the pixel of interest have a horizontal trend or a vertical trend, then counts the number of pixels with horizontal trends and the number of pixels with vertical trends among the surrounding pixels, and then determines the edge trend of the pixel of interest based on the trend of the pixel with the larger number.

Description

A Method and Apparatus for Determining the Edge Trend of a Pixel of Interest

technical field

The present invention relates to an image array interpolation method, in particular to a method for determining an edge trend (edge trend) of a pixel of interest in a color filter array interpolation system (Color Filter Array Interpolation; CFAI) with the device.

technical background

Color images can be obtained using a single sensor array. This sensor senses only a single color at each pixel location in its array to produce a color value, but the color values of the other colors that are missing at each pixel location are generated by interpolation of the color values at its neighboring pixel locations, A full-color reconstructed image is thus obtained. A sensor with a specific pattern of pixel positions corresponding to each color value facilitates and efficiently interpolates such additional color values. For example, in the color image array of US Pat. No. 3,971,065 issued to Bayer, the color that dominates the entire image is sensed by staggered pixel positions, and the other pixel positions sense the other two colors respectively. Such techniques are used in today's single-sensor cameras, where the sensor is overlaid with an array of color filters in a specific pattern so that each pixel position on the sensor produces only the corresponding red, colored or blue color. A signal of a single color value. In other words, although the original image plane generated by the sensor contains three colors of red, green and blue, each pixel position has only a single color value. To reconstruct a full-color image, the conventional CFAI algorithm transforms the original image plane so that each pixel position contains an image plane with three color information of red, green and blue. Since the lost color value of the pixel position of interest is calculated with reference to the color values of adjacent pixel positions, the CFAI method used will determine the quality of the final reconstructed full-color image.

The simplest and direct interpolation method is a bilinear interpolation method, which averages the color values of adjacent pixel positions of the pixel position of interest to obtain the color value of the pixel position of interest. However, this non-edge based CFAI method produces artifacts at the edges of the color image, thus losing resolution. Another edge-based CFA1 method can reduce the artifact effect of the edge of the color image, thereby improving the sharpness of the image (Sharpness). The edge-based CFAI method first judges whether the edge trend of the image at the pixel position of interest is a horizontal trend or a vertical trend, and then decides to use the color values of adjacent pixel positions in the horizontal or vertical direction for interpolation operations. Therefore, correctly Determining whether the image is trending horizontally or vertically at the pixel of interest will determine whether the interpolated color values produce better image quality. The edge tendency at a pixel location of interest can be determined by the color value gradient at that location. For example, a 6×6 pixel array 10 is shown in FIG. 1, wherein R, G and B respectively represent the color values of three colors of red, green and blue, and the subscripts represent the coordinates of the pixel position. The color value representation of is generated by interpolation. For example, there is only blue color value B ₄₃ at the pixel (4, 3) position, and its green color value G ₄₃ is generated by interpolation from the color values of adjacent pixel positions. . In a conventional CFAI method, when G ₄₃ is to be interpolated, the horizontal and vertical gradients at the position of pixel (4, 3) are first calculated

ΔH=|G ₄₂ -G ₄₄ |+|B ₄₃ -B ₄₁ +B ₄₃ -B ₄₅ |+|G ₃₃ -G ₃₂ +G ₃₃ -G ₃₄ |, Formula 1 and

ΔV=|G ₃₃ -G ₅₃ |+|B ₄₃ -B ₂₃ +B ₄₃ -B ₆₃ |+|G ₃₂ -G ₄₂ -G ₃₄ -G ₄₄ |, formula 2

If ΔH<ΔV, it is considered that the edge at the position of pixel (4,3) tends to be along the horizontal direction, so G ₄₃ is interpolated with the color values of adjacent pixel positions in the horizontal direction, for example:

G ₄₃ ＝(G ₄₂ +G ₄₄ )÷2 Formula 3

If ΔH>ΔV, it is considered that the edge trend at the pixel (4,3) position is along the vertical direction, so G ₄₃ is interpolated with the color values of adjacent pixel positions in the vertical direction, for example:

G ₄₃ ＝(G ₃₃ +G ₅₃ )÷2 Formula 4

This method is simple and effective when the edge trend is very obvious. However, when the values of the horizontal gradient ΔH and vertical gradient ΔV are similar, it is easy to make wrong judgments, especially when the noise is large, it will lead to misjudgment of the direction of the trend . Another disadvantage of this method is that crosses are often generated on images, for example, crosses are generated on nose images of human faces.

Accordingly, an improved method and apparatus for determining the edge trend of a pixel of interest in a CFAI system is desired.

Contents of the invention

One of the objectives of the present invention is to provide a method and device for determining the edge trend of a pixel of interest in the CFAI system, which determines the pixel of interest by the horizontal and vertical trends of the surrounding pixels of the pixel of interest marginal trend.

Another object of the present invention is to provide a method and device for determining the edge trend of a pixel of interest in the CFAI system, which can still correctly determine when the values of the horizontal gradient and the vertical gradient at the pixel of interest are close The edge trend of the pixel of interest.

According to the present invention, a method for determining the edge trend of a pixel of interest in the CFAI system is to first determine whether each surrounding pixel at the pixel of interest is a horizontal or vertical trend, which includes calculating each of the surrounding pixels in two pairs The difference between the color values in the angular direction, and then the product of the two differences is positive or negative to determine whether the surrounding pixels are horizontal or vertical, and count the number of surrounding pixels with a horizontal trend and the number of pixels with a vertical trend The number of surrounding pixels, when the number of surrounding pixels with a horizontal trend is greater than the number of surrounding pixels with a vertical trend, the edge trend of the pixel of interest is determined to be along the horizontal direction, otherwise, the edge of the pixel of interest is determined The trend is along the vertical direction.

The device for performing the above method includes a processor and a buffer, the original color value of a pixel array centered on the pixel of interest is stored in the buffer, and the processor reads the values of the surrounding pixels from the buffer respectively. The color values in the two diagonal directions are calculated and the product of the difference is stored in the buffer, and then the number of positive value products and negative value products in these products is counted, and the size of the two numbers is compared to judge the sense. The edge trend of the pixel of interest is along the horizontal or vertical direction.

Through the method and device provided by the present invention, even if the horizontal gradient and the vertical gradient at a pixel are equal or close, the present invention can correctly judge the edge trend at the pixel.

Description of drawings

Figure 1 shows a 6×6 pixel array;

Fig. 2 shows the green color value of a 5x5 pixel array centered on the pixel (3,3) which is desired to be interpolated to produce the green color value G ₃₃ ;

Figure 3A shows the green color values of a 3x3 array centered on pixel (2,3) in Figure 2;

Figure 3B shows the green color values of a 3x3 array centered on pixel (3,2) in Figure 2;

FIG. 3C shows the green color value of a 3×3 array centered on pixel (3,4) in FIG. 2;

Figure 3D shows the green color values of a 3x3 array centered on pixel (4,3) in Figure 2;

Fig. 4 is a device for performing interpolation G ₃₃ ;

Fig. 5 is a flowchart for performing the interpolation _G33 .

Description of figure number

10 6×6 pixel array

20 5×5 pixel array

22 A 3×3 array centered on pixel (2, 3)

24 A 3×3 array centered at pixel (3, 2)

26 3×3 array centered on pixel (3,4)

28 3×3 array centered on pixel (4, 3)

30 Means for performing interpolation G ₃₃

32 processors

34 buffer

34a section of buffer 34

34b section of buffer 34

34c Section of buffer 34

34d Section of buffer 34

34e Section of buffer 34

34f Section of buffer 34

40 Calculate the horizontal gradient ΔH and vertical gradient ΔV of the pixel of interest (3, 3)

42 Judging whether the absolute value of the difference between the horizontal gradient ΔH and the vertical gradient ΔV is less than the critical value TH

44 Determine whether the horizontal gradient ΔH is greater than the vertical gradient ΔV

46 Determine the trend of the surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ of the pixel of interest (3, 3)

48 Count the number of horizontal and vertical trends of surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃

50 Determine whether the number with a horizontal trend is greater than the number with a vertical trend

52 Pixels of interest (3, 3) are horizontal trends

54 Determine whether the number with a horizontal trend is smaller than the number with a vertical trend

56 pixels of interest (3, 3) are vertical trends

58 Use bilinear interpolation to find the green color value G that you want to interpolate to the pixel of interest (3, 3) ₃₃

Detailed ways

The method of the present invention is to first judge that the surrounding pixels at a pixel of interest are of horizontal trend or vertical trend, count the number of those with horizontal trend and the number of those with vertical trend in these surrounding pixels, compare the number of the two and obtain Determine whether the edge trend of the pixel of interest is along the horizontal direction or the vertical direction. To clearly illustrate this method, FIG. 2 provides a 5×5 pixel array 20, the green color value G ₃₃ at the center pixel (3, 3) position is intended to be generated by interpolation, and other green color values in the array 20 G _xy is the original color value directly generated by the image sensor, and is used to judge the edge tendency of the pixel of interest (3, 3). First, it is judged that the surrounding pixels (2, 3), (3, 2), (3, 4) and (4, 3) of the pixel of interest (3, 3) are in a horizontal or vertical trend. 3A, 3B, 3C, and 3D, each of which is centered on the relevant color values from the array 20 for easy understanding, and Fig. 3A shows a 3×3 The green color value of the array, Figure 3B shows the green color value of the 3×3 array centered on the pixel (3,2), and Figure 3C shows the green color value of the 3×3 array centered on the pixel (3,4), Figure 3D shows a 3x3 array of green color values centered at pixel (4,3).

In Fig. 3A, when judging the trend of pixel (2, 3), first calculate the difference between the green color values G ₁₂ and G ₃₄ of pixels (1, 2) and (3, 4) in the diagonal direction

Δ1=G ₁₂ -G ₃₄ , Formula 5

and the difference between the green color values G ₁₄ and G ₃₂ of pixels (1, 4) and (3, 2) in the other diagonal direction

Δ2=G ₁₄ -G ₃₂ , formula 6

Then multiply the two interpolation values Δ ₁ and Δ ₂ to get

Buffer ₁ = (G ₁₂ -G ₃₄ )×(G ₁₄ -G ₃₂ ), Formula 7

When Buffer ₁ > 0, the pixel (2, 3) has a horizontal trend;

When Buffer ₁ <0, pixel (2, 3) has a vertical trend.

In other words, the sign of the product of the color value gradient values in the two diagonal directions of the pixel (2, 3) is used to judge the trend of the pixel (2, 3).

In Fig. 3B, as mentioned above, when judging the trend of pixel (3, 2), first calculate the difference between the green color values G ₂₁ and G ₄₃ and the difference between G ₂₃ and G ₄₁ in the two diagonal directions , and then find its product

Buffer ₂ = (G ₂₁ -G ₄₃ )×(G ₂₃ -G ₄₁ ), formula 8

When Buffer ₂ > 0, the pixel (3, 2) has a horizontal trend;

When Buffer ₂ <0, pixel (3, 2) has a vertical trend.

Similarly, from Figure 3C and Figure 3D respectively

Buffer ₃ = (G ₂₃ -G ₄₅ )×(G ₂₅ -G ₄₃ ), Formula 9

When Buffer ₃ > 0, the pixel (3, 4) has a horizontal trend;

When Buffer ₃ <0, pixel (3, 4) has a vertical trend.

as well as

Buffer ₄ = (G ₃₂ -G ₅₄ )×(G ₃₄ -G ₅₂ ), Formula 10

When Buffer ₄ > 0, the pixel (4, 3) has a horizontal trend;

When Buffer ₄ <0, pixel (4, 3) has a vertical trend.

After obtaining the horizontal or vertical trend of the surrounding pixels (2, 3), (3, 2), (3, 4) and (4, 3) of the pixel of interest (3, 3), follow the statistics of these 4 surrounding pixels The number of those with a horizontal trend and the number of those with a vertical trend, if the number of surrounding pixels S _I with a horizontal trend is greater than the number S _V of surrounding pixels with a vertical trend, then the edge of the pixel of interest (3, 3) is considered The trend is along the horizontal direction, on the contrary, if the number S _I of the surrounding pixels with the horizontal trend is less than the number S _V of the surrounding pixels with the vertical trend, it is considered that the edge trend of the pixel of interest (3, 3) is along the vertical direction . If S _I and S _V are equal, bilinear interpolation is used, and the average color value of pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ is used as the green color value of pixel (3, 3).

FIG. 4 is an apparatus 30 for performing interpolation G ₃₃ , which includes a processor 32 and a buffer 34 . The original color value of the array 20 centered on the pixel (3, 3) is first stored in the buffer 34, as shown in section 34a, and then the above-mentioned judgment method is performed, and the processor 32 reads the color value in the section 34a The original color values of the array 20 are subjected to operations such as difference and product and counting. In order to reduce the operation of the device 30, in one embodiment, the buffer 34 further includes a threshold value TH, which is predetermined by the input SET and stored in the section 34b of the buffer 34, and the processor 32 from the section of the buffer 34 34a reads the original color value of the array 20, calculates the horizontal gradient ΔH and the vertical gradient ΔV of the color value of the pixel (3, 3) and stores them in the section 34c of the buffer 34, and calculates the difference |ΔH- ΔV| is compared with the critical value TH, if |ΔH-ΔV|≥TH, then judge the edge trend of the pixel of interest (3, 3) according to the known method, and determine the pixel (3, 3) by the relative size of ΔH and ΔV The edge trend of , only when |ΔH-ΔV|<TH, the operation of formulas 7 to 10 will be performed. When |ΔH-ΔV|<TH, the processor 32 reads the color value from section 34a and generates Buffer ₁ to Buffer ₄ according to formulas 7 to 10, and stores them in section 34d of buffer 34, and then counts Buffer ₁ The number S _I of pixels with a horizontal trend and the number S _V of pixels with a vertical trend in Buffer ₄ are stored in the section 34e of the buffer 34, and the pixel of interest is determined after comparing the sizes of S _I and S _V (3, 3 ), then calculate G ₃₃ according to the interpolation formula in section 34f, for example:

When S _I >S _V , G ₃₃ =(G ₃₂ +G ₃₄ )÷2; Formula 11

When S _I <S _V , G ₃₃ =(G ₂₃ +G ₄₃ )÷2; Formula 12

When S _I =S _V , G ₃₃ =(G ₂₃ +G ₃₂ +G ₃₄ +G ₄₃ )÷4. Formula 13

The interpolation formulas in section 34f can be determined by input F, and these interpolation formulas can use known techniques or modifications thereof. In a different embodiment, the processor 32 does not calculate ΔH and ΔV and compare the difference with the threshold value TH, but directly uses the formulas 7 to 10 to calculate Buffer ₁ to Buffer ₄ , completely from the surrounding of the pixel (3, 3) The horizontal and vertical trends of pixels (2,3), (3,2), (3,4) and (4,3) determine the edge trend of pixel (3,3).

A flow process for performing the above operations is shown in FIG. 5. When determining the edge tendency of the pixel (3, 3) in FIG. Horizontal gradient ΔH and vertical gradient ΔV, for example, use formulas 1 and 2, or use simpler formulas, for example, refer to Figure 2:

ΔH=|G ₃₄ −G ₃₂ |, Formula 14

and

ΔV=|G ₄₃ −G ₂₃ |, Formula 15

Then in step 42 it is judged whether the difference |ΔH-ΔV| of the two is less than the critical value TH, if not, proceed to step 44, further judge the relative size of the horizontal gradient ΔH and the vertical gradient ΔV, if ΔH<ΔV, then consider It is a horizontal trend at the pixel (3, 3), and in step 52, the corresponding formula is used to interpolate G ₃₃ , such as formula 11. If ΔH>ΔV, then it is considered to be a vertical trend at the pixel (3, 3). In step 56, use the corresponding formula to interpolate G ₃₃ , such as formula 12. If the result obtained in step 42 is |ΔH-ΔV| The trend of the surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ , followed by counting the numbers S _I and S _V of those with horizontal and vertical trends in the surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ in step 48, Next, steps 50 and 54 are to compare the size of S _I and S _V , when the number with horizontal trend is greater than the number with vertical trend, it means that it is a horizontal trend at the pixel (3,3) place, to step 52 to adopt the corresponding The formula interpolates G ₃₃ , such as formula 11. On the contrary, when the number with the horizontal trend is less than the number with the vertical trend, it means that the pixel (3, 3) is a vertical trend, and the corresponding formula is used in step 56 for interpolation G ₃₃ , such as formula 12, if the number S _I and S _V of those with horizontal and vertical trends in the surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ are equal, then proceed to step 58, such as formula 13, with bilinear Interpolation calculates the green color value _G33 for pixel (3,3). In a different embodiment, steps 40 to 44 are not performed, but directly from step 46, completely by the horizontal and vertical trends in the surrounding pixels G ₂₃ , G ₃₂ , G ₃₄ and G ₄₃ of pixel (3,3) The number of them S _I and S _V determines whether the trend is horizontal or vertical at pixel (3,3).

Through the methods and devices exemplified above, even if the horizontal gradient and the vertical gradient at a pixel are equal or close, the present invention can correctly determine the edge tendency at the pixel.

The above descriptions of the preferred embodiments of the present invention are for the purpose of illustration, and are not intended to limit the precise disclosed form of the present invention. It is possible to modify or change based on the above teachings or learning from the embodiments of the present invention. The embodiment is to explain the principle of the present invention and to allow those familiar with the art to use various embodiments to select and describe the present invention in practical applications. Decide.

Claims (11)

1. A method for determining the edge tendency of a pixel of interest in a color filter image array interpolation system, characterized in that it comprises the following steps: Determine the edge trend of the surrounding pixels of the pixel of interest; Count the number of those with different edge tendencies among the surrounding pixels; and The number of different edge trends is compared, and the trend of the one with the larger number determines the edge trend of the pixel of interest. 2. The method for determining an edge tendency of a pixel of interest in a color filter image array interpolation system as claimed in claim 1, wherein the edge tendency of the pixels surrounding the pixel of interest is determined to include horizontal Trends and Vertical Trends. 3. The method for determining the edge trend of a pixel of interest in a color filter image array interpolation system as claimed in claim 1, wherein the step of determining the edge trend of surrounding pixels comprises the following steps: calculating the difference of the color value of each surrounding pixel position in the first direction; calculating the difference in color value in the second direction for each of the surrounding pixel positions; and The edge trend of each surrounding pixel is determined according to whether the product of the two difference values is positive or negative. 4. The method for determining an edge tendency of a pixel of interest in a color filter image array interpolation system as claimed in claim 3, wherein the first direction and the second direction are orthogonal to each other. 5. The method for determining an edge tendency of a pixel of interest in a color filter image array interpolation system as claimed in claim 4, wherein the first direction and the second direction are in a diagonal direction . 6. A device for determining the edge trend of a pixel of interest in a color filter image array interpolation system, characterized in that it comprises: a buffer for storing the color values of the pixel array including the pixel of interest; and A processor, using the color value calculation of the pixel array to determine the edge trend of the surrounding pixels of the pixel of interest, and counting the number of pixels with different edge trends in the surrounding pixels to determine the edge trend of the pixel of interest; Wherein, the processor calculates the difference between the color values of each surrounding pixel position in the first direction and the second direction, and then determines the edge of each surrounding pixel according to whether the product of the two differences is positive or negative. trend. 7. A method for determining the edge tendency of a pixel of interest in a color filter image array interpolation system comprising the steps of: storing the color value of the pixel array including the pixel of interest; calculating gradient values in different directions at the pixel of interest by using the color values of the pixel array; Calculate the difference between the color gradient values in the different directions; Comparing the difference with a critical value, when the difference is smaller than the critical value, performing subsequent steps; Determine the edge trend of the surrounding pixels of the pixel of interest; Count the number of those with different edge tendencies among the surrounding pixels; and Comparing the number, the trend of the one with the larger number determines the edge trend of the pixel of interest. 8. The method for determining the edge trend of a pixel of interest in a color filter image array interpolation system as claimed in claim 7, wherein the step of determining the edge trend of surrounding pixels comprises the following steps: calculating the difference of the color value of each surrounding pixel position in the first direction; calculating the difference in color value in the second direction for each of the surrounding pixel positions; and The edge trend of each surrounding pixel is determined according to whether the product of the two difference values is positive or negative. 9. The method for determining the edge tendency of a pixel of interest in the color filter image array interpolation system as claimed in claim 7, wherein the gradient values in different directions of the surrounding pixels of the pixel of interest are judged to be Including at least a gradient value in the horizontal direction and a gradient value in the vertical direction. 10. The method for determining the edge trend of a pixel of interest in a color filter image array interpolation system as claimed in claim 8, wherein the first direction and the second direction are in a diagonal direction . 11. A device for determining the edge trend of a pixel of interest in a color filter image array interpolation system, characterized in that it comprises: a buffer for storing the color values of the pixel array including the pixel of interest; and A processor, using the color value of the pixel array to calculate the difference between the gradient values in different directions at the pixel of interest and compare it with a critical value, and when the difference is smaller than the critical value, determine the surrounding pixels of the pixel of interest edge trend, and count the number of people with different edge trends in the surrounding pixels to determine the edge trend of the pixel of interest; Wherein, the processor calculates the difference between the color values of each surrounding pixel position in the first direction and the second direction, and then determines the edge of each surrounding pixel according to whether the product of the two differences is positive or negative. trend.

Images