TW201738841A - Method of dynamic adjustment to automatic white balance under hybrid light sources - Google Patents

Abstract

A method of dynamic adjustment to automatic white balance under hybrid light sources includes determining whether a difference of a primary luminance boundary and a secondary luminance boundary of an image is greater than a first threshold, and determining whether a ratio of a primary luminance area and a total area of the image is greater than a second threshold when the difference is not greater than the first threshold, to perform white balance to the image according to a primary color temperature associated with primary luminance boundary and a secondary color temperature associated with the secondary luminance boundary.

Description

Method for dynamically adjusting white balance of mixed color light source

The invention relates to a method for dynamically adjusting the white balance of a mixed color light source, in particular to a method for recognizing a scene light source and its corresponding color temperature in a mixed color light source scene for dynamic white balance compensation.

Depending on the color and characteristics of the light source, the hue of the object will change accordingly. For example, the incandescent light is yellowish and the sunlight has a bluish hue, so the white object may appear yellowish or bluish under different light sources. Tone. Fortunately, the human eye can color compensate for the image seen in front of the eye according to the concept that the white object should be white. Therefore, the naked eye can still restore the image to the desired color regardless of the light source.

However, the photographic device will faithfully display the captured color on the photo. Compared with the image seen by the naked eye, the white color captured by the photographic device will take a yellowish-colored photo under the light according to the ambient light source, or Photographs with a bluish tint in the sun. The White Balance function compensates for the effects of the color of the light source in the shooting environment so that the white objects in the photo appear white.

Most of the current camera devices have Auto White Balance (AWB) function, which means that the camera device can detect the ambient light source or shoot the scene, and automatically compensate the captured image for color compensation, but the application performance in some scenes. Still not able to meet the expected color effect. For example, if the shooting scene contains a mixed color light source, such as indoor lighting and outdoor sunlight, the two light sources respectively correspond to different colors and color temperatures, and some specifications regarding a standard light source can be seen in Table 1 below.

In general, the incandescent light can be a TL84 light source or an A light source with a yellowish color and a lower color temperature, and the sunlight can be a D50 light source or a D65 light source with a blue color and a higher color temperature. Since the naked eye is sensitive to high-brightness colors, the current automatic white balance algorithm performs color compensation with a high-brightness light source and its corresponding color temperature. In the case of high outdoor image brightness, the automatic white balance algorithm will compensate more red for the whole image according to the bluish D50 light source or D65 light source, which will cause the indoor image to be reddish.

Therefore, in the mixed color light source scene, the current automatic white balance algorithm cannot recognize the scene light source and cause improper color compensation, resulting in color shift of the shooting result. In view of this, how to identify the scene light source and its corresponding color temperature for white balance compensation in the mixed color light source scene is one of the important topics in the field.

Therefore, the main object of the present invention is to provide a method for recognizing a scene light source and its corresponding color temperature for dynamic white balance compensation in a mixed color light source scene.

The invention determines the corresponding color temperature of the mixed color light source by analyzing the brightness distribution interval of the image, so that the image blocks in the image can be white balance compensated for different light sources and corresponding color temperatures to restore the image to an ideal color. In addition, for images with small differences in brightness, the image area in the main brightness interval is large enough for the full image to be white balance compensated using only the main color temperature without color shift. On the other hand, for an image with a small difference in brightness, when the image area of the main brightness interval is not large enough, the white balance compensation is repeatedly performed using the dynamic mixed color temperature until the image area of the main and minor brightness intervals is substantially equal or When they are nearly equal, the white color balance is compensated for the whole image by using the main color temperature. This avoids the traditional white color compensation using only the single color temperature of the high-brightness light source. Therefore, the present invention can solve the problem that the conventional white balance function uses only a single color temperature to perform color compensation due to color compensation under a mixed light source.

1 is a flow chart of a process 10 of an embodiment of the present invention. The process 10 can be used for a photographing device or an electronic device having a photographing function. The process 10 can be compiled into a code and stored in a memory to instruct a photographing device or a processor or an arithmetic unit of the electronic device having the photographing function to perform image processing. To achieve white balance compensation. Process 10 includes the following steps.

Step 100: Start.

Step 101: Divide an image into a plurality of blocks B1 B BN.

Step 102: Set a main brightness limit HY and a primary brightness limit LY according to the brightness Y1 YN of the plurality of blocks B1 B BN.

Step 103: Determine whether a difference between the main brightness limit HY and the secondary brightness limit LY is greater than a threshold TH1. If yes, go to step 104; if no, go to step 105.

Step 104: When the brightness YX of one of the plurality of blocks BX is greater than or equal to the main brightness limit HY, white balance compensation is performed on the block BX according to the main color temperature CT_HY corresponding to the main brightness limit HY; when the brightness of the block BX is YX When less than or equal to the secondary brightness limit LY, white balance compensation is performed on the block BX according to the secondary color temperature CT_LY corresponding to the secondary brightness limit LY; and when the brightness YX of the block BX is smaller than the main brightness limit HY and larger than the secondary brightness limit In LY, the white balance compensation is performed on the block BX according to the main color temperature CT_HY corresponding to the main brightness limit, the secondary color temperature CT_LY corresponding to the secondary brightness limit, and a weight W1.

Step 105: Calculate a ratio R_HY between a main luminance area A_HY and an area of the image A_IMG according to the brightness Y1 YN of the plurality of blocks B1 B BN, and a ratio between the primary brightness area A_LY and the area of the image A_IMG The value is R_LY.

Step 106: Determine whether the ratio value R_HY is greater than a threshold TH2. If yes, go to step 107; if no, go to step 108.

Step 107: Perform white balance compensation on the image according to the main color temperature CT_HY.

Step 108: Perform white balance compensation on the image according to the product of the primary color temperature CT_HY and a weight W2 plus the product of the secondary color temperature CT_LY and a weight W3.

Step 109: Reduce the weight W2 and increase the weight W3.

Step 110: Calculate the proportional value R_HY and the proportional value R_LY to determine whether the proportional value R_HY is equal to the proportional value R_LY. If yes, go back to step 107; if no, go back to step 108.

Step 111: End.

According to the process 10, the processor or the computing unit of the photographing apparatus divides an image into a plurality of blocks B1 to BN (step 101); and sets the main according to the brightness Y1 to YN (for example, Y-level) of the blocks B1 to BN. The brightness limit HY and the secondary brightness limit LY (step 102); next, it is determined whether the difference between the main brightness limit HY and the secondary brightness limit LY is greater than the threshold TH1 (step 103) to determine the difference in the color mixture distribution in the image. size. Specifically, if the image contains two kinds of light sources, the brightness Y1 ～ YN distribution of most of the blocks B1 BBN will be concentrated in two brightness interval ranges, for example, a main brightness interval greater than the main brightness limit HY and less than The brightness interval to be the brightness limit LY. Therefore, when the difference between the main brightness limit HY and the secondary brightness limit LY is greater than the threshold TH1, it means that the color temperature difference between the two light sources is large, so that the distribution difference between the main brightness interval and the secondary brightness interval is large.

Therefore, in step 104, when the luminance YX of the block BX (X is an integer from 1 to N) is greater than or equal to the main luminance limit HY, it indicates that the block BX falls within the main luminance interval, and therefore the main corresponding to the main luminance limit HY The color temperature CT_HY performs white balance compensation on the block BX; when the brightness YX of the block BX is less than or equal to the secondary brightness limit LY, it indicates that the block BX falls within the secondary brightness interval, so corresponding to the secondary brightness limit LY For the color temperature CT_LY, the white balance compensation is performed on the block BX; and when the brightness YX of the block BX is smaller than the main brightness limit HY and larger than the secondary brightness limit LY, it indicates that the block BX falls within the main brightness interval and the secondary brightness interval. In the boundary interval between the two, therefore, according to the main color temperature CT_HY corresponding to the main brightness limit, the secondary color temperature CT_LY corresponding to the secondary brightness limit, and a weight W1, the white balance compensation is performed on the block BX to avoid the image between the mixed color temperatures. unexpected. In an embodiment, when the brightness YX of the block BX is smaller than the main brightness limit HY and greater than the secondary brightness limit LY, the mixed color temperature used may be expressed as: CT_LY*W1+CT_HY*(1-W1), wherein the weight W1 is the size Any value from 0 to 1 can be expressed as: (YX-CT_LY) / (CT_HY-CT_LY).

On the other hand, when the difference between the main brightness limit HY and the secondary brightness limit LY is not greater than the threshold TH1, it means that the color temperature difference between the two light sources is small, so that the distribution difference between the main brightness interval and the secondary brightness interval is small. In this case, it is necessary to further consider whether the area ratio R_HY occupied by the main brightness interval in the full image is greater than the threshold TH2, so as to determine whether the image area belonging to the main brightness interval is large enough to prevent color shift of the entire image. Under the premise, only the main color temperature CT_HY is used for white balance compensation. Therefore, according to the brightnesses Y1 to YN of B1 to BN, the ratio R_HY between the main luminance area A_HY and the area A_IMG of the image and the ratio R_LY between the minor luminance area A_LY and the area A_IMG of the image are calculated (step 105); Whether the scale value R_HY is greater than the threshold TH2 (step 106); when the scale value R_HY is greater than the threshold TH2, the white balance compensation is performed on the block BX according to the primary color temperature CT_HY (step 107). The scale value R_HY can be expressed as A_HY/A_IMG, and the scale value R_LY can be expressed as A_LY/A_IMG. In an embodiment, determining whether the ratio value R_LY is smaller than another threshold value, determining whether the image area belonging to the secondary brightness interval is small enough to be ignored, so that the entire image uses only the main color temperature without causing color shift. CT_HY performs white balance compensation.

In addition, when the area ratio value R_HY corresponding to the main brightness limit HY is not greater than the threshold value TH2, it is determined that the image area belonging to the main brightness interval is not large enough, and if the white color balance compensation is performed on the entire image using only the main color temperature CT_HY, color may occur. Partial. In this case, it is necessary to repeatedly perform white balance compensation on the entire image using the dynamic mixed color temperature until the area ratio value R_HY corresponding to the main brightness limit HY is substantially equal to or close to the area ratio value R_LY corresponding to the secondary brightness limit LY. Finally, white balance compensation is performed on the entire image using the main color temperature CT_HY. It is worth noting that the dynamic mixed color temperature is synthesized by the decreasing primary color temperature CT_HY and the increasing secondary color temperature CT_LY, which can gradually weaken the white color balance of the main color temperature CT_HY for the whole image, and gradually strengthen the secondary image to the secondary color temperature CT_LY. The white balance compensation can avoid the occurrence of color shift when the white balance compensation is performed on the whole image with the main color temperature CT_HY.

Therefore, when the scale value R_HY is not greater than the threshold TH2, the product of the dynamic color temperature (which is the product of the primary color temperature CT_HY and the weight W2 corresponding to the main brightness limit and the secondary color temperature CT_LY and the weight W3 corresponding to the secondary brightness limit) Performing white balance compensation on the image (step 108); reducing the weight W2 to reduce the component of the main color temperature CT_HY in the dynamic mixed color temperature, and increasing the weight W3 to increase the component of the secondary color temperature CT_LY in the dynamic mixed color temperature (steps) 109); and determining whether the scale value R_HY is equal to the ratio value R_LY (step 110). When the ratio value R_HY is substantially equal to the ratio value R_LY, white balance compensation is performed on the image according to the main color temperature CT_HY (return to step 107); when the scale value R_HY is substantially not equal to the scale value R_LY, the dynamic mixed color temperature is used to image White balance compensation is performed (back to step 108). In an embodiment, the dynamic blending color temperature may be expressed as: CT_HY*W2+CT_LY*W3, wherein the weight W2 may be equal to the proportional value R_HY, and the weight W3 may be equal to the proportional value R_LY or expressed as (1-W2). In an embodiment, at step 109, the weight W2 may be decreased and the weight W3 may be decreased by a unit ratio value, for example, 5%.

In addition, in step 101, after the image is divided into blocks B1 B BN, the conventional automatic white balance algorithm converts the blocks B1 B BN into RGB color space coordinates to represent YCbCr color space coordinates, thereby finding out The image pixels whose brightness is too high or too low are regarded as invalid image pixels; then the blocks B1 to BN are converted into HSV color space coordinates to find the image pixels with too high saturation. And treat it as an invalid image pixel. Through the above color space coordinate conversion, screening and counting the image pixels belonging to the neutral color and removing the unnecessary image pixel information, determining whether the main brightness pixels of the blocks B1 B BN belong to a specific color temperature and the secondary brightness Whether the pixel belongs to another specific color temperature, wherein the specific color temperature corresponding to the primary and secondary luminance pixels can be set according to the user or automatically detected. When the main luminance pixels of the blocks B1 B BN belong to the specific color temperature and the secondary luminance pixels belong to the other specific color temperature, white balance compensation is performed on the entire image according to the specific color temperature of the main luminance pixels. On the other hand, when the main luminance pixels of the blocks B1 B BN do not belong to the specific color temperature and the secondary luminance pixels do not belong to the other specific color temperature, steps 102 to 110 are performed to perform dynamic white balance compensation.

In short, the present invention determines the corresponding color temperature of the mixed color light source by analyzing the brightness distribution interval of the image, so that the image blocks in the image can be white balance compensated for different light sources and corresponding color temperatures to restore the image to the desired color. The image block located in the boundary between the main brightness interval and the secondary brightness interval may perform white balance compensation according to the weighted primary color temperature and the secondary color temperature to prevent the image between the mixed color temperatures from being too abrupt (step 101~) 104). In addition, for images with small differences in brightness, the image area in the main brightness interval is large enough to allow white balance compensation using only the main color temperature without premature color shift (steps 105-107). On the other hand, when the image area of the main brightness interval is not large enough, the white balance compensation of the entire image is repeatedly performed using the dynamic mixed color temperature until the image areas of the main and minor brightness intervals are substantially equal or nearly equal, and finally Then use the main color temperature to perform white balance compensation on the whole image (steps 107-110), so that the white balance compensation can be avoided by using a single color temperature of the high-brightness light source. Therefore, through the process 10, the present invention can solve the problem that the conventional white balance function uses only a single color temperature to perform color compensation caused by color compensation under the mixed light source.

Please note that the above-mentioned thresholds TH1, TH2, the weights W1, W2, W3, the adjustment weight W2, and the unit ratio value of the weight W3, and other related condition parameters can be adjusted according to different application scenarios, but are not limited.

Figure 2 shows an original image IMG_RAW. FIG. 3 illustrates a compensated image IMG_WB according to an embodiment of the present invention. After the processor or the arithmetic unit of the photographing device receives the original image IMG_RAW, it is divided into a plurality of blocks B1 B BN (for example, 64 blocks are divided into 8 blocks, that is, blocks B1 B B64), and then Calculate the brightness of each block separately (Y-level, taking 8-bit image pixels as an example, which is a value from 0 to 255), assuming that the brightness of blocks B30-B32, B37-B40, B45-B48 is between 158 to 179 (the main brightness limit HY is assumed to be 158), and the remaining blocks have a brightness between 20 and 65 (the secondary brightness limit LY is assumed to be 65). It can be seen from the above information that the main luminance area A_HY (that is, the total area of the block with the brightness between 158 and 179) accounts for 11/64 (about 17.2%) of the total area of the original image IMG_RAW, and the secondary brightness area A_LY (ie, The total area of the block with a brightness between 20 and 65) is 53/64 (about 82.8%) of the total area of the original image IMG_RAW. After color analysis is performed for each block BX (X represents an integer of 1 to 64), it is assumed that the block whose color temperature is the D50 light source is determined to belong to the main luminance interval, and the block whose color temperature is the A light source is determined to belong to the secondary luminance interval. Therefore, through the above analysis, each block BX in the original image IMG_RAW can dynamically perform white balance compensation for different color temperatures.

When the difference between the main brightness limit HY (for example, 158) and the secondary brightness limit LY (for example, 65) is greater than the threshold TH1, each block BX performs white balance compensation with the color temperature corresponding to the respective brightness. For example, the block BX with brightness between 20 and 65 can perform white balance compensation according to the color temperature corresponding to the A light source, and the block BX with brightness between 158 and 179 can perform white balance compensation according to the color temperature corresponding to the D50 light source. The block BX having a brightness between 159 and 178 can perform white balance compensation according to the color temperature corresponding to the D50 light source, the color temperature corresponding to the A light source, and the weight W1.

On the other hand, when the difference between the main brightness limit HY and the secondary brightness limit LY is not greater than the threshold TH1, it is further determined whether the ratio of the main brightness area A_HY to the total area of the original image IMG_RAW R_HY is greater than the threshold TH2 (eg , 80%). When the scale value R_HY is greater than the threshold TH2, the white balance compensation is performed on the original image IMG_RAW according to the color temperature corresponding to the D50 light source. Conversely, when the ratio value R_HY is not greater than the threshold value TH2, the white balance compensation is performed according to the color temperature corresponding to the decreasing D50 light source and the color temperature corresponding to the increasing A light source (for example, each decrement and increment by 5%) until the ratio The value R_HY is substantially equal to or close to the proportional value R_LY, and finally the white balance compensation is performed according to the color temperature corresponding to the D50 light source. In this way, the compensated image IMG_WB of FIG. 3 can be generated by the dynamic white balance compensation described above.

4 is a chromatic level distribution diagram of the original image IMG_RAW, and FIG. 5 is a chromatic level distribution diagram of the compensation image IMG_WB according to an embodiment of the present invention, wherein the gradation distribution of the red pixel R is represented by a diagonal line pattern, and the green pixel G The gradation distribution is indicated by a space, and the gradation distribution of the blue pixel B is represented by a dot pattern.

In Fig. 4, when the original image IMG_RAW uses the traditional white balance algorithm, the gradation distribution of the red pixel R accounts for about 79.13% of the total image pixels, and the gradation distribution of the green pixel G accounts for the full image. The 10.19% of the pixels and the gradation distribution of the blue pixel B account for about 0.57% of the total image pixels. The ratio R/G of the red pixel R to the green pixel G is about 7.13, and the ratio B/G of the blue pixel B to the green pixel G is about 0.57. It can be seen that under the mixed light source, if the white balance compensation is performed only by the color temperature corresponding to the single light source with the bluish tone, the color shift will be caused, and the ratio of the red and blue pixels R and B will be greatly different, resulting in the original image IMG_RAW. red.

In Fig. 5, the gradation distribution of the red pixel R of the compensated image IMG_WB accounts for about 68.05% of the total image pixel, and the gradation distribution of the green pixel G accounts for 31.94% of the total image pixel, the blue pixel The gradation distribution of B accounts for about 11.46% of the total image pixels. The ratio R/G of the red pixel R to the green pixel G is about 2.13, and the ratio B/G of the blue pixel B to the green pixel G is about 0.35. It can be seen that under the mixed light source, the dynamic white balance compensation of the flow 10 can make the gradation distribution of the compensated image IMG_WB relatively uniform, and the difference ratio of the red and blue pixels R and B is small to restore the compensated image IMG_WB. Into the ideal color.

In summary, the present invention determines the corresponding color temperature of the mixed color light source by analyzing the brightness distribution interval of the image. Therefore, the white balance compensation can be performed on the image blocks in the image for different light sources and corresponding color temperatures to restore the image to an ideal image. colour. In addition, for an image with a small difference in brightness, the present invention uses the dynamic mixed color temperature to perform white balance compensation on the entire image according to the ratio of the image area of the main brightness interval to the total image area, thereby avoiding the traditional single color temperature using only the high-intensity light source. Perform white balance compensation. Therefore, in the above manner, the present invention can solve the problem that the conventional white balance function uses only a single color temperature for color compensation to cause color shift under a mixed light source. The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.

10‧‧‧ Process
100～110‧‧‧Steps
Block B1~BN, BX‧‧‧
HY, LY, Y1 ~ YN, YX‧‧‧ brightness
TH1, TH2‧‧‧ threshold
A_HY, A_LY, A_IMG‧‧‧ area
R_HY, R_LY‧‧‧ ratio
CT_HY, CT_LY‧‧‧ color temperature
W1, W2, W3‧‧‧ weights
IMG_RAW‧‧‧ original image
IMG_WB‧‧‧Compensated image
R, G, B‧‧ ‧ pixels

FIG. 1 is a flow chart of a process of an embodiment of the present invention. Figure 2 depicts an original image. FIG. 3 illustrates a compensated image according to an embodiment of the present invention. Figure 4 is a diagram showing the gradation distribution of the original image of Figure 2. FIG. 5 is a diagram showing the gradation distribution of the compensated image in the third embodiment of the embodiment of the present invention.

10‧‧‧ Process

100~110‧‧‧Steps

Claims (8)

A method for dynamically adjusting white balance compensation of a mixed color light source, comprising: determining whether a difference between a primary brightness limit of an image and a primary brightness limit is greater than a first threshold; and when the difference is not greater than the first a threshold value, determining whether a first ratio between a major luminance area of the image and an area of the image is greater than a second threshold, to correspond to one of the primary color temperature and the secondary luminance limit according to the primary luminance limit Corresponding to the color temperature, white balance compensation is performed on the image. The method of claim 1, further comprising: dividing the image into a plurality of blocks; and setting the primary brightness limit and the secondary brightness limit based on the plurality of brightnesses of the plurality of blocks. The method of claim 2, wherein when the difference is greater than the first threshold, the method further comprises: when the brightness of one of the plurality of blocks is greater than or equal to the main brightness limit, according to the main Color temperature, performing white balance compensation on one of the plurality of blocks; when the brightness of one of the plurality of blocks is less than or equal to the secondary brightness limit, one of the plurality of blocks according to the secondary color temperature Performing white balance compensation; and when the brightness of one of the plurality of blocks is less than the primary brightness limit and greater than the secondary brightness limit, the plurality of colors according to the primary color temperature, the secondary color temperature, and a first weight One of the blocks performs white balance compensation. The method of claim 2, wherein when the difference is not greater than the first threshold, the method further comprises: calculating the main brightness area and the area of the image according to the plurality of brightnesses of the plurality of blocks The first ratio value between the first ratio value and a second ratio between the area of the luminance area and the area of the image. The method of claim 4, wherein determining whether the first ratio of the major luminance area of the image and the area of the image is greater than the second threshold is based on the primary color temperature and the secondary color temperature. Performing white balance compensation on the image includes: when the first ratio value between the main brightness area and the area of the image is greater than the second threshold value, performing white balance compensation on the image according to the main color temperature. The method of claim 4, wherein determining whether the first ratio of the major luminance area of the image and the area of the image is greater than the second threshold is based on the primary color temperature and the secondary color temperature. Performing white balance compensation on the image includes: when the first brightness value between the main brightness area and the area of the image is not greater than the second threshold value, performing white balance compensation on the image according to a dynamic mixed color temperature The dynamic mixed color temperature is a product of the primary color temperature and a second weight plus a product of the secondary color temperature and a third weight. The method of claim 6, wherein when the first ratio value between the main luminance area and the area of the image is not greater than the second threshold, white balance compensation is performed on the image according to the dynamic mixed color temperature. Having: reducing the second weight and increasing the third weight; and calculating the first proportional value and the second proportional value to determine whether the first proportional value is substantially equal to the second proportional value. The method of claim 7, further comprising: when the first proportional value is substantially equal to the second proportional value, performing white balance compensation on the image according to the primary color temperature; and when the first proportional value Whether or not the second ratio value is substantially equal to the white balance compensation is performed on the image according to the dynamic mixed color temperature.