JP2015005927A - Image processing apparatus and control method of the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can obtain a white balance gain accurately from an input image, and a control method of the same.SOLUTION: The image processing apparatus extracts a specular reflection component from an input image and corrects a method of white extraction and a value of a white balance gain on the basis of the extracted specular reflection component. For example, the image processing apparatus corrects a white extraction range to a range determined in consideration of the specular reflection component, weighs pixel values within the white extraction range according to the relationship with the specular reflection component, and corrects a possible range of the white balance gain according to the color temperature of the specular reflection component.

Description

  The present invention relates to an image processing apparatus and a control method thereof, and more particularly to a technique for adjusting white balance of an image.

  Conventionally, an auto white balance control function used in a digital camera has been mainly used to adjust white balance using an output of an image sensor without using an external sensor. For example, in Patent Document 1, as a method of adjusting white balance using the output of an image sensor, a color that is considered to be white (achromatic color) is extracted from an input image, and the extracted color becomes white. A method (white extraction method) for calculating a white balance gain is disclosed.

  Also known is a dichroic reflection model that separates subject object light into a specular reflection component and a diffuse reflection component.

  Patent Document 2 discloses a method for specifying a light source color for irradiating a subject by calculating a difference between adjacent pixels having different brightnesses. In this method, a first pixel having a high luminance (including both a diffuse reflection component reflecting the object color and a specular reflection component reflecting the light source color) to a pixel having a lower luminance than the first pixel (only the object color is detected). This is based on the principle of extracting the light source color component by calculating the difference of the diffuse reflection component.

  Specifically, the difference between the RGB value of the first pixel with high luminance and the RGB value of the second pixel with lower luminance than the first pixel is calculated. Since the difference RGB value is an RGB value having the same vector component as the light source color, the white balance gain can be calculated from the component ratio of the difference RGB value.

JP-A-5-83728 JP 2007-13415 A

  In the method described in Patent Document 1, there is a problem that an incorrect color is extracted as white depending on conditions, particularly when the subject is dark, and the accuracy of white balance is lowered. Moreover, although the technique of Patent Document 2 can roughly estimate the light source color, there is a problem that it is difficult to estimate the color temperature of the light source with high accuracy.

  The present invention has been made in view of the above-described problems of the prior art, and an object thereof is to provide an image processing apparatus capable of obtaining a white balance gain from an input image with high accuracy, and a control method therefor.

  The above-described object is to set a white extraction range in a predetermined color space, a first extraction unit that extracts a specular reflection component included in an input image, and a white extraction based on the extracted specular reflection component. Correction means for correcting the range, second extraction means for extracting pixels having a color included in the corrected white extraction range from the input image, and white balance for the input image based on the value of the extracted pixel This is achieved by an image processing apparatus comprising a calculation means for calculating a gain.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus which can obtain a white balance gain from an input image accurately, and its control method can be provided.

1 is a block diagram illustrating a configuration example of a digital camera as an example of an image processing apparatus according to an embodiment. The block diagram which shows the structural example of the image process part of FIG. The flowchart for demonstrating the white balance gain calculation process in 1st Embodiment. Flowchart for explaining specular reflection component extraction processing in S301 of FIG. The figure which shows the example of the input image in 1st Embodiment The figure which shows typically the storage method of the difference and count value in 1st Embodiment. The figure for demonstrating the estimation method of the light source color in 1st Embodiment. The figure which shows the example of the setting and correction | amendment of the white extraction range in 1st Embodiment Flowchart for explaining white balance gain calculation processing in the second embodiment The figure which shows the example of the weight determined in S904 and S905 of FIG. Flowchart for explaining white balance gain calculation processing in the third embodiment The figure which shows the example of a relationship between the block average color of a specular reflection component in 3rd Embodiment, and the block average color of a pixel, and an example of a weight Flowchart for explaining white balance gain calculation processing in the fourth embodiment The figure which shows the example of the limit value of the white balance gain in 4th Embodiment The figure which shows the example of control of the limit value of a white balance gain according to the color temperature of a specular reflection component in 4th Embodiment

  Exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following embodiments, a digital camera as an example of an image processing apparatus according to the present invention will be described. However, a configuration related to shooting is not essential in the present invention, and the present invention is implemented by any apparatus capable of acquiring image data. Is possible.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of a digital camera 100 according to the first embodiment of the present invention.
In FIG. 1, a lens group 101 is a zoom lens including a focus lens. A shutter 102 having a diaphragm function is provided between the lens group 101 and the imaging unit 103. The imaging unit 103 includes an imaging element represented by a CCD / CMOS image sensor that converts an optical image formed on the imaging surface by the lens group 101 into an electrical signal in pixel units. The A / D converter 104 converts the analog signal output from the imaging unit 103 into a digital signal (image data).

  The image processing unit 105 performs various image processing such as color interpolation (demosaic), white balance adjustment, γ correction, contour enhancement, noise reduction, and color correction on the image data output from the A / D converter 104. The image memory 106 temporarily stores image data. A memory control unit 107 controls reading and writing of the image memory 106. The D / A converter 108 converts the image data into an analog signal. The display unit 109 includes a display device such as an LCD or an organic EL display, and displays various GUIs, live view images, images read out from the recording medium 112 and reproduced. The codec unit 110 encodes the image data stored in the image memory 106 by a predetermined method for recording on the recording medium, or decodes the encoded image data included in the image file for display, for example. Or

  The interface (I / F) 111 mechanically and electrically connects a detachable recording medium 112 such as a semiconductor memory card or a card type hard disk to the digital camera 100. The system control unit 50 may be a programmable processor such as a CPU or MPU. The system control unit 50 executes a program recorded in, for example, the non-volatile memory 124 or the built-in non-volatile memory to control necessary blocks and circuits, so that the digital camera 100 including the light source estimation process described later is started. Realize the function.

  The operation unit 120 collectively describes buttons and switches for the user to input various instructions to the digital camera 100. The power switch 121 is one of the switches included in the operation unit 120, but is illustrated in different blocks for convenience. The power control unit 122 is configured by a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, for example, information on presence / absence of installation, type, remaining battery level, etc. To detect. Further, the power control unit 122 controls the DC-DC converter based on the detection result related to the power supply unit 123 and the instruction of the system control unit 50, and each unit of the digital camera 100 including the recording medium 112 for a necessary voltage for a necessary period. To supply.

  The nonvolatile memory 124 may be an electrically erasable / recordable, for example, an EEPROM. The nonvolatile memory 124 stores various setting values, GUI data, and programs when the system control unit 50 is an MPU or CPU.

  The system timer 125 measures the time used for various controls and the time of the built-in clock. The system memory 126 is, for example, a volatile memory that is used for storing constants and variables for the operation of the system control unit 50 and for developing programs read from the nonvolatile memory 124.

Next, the operation at the time of shooting in the digital camera 100 will be described.
For example, the imaging unit 103 photoelectrically converts a subject image formed on the imaging surface by the lens group 101 when the shutter 102 is opened, and outputs the subject image to the A / D converter 104 as an analog image signal. The A / D converter 104 converts the analog image signal output from the imaging unit 103 into a digital image signal (image data) and outputs the digital image signal to the image processing unit 105.

  The image processing unit 105 performs various processes such as color interpolation (demosaic), γ correction, edge enhancement, noise reduction, and color correction on the image data from the A / D converter 104 or the image data from the memory control unit 107. Perform image processing.

  In addition, the image processing unit 105 performs predetermined calculation processing relating to luminance, contrast, and the like using image data obtained by photographing, and the system control unit 50 performs distance measurement control and exposure control based on the obtained calculation result. Do. As described above, the digital camera 100 according to the present embodiment performs TTL (through the lens) AF (autofocus) processing and AE (automatic exposure) processing. The image processing unit 105 further estimates a light source color that illuminates the subject at the time of shooting using image data obtained by shooting, and also performs auto white balance (AWB) adjustment based on the estimated light source color.

  The image data output from the image processing unit 105 is written into the image memory 106 via the memory control unit 107. The image memory 106 stores image data output from the imaging unit 103 and image data to be displayed on the display unit 109.

  The D / A converter 108 converts the image display data stored in the image memory 106 into an analog signal and supplies the analog signal to the display unit 109. The display unit 109 performs display in accordance with an analog signal from the D / A converter 108 on a display device such as an LCD.

  The codec unit 110 encodes the image data recorded in the image memory 106 based on a standard such as JPEG or MPEG. The system control unit 50 assigns a predetermined header or the like to the encoded image data, forms an image file, and records the image file on the recording medium 112 via the interface 111.

  Note that with current digital cameras, it is common to perform moving image shooting in the shooting standby state, and keep the captured moving image displayed on the display unit 109 so that the display unit 109 functions as an electronic viewfinder (EVF). is there. In this case, the shutter 102 is opened and a so-called electronic shutter of the imaging unit 103 is used to capture an image at, for example, 30 frames / second.

  When the shutter button included in the operation unit 120 is half-pressed, the above-described AF and AE control is performed. When the shutter button is fully pressed, the still image shooting for recording is executed by the main shooting and is recorded on the recording medium 112. The Also, when moving image shooting is instructed by a moving image shooting button or the like, moving image recording on the recording medium 112 is started.

FIG. 2 is a block diagram illustrating a functional configuration example of the image processing unit 105.
The image data output from the A / D converter 104 in FIG. 1 is input to the luminance / color signal generation unit 200. The image data has a value corresponding to one of the color components constituting the color filter provided in the image sensor. When a commonly used primary color filter with a Bayer array is used, the image data is composed of R pixel, G pixel, and B pixel data.

  The luminance / color signal generation unit 200 performs demosaic processing on such image data, and generates a luminance signal Y and color signals R, G, and B for each pixel. The luminance / color signal generation unit 200 outputs the generated color signals R, G, and B to the white balance (WB) amplification unit 201 and the luminance signal Y to the luminance γ processing unit 205. Further, the luminance / color signal generation unit 200 outputs the color signals R, G, and B as image data (R, G, and B) to the image memory 106 via the memory control unit 107.

The color signals R, G, and B generated by the luminance / color signal generation unit 200 are also input to the color space conversion processing unit 207. The color space conversion processing unit 207 converts the color signals R, G, and B into chromaticities Cx and Cy using the following equations, and the image memory 106 via the memory control unit 107 as image data (Cx, Cy). Output to.
Cx = (R-B) / Y
Cy = (R + B -2G) / Y
Y = (R + G + B) / 2

  The WB amplifying unit 201 adjusts the white balance by applying gain to the color signals R, G, and B based on the white balance gain value calculated by the system control unit 50 through processing described later. The color γ processing unit 202 performs gamma correction on the color signals R, G, and B. The color difference signal generation unit 203 generates color difference signals RY and BY from the color signals R, G, and B and outputs them to the color correction unit 204. The color correction unit 204 adjusts the hue and saturation by applying a gain to the color difference signals RY and BY. The color correction unit 204 outputs the corrected color difference signals RY and BY to the image memory 106 via the memory control unit 107.

  On the other hand, the luminance γ processing unit 205 performs gamma correction on the luminance signal Y and outputs it to the contour enhancement processing unit 206. The contour emphasis processing unit 206 performs contour emphasis processing on the luminance signal Y and outputs it to the image memory 106 via the memory control unit 107. The luminance signal Y output from the contour enhancement processing unit 206 and the color difference signals RY and BY output from the color correction unit 204 are encoded by the codec unit 110 and finally recorded on the recording medium 112.

  The system control unit 50 reads the image data (RGB) and the image data (CxCy) output from the image processing unit 105 to the image memory 106 and develops them in the system memory 126. Then, the system control unit 50 analyzes these image data to estimate a light source color component, and calculates a white balance gain corresponding to the estimated light source color.

  The white balance gain calculation process will be described in detail with reference to the flowchart of FIG. First, in S301, the system control unit 50 as the first extraction unit extracts a specular reflection component. Details of the processing in S301 will be described using the flowchart shown in FIG.

  4, S401 and S407 indicate that the processes of S402 to S406 are performed on all the pixels while raster scanning the target pixel.

  In S402, the system control unit 50 checks whether the processing target pixel is saturated. Specifically, the system control unit 50 compares each of the RGB components with a predetermined threshold value, and if there is a component larger than the threshold value, determines that the pixel is saturated and skips processing of the current processing target pixel. Then, the saturation is confirmed for the next pixel. On the other hand, if all of the RGB components are equal to or smaller than the threshold value, it is determined that the pixel is not saturated, and the system control unit 50 advances the process to S403.

  In S403, the system control unit 50 calculates a difference between the processing target pixel and the comparison pixel. The processing target pixel and the comparison pixel will be described with reference to FIG. In FIG. 5A, reference numeral 500 denotes an input image, and reference numeral 501 schematically shows a state in which a part of the input image 500 is enlarged. A mesh in 501 indicates a pixel boundary. Reference numeral 502 denotes a processing target pixel in the current processing, and the processing in FIG. 4 is performed for each pixel while sequentially raster-scanning the position of the processing target pixel 502. Reference numerals 503, 504, and 505 denote comparison pixels for the processing target pixel 502. In this embodiment, a plurality of pixels that are separated from the processing target pixel 502 by a predetermined distance (number of pixels) in a predetermined direction are used as comparison pixels. .

  Although the positional relationship to be satisfied by the processing target pixel and the comparison pixel can be determined in advance, basically, pixels at a plurality of positions close to the processing target pixel are set as the comparison pixels. Here, as an example, the remaining three corner pixels in the 4 × 4 pixel block having the processing target pixel as the upper left corner pixel are used as comparison pixels. If the distance between the pixel to be processed and the comparison pixel is too short, there is a high possibility that the difference between the pixels will be small, and if the distance is too long, there is a high possibility that the pixel does not correspond to the same object. Therefore, the distance is experimentally determined within a range of, for example, several tens of pixels. Note that the positional relationship between the processing target pixel and the comparison pixel may be appropriately changed when the position of the processing target pixel is an end of the input image.

  In step S <b> 403, the system control unit 50 calculates a difference between the processing target pixel 502 and the comparison pixel 503. That is, the system control unit 50 determines the difference (subR, subG, subB) for each color component between the RGB value (R1, G1, B1) of the processing target pixel 502 and the RGB value (R2, G2, B2) of the comparison pixel 503. Is calculated.

In S404, the system control unit 50 determines whether the magnitude S of the differences (subR, subG, subB) calculated in S403 is larger than a predetermined threshold T1 (T1 <S). The size S of the difference (subR, subG, subB) is calculated from the following equation.
S = √ {(subR) ² + (subG) ² + (subB) ² }
The system control unit 50 proceeds to S405 when the difference size S is larger than the threshold T1, and proceeds to S406 when the difference is equal to or smaller than the threshold T1.

  In S405, the system control unit 50 saves (adds) the difference (subR, subG, subB) between the processing target pixel and the comparison pixel in the system memory 126, and calculates a difference sum value sumS. An example of the storage format of the difference (subR, subG, subB) is shown in FIG. In this embodiment, when the magnitude S of the difference is larger than the threshold value T1, as shown in FIG. 6, the differences (subR, subG, subB) are summed for each block to which the pixel to be processed belongs, and the difference sum value sumS is calculated. obtain.

  Here, the block indicates a divided area of the input image, and FIG. 5A shows an example in which the input image 500 is divided into 64 in the horizontal direction 8 × vertical direction 8. Thus, for each partial area of the input image, the difference (subR, subG, subB) calculated for the pixels included in the partial area is summed up for each block, and a difference value (block difference) representing the block is added. Value). Each time the difference (subR, subG, subB) is added, the count value CountS associated with the difference sum value sumS is incremented by one. Therefore, the count value CountS indicates the number of difference values used for calculating the block difference value.

  In S404 and S405, the reason for summing the differences (subR, subG, subB) only when the difference size S is larger than the threshold value T1 is that only the difference value that is considered to have a high degree of characteristics of the light source color. It is for extracting. This will be described more specifically with reference to FIGS. 5B and 7.

  FIG. 5B shows an image of a subject including specular reflection, a processing target pixel 506, and comparison pixels 507 to 509. Here, it is assumed that the comparison pixels 507 and 509 having high luminance include a relatively large amount of specular reflection components, and the processing target pixel 506 and the comparison pixel 508 are pixels that are substantially composed of only diffuse reflection components.

  FIG. 7A schematically shows an example of the relationship between the specular reflection component and the diffuse reflection component included in the pixel in the RGB color space. In the RGB space, the RGB value of a certain pixel can be represented by a vector from the origin to the coordinates corresponding to the RGB value. A vector 701 indicates a case where the RGB value is composed only of an object color (diffuse reflection component), and a vector 702 indicates a case where the RGB value is composed of an object color (diffuse reflection component) and a light source color (specular reflection component). . In this case, a vector 703 indicating the RGB value of the light source color (specular reflection component) to be obtained can be obtained as a difference vector between the vectors 702 and 701. Note that since the starting point of each vector is the coordinate origin, the RGB value obtained as the difference between the coordinates is actually the RGB value of the light source. Therefore, the difference between the RGB value of the pixel consisting only of the object color and the RGB value of the pixel consisting of the object color and the light source color can be obtained as the RGB value of the light source color. In the example of FIG. 5B, the RGB value difference between the processing target pixel 506 and the comparison pixel 507 or 509 is obtained as an estimated value of the RGB value of the light source color.

  On the other hand, since the diffuse reflection component is dominant in both the processing target pixel 506 and the comparison pixel 508 in FIG. 5B, the RGB values of these pixels are close to the vector 701 in the example of FIG. For example, in FIG. 7B, if the RGB values of the processing target pixel 506 and the comparison pixel 508 are vectors 711 and 712, respectively, the difference between the RGB values is very small. Although not shown in the figure, the comparison pixels 509 and 509 both have a diffuse reflection component and a specular reflection component, and the RGB values are both represented by vectors similar to the vector 702. Very small value.

  If the RGB value of the comparison pixel 507 is represented by the vector 713 in FIG. 7B, the difference between the vector 711 corresponding to the RGB value of the processing target pixel 506 and the vector 713 corresponding to the RGB value of the comparison pixel 507 is growing. Since this difference is a vector close to the RGB value of the light source color, it is a difference to be extracted.

  As described above, when the magnitude S of the differences between the RGB values of the pixels is small, it is considered that the two often have similar reflection components, and the contribution of the light source characteristics to the difference values is considered to be small. In the embodiment, the light source color is not used for estimation. Therefore, the threshold value T1 is provided as described above, and only when the magnitude S of the difference is larger than the threshold value T1, the differences (subR, subG, subB) are summed in the system memory 126, and the difference sum value sumS (R, G, B )

  The system control unit 50 calculates the difference sum value sumS for each of the 64 blocks. Further, the sum of differences of block 1 is sumS1 (R1, G1, B1), the sum of differences of block 2 is sumS2 (R2, G2, B2),..., The sum of differences of block 64 is sumS64 (R64, G64 , B64). When the difference is added to the difference total value sumS, the system control unit 50 adds 1 to the difference count value countS. FIG. 6 shows the difference sum values sumS1 to sumS64 and the difference count value countS for each block.

  Returning to FIG. 4, in S406, the system control unit 50 confirms whether or not the processing of S402 to S405 has been performed for all the comparison pixels 503 to 505. As shown in FIG. 5, in this embodiment, there are three comparison pixels 503, 504, and 505 for one processing target pixel 502. Therefore, the difference calculation (S403) is performed on all the processing target pixels, and the difference (subR, subG, subB) is added in the system memory 126 when the difference size S is larger than the predetermined threshold T1. 1 is added to the difference count value.

  The processes of S402 to S406 described above are performed on all the pixels of the input image while raster scanning the processing target pixel.

  When the summation of the difference values for all the pixels of the input image is completed by the processing up to S407, the system control unit 50 calculates the average difference value for each of 8 × 8 = 64 blocks in S408 to S413.

  First, in step S409, the system control unit 50 refers to the difference count value countS of the block to be processed, and determines whether the difference count value countS is greater than a predetermined threshold T2. The system control unit 50 determines that the block having the difference count value countS greater than the threshold T2 includes a specular reflection component, and advances the processing to S410. Further, the system control unit 50 determines that the block whose difference count value countS is equal to or less than the threshold T2 does not include the specular reflection component, and moves to the processing of the next block.

  In S410, the system control unit 50 calculates average difference values (blkR, blkG, blkB). Specifically, the difference total value sumS is divided by the difference count value countS for each RGB component. As a result, an average difference value (blkR, blkG, blkB) for the processing target block is obtained.

  In S411, the system control unit 50 adds the average difference value (blkR, blkG, blkB) of the processing target block calculated in S410 to the block total value (sumBlkR, sumBlkG, sumBlkB) in the system memory 126. The block total value is used to calculate an average difference value of the entire image, which will be described later.

  In S412, the system control unit 50 adds 1 to the number of specular reflection blocks blkCountS. The number of specular reflection blocks blkCountS indicates the number of blocks determined to have a specular reflection component among the 64 blocks obtained by dividing the input image.

  The system control unit 50 calculates the average difference value (aveR, aveG, aveB) of the entire input image in S414 when the above-described processing of S409 to 612 is performed on all 8 × 8 = 64 blocks. Specifically, the system control unit 50 divides each component of the block total value (sumBlkR, sumBlkG, sumBlkB) by the number of specular reflection blocks blkCountS to obtain the average difference value (aveR, aveG, aveB) of the entire input image. calculate. The average difference values (aveR, aveG, aveB) of the entire input image indicate the average value of the specular reflection component as described with reference to FIG.

  Returning to FIG. 3, in S <b> 302, the system control unit 50 determines a white extraction range (a range regarded as white (achromatic color)) based on the brightness (EV value) of the subject. FIG. 8 is a diagram showing an example of the white extraction range on the CxCy plane, and FIGS. 8A and 8B show examples of the white extraction range that change according to the brightness of the subject.

  FIG. 8A shows an example of the white extraction range 801 set when the subject is dark. When the subject is dark, a white extraction range is set wide because light sources with various color temperatures such as artificial lighting such as incandescent lamps and fluorescent lamps, or shade are assumed. On the other hand, FIG. 8B shows an example of the white extraction range 802 set when the subject is bright. When the subject is bright, the light source has a high possibility of sunlight in the daytime, so a narrow white detection range corresponding to the color temperature of daylight sunlight is set. In this embodiment, the white extraction ranges 801 and 802 are switched based on whether the brightness of the subject is equal to or lower than the threshold value, but the subject brightness range is divided into three or more and divided into three. The above white detection range may be switched.

  There is no particular limitation on the type of value indicating the brightness of the subject and the calculation method, and any known method can be used. For example, the average luminance value of the entire image, the average luminance value of the focus detection area, or the average luminance value of an area corresponding to a specific subject such as a human face may be used. Further, instead of a method of obtaining the brightness of the subject from the input image, the brightness of the subject may be obtained using a photometric sensor. In addition, various calculation methods such as assigning a weight to a specific area when obtaining the luminance average value can be used. Further, the threshold value of the brightness of the subject can be experimentally determined in advance.

  In step S303, the system control unit 50 determines whether the input image includes a specular reflection component. Specifically, the system control unit 50 determines whether or not the number of specular reflection blocks blkCountS of the specular reflection component calculated in S301 is larger than the threshold T3. If the number of specular reflection blocks blkCountS is larger than the threshold T3, it is determined that the input image includes a specular reflection component at a predetermined amount or more, and the process proceeds to S304. If the number of specular reflection blocks blkCountS is equal to or smaller than the threshold T3, the system control unit 50 determines that the input image has a small amount of specular reflection component, skips S304, and proceeds to S305.

In S304, the system control unit 50 corrects the white extraction range set in S302 based on the specular reflection component. This process will be described with reference to FIG. FIG. 8C shows an example of the relationship between the white extraction range and the specular reflection color.
Here, a case where the subject is dark will be described as an example. As described above, when the subject is dark, a wide white extraction range 801 (FIG. 8A) is set. In the present embodiment, the system control unit 50 corrects the white extraction range 801 based on the average difference values (aveR, aveG, aveB) indicating the average value of the specular reflection component calculated in S301.

  First, the system control unit 50 converts the average difference values (aveR, aveG, aveB) in the RGB format into values (chromaticity coordinates) on the CxCy plane. In FIG. 8C, the coordinates 803 indicate the average difference value after conversion. Then, the system control unit 50 sets, as a white extraction range 804 based on the specular reflection component, a range obtained by moving the range of ± d2 in the Cy direction with respect to the coordinates 803 and a range of ± d1 in the Cx direction along the black body radiation axis. To do. Then, the system control unit 50 sets a region that overlaps the white extraction range 801 calculated based on the brightness and the white extraction range based on the specular reflection component as the corrected white extraction range. In the example of FIG. 8C, since the white extraction range 804 based on specular reflection is entirely included in the white extraction range 801 based on brightness, the white extraction range 804 based on specular reflection is directly used as the corrected white extraction range. Become. When the correction of the white extraction range is completed, the system control unit 50 advances the process to S305.

  In addition, the value of d1 and d2 can be determined based on the distribution range of a specular reflection component, for example. For example, the difference count value countS can be set as a value that falls within a range that includes an average difference value (blkR, blkG, blkB) for a block that is greater than the threshold T2 within a predetermined ratio (for example, 80%).

In step S <b> 305, the system control unit 50 as the second extraction unit determines whether all pixels of the input image have a color included in the white extraction range 804. Pixel values (Cx, Cy) included in the white extraction range are added to a buffer that may be the system memory 126, for example.
Further, the system control unit 50 adds the number of pixels having pixel values (Cx, Cy) included in the white extraction range to the white pixel count that is a variable.

In S306, the system calculates a white balance gain based on the result of the white extraction process in S305. First, the system control unit 50 calculates the white balance gain by dividing the total value of the extracted pixel values (Cx, Cy) by the white pixel count value. This calculation is an average calculation of pixel values having colors included in the white extraction range 401. A coordinate 805 in FIG. 8C indicates an example of the calculated average value.
Next, the system control unit 50 converts the color space of the calculated average value (Cx, Cy) and converts it into a value in the RGB color space. This RGB value is called the white extraction average RGB value (wR, wG, wB).

Then, the system control unit 50 calculates a white balance gain from the white extracted average RGB values (wR, wG, wB). When the G white balance gain wG-Gain is fixed, the system control unit 50 calculates the R and B white balance gains (wR-Gain, wB-Gain) by the following equations.
wR-Gain = (wG / wR) × wG-Gain
wB-Gain = (wG / wB) × wG-Gain
Thus, wR-Gain and wB-Gain indicate the white balance gain (first white balance gain) based on white extraction.

  In S307, the system control unit 50 sets the calculated white balance gain in the WB amplification unit 201 (FIG. 2). The WB amplifying unit 201 amplifies input RGB values based on the set white balance gain.

  As described above, the present embodiment corrects the white extraction range set in accordance with the brightness of the subject based on the specular reflection component extracted from the input image, and thereby performs white extraction particularly when the subject is dark. Accuracy can be improved. As a result, the accuracy of white balance adjustment can be improved.

  In the present embodiment, the white extraction range is corrected based on the average value (average color) of the specular reflection components. However, the white extraction range may be corrected using information on specular reflection components other than the average value. For example, the standard deviation may be calculated from the distribution of the specular reflection components (Cx, Cy), and a range that is a constant multiple of the standard deviation may be used as the white extraction range based on the specular reflection component. Further, the Cx maximum value, Cx minimum value, Cy maximum value, and Cx maximum value of the specular reflection component are obtained and determined by (Cx maximum value, Cy maximum value), (Cx minimum value, Cy minimum value), and the black body radiation axis. The range may be a white extraction range based on a specular reflection component.

  In the present embodiment, the configuration in which the white extraction range is corrected based on the specular reflection component whenever the input image has a certain amount of specular reflection component has been described. However, the white extraction range based on the specular reflection component may be corrected only when the condition that the reliability of the white balance adjustment based on the white extraction range corresponding to the brightness of the subject is lowered is satisfied. For example, the white extraction range may be corrected based on the specular reflection component only when the brightness of the subject is less than a predetermined value.

  In this embodiment, the method of calculating the specular reflection component from the difference between the processing target pixel and the comparison pixel has been described. However, the specular reflection component may be extracted using any other method.

  In the present embodiment, an example in which an input image is divided into eight blocks by dividing the input image into eight in both the horizontal and vertical directions has been described, but there is no limitation on the block division method and the number of divisions. For example, even if the image is divided into 64 or more blocks, the entire image may be handled as one block. Also, the block sizes need not be the same.

  In this embodiment, white extraction is performed using the value (chromaticity) of the CxCy plane. However, in another color space (color system), for example, another color space such as an L * a * b * color space. The value of may be used.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. The first embodiment has described the configuration for controlling the white extraction range based on the color information of the specular reflection component. In the second embodiment, the white extraction range is not corrected, and the extracted white (achromatic) pixel value is weighted based on the color of the specular reflection component.

  Since this embodiment can also be implemented in the digital camera 100 described in the first embodiment, a description of the configuration of the digital camera 100 is omitted, and a white balance gain calculation process unique to this embodiment is illustrated in FIG. This will be described with reference to FIGS.

FIG. 9 is a flowchart for explaining the white balance gain calculation process in the present embodiment. The same operation steps as those in the first embodiment are denoted by the same reference numerals as those in FIG.
If it is determined in S303 that the input image contains a predetermined amount or more of the specular reflection component, the system control unit 50 determines the white extraction weight based on the specular reflection component in S904. 10A shows an example of the white extraction range 1001 on the CxCy plane and the CxCy plane coordinates 1002 of the difference average values (aveR, aveG, aveB), and FIG. 10B shows an example of the relationship between the Cx coordinate values and the weights. , Respectively.

The system control unit 50 converts an average difference value (aveR, aveG, aveB), which is an average value of specular reflection components, into a CxCy color space value (Cx, Cy), and determines a weight based on the Cx coordinate value. Specifically, as shown in FIG. 10B, the CxCy plane coordinate (Cx, Cy) 1002 of the difference average value has a large weight (3.0) in the vicinity of the Cx coordinate value, and the coordinate value separated by a certain value or more. Determines a small weight (1.0). Since 1.0 is a weight that does not change the original value, it is equivalent to no weighting.
On the other hand, if it is determined in S303 that the input image does not contain a specular reflection component at a predetermined amount or more, the system control unit 50 sets a constant and small weight (1.0) as shown in FIG. 10C in S905. To do.

  Thus, in this embodiment, when the input image includes a specular reflection component at a predetermined amount or more, a weight greater than 1 is determined in the vicinity of the Cx coordinate value of the CxCy plane coordinate 1002 of the difference average value. In addition, when the input image does not include a specular reflection component at a predetermined amount or more, and when the input image includes a specular reflection component at a predetermined amount or more, the difference average value is not near the Cx coordinate value of the CxCy plane coordinate 1002. Also determine a weight of 1. As a result, the pixel value near the light source color estimated from the specular reflection component is weighted larger than 1.

In step S906, the system control unit 50 performs white extraction using the weights calculated in steps S904 and S905. Specifically, it is determined whether or not the color of the pixel falls within the white extraction range 1001 for all the pixels of the input image. The white extraction range 1001 is set according to the brightness of the subject in S302. The system control unit 50 obtains the weight w (Cx) corresponding to the Cx coordinate value of the pixel value (Cx, Cy) determined to fall within the white extraction range 1001, and calculates the weighted pixel value. In particular,
Weighted Cx = w (Cx) × Cx
Weighted Cy = w (Cx) × Cy
Is calculated. The system control unit 50 adds weighted pixel values (w (Cx) Cx, w (Cx) Cy) to a buffer that may be the system memory 126, for example. Further, the system control unit 50 adds the weight w (Cx) calculated for each pixel to the white pixel count that is a variable. For example, when the weight is 3.0, 3 is added to the white pixel count.

  In step S306, the system control unit 50 calculates a white balance gain based on the weighted white extraction result. Specifically, the total weighted pixel value stored in the buffer is divided by the white pixel count value for each Cx component and Cy component. This process corresponds to the calculation of the weighted average value of the pixels included in the white extraction range 1001. An example of the weighted average value (Cx, Cy) is shown at 1003 in FIG. The system control unit 50 uses a value obtained by converting the calculated weighted average value (Cx, Cy) into a value in the RGB color space, similarly to the white extraction average RGB value (wR, wG, wB) in the first embodiment. Calculate the white balance gain.

  In this embodiment, white extraction is performed by assigning a weight greater than 1 to a pixel value close to the specular reflection component extracted from the input image, thereby weighting a color that is likely to be close to the light source color. Is obtained. Therefore, as in the first embodiment, it is possible to improve the accuracy of the white balance gain in a situation where the accuracy of white extraction decreases, particularly when the subject is dark.

In the present embodiment, the pixel values near the average value of the specular reflection component are weighted greater than 1, but may be weighted by other methods. For example, a frequency distribution of the Cx coordinate value of the specular reflection component may be generated, and a greater weight may be given to a Cx coordinate value having a larger frequency distribution. Further, the weight may be reduced in a finer stepwise manner as the distance from the average Cx coordinate value increases.
In this embodiment, the weights of both the Cx component and the Cy component are determined based on only the Cx coordinate value. However, it is also possible to determine an independent weight for the Cy coordinate value and apply it to the Cy component.

  In FIG. 10B, there is no limitation on the setting method of the range in which the weight is set to 3.0, the size of the range in which the weight changes from 3.0 to 1.0, and the setting method of the maximum value of the weight. For example, it can be determined experimentally based on the accuracy of white balance gain obtained using images of various subjects photographed in a dark situation.

(Third embodiment)
Next, a third embodiment of the present invention will be described. In the second embodiment, the weight is determined based on the specular reflection component extracted from the entire image. However, the third embodiment is characterized in that the weight is determined and applied for each local region of the input image.

  Since this embodiment can also be implemented in the digital camera 100 described in the first embodiment, a description of the configuration of the digital camera 100 is omitted, and a white balance gain calculation process unique to this embodiment is illustrated in FIG. 11 will be used for explanation.

  FIG. 11 is a flowchart for explaining the white balance gain calculation processing in the present embodiment. The same operation steps as those in the first embodiment are denoted by the same reference numerals as those in FIG.

  In S1103, the system control unit 50 calculates an average color block_C (R, G, B) for each block. Here, the block is one of 8 × 8 mesh blocks as shown in FIG. 5A, as in the first and second embodiments. The system control unit 50 averages the RGB data of the input image accumulated in the image memory 106 in units of blocks to calculate an average color block_C (R, G, B), and calculates the average color block_C (R, G, B) is stored in the system memory 126.

S1104 and S1110 indicate that the processing of S1105 to S1109 is performed on all blocks in units of blocks.
In step S1105, the system control unit 50 converts the average color of the block into a value in the CxCy color space, and determines whether or not it is included in the white extraction range. FIG. 12A shows an example of the white extraction range 1210. If it is determined that the average color of the block is included in the white extraction range, the system control unit 50 advances the process to step S1106. On the other hand, if it is not determined that the average color of the block is included in the white extraction range, the system control unit 50 ends the process for this block, and performs the process from S1105 for the next unprocessed block. If there is no unprocessed block, the process proceeds to S1111.

  The processing of S1106 to S1109 is executed for blocks (white blocks) whose average color is included in the white extraction range. In step S1106, the system control unit 50 determines whether the processing target block includes a specular reflection component. Similar to the first embodiment, if the difference count value countS of the block to be processed is larger than the predetermined threshold T2, it can be determined that the block includes a specular reflection component. If the system control unit 50 determines that the processing target block includes a specular reflection component, the system control unit 50 proceeds to S1107. If the system control unit 50 determines that the processing target block does not include a specular reflection component, the system control unit 50 proceeds to S1108.

In step S1107, the system control unit 50 calculates a weight based on the difference between the specular reflection component and the block average color as the weight based on the relationship between the specular reflection component and the block average color.
Specifically, the system control unit 50 divides the difference total value sumS (R, G, B) of the processing target block by the associated count value countS, and the block average color block_S (R of the specular reflection component , G, B).

  The system control unit 50 converts the block average color block_S (R, G, B) and the block average color Block_C (R, G, B) of the specular reflection component into values (chromaticity coordinates) on the CxCy plane, and blocks_S (Cx , Cy), block_C (Cx, Cy). FIG. 12A shows the relationship between block_S (Cx, Cy) and block_C (Cx, Cy). In FIG. 12A, the input image 1200 is divided into 8 × 8 blocks. Here, the average color (block_S) and the block average value (block_C) of the specular reflection component of the block 1201 are 1203 and 1204. In addition, the average color (block_S) and the block average value (block_C) of the specular reflection component of the block 1202 are 1205 and 1206, respectively.

The system control unit 50 calculates a distance D between block_S (Cx, Cy) and block_C (Cx, Cy) calculated for the same block from the following formula, and determines a weight for each block based on the calculated distance.
D = | block_S (Cx)-block_C (Cx) | + | block_S (Cy)-block_C (Cy) |

  In this embodiment, as shown in FIG. 12B, the weight of the block unit is a large weight (here, 3.0) when the distance D is from 0 to a predetermined value, and thereafter, the weight decreases as the distance increases. In the end, control is performed to be 1.0. When the distance D is large, that is, when the difference between the average color of the specular reflection component and the average color of the pixel values in the same block is large, the white balance gain is calculated even if the average color of the pixels in the block is within the white extraction range. Sometimes it is better not to focus on it. On the other hand, if the distance D is small, the average color of the specular reflection component and the average color of the pixel values are close, and the block average color is likely to be white. Emphasize on calculation.

  On the other hand, in S1108, the system control unit 50 determines a weight for a block that includes a specular reflection component or is determined to be a block. In this embodiment, the system control unit 50 uniformly determines a weight of 1.5 for blocks determined not to include a specular reflection component. This weight value is an example, and an intermediate value of a weight range (here, 1.0 to 3.0) based on the relationship between the specular reflection component and the block average color as a value corresponding to medium reliability. Can be determined as appropriate.

  In S1109, the system control unit 50 multiplies the block average color block_C (R, G, B) by the weight determined in S1107 or S1108, and adds it to the buffer. Further, the weight value is added to the block total count value which is a variable. These values are used when calculating the weighted average value of the entire white block included in the image.

  After performing the processing of S1105 to S1109 for all the blocks, the system control unit 50 advances the processing to S1111. In S1111, the system control unit 50 divides the weighted addition value of the block average color added in S1109 by the block total count value, and calculates the weighted average value (R, G, B) of the entire white block included in the input image. To do. Subsequently, using this weighted average value (R, G, B) as the white extraction average RGB value (wR, wG, wB) of the first embodiment, the white balance gain is calculated in S306, and the white balance gain is calculated in S307. Apply.

  In the present embodiment, among the blocks that are partial areas of the input image, the average color and the average color of the specular reflection component are calculated for each white block whose average color is within the white extraction range, and the average color of the blocks having a small difference is calculated. The white extraction average of the input image is calculated with a greater weight. Therefore, the white balance gain can be calculated by weighting the average color close to the specular reflection component with a large weight, and the accuracy of the white balance gain can be improved. This embodiment is also very effective under conditions where the accuracy of white extraction is reduced, such as when the subject is dark.

  In the present embodiment, the color average value and the average value of the specular reflection component are compared in units of blocks. However, the size and shape of the partial areas are not limited as long as they are in units of partial areas.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. In the first to third embodiments, the white extraction range and the weight of the white extraction result are controlled based on the specular reflection component. In the fourth embodiment, the white balance gain range is controlled based on the specular reflection component. It is characterized by doing.

  Since this embodiment can also be implemented in the digital camera 100 described in the first embodiment, a description of the configuration of the digital camera 100 is omitted, and a white balance gain calculation process unique to this embodiment is illustrated in FIG. This will be described with reference to FIGS.

  FIG. 13 is a flowchart for explaining the white balance gain calculation processing in the present embodiment. The same operation steps as those in the first embodiment are denoted by the same reference numerals as those in FIG.

  In step S1405, the system control unit 50 determines the limit value of the white balance gain based on the brightness of the subject. The limit value of the white balance gain will be described with reference to FIG. FIG. 14A is a diagram showing an example of the relationship between R and B white balance gains (wR-Gain, wB-Gain). In the present embodiment, it is assumed that the white balance gains (wR-Gain, wB-Gain) of R and B are controlled to take values on the black body radiation axis 1501. In some cases, such as when the light source is a fluorescent lamp, the white balance gain may actually deviate from the black body radiation axis. However, for ease of explanation and understanding, the white balance gain is all black body. A case of having a value on the radial axis 1501 will be described. In FIG. 14A, Lo_lim is a limit value on the low color temperature side, and Hi_lim is a limit value on the high color temperature side.

  In S1405, the system control unit 50 sets the white balance gain limit values to Lo_lim1 and Hi_lim1 when the brightness of the subject (for example, the EV value) is dark below a predetermined threshold. Further, the system control unit 50 sets the limit value of the white balance gain to Lo_lim2 and Hi_lim2 when the brightness of the subject is larger than a predetermined threshold. The reason for this setting is that when the subject is bright, the light source has a high possibility of sunlight in the daytime, and the range that the white balance gain can take is limited to a narrow range corresponding to daylight in the daytime. On the other hand, when the subject is dark, various light sources may exist, and therefore the limit value is set so that the range that the white balance gain can take is wider than the range corresponding to daylight in the daytime.

  In S1406, the system control unit 50 determines whether or not the input image includes a specular reflection component in the same manner as in S303 of the first embodiment. If it is determined that the input image includes a specular reflection component, the system control unit 50 proceeds to S1407. If it is determined that it does not contain, the process proceeds to S1409.

  In step S1407, the system control unit 50 calculates a difference from the reference value of the specular reflection component. Specifically, the system control unit 50 calculates the difference between the average Cx value of the specular reflection component and the reference value Cx_base of Cx as the distance d_base. That is, the distance d_base = Cx−Cx_base.

  FIG. 14B shows an example of the distance d_base. FIG. 14B shows an example of a white extraction range 1502 on the CxCy plane and average values 1503 and 1504 of specular reflection components. FIG. 14B shows an example in which the reference value Cx_base = 0 is set. The distance for the average value 1503 of the specular reflection component is indicated by d_base1, and the distance for the average value 1504 of the specular reflection component is indicated by d_base2. The reference value Cx_base is a threshold value for determining whether the color temperature of the specular reflection component is high or low, and other values may be used. For example, the reference value may be a Cx value of sunlight (D50 light source) other than Cx_base = 0. If the color temperature of the specular reflection component is determined to be high, the limit value on the high color temperature side of the white balance gain is controlled, and if the color temperature of the specular reflection component is determined to be low, the low color temperature side of the white balance gain is controlled. Control limit values.

  In step S1408, the system control unit 50 corrects the white balance gain limit value calculated in step S1405 based on the distance d_base calculated in step S1407. This process will be described with reference to FIG. 14C and FIG.

  FIG. 14C is a diagram showing an example of the relationship between R and B white balance gains (wR-Gain, wB-Gain) as in FIG. 14A, but the low color temperature limit value Lo_lim3 and high A color temperature limit value Hi_lim3 has been added. The system control unit 50 corrects the limit value of the white balance gain based on the distance d_base calculated in S1407 and the characteristics shown in FIG.

  15A shows a case where the distance d_base is positive (when the specular reflection component is a low color temperature), and FIG. 15B shows a case where the distance d_base is negative (when the specular reflection component is a high color temperature). The control characteristic of the limit value is shown. When the distance d_base is positive, the color temperature of the specular reflection component is low. Therefore, the system control unit 50 decreases the limit value on the high color temperature side as the distance d_base increases according to the characteristics of FIG. To control. That is, since there is a high possibility that the light source has a low color temperature due to the specular reflection component, control is performed to reduce the limit value on the high color temperature side so that it is difficult to capture colors on the higher color temperature side. Then, the system control unit 50 compares the limit value on the high color temperature side according to the distance d_base with the limit value Hi_lim on the high color temperature side based on the brightness determined in S1405, and the smaller one is the final high color It is determined as the temperature limit value.

  On the other hand, when the distance d_base is negative, the color temperature of the specular reflection component is high. Therefore, the system control unit 50 limits the lower color temperature side as the absolute value of the distance d_base increases according to the characteristics of FIG. Control to increase the value. That is, since there is a high possibility that the light source has a high color temperature due to the specular reflection component, control is performed to increase the limit value on the low color temperature side so that it is difficult to capture colors on the low color temperature side. Then, the system control unit 50 compares the limit value on the low color temperature side corresponding to the distance d_base with the limit value Lo_lim on the low color temperature side based on the brightness determined in S1405, and the larger one is the final high color It is determined as the temperature limit value.

  In step S1409, the system control unit 50 performs limit processing for correcting the calculated white balance gain as necessary so that the calculated white balance gain is within a range determined by the corrected limit value. For example, in FIG. 14A, the limit process when the white balance gain calculated in S306 is indicated by 1505 will be described. Since the white balance gain 1505 is not a value on the black body radiation axis 1501, the system control unit 50 first corrects only the white balance gain (B gain) of B to a value 1506 on the black body radiation axis 1501. Next, the system control unit 50 determines whether or not the corrected white balance gain 1506 is included in the range determined by the limit value. Here, it is assumed that the limit values of the white balance gain are Lo_lim2 and Hi_lim2. In this case, since the white balance gain 1506 corrected to the value on the blackbody radiation axis 1501 is equal to or smaller than Lo_lim2, the white balance gain 1507 corresponding to Lo_lim2 is corrected on the blackbody radiation axis 1501. On the other hand, when the white balance gain calculated in S306 is equal to or higher than Hi_lim2 on the black body radiation axis 1501, the system control unit 50 corrects the white balance gain 1508 corresponding to Hi_Lim2 on the black body radiation axis 1501. .

  If the white balance gain calculated in S306 is a value on the blackbody radiation axis 1501 and is included in the range determined by the limit value, the system control unit 50 does not correct the white balance gain calculated in S306 without any correction. Use.

  In step S307, the system control unit 50 sets the white balance gain calculated in step S306 or the white balance gain corrected in step S1409 in the WB amplification unit 201 (FIG. 2).

In this embodiment, the white balance gain calculated based on the white extraction is corrected based on the specular reflection component information of the input image. Specifically, the range of the white balance gain is controlled in consideration of the color of the specular reflection component.
By using the specular reflection component information indicating the light source color to control the range of white balance gain based on white extraction, it is possible to suppress the white balance gain from becoming an inappropriate value. Accuracy can be improved.

  In the present embodiment, the configuration in which the range of the white balance gain is controlled according to the color temperature of the specular reflection component is described. However, the limit value that defines the range is determined by using the color information of the specular reflection component. You may set by the method.

  In this embodiment, the limit value is set while setting the white balance gain value on the blackbody radiation axis. However, the present invention is not limited to the configuration for controlling the range in the direction along the blackbody radiation axis. . For example, it is possible to use a configuration in which the range of white balance gain in the direction orthogonal to the blackbody radiation axis is controlled using color information of the specular reflection component.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (16)

Setting means for setting a white extraction range in a predetermined color space;
First extraction means for extracting a specular reflection component included in the input image;
Correction means for correcting the white extraction range based on the extracted specular reflection component;
Second extraction means for extracting pixels having a color included in the corrected white extraction range from the input image;
Calculation means for calculating a white balance gain for the input image based on the extracted pixel value;
An image processing apparatus comprising:   The image processing according to claim 1, wherein the correction unit corrects the white extraction range based on coordinates in the color space of the specular reflection component so as to be smaller than a range set by the setting unit. apparatus.   The correction unit corrects the white extraction range to a region where a region including coordinates of the specular reflection component in the color space overlaps with the white extraction range set by the setting unit. The image processing apparatus according to claim 1 or 2.   The correction means performs the correction when it is determined that the specular reflection component is included in the input image, and performs the correction when it is determined that the input image does not include the specular reflection component. The image processing apparatus according to claim 1, wherein the image processing apparatus is not provided. Setting means for setting a white extraction range in a predetermined color space;
First extraction means for extracting a specular reflection component included in the input image;
Second extraction means for extracting pixels having a color included in the white extraction range from the input image;
Correction means for weighting the color of the pixel extracted by the second extraction means based on the relationship with the extracted specular reflection component;
Calculation means for calculating a white balance gain for the input image based on the weighted pixel color;
Have
The image processing apparatus according to claim 1, wherein the correction unit weights a color of a pixel having a smaller distance from the specular reflection component in the color space. The first extraction means extracts the specular reflection component for each partial region of the input image;
The second extracting means extracts the pixel for each partial region;
The correction means performs the weighting for each of the partial areas;
The image processing apparatus according to claim 5.   The correction means performs weighting based on the relationship with the extracted specular reflection component for the partial region in which the average color is included in the white extraction range and the specular reflection component is included in the partial region. The image processing apparatus according to claim 6.   The correction means weights a partial area in which the average color is included in the white extraction range of the partial area but does not include a specular reflection component based on a relationship with the extracted specular reflection component. The image processing apparatus according to claim 7, wherein weighting is uniformly performed with an intermediate value in a range of weights according to claim 8.   The said correction | amendment means does not perform the said weighting with respect to the partial area | region where an average color is not contained in the said white extraction range among the said partial area | regions. An image processing apparatus according to 1. First extraction means for extracting pixels having a color included in a white extraction range in a predetermined color space from an input image;
Calculation means for calculating a white balance gain for the input image based on the extracted pixel value;
Second extraction means for extracting a specular reflection component included in the input image;
Determining means for determining a limit value of white balance gain based on the brightness of the input image and the extracted specular reflection component;
Correction means for correcting the white balance gain calculated by the calculation means so as to be within the range of the limit value;
An image processing apparatus comprising:   The image processing apparatus according to claim 10, wherein the determination unit determines the limit value such that the range of the limit value becomes narrower as the input image becomes brighter. The determination means is configured such that the higher the color temperature of the specular reflection component, the smaller the limit value on the high color temperature side, and the lower the color temperature of the specular reflection component, the larger the limit value on the low color temperature side. Correct the limit value based on the brightness of the image,
12. The image processing apparatus according to claim 11, wherein the correction unit corrects the white balance gain calculated by the calculation unit so as to be within the range of the corrected limit value. A setting step in which the setting means sets a white extraction range in a predetermined color space;
A first extraction means for extracting a specular reflection component included in the input image;
A correcting step for correcting the white extraction range based on the extracted specular reflection component;
A second extraction step in which a second extraction unit extracts pixels having a color included in the corrected white extraction range from the input image;
A calculating step of calculating a white balance gain for the input image based on the value of the extracted pixel;
A control method for an image processing apparatus, comprising: A setting step in which the setting means sets a white extraction range in a predetermined color space;
A first extraction means for extracting a specular reflection component included in the input image;
A second extraction step in which a second extraction unit extracts pixels having a color included in the white extraction range from the input image;
A correction step in which the correction means weights the color of the pixel extracted by the second extraction means based on the relationship with the extracted specular reflection component;
A calculating step of calculating a white balance gain for the input image based on the weighted pixel color;
Have
The image processing apparatus control method according to claim 1, wherein in the correction step, the correction unit performs greater weighting on a color of a pixel having a smaller distance from the specular reflection component in the color space. A first extraction unit for extracting pixels having a color included in a white extraction range in a predetermined color space from an input image;
A calculating step of calculating a white balance gain for the input image based on the value of the extracted pixel;
A second extraction unit for extracting a specular reflection component included in the input image;
A determining step for determining a limit value of a white balance gain based on the brightness of the input image and the extracted specular reflection component;
A correcting step for correcting the white balance gain calculated by the calculating unit so as to be within the range of the limit value;
A control method for an image processing apparatus, comprising:   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1-10.