JP2017068513A - Image processing device and method thereof, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device with which it is possible to perform appropriate gradation control while maintaining the local contrast of an image, with a photographed scene taken into account.SOLUTION: The image processing device comprises: a division unit for dividing an inputted image into a plurality of regions; a determination unit for determining gradation characteristics for applying gradation correction to the image on the basis of luminance information on each of a plurality of regions divided by the division unit; a generation unit for generating a plurality of images differing in frequency band from the inputted image; a conversion unit for converting the signals of the plurality of generated images differing in frequency band into gains on the basis of the gradation characteristics; an addition unit for adding up the gain-converted signals of the plurality of images differing in frequency band that are weighted; and a correction unit for multiplying the image by the gain added up by the addition means and thereby correcting the image. The addition unit determines weighting of the signals of plurality of images differing in frequency band on the basis of at least one of a photographing condition when the inputted image was photographed and information on the plurality of regions.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an exposure control technique in an imaging apparatus such as a digital camera and an image processing technique for performing gradation correction on a signal after photographing.

  2. Description of the Related Art Conventionally, processing for gradation-compressing a signal with an expanded dynamic range of an input signal, such as dodging processing for correcting a partial contrast of an image or HDR (High Dynamic Range) processing, is known. In such a technique, a method of outputting an image retaining the atmosphere of the scene by further applying the result of determining the scene at the time of shooting to exposure control and gradation correction is known.

  Japanese Patent Application Laid-Open No. 2004-151561 discloses a technique for setting gradation correction processing conditions for an input image based on a scene discrimination result. Specifically, for example, a method is disclosed in which gamma characteristics and offset correction are applied so as to lower the luminance value when the main subject is too bright. Patent Document 2 discloses a technique for controlling a correction amount based on a difference in dodging correction effects based on different frequency components.

JP 2007-293528 A JP 2010-244360 A

  However, with the technique disclosed in Patent Document 1, even when information that identifies a scene is used, tone control based on gamma characteristics results in correction that is sensitive to pixel values, and local contrast may be reduced.

  In the technique disclosed in Patent Document 2, for example, if the correction effect by the low frequency component is suppressed for a low luminance part adjacent to the high luminance, the correction effect is sensitive to the pixel value, resulting in a decrease in contrast. there's a possibility that.

  The present invention has been made in view of the above-described problems, and an object thereof is to perform appropriate gradation control while maintaining local contrast of an image in consideration of a photographed scene. An image processing apparatus is provided.

  An image processing apparatus according to the present invention includes a dividing unit that divides an input image into a plurality of regions, and gradation correction for the image based on luminance information of each of the plurality of regions divided by the dividing unit. Determining means for determining gradation characteristics for performing image generation, generating means for generating a plurality of images having different frequency bands from the input image, and signals of the generated images having different frequency bands from Conversion means for converting to gain based on tonal characteristics, addition means for weighting and adding signals of a plurality of images having different frequency bands converted to gain, and gain added to the image by the addition means Correcting means for correcting the image by multiplying, wherein the adding means reduces a shooting condition when the input image is shot and information on the plurality of areas. It is based on one, and determines the weighting of the signals of multiple different images of the frequency band.

  According to the present invention, it is possible to provide an image processing apparatus that can perform appropriate gradation control while maintaining local contrast of an image in consideration of a photographed scene.

1 is a diagram showing a configuration of a digital camera that is a first embodiment of an image processing apparatus of the present invention. FIG. FIG. 2 is a block diagram showing a configuration of an image processing unit 104. 6 is a flowchart showing a flow of processing up to determining an exposure amount for main photographing when a photographing instruction is given by a photographer. The figure which shows an example of the weight at the time of luminance value calculation. The figure which shows the example of an input image. The figure which shows an area | region division result. The figure which shows the exposure amount calculation method in 1st Embodiment. 6 is a flowchart showing a flow of gradation correction processing performed on a real captured image. The figure which shows the gain amount calculation method in 1st Embodiment. The figure which shows the calculation method of the gain application range in 1st Embodiment. FIG. 6 is a diagram illustrating an example of gradation characteristics according to the first embodiment. The block diagram which shows the structure of the gain process part in 1st Embodiment, a gain image generation part, and a gain addition process part. The figure which shows an example of the bad effect which generate | occur | produces in the brightness | luminance boundary in a dodging process. The figure which shows the gain addition process in 1st Embodiment. The figure which shows the gain addition process in 1st Embodiment. The flowchart which shows the processing content of a 2nd addition process part. The figure which shows an example of the bad effect which generate | occur | produces in the brightness | luminance boundary in a dodging process. 10 is a flowchart illustrating an operation of the image processing apparatus according to the second embodiment. The figure which shows an example of the upper hierarchy and lower hierarchy gain in 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
In the first embodiment, exposure control is performed using information related to a shooting scene such as information on a subject region and information related to shooting conditions, and gradation correction processing is performed by dodging processing. In the present embodiment, a description will be given assuming a night view, which is one of the scenes where an effect can be expected by using the method of the present embodiment. However, the method of the present embodiment can be applied to scenes other than night views.

  FIG. 1 is a diagram showing a configuration of a digital camera which is a first embodiment of an image processing apparatus of the present invention. In FIG. 1, an optical system 101 includes a lens group including a zoom lens and a focus lens, a diaphragm, and a shutter. The optical system 101 adjusts the magnification, focus position, or light amount of the subject image formed on the imaging unit 102. The imaging unit 102 includes an imaging element such as a CCD or a CMOS sensor that photoelectrically converts a light beam from a subject that has passed through the optical system 101 and converts it into an electrical signal.

  The A / D converter 103 converts the input analog video signal into a digital video signal. The image processing unit 104 performs gradation correction processing in the present embodiment in addition to normal signal processing. The image processing unit 104 can perform similar image processing not only on the image output from the A / D conversion unit 103 but also on the image read out from the recording unit 110.

  The exposure amount calculation unit 105 is a part that calculates an exposure amount at the time of shooting in order to obtain an input image suitable for performing the gradation correction processing of the present embodiment. An exposure amount is calculated using the processing result of the image processing unit 104 as an input, and input to the exposure amount control unit 106. In order to realize the exposure amount calculated by the exposure amount calculation unit 105, the exposure amount control unit 106 sends a command to the optical system 101 and the imaging unit 102 to control the aperture value, shutter speed, and analog gain of the imaging element. .

  The display unit 107 functions as an electronic viewfinder (EVF) by sequentially displaying an image output from the image processing unit 104 on a display device such as an LCD. The recording unit 110 has a function of recording an image, and may include, for example, an information recording medium using a memory card on which a semiconductor memory is mounted, a package containing a rotating recording body such as a magneto-optical disk, or the like. .

  FIG. 2 is a block diagram illustrating a configuration of the image processing unit 104. Hereinafter, processing performed by each block of the image processing unit 104 will be described together with the flow thereof. First, an exposure amount calculation unit 105 that controls exposure before actual photographing based on luminance information in the screen and calculates an appropriate exposure amount during main photographing for performing gradation correction processing, and a gradation characteristic calculating unit The processing content of 204 will be described.

  FIG. 3 is a flowchart showing the flow of processing until the exposure amount for the main photographing when the photographing instruction is given by the photographer.

  First, in step S601, the exposure amount calculation unit 105 continuously acquires images for calculating the exposure amount by the live view operation, and calculates the luminance value of the entire image at a predetermined interval (for example, once every two frames). And change the exposure. Specifically, the image is divided into a plurality of block areas as shown in FIG. 4, and the average luminance value of each block is obtained. Then, the luminance value of the entire screen is obtained by performing weighted addition so as to weight the central block with respect to the average luminance of each block. The amount of exposure is determined so that the weighted and added luminance value of the entire screen approaches an appropriate value, that is, a preset target luminance value (for example, 130 at 10 bits).

  FIG. 5 is a diagram illustrating an example of an image obtained when exposure is performed with the exposure amount obtained in step S601. This is an image showing a night view scene, while high brightness areas such as the interior of a building and point light sources are whiteout, while most of the other background areas are blacked out and have no gradation. .

  In step S602, the subject area is determined whether the subject area detection unit 201 is a high-luminance area or a low-luminance area from the image acquired with the exposure amount obtained in step S601. In the present embodiment, the integral value of luminance is calculated for each block area as shown in FIG. The block is determined as a low luminance area. FIG. 6 is a diagram illustrating an example of a result of performing region discrimination on an input image. In the figure, the shaded portion indicates the high luminance region, and the black portion indicates the low luminance region.

  In step S603, the exposure amount calculation unit 105 determines the main exposure amount at the time of shooting based on the distribution of the high luminance region and the low luminance region determined in step S602. FIG. 7 is a diagram showing a method for determining the exposure amount, in which the horizontal axis of the graph represents the ratio of the high luminance area to the entire image, and the vertical axis of the graph represents the amount of change in exposure with respect to the appropriate exposure amount. . Specifically, the ratio on the horizontal axis is a value obtained by dividing the number of blocks in the high luminance area by the total number of divided blocks and multiplying by 100, and the exposure change amount is an appropriate exposure that achieves the target luminance value described above. It is a value that represents how much the exposure changes from the amount. As shown in the drawing, the underexposure is controlled with respect to the appropriate exposure as the ratio of the high luminance region is larger. The purpose of this is to suppress the amount of gain so that the noise component does not increase by the gain processing described later, while avoiding losing the gradation of a high-luminance subject (such as a light source or the inside of a building) that is easily overexposed in a night scene. is there.

  In step S604, the actual exposure is performed by the exposure control unit 106 based on the exposure determined in step S603. Thus, the description of the processing from the acquisition of the input image to the calculation of the exposure amount by the exposure amount calculation unit 105 until the actual photographing is finished.

  Next, FIG. 8 is a flowchart showing the flow of gradation correction processing performed on the image actually captured in step S604 of FIG.

  In step S701, the image captured in step S604 is acquired. In step S702, similarly to step S602, a high luminance region and a low luminance region are determined from the input image. Since the determination method is the same as that in step S602, description thereof is omitted.

  In step S703, the gradation characteristic calculation unit 204 performs a process of generating gradation characteristics that are uniformly applied to the entire screen. First, a gain amount and a luminance range to which the gain amount is applied are calculated for each of the high luminance region and the low luminance region detected in step S702. The gain LG for the low luminance region is calculated based on the difference between the exposure at the time of actual photographing and the appropriate exposure determined in step S603. FIG. 9 is a diagram illustrating a method of calculating the gain amount LG for the low luminance region. When the exposure at the time of actual photographing is larger than the appropriate exposure, the gain amount is set to 1 time or less (that is, gain down), and when the exposure is small. The gain amount is set to 1 or more (that is, gain up). This is to prevent the dark portion from being brightened more than necessary in the night scene and making it look unnatural and preventing noise from becoming noticeable due to gain increase. Further, the gain amount HG for the high luminance region is set to 1 time (that is, equal gain) in order to suppress overexposure. Subsequently, a luminance range to which the calculated gain amounts LG and HG are applied is calculated. FIG. 10 is a diagram showing the calculation method. First, the region-specific luminance value calculation unit 203 calculates histograms for each of the low luminance region and the high luminance region. FIG. 10A is a diagram showing a luminance histogram of a low luminance region, where the horizontal axis represents the luminance value and the vertical axis represents the frequency of the corresponding luminance value. In FIG. 10A, first, the frequency number is cumulatively added from the minimum luminance value. FIG. 10B is a diagram in which the cumulative addition value is plotted on the vertical axis, and the luminance value when the cumulative addition value reaches a predetermined value (LOW_TH) is the upper limit luminance value of the gain application range for the low luminance region. LP. Similarly, the frequency number of the luminance histogram of the high luminance region is cumulatively added from the high luminance side, and the luminance value when reaching a predetermined value (HI_TH) is the lower limit luminance value HP of the gain application range for the high luminance region. And

FIG. 11 shows the gradation characteristics generated based on these four values LG, HG, LP, HP. In the figure, the horizontal axis represents input luminance, and the vertical axis represents output luminance. The slope of the straight line represents the gain amount, and the gradation characteristic is expressed by an equation using the gain amount GAIN.
Gradation characteristics: Output Y = GAIN × input Y (1)
It becomes. A straight line indicated by a dotted line indicates a 1 × gain (equal gain), a solid line LG is a gain reduction, and a solid line HG is an equal gain. Using this gradation characteristic, the gain table generation unit 205 generates a gain table for each luminance, that is, a table representing a gain amount with respect to input luminance.

  In step S704, a process for generating a gain image is performed. FIG. 12 is a block diagram illustrating the configuration of the gain processing unit 202 that executes the generation of the gain image in step S704 and the gain processing in step S705. Hereinafter, the processing in steps S704 and S705 will be described with reference to FIG.

  FIG. 12A shows the configuration of the gain processing unit 202, in which 1201 is an input image, 1202 is a gain image generation unit, and 1203 is a gain multiplication unit. Further, FIG. 12B shows the configuration of the gain image generation unit 1202. In the figure, a luminance signal generator 1212 generates a luminance signal from an input image 1201. The band limiting unit 1213 applies a predetermined low pass filter (LPF) to the luminance signal output from the luminance signal generating unit 1212 in order to limit the frequency band of the image. As a result, an image signal of a hierarchy including a high frequency band output from the luminance signal generation unit 1212 is input to the blurring processing unit 1214 described later, and only the low frequency band that has passed through the band limiting unit 1213 is input to the blurring processing unit 1215. The image signal of the hierarchy that leaves is input. Although a low-pass filter is used here as a band limiting method, the present invention is not limited to this, and may be realized by performing a resizing process such as reduction or enlargement.

  The blurring processing units 1214 and 1215 execute predetermined blurring processing on the layer image including the high frequency band from the luminance signal generation unit 1212 and the layer image remaining only from the low frequency band from the band limiting unit 1213. This is intended to reduce gain sensitivity to fine textures and maintain better contrast. On the other hand, for example, it is better not to perform the blurring process at the boundary between the high luminance region and the low luminance region as shown in FIG. The reason for this is that, in the dodging gain process, if the blur process is performed at such a boundary part, it is considered that the generation of a pseudo contour becomes conspicuous without applying a desired gain to the area near the boundary part. . Therefore, it is necessary to perform the blurring process while keeping an edge having a large luminance difference.

The luminance → gain conversion processing units 1216 and 1217 convert each luminance image output from the blur processing units 1214 and 1215 into a gain image using the luminance-specific gain table 1210 calculated by the gain table generation unit 205. The gain image is output from the input signal Yin (x, y), where (0, 0) is the pixel position located in the upper left corner of the image and (x, y) is the pixel position of the captured input image. It is an image which made gain to be Gain (x, y). That is, when the gain table function GainTbl shown below is used,
Gain (x, y) = GainTbl (Yin (x, y)) (2)
Gain (x, y) in the gain image represented by is a gain corresponding to the pixel located at (x, y) of the photographed input image.

  The image size adjustment unit 1218 adjusts the output image size of the luminance → gain conversion processing unit 1217 to be the same as that of the input image 1201 when the band limiting unit 1213 performs a process of changing the size of the input image. Process. In the present embodiment, the band limiter 1213 does not change the image size, and only the low-pass filter is applied, so the size is not adjusted.

  The gain addition processing unit 1219 performs processing for adding the output gain images of the luminance → gain conversion processing unit 1216 and the image size adjustment unit 1218 for each pixel. FIG. 12C is a diagram illustrating a configuration of the gain addition processing unit 1219. A gain image generated from an image of a layer including a relatively high frequency band that is not subjected to band limitation on the input luminance and a gain image generated from an image of a layer leaving only the low frequency band are added. Hereinafter, each of these will be referred to as an upper layer image 1251 and a lower layer image 1252, and the gain addition processing will be described in detail.

  As shown in FIG. 13, simply adding and averaging the gains of the upper layer and the lower layer generates a pseudo contour depending on the shape of the gain table near the edge portion where the luminance difference is large. It is necessary to devise.

  FIG. 14 is a diagram schematically illustrating the first addition processing unit 1253, the second addition processing unit 1254, and the weighted addition unit 1256. As shown in FIG. 14, the first addition processing unit 1253 multiplies the gain of the target position (target pixel) in the upper hierarchy (hereinafter referred to as the target gain) by a predetermined weight W1 and the gain of the target position in the lower hierarchy. Add weight. However, the pseudo contour as described above may be generated only by this result. A method for calculating the weight W1 will be described later.

  Next, processing contents in the second addition processing unit 1254 will be described. As shown in FIG. 14, the second addition processing unit 1254 uses a higher-level target gain and a gain in a peripheral area in the lower hierarchy (hereinafter referred to as a peripheral gain). Here, the peripheral region is a set of pixels within a predetermined range in the vertical and horizontal directions from the position of interest. The purpose of the second addition processing unit 1254 is to reduce the aforementioned pseudo contour. FIG. 15 is a diagram showing the shape of the upper layer gain and the lower layer gain when the pseudo contour is generated near the edge portion and the processing image of the second addition processing unit 1254 in the same way as FIG. In the example shown in FIG. 15, the lower layer gain becomes extremely small in the vicinity of the edge portion where the luminance difference is large. Therefore, when gains are added at the position X, if the gains of interest indicated by square marks are added together, the adverse effect of the pseudo contour increases. Therefore, it is better to search for a range of L from the position X to the left and right from the position X, and select a gain having a value closest to the target gain of the upper hierarchy, not the position of interest. In the example of the figure, by using the gain at the position XL indicated by a circle, it is possible to reduce the influence received from an extremely small gain in the lower layer. By the above addition processing, generation of a pseudo contour due to multiplication of extremely different gains in the vicinity of the edge portion is suppressed. In FIG. 15, a one-dimensional example is described for easy understanding, but the actual second addition processing unit 1254 performs the same processing two-dimensionally.

  FIG. 16 is a flowchart showing the processing contents of the second addition processing unit. In step S1601, for each of the peripheral gains in the lower layer, an absolute difference value from the target gain in the upper layer is calculated. In step S1602, M peripheral gains are selected in ascending order of the absolute difference value calculated in step S1601. Here, M is a predetermined number. By this processing, as shown in FIG. 15, it is possible to select the lower-layer peripheral gain having a gain value close to the target gain of the upper layer. In the present embodiment, when the peripheral gain is selected, a predetermined number is selected in ascending order of the absolute difference value. However, another method may be used. For example, in order to suppress the calculation cost, a method may be used in which all those whose difference absolute values are equal to or less than a predetermined threshold value are selected.

  In step S1603, the M peripheral gains selected in step S1602 are weighted and added. Here, the weight at the time of weighted addition is determined by the following equation (1).

W (k) = A / | GL (k) −GH | (3)
In Expression (1), GL (k) represents the k-th gain among the peripheral gains of the selected lower layer, and W (k) represents the weight of weighted addition corresponding thereto. GH represents the gain of interest in the upper layer, and A is a predetermined constant. As can be seen from equation (1), the weight W (k) increases as the difference from the target gain in the upper layer is smaller.

  Further, after calculating the weight W (k) for each peripheral gain by the equation (1), the weighted addition is performed by the following equation (4).

... (4)
In Expression (2), GL ′ is a lower layer gain after weighted addition.

  In step S1604, the lower layer peripheral gain GL ′ obtained by weighted addition and the higher layer target gain GH multiplied by the predetermined weight W2 are weighted and the added gain value is the second addition. This is an output of the processing unit 1254.

  Here, a calculation method of the weight W1 used in the first addition processing unit 1253 and the weight W2 used in the second addition processing unit 1254 will be described. FIG. 17 is a diagram showing a region where the input luminance is saturated and the gain signals of the upper and lower layers corresponding thereto. Referring to FIG. 17, the upper layer has a gain reduction for the low luminance part, and the high luminance part has the same gain, whereas the lower layer has not only the low luminance part but also the high luminance region. It can be seen that the gain is also reduced. This is because the lower layer has a gain that is less sensitive to luminance due to band limitation. As a result, the high luminance around the saturated portion is reduced in gain, resulting in a problem that the balance of RGB does not become a desired value.

  Therefore, in the present embodiment, it is considered that the above-described adverse effects are suppressed by determining the two weights W1 and W2 based on several evaluation values. Specifically, the values of the weights W1 and W2 applied to the upper layer are increased as the Bv (bright value) value calculated by the exposure amount calculation unit 105 before photographing is increased and as the area ratio of the high luminance region is increased. To control. This is because the smaller the Bv value at the time of shooting, the higher the possibility that the scene is a night scene, and the smaller the area of the high luminance region, the more likely the tone characteristics are to reduce the gain. The purpose is to reduce the weight of the lower layer so that the down does not occur.

  The weighted addition unit 1256 performs weighted addition of the result of the first addition processing unit 1253 and the result of the second addition processing unit 1254, and the weight at that time is based on the gain difference between the upper layer and the lower layer at the position of interest. → Determined by the weight calculation unit 1255. Specifically, the setting is made such that the result of the second addition processing unit 1254 is emphasized as the difference value between the target gains of the upper layer and the lower layer is larger. This is because it is desirable to suppress the pseudo contour using the result of the second addition processing unit 1254 in the vicinity of the edge portion where the luminance difference is large and the gain difference is large. On the other hand, in the texture portion where the gain difference is small, if the result of the second addition processing unit 1254 is used, the contrast of the image is lowered. If the gain difference is small, the pseudo contour does not appear, so it is desirable to suppress the decrease in contrast using the result of the first addition processing unit 1253.

  Above, description of the gain addition process part 1219 and the gain image generation part 1202 in step S704 of FIG. 8 is complete | finished. As a result, an output gain image 1221 is obtained.

  Returning to the description of FIG. 8, in step S705, the gain image generated by the gain image generation unit 1202 is multiplied by the original image subjected to predetermined signal processing by the gain multiplication unit 1203. The output of the gain multiplication unit 1203 is tone-compressed to a predetermined output range by a tone compression processing unit 206 by gamma conversion processing or the like. Finally, the image subjected to the gradation compression in the gradation compression processing unit 206 is output to the display unit 107 and the recording unit 110.

  As described above, according to the present embodiment, in the process of locally controlling the gradation characteristics, the pseudo contour due to the use of a low-frequency image and the color balance of the saturated portion that is likely to occur mainly in night scenes. Detrimental effects such as deviation can be suppressed. Further, based on the gain difference of the position of interest, (1) weighted addition of each of the first addition result prioritizing contrast maintenance and (2) the second addition result prioritizing suppression of pseudo gradation, It is possible to suppress the pseudo gradation while suppressing the decrease in contrast.

  In the present embodiment, the upper layer image and the lower layer image having two types of frequency bands are generated from the input image during the gain addition process. However, the present invention is not limited to this, and images having three or more different frequency bands are generated. They may be generated and sequentially added.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. FIG. 18 is a flowchart illustrating the operation of the image processing apparatus according to the second embodiment. In the present embodiment, an example in a scene where a distant subject area is greatly blurred by focusing on a close subject will be described. Note that a description of processes similar to those in the first embodiment and processes using known techniques is omitted, and the characteristic processes of the present embodiment are mainly described. FIG. 18A is a flowchart showing a flow of processing up to determining the exposure amount of the main photographing when the photographing instruction is given by the photographer.

  First, in step S1801, images for calculating an exposure amount are continuously acquired as in step S601 in FIG. In steps S1802 to S1803, the subject area is determined by incorporating the concept of area-specific tone mapping, and the exposure amount is determined. That is, the input image is divided into a sky region and a background region using feature amounts such as luminance, hue, and saturation, and the main exposure amount is determined based on the representative luminance value and the luminance value step of each region. In step S1804, actual exposure is performed by the exposure control unit 106 based on the exposure determined in step S1803.

  Next, FIG. 18B is a flowchart showing the flow of gradation correction processing performed on the actually captured image. In step S1811, the image actually captured in step S1804 is acquired. In steps S1812 to S1813, the tone characteristic calculation unit 204 generates the tone characteristics in consideration of the brightness of the subject by incorporating the above-described tone mapping by area. That is, based on the brightness information of the sky area and the background area calculated by the area-specific brightness value calculation unit 203, a gradation is generated so that each brightness and its balance are appropriate.

  In step S1814, a process for generating a gain image is performed. Hereinafter, the process of step S1814 will be described with reference to FIG. First, in the gain image generation unit 1202, processing performed by parts other than the gain addition processing unit 1219 is the same as that in the first embodiment, and thus description thereof is omitted. Two gain images having different bands, that is, an upper layer image 1251 and a lower layer image 1252 are input to the gain addition processing unit 1219. The first addition processing unit 1253 multiplies the target gain of the upper layer by a predetermined weight W1, and then performs weighted addition with the target gain of the lower layer. A method for calculating the weight W1 will be described later.

  FIG. 18C is a flowchart showing the flow of processing performed by the second weighted addition unit. Steps S1821 to S1824 in the figure are the same as steps S1601 to S1604 of FIG. 16 described in the first embodiment, and thus description thereof is omitted. In step S1824, the weighted addition of the lower layer peripheral gain GL ′ and the higher layer of interest gain GH multiplied by the predetermined weight W2 are weighted and the added gain value is the second addition processing unit 1254. Output.

  Here, a calculation method of the weight W1 used in the first addition processing unit 1253 and the weight W2 used in the second addition processing unit 1254 will be described. FIG. 19 is a diagram illustrating a part of the background area where the boundary between the high luminance part and the low luminance part is blurred in the input luminance, and the upper layer and lower layer gain signals corresponding thereto. As can be seen, the gain value in the blurred area of the boundary changes smoothly in the lower layer, whereas the gain value changes sharply at the boundary of the predetermined luminance value in the upper layer. Recognize. This is because the gain value changes sharply with a predetermined luminance value as a boundary in the gradation characteristics generated in step S1813. As the higher-layer gain is applied to this boundary portion, an unnatural brightness switch occurs in the image after gain application.

  Therefore, in the present embodiment, it is considered to suppress the above-described adverse effects by determining the two weights W1 and W2 based on several evaluation values. Specifically, control is performed such that the lower layer weights W1 and W2 are decreased as the aperture value set by the exposure amount control unit 106 during actual photographing is smaller and as the focal length during focusing is shorter. This is because the closer the aperture is to the open value and the shorter the focal distance, the more easily the area other than the focus distance is blurred, and the above-mentioned adverse effects are likely to occur, so the weight of the lower layer where the gain changes smoothly is increased. The purpose is to suppress the occurrence of harmful effects.

  Above, description of operation | movement of the gain addition process part 1219 and the gain image generation part 1202 in step S1814 is complete | finished. Thereby, an output gain image is obtained.

  Returning to the description of FIG. 18B, in step S1815, the gain image generated by the gain image generation unit 1202 is multiplied by the original image subjected to the predetermined signal processing in the gain multiplication unit 1203. The output of the gain multiplication unit 1203 is tone-compressed to a predetermined output range by a tone compression processing unit 206 by gamma conversion processing or the like. Finally, the image subjected to the gradation compression in the gradation compression processing unit 206 is output to the image display unit 107 and the image recording unit 110.

  As described above, according to the present embodiment, it is possible to suppress adverse effects such as unnatural switching of a blurred luminance boundary portion, which is likely to occur mainly in close-up photography, in processing for locally controlling gradation characteristics. it can.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101: Optical system, 102: Imaging unit, 103: A / D conversion unit, 104: Image processing unit, 105: Exposure amount calculation unit, 106: Exposure amount control unit, 107: Display unit, 110: Recording unit

Claims (12)

Dividing means for dividing the input image into a plurality of regions;
Determining means for determining gradation characteristics for performing gradation correction on the image based on luminance information of each of the plurality of regions divided by the dividing means;
Generating means for generating a plurality of images having different frequency bands from the input image;
Conversion means for converting the generated signals of the plurality of images having different frequency bands into gains based on the gradation characteristics;
Adding means for weighting and adding signals of a plurality of images having different frequency bands converted into gain;
Correction means for correcting the image by multiplying the image by the gain added by the addition means,
The adding means determines weights of signals of a plurality of images having different frequency bands based on at least one of imaging conditions when the input image is captured and information on the plurality of regions. A featured image processing apparatus.   The image processing apparatus according to claim 1, further comprising an imaging unit configured to capture the image.   The image processing apparatus according to claim 2, wherein the photographing condition is at least one of a Bv value, an aperture value, and a focal length at the time of photographing.   The image processing apparatus according to claim 2, wherein the determination unit changes the gradation characteristic based on the photographing condition.   The generation unit generates at least two images: an upper layer image including a relatively high frequency band and a lower layer image including a lower frequency band than the upper layer image. Item 5. The image processing apparatus according to any one of Items 1 to 4.   The adding means includes a first addition result that is a result of adding the target gains corresponding to the target pixels of the upper layer image and the lower layer image, and the upper layer target gain and the lower layer. In a peripheral region within a predetermined range from the target pixel, the second addition result that is the addition result of the lower layer gain having a value close to the upper layer target gain is weighted and added. The image processing apparatus according to claim 5.   The image processing apparatus according to claim 1, wherein the information on the plurality of areas is a ratio of a high luminance area or a low luminance area in the input image. .   The image processing apparatus according to claim 1, wherein the generation unit generates a plurality of images having different frequency bands by filtering the image.   The image processing apparatus according to claim 1, wherein the generation unit generates a plurality of images having different frequency bands by performing a resizing process on the image. A division step of dividing the input image into a plurality of regions;
A determination step of determining gradation characteristics for performing gradation correction on the image based on luminance information of each of the plurality of regions divided in the division step;
Generating a plurality of images having different frequency bands from the input image; and
A conversion step of converting the generated signals of the plurality of images having different frequency bands into gains based on the gradation characteristics;
An adding step of weighting and adding signals of a plurality of images having different frequency bands converted into gains;
A correction step of correcting the image by multiplying the image by the gain added in the addition step,
In the adding step, the weighting of the signals of the plurality of images having different frequency bands is determined based on at least one of photographing conditions when the input image is photographed and information on the plurality of regions. A featured image processing method.   The program for making a computer perform each process of the image processing method of Claim 10.   A computer-readable storage medium storing a program for causing a computer to execute each step of the image processing method according to claim 10.