JP2012119818A - Image processing device, image processing method, and image processing program - Google Patents

Abstract

An image processing apparatus, an image processing method, and an image processing program capable of accurately performing saturation adjustment for making colors vivid are provided. A hue area processing unit includes a plurality of hue areas. The saturation peak value indicating the state of the saturation value in the associated hue region is calculated from the input image. The peak saturation addition amount calculation unit 40 calculates a saturation addition value for each hue region based on a predetermined upper limit saturation value for each hue region and a saturation peak value for each hue region. The saturation gain adjustment unit 70 adjusts the saturation value of the pixels included in the input image with the saturation gain determined based on the saturation addition value.
[Selection] Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program, and more particularly to an image processing apparatus, an image processing method, and an image processing program for adjusting the saturation of an input image.

  A general input image such as a landscape image often has low saturation in an unprocessed state. Therefore, the user often feels that the color of the image is light. Even if the input image is reproduced in saturation that is faithful to the original image, the input color is higher because the human memory color (the color that humans store as an image) is higher than the actual saturation. It is widely known that it is preferable to increase the saturation of.

  For this reason, display devices such as displays and printers that are widely used are set to increase the saturation of the image displayed on the display screen. However, the optimum amount of saturation adjustment varies depending on the type of input content and the viewing environment. Therefore, in these display devices, the manufacturer or end user is most suitable for the display mode (for example, standard mode, cinema mode, dynamic mode, etc.) and the input image type (for example, video image, PC image, tuner input, etc.). It is common to set a fixed value. In the following description, the saturation adjustment method for fixedly setting the saturation adjustment amount in this way is referred to as “fixed saturation adjustment”.

  Patent Document 1 discloses a technique related to one aspect of fixed saturation adjustment. In Patent Document 1, an LUT (Look Up Table) for determining a saturation adjustment amount is determined from the relationship between the hue angle and the saturation value, and the saturation adjustment amount of each pixel of the input image is calculated. That is, although the hue angle and the saturation value of each pixel of the input image are referred to, the saturation adjustment amount determined by the reference process is a predetermined fixed amount.

  Patent Documents 2 to 4 provide a vivid image with high saturation by analyzing the saturation of the input video signal and automatically increasing the saturation when an image with low saturation is input. A method has been proposed. In the methods of Patent Documents 2 to 4, control is performed so that the saturation value of each pixel is uniformly increased regardless of the hue region. In the following description, a saturation adjustment method for uniformly increasing the saturation value of each pixel in accordance with the saturation analysis result is described as “all hue uniform automatic adjustment”.

  Patent Document 5 discloses a technique for calculating gain for each 6-axis component for 6 axes of RGB and YCM, and performing saturation adjustment using the gain.

  Patent Document 6 proposes an image processing technique generally called histogram flattening that can obtain an image with high contrast. Specifically, a two-dimensional or one-dimensional histogram is generated from the three color vision signals, and the histogram is divided into a plurality of regions, and an optimum saturation is determined based on the divided regions, thereby obtaining an image with high contrast. Generate. Here, an image with high contrast is realized by averaging the saturation (average of occurrence ratio (frequency) of each saturation level).

  Patent Document 7 discloses a technique for detecting the luminance, saturation, and hue of an input video signal, and performing color space enlargement / reduction adjustment using the detected information. In this method, a histogram is detected for each luminance, saturation, and hue of the input video signal, but color adjustment processing is performed by RGB matrix conversion.

  Note that the image adjustment is not limited to saturation adjustment, and various image adjustments are performed according to the purpose. For example, Patent Document 8 proposes a technique for performing highly accurate gradation compression processing and gray balance processing based on the hue value and saturation value of an input image.

  In Patent Document 9, the hue distribution of an important subject in a natural image is extracted, and a correction color is determined from the relationship between the extracted information and a predetermined representative color, thereby automatically performing good color adjustment (preferred color, A method of performing correction to correct colors) has been proposed.

  Note that the methods disclosed in Patent Documents 8 and 9 do not provide any teaching or suggestion regarding adjustment processing for making the color of each pixel vivid, that is, adjustment processing for increasing the saturation value of each pixel.

JP 2003-274210 A JP 2004-236169 A JP 2007-74063 A JP 2008-72253 A JP 2009-49633 A Japanese Patent Laid-Open No. 9-247483 JP 2009-81849 A JP 2005-192162 A JP 2001-92956 A

  Various image processing techniques have been proposed as described above. However, the above-described image processing method has a problem that the saturation adjustment accuracy for making the color vivid is not sufficient. This problem will be described below.

  In the fixed saturation adjustment, the saturation adjustment amount is fixed. Therefore, when the saturation of the input image is low, the saturation cannot be increased sufficiently, and a colorful output image cannot be obtained. On the other hand, when the saturation of the input image is high, the saturation is further increased, so that the color density is saturated and an unnatural output image is output.

  In the all-hue uniform automatic adjustment, even if the saturation of only a certain hue region of the input image is high, it can be determined that the saturation of the input image is also high. As a result, even when the saturation of other hue regions is low, adjustment cannot be made to increase the saturation. On the other hand, when adjusting saturation by paying attention to a hue region with low saturation, the saturation of pixels in the hue region with high saturation is also increased, and color saturation occurs. As described above, in the all-hue uniform automatic adjustment, the saturation adjustment accuracy is not sufficient.

  The state of this saturation adjustment is shown in FIGS. FIG. 15 and FIG. 16 are diagrams showing the highest saturation value of each hue before and after saturation adjustment by all-hue uniform automatic adjustment. FIG. 15 shows a state in which the saturation of another hue cannot be increased because the saturation of the hue region R is high. For example, the saturation values of the hue Y and the hue G cannot be increased like the dotted line portion in the figure. FIG. 16 shows how the saturation is adjusted by paying attention to a hue with low saturation. In this case, the color density of the hue region R having a sufficiently high saturation is saturated as indicated by the dotted line in the figure.

  In the method of Patent Document 5, since the six axes of RGB and YCM are the basis of processing, saturation adjustment for the intended hue region cannot be performed. For example, the skin color that is a target of typical color adjustment is defined between R and Y, but it is difficult to adjust the skin color with the method of Patent Document 5. If the processing target axis is divided more than six axes, the number of samples decreases due to an increase in the number of divisions, and the processing accuracy becomes low.

  In the method of Patent Document 6, since the saturation is averaged (the generation ratio (frequency) of each saturation level), pixels with low saturation are generated. For this reason, an improvement in contrast can be realized, but there is a risk that it may be counterproductive to making the image vivid.

  In the method of Patent Document 7, since color adjustment is realized by RGB matrix conversion as described above, only color (saturation) adjustment from colorless (black-white) to the R / G / B hue axis direction is possible. There are constraints. For example, even if the saturation of the yellow hue region is insufficient, if the saturation of red or green is already sufficiently high, the saturation of yellow cannot be increased any more. become. Therefore, it is not possible to perform effective adjustment for many input images.

  As described above, the above-described image processing technique has a problem that the saturation adjustment accuracy for making the color vivid is not sufficient.

One aspect of the image processing apparatus of the present invention is:
At least one hue region processing unit that is provided corresponding to one of the plurality of hue regions and calculates a saturation peak value indicating a state of the saturation value in the associated hue region from the input image;
Based on a predetermined upper limit saturation value for each hue region and a saturation peak value calculated by the hue region processing unit, peak saturation for calculating a saturation addition value for the hue region corresponding to the hue region processing unit A degree addition amount calculation unit;
A saturation gain adjustment unit that adjusts a saturation value of a pixel included in the input image with a saturation gain determined based on the saturation addition value.

One aspect of the image processing method of the present invention is:
From the input image, calculating a saturation peak value indicating a saturation value state in at least one of the plurality of hue regions,
Based on a predetermined upper limit saturation value for each hue region and the saturation peak value, a saturation addition value of pixels included in the hue region corresponding to the saturation peak value is calculated,
A saturation value of a pixel included in the input image is adjusted by a saturation gain determined based on the saturation addition value.

One aspect of the image processing program of the present invention is:
A program for causing a computer to execute a process of adjusting a saturation value of an input image,
The process is
From the input image, a saturation peak value indicating a saturation value state in at least one of the plurality of hue regions is calculated,
Based on a predetermined upper limit saturation value for each hue region and the saturation peak value, a saturation addition value of pixels included in the hue region corresponding to the saturation peak value is calculated,
A saturation value of a pixel included in the input image is adjusted by a saturation gain determined based on the saturation addition value.

  In the present invention, the image processing apparatus calculates the saturation peak values of a plurality of hue regions, and based on the saturation peak value and the upper limit saturation value determined to suppress color saturation, Calculate the saturation gain. Since the saturation gain corresponding to the saturation state of each hue region can be calculated, it is possible to realize highly accurate saturation adjustment in consideration of the state of each hue region.

  According to the present invention, it is possible to provide an image processing apparatus, an image processing method, and an image processing program capable of accurately performing saturation adjustment for making colors vivid.

1 is a block diagram illustrating an overall configuration of an image processing apparatus according to a first embodiment; 2 is a block diagram showing an internal configuration of a luminance processing unit according to the first embodiment; FIG. 3 is a block diagram showing an internal configuration of a hue area processing unit according to the first embodiment; FIG. 6 is a diagram illustrating a method of calculating a saturation peak addition amount by a saturation peak addition amount calculation unit according to the first embodiment. FIG. 6 is a diagram illustrating a method of calculating a saturation peak addition amount by a saturation peak addition amount calculation unit according to the first embodiment. FIG. 3 is a block diagram illustrating an internal configuration of a saturation peak addition amount calculation unit according to the first embodiment; FIG. 6 is a diagram illustrating an example of a method of calculating a luminance APL modulation gain APL_mod according to the first embodiment. FIG. 6 is a diagram illustrating an example of a method for calculating a modulation gain RUI_mod according to the first embodiment; FIG. 6 is a diagram illustrating an example of a method for calculating a modulation gain SAT_mod according to the first embodiment; FIG. 6 is a diagram illustrating an example of a method for calculating a modulation gain HUE_mod according to the first embodiment; FIG. FIG. 3 is a diagram illustrating a concept of saturation adjustment processing by the image processing apparatus according to the first embodiment. It is a figure which shows the difference in the color reproduction range by a some display apparatus. It is a figure which shows the deviation ellipse of MAC ADAM. And FIG. 11 is a block diagram illustrating a configuration example of a computer that executes a program. It is a figure which shows the concept of the saturation adjustment process of the image by all the hue uniform automatic adjustment. It is a figure which shows the concept of the saturation adjustment process of the image by all the hue uniform automatic adjustment.

<Embodiment 1>
Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image processing apparatus 1 according to the present embodiment. The image processing apparatus 1 can be mounted on, for example, a television receiver, a projector, a liquid crystal display for a personal computer, a digital still camera, or the like.

  An input image in which each pixel is expressed in RGB format is input to the image processing apparatus 1. Note that an input image in which each pixel is expressed in the YUV format may be input to the image processing apparatus 1.

  The image processing apparatus 1 includes a luminance / saturation / hue conversion unit 10, a luminance processing unit 20, a hue region 1 (R) processing unit 30, a hue region 2 (Y) processing unit 31, and a hue region 3 (G ) Processing unit 32, hue region 4 (C) processing unit 33, hue region 5 (B) processing unit 34, hue region 6 (M) processing unit 35, saturation peak addition amount calculation unit 40, pixel A unit saturation peak modulation gain calculation unit 50, a saturation gain calculation unit 60, a saturation gain adjustment unit 70, and an RGB / YUV conversion unit 80 are provided.

  The luminance / saturation / hue conversion unit 10 converts the RGB representation or YUV representation of each pixel of the input image into luminance / saturation / hue component amounts. For the expression of luminance / saturation / hue, an arbitrary color system such as HSV, Lch (Lab), YSH, or the like may be used. This is because there is no theoretical absolute solution for the saturation adjustment value adopted by the image processing apparatus 1, and thus it is not possible to determine which color space expression is optimal. Therefore, the expression of luminance / saturation / hue may adopt an optimum color system in consideration of the age, viewing environment, position, etc. of the user who refers to the image.

  A hue area classification unit (not shown) determines a hue area to which each pixel belongs from the hue component amount (hereinafter referred to as a hue value) of each pixel calculated by the luminance / saturation / hue conversion unit 10. In this example, six hue regions of R / G / B / Y / C / M are set as the hue region. Based on the determination of the hue region classification unit (not shown), the luminance / saturation / hue conversion unit 10 performs a hue region processing unit (hue region 1 (R) processing unit 30 to 30) corresponding to the hue region to which each pixel belongs. The hue value of the pixel is supplied to any one of the hue area 6 (M) processing unit 35. Similarly, based on the determination of the hue area classification unit (not shown), the luminance / saturation / hue conversion unit 10 determines the luminance value / saturation value / hue value of each pixel as the luminance processing unit 20 and the pixel unit saturation. This is supplied to the degree peak modulation gain calculation unit 50 and the saturation gain adjustment unit 70.

  Note that the number of divisions of the hue region can be set to an arbitrary number as long as it is two or more. However, when the hue area is divided more finely than necessary, the stability of the histogram acquisition result of each divided hue area performed by the hue area processing unit described later decreases. In particular, the stability of the histogram acquisition result of each divided hue area during moving image reproduction decreases. Lack of stability in the histogram acquisition result of each divided hue region may cause gradation performance degradation, unstable operation during moving images, and S / N degradation. Therefore, in general, it is desirable that the maximum number of divisions is about six hues of R / G / B / Y / C / M. If the resolution in the hue direction and the resolution of the saturation peak calculation value are high, seven or more hue regions may be set.

  The luminance processing unit 20 is a processing unit that mainly calculates a luminance APL (Average Peak Level) value of the input image. The luminance processing unit 20 is supplied with the luminance value of each pixel of the input image.

  As shown in FIG. 2, the luminance processing unit 20 includes a luminance histogram calculation unit 210, a luminance APL calculation unit 220, and a time constant processing unit 230.

  The luminance histogram calculation unit 210 calculates a luminance histogram indicating the distribution of luminance values of each pixel in the input image by level from the luminance value of each pixel of the input image. The luminance histogram calculation unit 210 supplies the calculated luminance histogram to the luminance APL calculation unit 220. Here, as the resolution in the level direction of the luminance histogram increases, the luminance APL calculation unit 220 described later can calculate a luminance APL value with high accuracy.

  The luminance APL calculation unit 220 is a processing unit for calculating the average brightness of the input image, that is, the luminance APL value. The luminance APL calculation unit 220 calculates a luminance APL value using the luminance histogram calculated by the luminance histogram calculation unit 210. The luminance APL calculation unit 220 supplies the calculated luminance APL value to the time constant processing unit 230.

  The calculation method of the luminance APL may be realized by an arbitrary method. For example, the luminance APL calculation unit 220 may cumulatively add the frequency values in order from the lowest level frequency value of the luminance histogram, and calculate the level at which the cumulative addition value becomes about 50% of the total as the luminance APL value. Good. The luminance APL calculation unit 220 can also calculate the average luminance level of all the pixels included in the input image as the luminance APL value.

  The time constant processing unit 230 performs time constant processing of the luminance APL value calculated by the luminance APL calculation unit 220 and supplies the luminance APL value to the saturation peak addition amount calculation unit 40 after the processing. The image processing apparatus 1 according to the present embodiment performs image processing in units of frames, specifically, input image analysis and saturation adjustment. That is, the luminance APL value of each frame is calculated, and the saturation adjustment is performed by the processing unit described later using the luminance APL value. Here, when the luminance APL value changes greatly between consecutive frames, the saturation adjustment value in the processing unit described later changes greatly. When the input image is a moving image, at the timing of the scene change, in order to prevent overcorrection of the saturation gain, if the maximum saturation changes greatly from a low state to a high state, the saturation adjustment value is also Following this, it is preferable that the gain value changes greatly from a large gain value to a small gain value. On the other hand, if the saturation adjustment value changes greatly in the same scene, the screen stability is impaired. Therefore, the time constant processing unit 230 performs time constant processing on the calculated luminance APL value and adjusts the value so that the value does not change suddenly from the luminance APL value calculated in the previous frame. The time constant processing unit 230 realizes time constant processing by general IIR (Infinite Impulse Response) filter processing and moving average filter processing.

  However, if a time constant process is performed by referring to the luminance APL value of the previous frame when a scene change occurs, the saturation adjustment value becomes an excessive value. As a result, there is a risk that a processing unit described later performs overcorrection of saturation and color density saturation occurs. Therefore, the image processing apparatus 1 includes a general scene change detection unit (not shown) that detects a scene change by detecting a difference between frames. When the scene change detection unit detects a scene change, it notifies the time constant processing unit 230 of the scene change. The time constant processing unit 230 notified of the detection of the scene change performs control so as to prevent color density saturation by not performing the time constant processing or adjusting the coefficient of the time constant processing.

  Subsequently, processing of the hue region 1 (R) processing unit 30 to the hue region 6 (M) processing unit 35 will be described with reference to FIGS. 1 and 3. The hue region 1 (R) processing unit 30 is supplied with the saturation values of all the pixels belonging to the hue region R in the processing target frame and the total number of pixels of the input image.

  The hue region 1 (R) processing unit 30 is a processing unit that calculates a saturation peak value from a pixel group belonging to the hue region R. A detailed configuration of the hue region 1 (R) processing unit 30 is shown in FIG.

  The hue region 1 (R) processing unit 30 includes a hue region 1 (R) saturation histogram calculation unit 310, a saturation peak value calculation unit 320, a time constant processing unit 330, a cumulative pixel rate calculation unit 340, a time A constant processing unit 350.

  The hue region 1 (R) saturation histogram calculation unit 310 calculates a saturation histogram of the pixel group belonging to the hue region R. That is, the hue region 1 (R) saturation histogram calculation unit 310 calculates a histogram (a graph showing the distribution of saturation values) for each saturation level in the hue region R. The resolution in the histogram (frequency) direction does not need to be highly accurate due to the visual characteristic with respect to the saturation. For example, if a resolution of about 8 steps (3 bits) to 16 steps (4 bits) is employed, a sufficiently practical histogram can be calculated. The hue region 1 (R) saturation histogram calculation unit 310 supplies the calculated histogram to the saturation peak value calculation unit 320 and the cumulative pixel rate calculation unit 340.

  The saturation peak value calculation unit 320 is a processing unit that calculates a saturation peak value related to the hue region R from the saturation histogram related to the input hue region R. The saturation peak value is an index value indicating a sufficiently high saturation value among the pixels belonging to the hue region R. The saturation peak value is a value that serves as a reference for calculating the amount of saturation addition by a processing unit that will be described later. Therefore, a sufficiently high value is desirable to avoid color saturation. For example, the saturation peak value is set to the maximum value among the saturation values of the pixels belonging to the hue region R.

  The method of calculating the saturation peak value can be switched according to, for example, the S / N ratio (Signal to Noise ratio) of the input image. In this case, the image processing apparatus 1 includes an S / N ratio detection circuit that detects an S / N ratio of the input image.

  When the S / N ratio (Signal to Noise ratio) of the image is good, since there are few pixels that become noise components, there is little possibility that pixels with unnaturally high saturation exist. Therefore, if the saturation peak value is set to the maximum value among the saturation values of the pixels belonging to the hue region R, it is possible to perform highly accurate saturation adjustment in the processing unit described later.

  On the other hand, when the S / N ratio (Signal to Noise ratio) of the image is poor, there are many pixels that are noise components, and therefore there is a high possibility that pixels with unnaturally high saturation exist. Therefore, for example, the saturation peak value is designated as the saturation value located in the upper 10% of the saturation values of the pixels belonging to the hue region R. Specifically, the frequency values are added in order from the lowest level frequency value of the saturation histogram for the hue region R, and the saturation value at the level where the total accumulated value becomes 90% is set as the saturation peak value. . By setting the saturation peak value in this way, it is possible to set the saturation peak value that is not affected by the noise component, and to adjust the saturation with high accuracy even for an image with a low S / N ratio.

  The saturation peak value calculation unit 320 supplies the calculated saturation peak value of the hue region R to the time constant processing unit 330.

  The time constant processing unit 330 is a processing unit that performs time constant processing on the supplied saturation peak value and supplies the processed saturation peak value to the saturation peak addition amount calculation unit 40. The time constant processing unit 330 differs from the time constant processing unit 230 only in that the processing target is a saturation peak value. Therefore, detailed description of the time constant processing unit 330 is omitted.

  The cumulative pixel rate calculation unit 340 is a processing unit that calculates a ratio (cumulative pixel rate) of pixels belonging to the hue R among all the pixels of the input image. The cumulative pixel rate calculation unit 340 calculates the cumulative pixel rate by dividing the number of pixels belonging to the hue R by the number of pixels for one screen (the total number of pixels of the input image). The cumulative pixel rate calculation unit 340 supplies the calculated cumulative pixel rate to the time constant processing unit 350.

  The time constant processing unit 350 is a processing unit that performs time constant processing on the supplied cumulative pixel rate and supplies the processed cumulative pixel rate to the saturation peak addition amount calculation unit 40. The time constant processing unit 350 is different from the time constant processing unit 230 only in that the processing target is the cumulative pixel rate. Therefore, detailed description of the time constant processing unit 350 is omitted.

  The hue region 2 (Y) processing unit 31 to the hue region 6 (M) processing unit 35 are processing units corresponding to hues Y / G / C / B / M, respectively. This is substantially the same as 1 (R) 30. Therefore, detailed description of the hue region 2 (Y) processing unit 31 to the hue region 6 (M) processing unit 35 is omitted.

  In the above description, six hue regions are assumed, and it is assumed that the saturation of pixels belonging to all the hue regions is adjusted. However, the present invention is not necessarily limited to this, and only the pixels belonging to some of the hue regions may be subjected to saturation adjustment. In this case, it is only necessary to provide a hue area processing unit corresponding to the hue area for which saturation adjustment is performed. For example, when it is desired to increase only the saturation of the skin color, the image processing apparatus 1 may include only the hue region 1 (R) processing unit 30 and the hue region 2 (Y) processing unit 31. When the image processing apparatus 1 is configured in this way, the overall processing amount is reduced, so that the processing speed can be improved as well as the effect of increasing the saturation for a desired hue region.

  Furthermore, the hue area processed by each hue area processing unit can be arbitrarily set. For example, when a hue area intermediate between R and Y is defined and a hue area processing section for processing this hue area is provided, the skin color saturation increase adjustment can be performed by only one hue area processing section.

  Returning to FIG. 1, the saturation peak addition amount calculation unit 40 will be described. The saturation peak addition amount calculation unit 40 is a luminance APL value supplied from the luminance processing unit 20, and a saturation peak value calculated from each of the hue region 1 (R) processing unit 30 to the hue region 6 (M) processing unit 35. Based on the cumulative pixel ratio, the saturation addition amount ΔS peak_V is calculated for each hue region (each of the six hue regions in this example). The saturation addition amount ΔS peak_V is an addition amount to be added to the saturation peak value in order to obtain the saturation peak value after the saturation adjustment is calculated.

  With reference to FIG.4 and FIG.5, the outline | summary of the calculation process of saturation peak addition amount (DELTA) Speak_V by the saturation peak addition amount calculation part 40 is demonstrated. First, the maximum saturation value S max is set for each hue region. The maximum saturation value Smax is the maximum saturation value that can be obtained by performing saturation adjustment. In principle, the maximum saturation value Smax can be arbitrarily set to a value that does not cause color saturation for each hue region. However, the maximum saturation value that can be taken for each hue region can be calculated from the color space used by the luminance / saturation / hue conversion unit 10 and the color space of the input signal standard. Therefore, the maximum saturation value Smax is equal to or less than the maximum saturation value that can be taken, and it is necessary for the user to set a value to the highest saturation as a result after adjusting the saturation.

  The saturation addition amount ΔS peak_V is basically calculated by subtracting the peak saturation value S peak from the maximum saturation value S max (FIG. 4). That is, the saturation is added so that the peak saturation value Peak becomes the maximum saturation value Smax. As described above, the maximum saturation value Smax is set in advance to a value that does not cause color saturation. Therefore, by subtracting the peak saturation value S peak from the maximum saturation value Smax, it is possible to achieve image adjustment that makes the color vivid without causing color saturation.

  However, when the value obtained by subtracting the peak saturation value S peak from the maximum saturation value S max (FIG. 4) is used as it is as the saturation addition amount ΔS peak_V, the saturation becomes too high in terms of visibility, which is unnatural. There is a risk that the image may appear and color noise may be increased. Therefore, as shown in FIG. 5, the final saturation addition amount ΔS peak_V is calculated by performing various modulations on the calculated subtraction value. Details of the saturation peak addition amount calculation unit 40 including these various modulation processes will be described with reference to FIG.

  The saturation peak addition amount calculation unit 40 includes a subtractor 410, a luminance APL modulation gain calculation unit 420, a multiplier 430, a cumulative pixel rate modulation gain calculation unit 440, a multiplier 450, and a multiplier 460. Prepare.

The subtractor 410 receives the peak saturation value S peak and the maximum saturation value S max of each hue region. The subtractor 410 subtracts the peak saturation value S peak from the maximum saturation value S max to calculate a maximum saturation amount S1 that can be added.
S1 = Smax−Speak

  The calculation of S1 is performed for each hue region as described above. In the configuration of FIG. 1, six S1 values are calculated per frame. The subtractor 410 supplies the calculated S1 to the multiplier 430.

  The luminance APL modulation gain calculation unit 420 is supplied with the luminance APL value for each frame. The luminance APL modulation gain calculation unit 420 calculates the modulation gain APL_mod from the luminance APL value, and supplies the calculated APL_mod to the multiplier 430.

  When the luminance APL value is high, the maximum saturation amount S1 that can be added may be added to the peak saturation S peak as it is. On the other hand, when the luminance APL value is low, it is necessary to reduce the maximum saturation amount S1 that can be added. This is because the sensitivity of photoelectric conversion of the camera is high in an image scene such as a night view or a dark room, and there is a high possibility that color noise is generated. This is because, if the saturation is increased with respect to the input image recorded by such an image scene, the possibility of increasing the noise of the color system increases.

FIG. 7 is a diagram illustrating a calculation example of the luminance APL modulation gain APL_mod by the luminance APL modulation gain calculation unit 420. The luminance APL modulation gain APL_mod has the following value range.
0 <= APL_mod <= 1

  As shown in FIG. 7, the luminance APL modulation gain calculation unit 420 calculates the luminance APL modulation gain APL_mod using a LUT (Look Up Table) or the like, or a primary / secondary function with the luminance APL value as an input. In any calculation method, the luminance APL modulation gain APL_mod is set to be low when the luminance APL value is low. The luminance APL modulation gain calculation unit 420 supplies the calculated luminance APL modulation gain APL_mod to the multiplier 430.

The multiplier 430 calculates S2 obtained by multiplying the maximum saturation amount S1 that can be added by the luminance APL modulation gain APL_mod. That is, S2 is a value obtained by modulating the maximum chroma amount S1 that can be added by the luminance APL modulation gain APL_mod.
S2 = S1 * APL_mod = (Smax-Speak) * APL_mod

  The multiplier 430 supplies the modulated saturation amount S2 to the multiplier 450.

  The cumulative pixel rate modulation gain calculation unit 440 is a processing unit that calculates the modulation gain RUI_mod from the cumulative pixel rate of each hue region calculated in units of frames. In the configuration of FIG. 1, since there are six hue regions, the cumulative pixel rate modulation gain calculation unit 440 calculates six modulation gains RUI_mod per frame. The modulation gain RUI_mod is used by the multiplier 450 to modulate the above-described saturation amount S2. Details of the modulation process using the modulation gain RUI_mod and the reason for performing the modulation process are as follows.

  In a hue region with a high cumulative pixel rate, the saturation amount S2 may be added to the peak saturation S peak as it is. On the other hand, in a hue region with a low cumulative pixel rate, it is necessary to add to the peak saturation S peak after reducing the saturation amount S2. The fact that there are few pixels in the target hue area means that there are not many objects of the target color on the screen. This is because if the saturation is increased in a state where there are few objects of the target color, there is a possibility that only an unintended component such as a noise component of the color may be increased although there is no object for which the color is to be vivid.

FIG. 8 is a diagram illustrating a calculation example of the modulation gain RUI_mod by the cumulative pixel rate modulation gain calculation unit 440. The modulation gain RUI_mod has the following value range.
0 <= RUI_mod <= 1
Similar to the luminance APL modulation gain calculation unit 420 described above, the modulation gain RUI_mod is calculated by a linear function having the modulation gain RUI_mod as an input. The cumulative pixel rate modulation gain calculation unit 440 supplies the calculated modulation gain RUI_mod to the multiplier 450.

The multiplier 450 calculates S3 obtained by multiplying the saturation amount S2 by the modulation gain RUI_mod calculated for the hue region to be processed.
S3 = S2 * RUI_mod = (Smax-Speak) * APL_mod * RUI_mod
The multiplier 450 supplies the modulated saturation amount S3 to the multiplier 460.

  The multiplier 460 receives the saturation amount S3 of each hue region and the user gain USER_gain. The multiplier 460 is a processing unit that modulates the saturation amount S3 with the user gain USER_gain. The user gain USER_gain is a modulation parameter for adjusting the degree to which the user makes the color vivid for each hue region. In the conventional fixed saturation adjustment, it is the same as when the set maker or end user performs fixed saturation gain adjustment according to the display mode (standard, cinema, dynamic, etc.) and input (video, PC, tuner, etc.) In addition, the end user can adjust the saturation addition amount according to the input content and the viewing environment by setting the user gain USER_gain.

  Since the user gain USER_gain is set for each hue area (each of the six hue areas in the configuration of FIG. 1) as described above, the user can set the saturation adjustment amount independently for each hue area. .

The multiplier 460 calculates the saturation addition amount ΔS peak_V for each hue region by multiplying the saturation amount S3 by the user gain USER_gain.
ΔS peak_V = S3 × USER_gain = (S max − S peak) × APL_mod × RUI_mod × USER_gain

  The multiplier 460 supplies the saturation addition amount ΔS peak_V for each hue region to the saturation gain calculation unit 60. In the configuration of FIG. 1, the multiplier 460 supplies the saturation addition amount ΔS peak_V for each of six hue regions to the saturation gain calculation unit 60 for each frame.

  Returning to FIG. 1, the pixel unit saturation peak modulation gain calculation unit 50 will be described. The saturation value and the hue value of each pixel constituting the input image are supplied from the luminance / saturation / hue conversion unit 10 to the pixel unit saturation peak modulation gain calculation unit 50. The pixel unit saturation peak modulation gain calculation unit 50 is a processing unit that converts the saturation value and the hue value of each pixel into modulation gains (SAT_mod, HUE_mod), respectively. These modulation gains are used as modulation parameters when calculating the saturation gain of each pixel. The pixel unit saturation peak modulation gain calculation unit 50 supplies the two calculated modulation gains to the saturation gain calculation unit 60. The reason why these modulation gains are used and the method for calculating the modulation gain by the pixel unit saturation peak modulation gain calculation unit 50 will be described below.

  First, the saturation modulation gain SAT_mod will be described. When the saturation value of a certain pixel is higher than a predetermined threshold, the saturation addition amount ΔS peak_V calculated by the saturation peak addition amount calculation unit 40 may be added to the saturation value of the pixel as it is. However, when the saturation value of a pixel is lower than a predetermined threshold, it is necessary to reduce the saturation addition amount ΔS peak_V calculated by the saturation peak addition amount calculation unit 40 and then add it to the saturation value of the pixel. is there. This is because when an object having a low saturation, that is, a light color that is correct (natural) is darkened, an unnatural color expression may occur or color noise may be increased.

Similar to the calculation of the luminance APL, the pixel-unit saturation peak modulation gain calculation unit 50 calculates the pixel-unit saturation modulation gain SAT_mod by a method using an LUT or a method using a primary / quadratic function. FIG. 9 shows a method for calculating the saturation modulation gain SAT_mod. The calculated saturation modulation gain SAT_mod has the following range.
0 <= SAT_mod <= 1

  Next, the hue modulation gain HUE_mod will be described. For a pixel located near the central hue to be adjusted, the saturation addition amount ΔS peak_V calculated by the saturation peak addition amount calculation unit 40 may be added to the saturation value of the pixel as it is. However, for a pixel located far from the center hue to be adjusted, it is necessary to reduce the saturation addition amount ΔS peak_V calculated by the saturation peak addition amount calculation unit 40 and then add it to the saturation value of the pixel. is there. This is to prevent the saturation adjustment that greatly changes the saturation with respect to a pixel having a hue value that is not intended to be greatly adjusted. In other words, the control is basically performed so that the saturation is adjusted only for the pixel located in the central hue. However, when setting the hue modulation gain HUE_mod that suddenly lowers the amount of saturation addition or setting the adjustment hue region narrow, the stability of the image when handling moving images may be impaired.

A method of calculating the hue modulation gain HUE_mod is shown in FIG. The hue pixel unit saturation peak modulation gain calculation unit 50 calculates the hue modulation gain HUE_mod for each pixel by a method using an LUT or a method using a linear / quadratic function with reference to the hue center to be adjusted. In the method of calculating the modulation gain HUE_mod, the coefficients of the LUT and the primary / secondary function are determined in consideration of the above-described image stability. As shown in the figure, by setting the hue modulation gain HUE_mod to be gradually lower from the central hue (near 90 degrees in the figure), the saturation adjustment amount between the central hue and the adjacent hue may change greatly. Disappear. This prevents the gradation characteristics and moving image stability from being impaired. The calculated hue modulation gain HUE_mod has the following value range.
0 <= HUE_mod <= 1

  Note that the hue region to be adjusted shown in FIG. 10 and the hue region handled by the hue region 1 (R) saturation histogram calculation unit 310 (and the corresponding processing unit of another hue region) can be set separately. Thereby, the freedom degree of saturation adjustment improves.

  Returning to FIG. 1, the saturation gain calculator 60 will be described. The saturation gain calculation unit 60 receives the saturation addition amount ΔS peak_V after modulation of each hue region, the saturation modulation gain SAT_mod of each pixel, and the hue modulation gain HUE_mod of each pixel from the saturation peak addition amount calculation unit 50. Entered. Based on these inputs, the saturation gain calculation unit 60 calculates a saturation gain SAT_gain that multiplies the saturation value of each pixel. Hereinafter, a method for calculating the saturation gain SAT_gain will be described.

The saturation gain calculator 60 calculates the modulation gain DOT_mod of each pixel by multiplying the saturation modulation gain SAT_mod and the hue modulation gain HUE_mod.
DOT_mod = SAT_mod × HUE_mod
The calculated modulation gain DOT_mod has the following range.
0 <= DOT_mod <= 1

The saturation gain calculation unit 60 calculates a final saturation addition amount ΔS peak_H / V obtained by modulating the saturation addition amount ΔS peak_V by the above-described DOT_mod. The saturation gain calculation unit 60 calculates a saturation addition amount ΔS peak_H / V for each pixel.
ΔS peak_H / V = ΔS peak_V × DOT_mod

The saturation gain calculation unit 60 calculates the SAT_gain of each pixel from the following expression using the final saturation addition amount ΔS peak_H / V and the peak saturation S peak of each hue region analyzed from the input image. To do.
SAT_gain = (ΔS peak_H / V + Speak) / S peak

  The saturation gain calculation unit 60 supplies the calculated saturation gain SAT_gain of each pixel to the saturation gain adjustment unit 70.

The saturation gain adjustment unit 70 is a processing unit that adjusts the saturation value of each pixel. The saturation gain adjustment unit 70 is supplied with the luminance / saturation / hue value of each pixel and the saturation gain SAT_gain of each pixel. The saturation gain adjustment unit 70 calculates the saturation value SAT_out after saturation adjustment by multiplying the saturation value SAT_in of each pixel by the saturation gain SAT_gain. By this multiplication, it is possible to increase the intended amount of saturation according to the design with respect to the intended saturation and hue.
SAT_out = SAT_in x SAT_gain

  The saturation gain adjustment unit 70 supplies the luminance / hue value of each pixel and the saturation value SAT_out after saturation adjustment of each pixel to the RGB / YUV conversion unit 80.

  The RGB / YUV conversion unit 80 converts each value of luminance / saturation / hue of each pixel into an RGB or YUV color space, and outputs the converted value to an arbitrary processing unit.

  Next, effects of the image processing apparatus according to this embodiment will be described. The image processing apparatus according to the present embodiment defines a plurality of hue regions and calculates a saturation addition amount for making a color vivid using a saturation peak value for each hue region. That is, the image processing apparatus according to the present embodiment adjusts the saturation for each hue region. By considering the peak value of saturation for each hue area, adjustment is made to increase the saturation value significantly when the saturation of the input image is low, and the saturation value is adjusted when the saturation of the input image is high. Only a slight increase can be made. In other words, because the saturation addition amount can be set dynamically according to the saturation state of the input image, the input image in any saturation state is vivid and has natural saturation. An image can be output.

  Furthermore, according to the image processing apparatus of the present embodiment, even if an image with uneven saturation is input for each hue region, a vivid and natural image can be output without causing color saturation. FIG. 11 is a diagram conceptually showing the effect of the saturation adjustment of the image processing apparatus according to the present embodiment. FIG. 11 shows the saturation amount of each hue before and after saturation adjustment by the image processing apparatus according to the present embodiment. As shown in the drawing, in the image processing apparatus according to the present embodiment, adjustment for increasing the saturation value is not performed for the hue region (R) in which color saturation may occur. On the other hand, in a hue area with low saturation (for example, hue area (Y) or hue area (G)), adjustment to increase the saturation value is performed.

  Furthermore, the saturation adjustment by the image processing apparatus of the present embodiment increases the range of saturation values that can be taken in each hue region, that is, the number of colors increases (color gradation increases).

  As described above, in the image processing apparatus according to this embodiment, the modulation gain (USER_gain) for modulating the saturation addition amount can be set for each hue region. The effect of being able to set different modulation gains for the saturation addition amount of each hue area will be described with reference to FIGS.

  FIG. 12 is a diagram showing the color reproduction range of the output display in the color region. FIG. 12 shows the color reproduction range of the display device corresponding to the color gamut standards of three types (Adobe RGB, s RGB, NTSC). As described above, the color reproduction range is generally greatly different depending on the output display device. For example, even in the state where the same green (green 100%) is displayed, the color of light emission is completely different depending on the output display device used.

  Next, FIG. 13 is a diagram illustrating a so-called MAC ADAM deviation ellipse. Specifically, FIG. 13 shows a MAC ADAM deviation ellipse in which a region in which a normal observer cannot distinguish from the central chromaticity is magnified 10 times with respect to the color region. FIG. 13 shows that the sensitivity with which a human recognizes a color change varies greatly depending on the hue. For example, humans are very sensitive to changes in blue, but not very sensitive to changes in green. For this reason, if the saturation is uniformly increased in each hue region without taking this property into consideration, it may be felt that the saturation is increased unevenly in terms of human visual sense.

  In the image processing apparatus according to the present embodiment, as described above, different modulation gains can be set for the saturation addition amount of each hue region. That is, in the image processing apparatus according to the present embodiment, the saturation addition amount can be adjusted according to the properties of the display used and the psychological properties of the user. Accordingly, it is possible to provide a colorful image that the user does not feel unnatural.

  Furthermore, the image processing apparatus according to the present embodiment can provide an image that feels natural even when a scene change or the like occurs by performing the above-described time constant processing. As described above, the time constant process can be performed on the luminance APL, the cumulative pixel rate, and the saturation peak value.

  In the image processing apparatus according to the present embodiment, it is possible to prevent an increase in color noise by considering the luminance of the input image.

  Furthermore, in the image processing apparatus according to the present embodiment, it is possible to prevent only the noise component from increasing by considering the cumulative pixel rate for each hue region.

  Furthermore, in the image processing apparatus according to the present embodiment, unnatural color expression can be prevented by modulating the saturation addition amount according to the saturation value of each pixel.

  In the image processing apparatus according to the present embodiment, the saturation peak value calculation method can be changed according to the quality of the S / N ratio of the input image. Thereby, an appropriate value according to the image quality can be set as the saturation peak value.

  The present invention has been described above based on the embodiment. The embodiment is an exemplification, and various modifications and changes may be made to the above-described embodiment without departing from the gist of the present invention. It will be understood by those skilled in the art that modifications in which these changes and increases / decreases are also within the scope of the present invention.

  For example, each processing unit (luminance / saturation / hue conversion unit 10 or the like) of the image processing apparatus 1 basically processes all pixels of the input image, but pixels located at some coordinates of the input image. (For example, upper left) may be excluded from the processing target.

  Furthermore, when the peak saturation value S peak is larger than the maximum saturation value S max, the saturation peak addition amount calculation unit 40 does not change the saturation value of the target hue region so that the saturation addition amount ΔS peak — V May be set. As a result, it is possible to prevent a pixel having a saturation value higher than the intended value.

  Similarly, when the peak saturation value S peak is larger than the maximum saturation value S max, the saturation peak addition amount calculation unit 40 notifies the saturation gain calculation unit 60 to that effect and calculates the saturation gain. The unit 60 may set the saturation gain SAT_gain to less than 1. By setting the saturation gain SAT_gain to less than 1, the saturation of the pixel in the target hue area is controlled to be light, and the saturation value of the pixel causing the color saturation can be lowered to an appropriate value.

  In the above description, the image processing apparatus 1 outputs the input component as it is for the luminance component and the hue component, and does not perform the adjustment process. However, by adjusting (increasing) the saturation, it may be assumed that the original input image and the corrected image cannot be seen in the same color (same color / luminance) in terms of visual sensitivity. In this case, the image processing apparatus 1 may adjust the luminance component and the hue component in order to cancel the amount of change in visual sensitivity corresponding to the amount of increase in saturation.

  Note that the above-described image processing technique can be applied to any apparatus that performs image processing. For example, the present invention can be applied to various devices such as a television receiver, a projector device, a display device for a personal computer, a digital still camera, a digital photo frame, a digital signage, a printer device, and a digital multifunction device.

  Part or all of the processing of each processing unit constituting the image processing apparatus 1 described above can be realized as a program that operates in an arbitrary computer. When performing the above-described image processing on a moving image, the image processing apparatus 1 needs to have a sufficient CPU (Central Processing Unit) in consideration of the image size such as the number of input pixels and the frame rate. is there. Note that the processing load may be reduced by limiting a part of the above-described image processing function (for example, omitting luminance processing). In other words, it is possible to mount only the minimum necessary processing unit according to the performance of the device.

  FIG. 14 shows a configuration example of a computer that operates the processing of each processing unit constituting the image processing apparatus 1 as a program. The computer 90 has a configuration in which, for example, a CPU 91, a main memory 92, a SCSI controller 93, and an HDD (hard disk drive) 94 are connected by a bus.

  The program may be stored using various types of non-transitory computer readable media and supplied to a computer. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W and semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (random access memory)) are included. The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Luminance / saturation / hue conversion unit 20 Luminance processing unit 210 Luminance histogram calculation unit 220 Luminance APL calculation unit 230 Time constant processing unit 30 Hue region 1 (R) processing unit 310 Hue region 1 (R) saturation Histogram calculation unit 320 Saturation peak value calculation unit 330 Time constant processing unit 340 Cumulative pixel rate calculation unit 350 Time constant processing unit 31 Hue region 2 (Y) processing unit 32 Hue region 3 (G) processing unit 33 Hue region 4 (C ) Processing unit 34 Hue region 5 (B) Processing unit 35 Hue region 6 (M) processing unit 40 Saturation peak addition amount calculation unit 410 Subtractor 420 Luminance APL modulation gain calculation unit 430 Multiplier 440 Cumulative pixel rate modulation gain calculation unit 450 Multiplier 460 Multiplier 50 Pixel unit saturation peak modulation gain calculation unit 60 Saturation gain calculation unit 70 Saturation gain adjustment unit 80 RGB / YUV modulation unit 90 Computer 91 CPU
92 Main memory 93 SCSI controller 94 HDD

Claims (13)

At least one hue region processing unit that is provided corresponding to one of the plurality of hue regions and calculates a saturation peak value indicating a state of the saturation value in the associated hue region from the input image;
Based on a predetermined upper limit saturation value for each hue region and a saturation peak value calculated by the hue region processing unit, peak saturation for calculating a saturation addition value for the hue region corresponding to the hue region processing unit A degree addition amount calculation unit;
An image processing apparatus comprising: a saturation gain adjustment unit that adjusts a saturation value of a pixel included in the input image based on a saturation gain determined based on the saturation addition value. A luminance processing unit that generates a luminance histogram from the input image and calculates a luminance APL from the histogram;
The image processing apparatus according to claim 1, wherein the peak saturation value addition amount calculation unit uses the luminance APL as a modulation parameter when calculating the saturation addition value. The hue region processing unit includes a cumulative pixel rate analysis unit that calculates a cumulative pixel rate that is a ratio of pixels included in the associated hue region to the input image,
The peak saturation value addition amount calculation unit uses the cumulative pixel rate of the hue area as a modulation parameter when calculating the saturation addition value for each hue area. 2. The image processing apparatus according to 2.   The peak saturation value addition amount calculation unit uses a user gain set for each hue region as a modulation parameter when calculating the saturation addition value for each hue region. The image processing device according to any one of items 3 to 4. A pixel unit peak saturation modulation gain calculation unit that converts the saturation value and hue value of the pixel unit included in the input image into a saturation modulation gain and a hue modulation gain;
The saturation gain calculation unit that calculates the saturation gain based on the saturation modulation gain and hue modulation gain of the pixel unit and the saturation addition value, further comprising: The image processing apparatus according to claim 4.   The hue region processing unit is a first time constant processing unit that performs time constant processing on the saturation peak value according to a change rate between frames of color components included in the input image, or the cumulative pixel. The image processing apparatus according to claim 1, further comprising at least one of a second time constant processing unit that performs time constant processing on the rate.  The luminance processing unit includes a third time constant processing unit that performs time constant processing on the luminance APL in accordance with an inter-frame change rate of a color component included in the input image. The image processing apparatus according to claim 2. A noise determination unit for determining whether the S / N ratio of the input image is good,
The image processing apparatus according to claim 1, wherein the hue region processing unit switches a calculation method of the saturation peak value based on determination by the noise determination unit. . The peak saturation value addition amount calculation unit is configured such that the upper limit saturation value for each hue region is S max, the maximum saturation value calculated by the hue region processing unit is S peak, the luminance APL is APL_mod, and the cumulative pixel rate Is RUI_mod and the user gain is USER_gain, ΔS peak_V which is the saturation addition value is calculated by the following equation (1),
The saturation gain calculation unit calculates SAT_gain, which is the saturation gain, by the following equation (2) when the saturation modulation gain and the hue modulation gain for each pixel are SAT_mod and HUE_mod, respectively. The image processing apparatus according to any one of claims 5 to 8.
ΔS peak_V = (S max-S peak) * APL_mod * RUI_mod * USER_gain --- Formula (1)
SAT_gain = ((ΔS peak_V * (SAT_mod * HUE_mod)) + S peak) / S peak --- Equation (2)   The said image processing apparatus is provided with the six said hue area | region process parts corresponding to six hue area | regions of R / Y / G / C / B / M, The any one of Claim 1 thru | or 9 characterized by the above-mentioned. The image processing apparatus according to item.   A display device equipped with the image processing device according to claim 1. From the input image, calculating a saturation peak value indicating a saturation value state in at least one of the plurality of hue regions,
Based on a predetermined upper limit saturation value for each hue region and the saturation peak value, a saturation addition value of pixels included in the hue region corresponding to the saturation peak value is calculated,
An image processing method of adjusting a saturation value of a pixel included in the input image by a saturation gain determined based on the saturation addition value. A program for causing a computer to execute a process of adjusting a saturation value of an input image,
The process is
From the input image, a saturation peak value indicating a saturation value state in at least one of the plurality of hue regions is calculated,
Based on a predetermined upper limit saturation value for each hue region and the saturation peak value, a saturation addition value of pixels included in the hue region corresponding to the saturation peak value is calculated,
An image processing program for adjusting a saturation value of a pixel included in the input image by a saturation gain determined based on the saturation addition value.

Images