JP2019165398A - Image processing apparatus, image processing method and program - Google Patents

Abstract

To accurately convert a device dependent color signal value to a device independent color signal value even when a huge number of color profiles are not preliminarily prepared.SOLUTION: An image processing apparatus that performs color conversion of an input image using a color profile comprises: determination means that determines a correction amount of gain correction on the basis of a difference between an imaging parameter at time when the input image is imaged and an imaging parameter assumed at time when the color profile is prepared; and output means that outputs an image after color conversion which image is obtained by performing color conversion of the input image on the basis of the correction amount of the gain correction and the color profile.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a color conversion technique for color image data photographed by a digital imaging device.

  Digital imaging devices such as digital still cameras are widely used. A digital imaging device generates color image data by converting light received by a photoelectric charge conversion device (imaging device) such as a CCD or CMOS sensor into a digital signal. Color filters having different transmittances are regularly arranged on the image sensor, and light having a specific wavelength is incident on each pixel. This is referred to as image data (hereinafter, “RAW image data”) obtained. ) Will follow the arrangement of the color filters. A series of image processing such as color correction and pixel interpolation is performed on the RAW image data, and general color image data such as RGB image data is obtained. Here, the pixel value (RGB value) of the RGB image data obtained by a general digital imaging device is a color signal value (= device-dependent color signal value) depending on the characteristics of the color filter used. Therefore, it is necessary to convert to color signal values (= device-independent color signal values) compliant with standard color space standards such as XYZ, L * a * b *, and sRGB. In general, data for recording color characteristics of each device called a color profile is used for this conversion process. The color profile is based on a correspondence relationship between a device-dependent color signal value obtained by photographing a color chart having a plurality of colors under a predetermined condition and a device-independent color signal value measured by a colorimeter. Created. By referring to a color profile prepared in advance, a device-dependent color signal value is converted into a device-independent color signal value. Here, in order to obtain an accurate color conversion result, it is important that there is no deviation between the shooting conditions assumed when the color profile is created and the shooting conditions when the subject is actually shot. In this regard, in Patent Document 1, color profiles are created and held under a plurality of exposure conditions, and a color profile that matches the exposure conditions when the subject is actually photographed is applied. A technique for realizing the above has been proposed.

JP 2014-42205 A

  In order to perform high-accuracy color conversion using the technique disclosed in Patent Document 1, it is necessary to create a color profile corresponding to any exposure condition in advance and store it in a memory or the like. However, the combination of exposure conditions is enormous, and creating such a color profile takes enormous effort and time. Also, a huge memory capacity is required to store a huge number of color profiles.

  Therefore, an object of the present invention is to realize highly accurate color conversion without preparing an enormous number of color profiles in advance.

  An image processing apparatus according to the present invention is an image processing apparatus for color-converting an input image using a color profile, and includes an imaging parameter when the input image is captured and an imaging parameter assumed when the color profile is created. A determination means for determining a correction amount for gain correction based on the difference, and an output for outputting a color-converted image obtained by color-converting the input image based on the correction amount for gain correction and the color profile And means.

  According to the present invention, it is possible to accurately convert device-dependent color signal values to device-independent color signal values without preparing a huge number of color profiles in advance.

The figure which shows an example of the hardware constitutions of an image processing apparatus 1 is a functional block diagram illustrating a software configuration of an image processing apparatus according to a first embodiment. The figure which shows an example of the color profile of 3D-LUT which converts RGB value into Lab value A flowchart showing a rough flow of image processing according to the first embodiment. The flowchart which shows the detail of the color processing condition determination process based on Embodiment 1. FIG. (A) is a correspondence table when a plurality of candidate profiles are prepared with different shutter speeds, (b) is a correspondence table when a plurality of candidate profiles are prepared with different aperture values, and (c) is an ISO sensitivity. Correspondence table when preparing multiple candidate profiles with different Flow chart showing the flow of color conversion processing in a broad sense Graph showing color profile characteristics The flowchart which shows the flow of the color conversion result display process based on Embodiment 1. FIG. (A) is an example of an image after color conversion, and (b) is an example of an image in which colors outside the definition area of the profile are highlighted. Functional block diagram showing the software configuration of the image processing apparatus according to the second embodiment A flowchart showing a rough flow of image processing according to the second embodiment. Example of UI screen for specifying the reference point The flowchart which shows the detail of the color processing condition determination process based on Embodiment 2. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and all the combinations of features described in the present embodiment are not necessarily essential to the solution means of the present invention.

Embodiment 1

  FIG. 1 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 100. The image processing apparatus 100 includes a CPU 101, a RAM 102, an HDD 103, a general-purpose interface (I / F) 104, a monitor 108, and a main bus 109. A general-purpose I / F 104 connects a digital imaging device 105, an input device 106 such as a mouse or a keyboard, and an external memory 107 such as a memory card to a main bus 109. FIG. 2 is a functional block diagram showing the software configuration of the image processing apparatus 100 according to the present embodiment. The color processing condition determination unit 201, the gain correction unit 202, the color conversion unit 203, the out-of-domain determination unit 204, and The display control unit 205 is configured. These units are realized by the CPU 101 developing a predetermined program stored in the HDD 103 in the RAM 102 and executing the program. In addition, the HDD 103 stores data of one or a plurality of color profiles 210, RGB images 211, and the like. As described above, the color profile is data referred to in the color conversion process for converting the device-dependent color signal value into the device-independent color signal value. FIG. 3 is an example of a 3D-LUT color profile that converts device-dependent RGB values (8 bits) into device-independent Lab values. Note that Lab means an L * a * b * color space defined by the CIE (International Commission on Illumination), which is composed of three elements: an L-axis representing brightness, an a-axis representing chromaticity, and a b-axis. To do. The 3D-LUT in FIG. 3 is a color profile for converting RGB into Lab, but a similar color profile can be created for conversion from RGB to XYZ and from RGB to sRGB. Hereinafter, the color profile is simply referred to as “profile”. In this embodiment, a plurality of such profiles are prepared in association with various exposure conditions. In other words, profiles are created separately when shooting parameters relating to exposure determination such as shutter speed, aperture value, and ISO sensitivity are made different in a plurality of stages, and are stored in the HDD 103 or the like. One or more profiles 210 prepared and held in advance are referred to as “candidate profiles” in the sense of profile candidates actually applied in color conversion processing.

  Image data to be subjected to image processing according to the present embodiment is input from the digital imaging device 105 or the external memory 107 and processed according to the flowchart shown in FIG. Of course, the input image data may be temporarily stored in a storage device such as the HDD 103 and read and processed in accordance with a user instruction. Hereinafter, image processing centering on color conversion processing according to the present embodiment will be described along the flow of FIG. 4. In the following description, the symbol “S” represents a step.

  In S401, a candidate profile and image data to be processed are input. Here, the image data to be processed is RGB image data in which predetermined image processing is performed on the RAW image data, and each pixel has a pixel value of 8 bits (0 to 255) for each device-dependent RGB. . Further, the number of candidate profiles to be input may be one or plural as described above. That is, if only one specific profile is prepared according to the expected exposure condition when photographing a subject, that profile is input, and a plurality of candidate profiles are prepared so as to correspond to a plurality of exposure conditions. In some cases, all or part of them are input. The input RGB image data is sent to the color processing condition determination unit 201 and the gain correction unit 202. The input candidate profile is sent to the color processing condition determination unit 201.

  In S402, the color processing conditions in the next S403 are determined based on the RGB image data and the candidate profile input in S401. The color processing conditions include a correction amount used in the gain correction process and profile information actually applied in the color conversion process. Details of the color processing condition determination processing will be described later.

  In S403, the RGB image data input in S401 is subjected to color processing (gain correction processing + color conversion processing) according to the conditions determined in S402. Details of the color processing will be described later.

  In S404, a process of displaying image information on the monitor 108 based on the result of the color conversion process in S403 is executed. Details of the color conversion result display processing will be described later.

  The above is a general flow of image processing centering on color conversion processing in the image processing apparatus 100 according to the present embodiment.

(Color processing condition determination processing)
Next, the color processing condition determination process in S402 will be described in detail. This process is performed in the color processing condition determination unit 201. FIG. 5 is a flowchart showing details of the color processing condition determination processing according to the present embodiment.

  In S501, shooting parameters at the time of shooting an input RGB image are acquired. RAW image data obtained by photographing with a digital camera includes exposure conditions at the time of photographing, that is, photographing parameters relating to exposure such as shutter speed, aperture value, ISO sensitivity, model name, photographing date, lens focal length, and the like. At the same time, it is added as photographed image information. This information is also inherited and held in the RGB image data, and in this step, shooting parameters relating to exposure are acquired from the held information. Here, it is assumed that shutter speed = 1/320 seconds, aperture value = F8, and ISO sensitivity = 200 are acquired as shooting parameters of the input RGB image.

  In S502, the profile actually applied in the color conversion process is determined from the candidate profiles input in S401 based on the shooting parameters acquired in S501. Here, the relationship between the shooting parameters related to exposure and the profile will be described. FIG. 6A is a correspondence table when seven types of candidate profiles (profile_s1 to profile_s7) are prepared by fixing the aperture value and the ISO sensitivity and changing the shutter speed in seven stages. FIG. 6B is a correspondence table when seven types of candidate profiles (profile_f1 to profile_f7) are prepared by fixing the shutter speed and ISO sensitivity and varying the aperture value in seven stages. FIG. 6C is a correspondence table when seven types of candidate profiles (profile_i1 to profile_i7)) are prepared by fixing the shutter speed and the aperture value and varying the ISO sensitivity in seven stages. The larger the number of candidate profiles, the better. However, as described in the solution problem, there are various restrictions in reality. Therefore, when preparing these candidate profiles, for example, “profile_s1” in FIG. 6A, “profile_f1” in FIG. 6B, and “profile_i1” in FIG. 6C may be the same. This is because the contribution ratio of the amount of light captured by the camera is large in the profile. Thus, by making several profiles common, the man-hours for creating candidate profiles can be reduced. In the present embodiment, an optimum one is selected as a profile to be actually applied from among a plurality of candidate profiles prepared in advance based on the shutter speed among the shooting parameters for determining the exposure. In this example, three profiles “profile_s1”, “profile_s4”, and “profile_s7” are acquired from the seven candidate profiles shown in FIG. 6A, and the profile to be applied is selected as an example. An explanation shall be given. Now, the shutter speed at the time of capturing the input RGB image is 1/220 seconds, whereas the assumed shutter speed of each candidate profile is 1/125 seconds for profile_s1, 1/250 seconds for profile_s4, and 1/500 seconds for profile_s7. It is. In this case, the closest shutter speed at which the absolute value of the difference between the shutter speed at the time of shooting (= 1/320 seconds) and the difference is minimum is 1/250 seconds. Therefore, profile_s4 is selected as a profile that is actually used in the color conversion process. If there is one candidate profile input in S401, this step can be skipped. In this case, the candidate profile is determined as a color conversion processing profile as it is.

In S503, a correction amount to be used in the gain correction process is determined based on the exposure condition at the time of capturing the input RGB image acquired in S501 and the exposure condition assumed in the candidate profile selected in S502. As described above, here, the gain correction amount is determined based on the shutter speed among the photographing parameters that determine the exposure. In this case, the gain correction amount α is calculated based on the assumed shutter speed Tv _{prf of} the candidate profile selected in S502 (hereinafter referred to as “selected profile”) and the shutter speed Tv _img at the time of shooting the input RGB image. To do. Specifically, the gain correction amount α is obtained from the following equation (1).
α = Tv _prf / Tv _img (1)
Since Tv _prf = 1/250 seconds and Tv _img = 1/320 seconds, the gain correction amount α = 1.28.
In S504, the information of the selection profile determined in S502 is output to the color conversion unit 203, and the information of the gain correction amount determined in S503 is output to the gain correction unit 202.

  In this embodiment, the case where the color processing condition is determined based on the shutter speed is described as an example, but the present invention is not limited to this. It is also possible to determine the color processing conditions based on the aperture value and ISO sensitivity.

For example, when based on the aperture value, the gain correction amount β can be obtained from the following aperture value Av _img of the input RGB image and the assumed aperture value Av _prf of the selected profile by the following equation (2). Provisional β = (Av _prf / Av _img ) ² ... Formula (2)
Further, when based on ISO sensitivity, the gain correction amount γ can be obtained by the following formula (3) from the ISO sensitivity ISO _img at the time of shooting the input RGB image and the assumed ISO sensitivity ISO _prf of the selected profile.
γ = ISO _prf / ISO _img (3)
Further, based on the following formula (4) based on the above formulas (1) to (3), the gain is obtained using any two or all three shooting parameters of shutter speed, aperture value, and ISO sensitivity. It is also possible to determine the correction amount Δ.
Δ = α × β × γ (4)
For example, when the shutter speed is fixed and the shutter speed is fixed and determined based on the aperture value and ISO sensitivity, α = 1.0 is set as a constant, and the gain correction amount Δ is obtained using the above equations (2) to (4). Good. Here, an example in which the gain correction amount is derived by calculation using a mathematical formula has been described. However, a lookup table (LUT) describing the relationship between various shooting parameters and the gain correction amount is prepared, and the LUT is referred to. By doing so, the gain correction amount may be determined.

The above is the content of the color processing condition determination process in the present embodiment.

(Color processing)
Next, the color processing in S403 will be described in detail. As described above, the color processing here includes gain correction processing for adjusting the gain of the input RGB image and color conversion for converting a device-dependent color signal value to a device-independent color signal value by applying a selection profile. Processing. The gain correction process is performed in the gain correction unit 202, and the color conversion process is performed in the color conversion unit 203. FIG. 7 is a flowchart showing the flow of color processing.

In S701, the gain correction amount and the selection profile information, which are the results of the color processing condition determination process, are respectively acquired. In subsequent S702, a gain correction process is performed on the input RGB image data based on the acquired gain correction amount. Specifically, the acquired gain correction amount is multiplied by each RGB value for each pixel of the input RGB image. For example, when the gain correction amount is determined based on the shutter speed, the RGB value (Ra, Ga, Ba) of each pixel after gain correction is expressed by the following equation (5).
Ra = R × α
Ga = G × α
Ba = B × α (5)
In this way, an RGB image subjected to gain correction (hereinafter referred to as “corrected RGB image”) is generated.

  Next, in S703, the acquired selection profile is applied, and the color signal value (Ra, Ga, Ba) of each pixel of the corrected RGB image generated in S702 is converted into a device-independent color signal value. Is done. In addition to the method using the LUT profile shown in FIG. 3 described above, this conversion method includes, for example, a method based on matrix calculation using a three-dimensional conversion matrix. Here, a case will be described as an example where device-dependent RGB color signal values of the corrected RGB image are converted into device-independent XYZ color signal values using a 3 × 3 conversion matrix profile.

  Now, it is assumed that a 3 × 3 transformation matrix M as a selection profile is expressed by the following equation (6).

このとき、変換後のＸＹＺ値は、以下の式（７）によって求められる。

At this time, the XYZ value after conversion is obtained by the following equation (7).

入力ＲＧＢ画像の画素値が８ビットの色信号値で表されている場合、各画素値は０から２５５までの整数値に限られる。そこで、入力値であるＲＧＢ値の全組み合わせについて、出力値であるＸＹＺ値を予め上記式（７）に基づいて算出しておき、対応するＲＧＢ値との組を記述したＬＵＴをプロファイルとして用意しておくことで、より簡便に色変換を実現できる。このとき、ＲＧＢ値の全組み合わせではなく、代表的な組み合わせのみ記述したＬＵＴを用意しておき、ＬＵＴに記述されていないＲＧＢ値に対応するＸＹＺ値については補間演算によって導出するようにしてもよい。こうして、入力ＲＧＢ画像に対して色変換処理が施された画像（以下、「色変換後画像」と呼ぶ。）が生成される。

When the pixel value of the input RGB image is represented by an 8-bit color signal value, each pixel value is limited to an integer value from 0 to 255. Therefore, for all combinations of RGB values that are input values, XYZ values that are output values are calculated in advance based on Equation (7) above, and an LUT that describes a pair with the corresponding RGB values is prepared as a profile. By doing so, color conversion can be realized more easily. At this time, an LUT describing only representative combinations instead of all combinations of RGB values may be prepared, and XYZ values corresponding to RGB values not described in the LUT may be derived by interpolation calculation. . In this way, an image obtained by performing color conversion processing on the input RGB image (hereinafter referred to as “color-converted image”) is generated.

  In step S <b> 704, the color-converted image data generated in step S <b> 703 is output to the out-of-domain determination unit 204 and the image information display unit 205.

The above is the content of the color processing.

Note that the order of the gain correction processing in S702 and the color conversion processing in S703 may be switched. Whichever is done first, the same result is obtained. Furthermore, the color signal value of the input RGB image data may be directly converted into a device independent color signal value using a profile obtained by correcting the selection profile using the gain correction amount. For example, in the case of a profile having a 3 × 3 conversion matrix M, the corrected conversion matrix Ma based on the gain correction amount α is expressed by the following equation (8).

また、本実施形態では、デバイス非依存の色信号値としてＸＹＺ値に変換する例を説明したがこれに限られるものではない。例えば、Ｌ*ａ*ｂ*、ｓＲＧＢ、ＬＵＶなどの色信号値に変換しても構わない。

In the present embodiment, the example of converting the device-independent color signal value into the XYZ value has been described, but the present invention is not limited to this. For example, it may be converted into color signal values such as L * a * b *, sRGB, and LUV.

(Color conversion result display process)
Next, the color conversion result display process in S404 will be described in detail. This process is performed in the display control unit 205. As described above, in the color processing in S403, gain correction processing is performed on the input RGB image data. By this gain correction processing, it may be converted into a color signal value outside the color gamut defined by the profile (in the case of conversion using the above-described equation (7), the color gamut in the XYZ color space). This will be described with reference to FIG. FIG. 8 is a graph showing the characteristics of “profile_s4” in FIG. 6A, in which the horizontal axis represents the luminance of the subject and the vertical axis represents the pixel value. Note that a three-dimensional correspondence between RGB values and XYZ values is originally required, but here it is one-dimensional for convenience of explanation. In the graph of FIG. 8, a solid line 801 corresponds to “profile_s4”, and a broken line 802 corresponds to a target input image for color conversion processing. The input image is an RGB image of 8 bits (maximum pixel value is 255). Note that supplementing the correspondence table shown in FIGS. 6A to 6C described above, the slope of the solid line 801 indicating the profile characteristics increases as the shutter speed decreases. For example, if the shutter speed is doubled, the slope of the solid line 801 is also doubled. In the case of FIG. 6C as well, the slope of the solid line 801 indicating the profile characteristics increases as the ISO sensitivity value increases. In the case of FIG. 6B, the slope of the solid line 801 decreases as the aperture value increases.

  Assume that the luminance value of the subject is 1000. In this case, it can be seen that the pixel value on the input image changes from the broken line 802 to “120”. When the pixel value “120” is multiplied by, for example, the gain correction amount α = 1.28 derived based on the above-described exposure difference, the pixel value becomes “154”. By applying “profile_s4” to the pixel value “154”, the luminance value “1000” of the subject can be obtained. On the other hand, it is assumed that the luminance value of the subject is “1500”. In this case, the pixel value on the input image is “220”. When the pixel value “220” is multiplied by the gain correction amount α = 1.28, the pixel value becomes “282”, which exceeds the maximum pixel value “255” of the input image. In this case, even if “profile_s4” is applied, the correct luminance value “1500” cannot be obtained. Therefore, the maximum luminance value “1400” that can be defined by “profile_s4” is output. As described above, the color signal value of the input RGB image is converted into a color signal value outside the color gamut defined by the profile due to the influence of the gain correction, and the color conversion accuracy may be lowered.

  Therefore, in the present embodiment, when a color conversion image is displayed on the monitor 108 or the like, if a location where the color conversion processing according to the profile cannot be performed correctly is present in the color-converted image, the user can grasp the location. Like that. This will be described in detail below with reference to the flowchart of FIG.

  In S901, image data after color conversion and a selection profile are acquired. In the subsequent S902, the pixel value (here, XYZ value) of the pixel of interest among the pixels of the acquired color conversion image is acquired. The pixel of interest in this case may be determined as the pixel of interest sequentially from the upper left pixel in the image after color conversion, for example.

In S903, it is determined whether or not the color represented by the pixel value of the target pixel acquired in S902 is a color outside the color gamut defined by the selection profile. For this determination, for example, information indicating the range of the color gamut is added to each profile in advance, and whether the color signal value of the target pixel is within the range of the color gamut by referring to the information added to the selected profile. Check if it is outside. When the color gamut defined by the profile is the color gamut of the XYZ color space, it can be determined using, for example, the following equations (9) and (10).
Y ₀ ≦ Y _min (9)
Y ₀ ≧ Y _max (10)
In the above formulas (9) and (10), Y ₀ represents the Y component signal value (Y signal) representing the luminance of the pixel of interest, and Y _max and Y _min represent the maximum and minimum values of the Y signal in the selected profile. Each value is shown. The X component and Z component signal values can be determined in the same manner. When the XYZ value of the target pixel satisfies any of the conditions expressed by the above formulas (9) and (10), it is determined that the color after conversion of the target pixel is a color outside the defined region. In addition,
Even when information on the color gamut defined by the profile is not added in advance, it can be determined by deriving as appropriate. For example, when the profile is a 3 × 3 transformation matrix, RGB = (255, 255, 255) where the signal value of each XYZ component is the maximum and RGB = the minimum where the signal value of each XYZ component is the above formula (7). Substitute (0,0,0) for operation. Thereby, the maximum value and the minimum value in the signal values of the XYZ components are obtained. When the profile is an LUT, the maximum value and the minimum value of the signal values in the XYZ components in the LUT may be used as they are. As a result of the determination, if the color represented by the pixel value of the target pixel is outside the definition area of the selection profile, the process proceeds to S904, and if it falls within the definition area, the process proceeds to S905.

  In S <b> 904, information indicating the position coordinates in the image after color conversion of the pixel of interest for which it is determined that the color is out of the definition area is stored and held in the RAM 102 or the like. In next step S905, it is determined whether or not the processing has been completed for all the pixels of the color-converted image. If the processing has been completed for all pixels, the process proceeds to S906. On the other hand, if there is an unprocessed pixel, the process returns to S902 and the process is continued with the next pixel as the target pixel.

  In S906, based on the coordinate information held in S904, image information that allows the user to grasp that a color outside the definition area of the profile exists in the color-converted image is generated and displayed on the monitor 108. For example, an image in which pixels out of the definition area of the profile among the pixels of the color-converted image are highlighted is generated, and the image is displayed on the monitor 108. This allows the user to know that a color outside the definition area of the profile used at that time has occurred as a result of the color conversion process. The highlighting method may be a display method that allows the user to easily recognize the part. For example, a portion outside the definition area is displayed in monochrome (white or black) regardless of the pixel value. With reference to FIGS. 10A and 10B, a specific example of an image in which pixels of a color outside the definition area of the profile are highlighted will be described. FIG. 10A shows an image obtained by performing the above-described color conversion processing on an RGB image obtained by shooting a glossy sphere 1001 as a subject. A region 1002 in the sphere 1001 indicates a glossy portion, and it is assumed that the luminance value of the pixel in the portion is determined to be out of the definition range by the profile. In FIG. 10B, the color of the area 1002 composed of pixels determined to be out of the definition area is converted to black and highlighted. Further, the user may be able to recognize more clearly by changing the color and highlighting and highlighting a location outside the definition area.

The above is the content of the color conversion result display process. Note that, in the out-of-gamut determination process of S903, if there is no pixel determined to be out of the definition area, the color-converted image as shown in FIG.

  According to the present embodiment, it is possible to estimate a correct color conversion value even when the exposure condition assumed when creating the profile is different from the shooting condition when the subject is actually shot. Further, when the gain correction process results in a color outside the profile definition area, the user can check the pixel position. As a result, the user can take measures such as changing the exposure condition and re-taking the subject, thereby facilitating the colorimetric evaluation.

Embodiment 2

  In the first embodiment, when a color out of the color gamut defined by the profile is generated by the color conversion process, the user can grasp which part of the image after color conversion is out of the definition gamut. . However, for example, it is desirable to avoid beforehand color conversion outside the definition area of such a profile in an important part of the subject. Therefore, an embodiment in which a predetermined pixel position in an image is designated in advance and color conversion processing is performed so that the color of the designated pixel position does not fall outside the definition area of the profile will be described as a second embodiment.

Note that description common to the first embodiment, such as the hardware configuration of the image processing apparatus, is omitted or simplified, and the following description will focus on differences.

FIG. 11 is a functional block diagram showing a software configuration of the image processing apparatus 100 according to the present embodiment. A reference point setting unit 1101 and an image evaluation unit 1102 are further added to the color processing condition determination unit 201, gain correction unit 202, color conversion unit 203, out-of-definition determination unit 204, and display control unit 205 common to the first embodiment. ing. The reference point setting unit 1101 sets a specific pixel position in the input RGB image data as a reference point. Information on the position coordinates of the set reference point (reference point information) is used by the color processing condition determination unit 201 and the image evaluation unit 1102. The color processing condition determination unit 201 determines a gain correction amount based on the input RGB image data, the reference point information and the candidate profile, and selects a profile to be applied to the color conversion process. The image evaluation unit 1102 evaluates the color-converted image based on the reference point information described above. Each unit illustrated in FIG. 11 is also realized by the CPU 101 developing a predetermined program stored in the HDD 103 in the RAM 102 and executing the program.

  FIG. 12 is a flowchart showing the flow of image processing in the image processing apparatus 100 according to the present embodiment. Hereinafter, the image processing centering on the color conversion processing according to the present embodiment will be described along the flow of FIG.

  In S1201, a candidate profile and RGB image data to be processed are input. In the case of this embodiment, in order to avoid as much as possible that the color after conversion becomes a color outside the definition area of the profile, the number of candidate profiles to be input is preferably plural, and more preferably as large as possible. Here, the following description will be given assuming that a plurality of candidate profiles are input. The input RGB image data is sent to the color processing condition determination unit 201 and the gain correction unit 202. The input candidate profile is sent to the color processing condition determination unit 201.

  In S1202, a pixel position that becomes a reference point is set. This pixel position is specified based on, for example, a user operation via a GUI (graphical user interface). FIG. 13A shows an example of a UI screen for the user to specify a reference point. The user designates a desired point on a subject (here, a sphere) as a reference when evaluating the color conversion result with a mouse or the like while viewing the input image 1301 displayed in the UI screen of FIG. To do. A black point 1302 in the input image 1301 represents a reference point designated by the user. Now, it is assumed that the color of the sphere that is the subject has a horizontal gradation that gradually becomes brighter from left to right, and the center is set as the reference point. When the user designates the reference point 1302, the coordinate information of the pixel position of the reference point (here, x = 1500, y = 1000) is displayed in the UI screen with the upper left corner of the input image as the origin (0, 0). Is done. Thus, a specific pixel position in the input image is set as a reference point. Note that the method of specifying the pixel position serving as the reference point is not limited to user input via the GUI. For example, the center pixel position of the input image may be determined in advance as the default position of the reference point, and the reference point may be set without the user specifying a specific pixel position. The coordinate information of the specific pixel position set in this way is sent to the color processing condition determination unit 201 and the image evaluation unit 1102 as information (reference point information) for specifying the reference point.

  In step S1203, color processing condition determination for determining color processing conditions (gain correction amount and profile to be actually applied) in next step S1204 based on the input RGB image, a plurality of candidate profiles, and the reference point set in step S1202. Processing is executed. Details of the color processing condition determination processing will be described later.

  In step S1204, color processing (gain correction processing + color conversion processing) is performed on the input RGB image according to the color processing conditions determined in step S1203. The contents of this color processing are as described in the flow of FIG.

  In S1205, the color-converted image is evaluated based on the reference point determined in S1202. For example, the distance e of chromaticity represented by the following equation (11) is calculated for each pixel from the difference between the color signal value of the pixel corresponding to the reference point and the color signal value of the other pixels.

上記式（１１）におけるｘ_0、ｘ_1、ｙ₀、ｙ₁は、以下の式（１２）で表される。

In the above formula (11), x _0, x _1, y ₀ and y ₁ are represented by the following formula (12).

上記式（１２）において、Ｘ_0、Ｙ_0、Ｚ₀は基準点情報が示す座標位置の画素におけるＸＹＺ成分の画素値を意味し、Ｘ_1、Ｙ_1、Ｚ₁は、評価値の算出対象となる基準点以外の画素におけるＸＹＺ成分の画素値を意味する。こうして得られた画素毎に評価値を算出した結果は、例えば、基準点を囲む一部領域１３０３に対応する部分画像（評価画像）で表現される。図１３（ｂ）で示す評価画像は、各画素が基準点１３０２に対する前述の式（１１）で表される評価値を持ち、その差が小さいほど暗く、差が大きいほど明るく表現されている。いま、横方向のグラデーションを持つ球体の中心が基準点として設定されているので、評価画像の中央は差が小さく（暗く）、両端は差が大きく（明るく）なっている。なお、評価画像の表現方法はこれに限られるものではない。例えば、明暗は逆であってもよいし、所望の閾値以下は青、閾値以上は赤などと色分けして表示してもよい。このような評価画像のデータが、表示制御部２０５に送られる。
なお、上述した色度の距離ｅは評価値の一例であり、これに限定されない。例えば、ＸＹＺ各成分それぞれの色信号値における差分の絶対値を評価値としてもよい。或いは、基準となる白色のＸＹＺ値を取得してＬａｂなどの視覚均等色空間の色信号値に変換した上で、明度差、色度差、色差などを評価値とすることもできる。また、図１３の例では、基準点の周辺の部分領域を対象に評価画像を生成したが、入力画像（色変換後画像）の全体に対応する評価画像を生成してもよい。

In the above equation (12), X _0, Y _0, Z ₀ mean the pixel value of the XYZ component in the pixel at the coordinate position indicated by the reference point information, and X _1, Y _1, Z ₁ are the evaluation value calculation targets Means a pixel value of an XYZ component in a pixel other than the reference point. The result of calculating the evaluation value for each pixel obtained in this way is expressed by, for example, a partial image (evaluation image) corresponding to a partial region 1303 surrounding the reference point. In the evaluation image shown in FIG. 13B, each pixel has an evaluation value represented by the above-described equation (11) with respect to the reference point 1302, and the smaller the difference is, the darker the expression is, and the larger the difference is, the brighter the expression is. Now, since the center of a sphere having a horizontal gradation is set as a reference point, the difference is small (dark) at the center of the evaluation image, and the difference is large (bright) at both ends. Note that the method of expressing the evaluation image is not limited to this. For example, the lightness and darkness may be reversed, or may be displayed in different colors such as blue below a desired threshold and red above the threshold. Such evaluation image data is sent to the display control unit 205.
The chromaticity distance e described above is an example of an evaluation value, and is not limited to this. For example, the absolute value of the difference in the color signal value of each XYZ component may be used as the evaluation value. Alternatively, after obtaining a white XYZ value as a reference and converting it into a color signal value in a visual uniform color space such as Lab, a brightness difference, a chromaticity difference, a color difference, or the like can be used as an evaluation value. In the example of FIG. 13, the evaluation image is generated for a partial region around the reference point, but an evaluation image corresponding to the entire input image (color-converted image) may be generated.

  In S1206, the evaluation image generated in S1205 is displayed on the monitor 108 instead of the color-converted image generated in S1204 (or together with the color-converted image).

The above is a general flow of image processing centering on color conversion processing in the image processing apparatus 100 according to the present embodiment.

(Color processing condition determination processing)
Next, the color processing condition determination process in S1203 will be described in detail. This process is performed in the color processing condition determination unit 201. FIG. 14 is a flowchart showing details of the color processing condition determination processing according to the present embodiment.

  In S1401, the shooting parameters related to the exposure at the time of shooting the input RGB image and the reference point information set in S1202 are acquired. In subsequent S1402, one candidate profile to be noticed is determined from the plurality of candidate profiles acquired in S1201 described above.

In S1403, the correction amount used in the gain correction process is determined based on the shooting parameters acquired in S1401 and the assumed shooting parameters of the target candidate profile determined in S1402. For example, when the gain correction amount is determined based on the shutter speed, the gain correction amount is set as the assumed shutter speed Tv _{prf of} the target candidate profile and the shutter speed Tv _img at the time of shooting the input RGB image using the above-described equation (1). α is determined.

  In S1404, based on the gain correction amount determined in S1403, the pixel value of the pixel specified by the reference point information is corrected using, for example, the above equation (5). In step S1405, it is determined whether the corrected pixel value is a color within the color gamut defined by the target candidate profile. This determination may be performed according to S903 in the flow of FIG. 9 of the embodiment. That is, when the color gamut defined by the profile is the color gamut of the XYZ color space, the determination may be made by using an expression in which the direction of the inequality sign is reversed in the expressions (9) and (10). As a result of the determination, if the corrected pixel value of the pixel specified by the reference point information is within the definition area of the target candidate profile, the process proceeds to S1406, and if it is outside the definition area, the process proceeds to S1407.

  In S1406, the RAM 102 and the like hold combination information of the profile and gain correction amount in which the attention candidate profile determined in S1402 and the gain correction amount determined in S1403 are associated with each other. In subsequent 1407, it is determined whether or not the processing has been completed for all of the plurality of candidate profiles acquired in S1201. If the processes for all candidate profiles have been completed, the process proceeds to S1408. On the other hand, if there is an unprocessed candidate profile, the process returns to S1402 and the next candidate profile is continued as a candidate profile to be noticed.

  In S1408, the combination having the minimum gain correction amount is selected from all the combination information held in S1406, and the gain correction amount and profile information related to the combination are output. That is, information on the gain correction amount included in the selected combination is output to the gain correction unit 202, and information on candidate profiles associated therewith is output to the color conversion unit 203.

The above is the content of the color processing condition determination process in the present embodiment.

According to the present embodiment, it is possible to perform color conversion so that the color at the designated pixel position in the input image surely falls within the color gamut defined by the profile. As a result, it is possible to avoid the occurrence of a color outside the definition area of the profile in an important part of the subject in advance, and it is possible to realize color conversion that is more desirable for the user.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 201 Color processing condition determination part 201 Gain correction part 203 Color conversion part

Claims (17)

An image processing apparatus for color-converting an input image using a color profile,
Determination means for determining a correction amount for gain correction based on shooting parameters when shooting the input image and shooting parameters when creating the color profile;
Output means for outputting a color-converted image obtained by color-converting the input image based on the correction amount of the gain correction and the color profile;
An image processing apparatus comprising: Correction means for performing the gain correction on the input image using the correction amount;
Color conversion means for performing color conversion using the color profile for the input image after performing the gain correction;
Further comprising
The image processing apparatus according to claim 1, wherein the output unit outputs a color-converted image output from the color conversion unit. Color conversion means for performing the color conversion on the input image using the color profile;
Correction means for performing the gain correction on the color-converted input image;
The image processing apparatus according to claim 1, wherein the output unit outputs an image output from the correction unit as a color-converted image. Correction means for correcting the color profile using the determined correction amount;
Color conversion means for performing the color conversion using the color profile after the correction is performed on the input image;
Further comprising
The image processing apparatus according to claim 1, wherein the output unit outputs a color-converted image output from the color conversion unit.   The image processing apparatus according to claim 2, wherein the photographing parameter is a parameter relating to exposure, and includes at least one of a shutter speed, an aperture value, and ISO sensitivity.   When the parameter relating to exposure is a shutter speed, the determining unit determines a value obtained by dividing a shutter speed assumed when the color profile is created by a shutter speed when the input image is taken as the correction amount. The image processing apparatus according to claim 5.   When the parameter relating to exposure is an aperture value, the determination unit obtains, as the correction amount, a square of a value obtained by dividing the aperture value assumed when the color profile is created by the aperture value when the input image is captured. The image processing apparatus according to claim 5, wherein the determination is performed.   When the parameter relating to exposure is ISO sensitivity, the determining unit determines, as the correction amount, a value obtained by dividing the ISO sensitivity assumed when the color profile is created by the ISO sensitivity when the input image is captured. The image processing apparatus according to claim 5. Storage means for storing a plurality of color profiles created by varying the exposure-related parameters at a plurality of levels;
The determination means selects a color profile whose shooting parameter level assumed at the time of creation is closest to the shooting parameter level at the time of shooting the input image from the plurality of color profiles,
The color conversion unit performs the color conversion using a selected color profile.
The image processing apparatus according to claim 5.   The image processing apparatus according to claim 2, further comprising display means for displaying the color-converted image.   When displaying the color-converted image, the display means enables the user to identify pixels whose color signal values after the color conversion are outside the defined area of the color profile used in the color conversion. The image processing apparatus according to claim 10, wherein the image processing apparatus displays the image. Storage means for storing a plurality of color profiles created by varying the parameters relating to exposure at a plurality of levels;
Setting means for setting a reference point for the input image;
Further comprising
The determination unit is configured to provide the plurality of color values on condition that a pixel value when gain correction is performed on a pixel corresponding to the reference point in the input image is within a color gamut defined by the color profile. The image processing apparatus according to any one of claims 5 to 8, wherein a color profile used for the color conversion is selected from profiles.   In the case where there are a plurality of color profiles in which the pixel value when the gain correction is performed on the pixel corresponding to the reference point in the input image is within the defined color gamut, The image processing apparatus according to claim 12, wherein a color profile having a minimum correction amount when gain correction is performed is selected as a color profile used for the color conversion. Evaluation means for generating an evaluation image obtained by evaluating the image after the color conversion based on the reference point;
Display means for displaying the evaluation image;
The image processing apparatus according to claim 12, further comprising:   The evaluation image is generated by calculating a difference between a color signal value of a pixel corresponding to the reference point and a color signal value of other pixels and calculating an evaluation value based on the obtained difference for each pixel. The image processing apparatus according to claim 14. An image processing method for color-converting an input image using a color profile,
Determining a correction amount for gain correction based on a difference between a shooting parameter when shooting the input image and a shooting parameter assumed when the color profile is created;
Outputting a color-converted image obtained by color-converting the input image based on the correction amount of the gain correction and the color profile;
An image processing method comprising:   A program for causing a computer to operate as the image processing apparatus according to any one of claims 1 to 15.

Images