JP2001184503A - Device and method for processing image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To cancel a difficulty in the application of a proper correction to any arbitrary area inside an object image. SOLUTION: An arbitrary mask area inside image data is designated by a UI part (S53), mask data corresponding to this mask area are prepared (S55), an LUT is prepared for performing the optimal exposure correction to the mask area on the basis of these mask data and the exposure correction is performed (S56).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing apparatus and method for correcting image data.

[0002]

2. Description of the Related Art In an image processing apparatus for forming a multi-valued image, it is possible to obtain a clearer image by adjusting the brightness value of the brightest highlight portion or the darkest shadow portion in the image. The intended so-called white balance adjustment is performed.

When performing white balance adjustment in a conventional image processing apparatus, pixels in a predetermined high-brightness region where the luminance is several percent from the highest in the image, excluding pixels whose luminance is equal to or higher than a predetermined threshold value, are excluded. R, G, B average values were calculated, and each pixel was corrected based on the average values.

[0004]

However, in the above-described conventional image processing apparatus, the image is corrected regardless of the scene of the target image, that is, without considering the image characteristics. Therefore, when white balance adjustment is performed on an image having a high luminance level as a whole, such as a portrait of a person against a white wall as a background, it is determined that the image is overexposed inside the apparatus. Although it is corrected to a proper state, there is a problem that the important person part becomes very dark.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problem, and has as its object to provide an image processing apparatus and an image processing method capable of appropriately performing exposure correction on an arbitrary area in a target image. And

[0006]

As one means for achieving the above object, the image processing apparatus of the present invention has the following arrangement.

That is, setting means for setting a mask area in an image, luminance distribution obtaining means for obtaining a first luminance distribution in the entire image and a second luminance distribution in the mask area, and the first and second luminance distribution obtaining means. And a correction unit that corrects the image based on the correction information.

[0008] For example, the correction information creating means creates first correction information for the entire image and second correction information for the mask area.

For example, the setting means has a display means for displaying the image, and a designating means for designating an arbitrary area on the displayed image as a mask area.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment according to the present invention will be described below in detail with reference to the drawings.

[Apparatus Outline Configuration] First, a configuration for performing image correction in the image processing apparatus of the present embodiment is shown in FIG. 1 and will be described in detail. The present embodiment is characterized in that optimal correction processing is performed on a set mask area.

The configuration of the image processing unit shown in FIG. 1 includes an image input unit 2, an image buffer 3, a histogram storage unit 4, a look-up table storage unit (hereinafter, LUT storage unit) 5, a histogram creation unit 6, a lookup table. Creation unit (hereinafter, LUT creation unit) 7, image correction unit 9, image output unit 1
0, and a user interface unit (hereinafter, UI unit) 12
including.

The image input unit 2 reads the image data and the mask data from the input image holding unit 1 and writes them into the image buffer 3. The image buffer 3 holds image data and mask data. The histogram holding unit 4
The histogram of the entire image data and the histogram of the mask area are held. The LUT holding unit 5 holds a look-up table (hereinafter, LUT) used in the correction processing. The histogram creating unit 6 creates a histogram based on the image data and the mask data stored in the image buffer 3 and stores the result in the histogram holding unit 4. The LUT creation unit 7 creates a LUT by calculating parameters required for correction based on the histogram stored in the histogram holding unit 4 and creates an LUT.
It is stored in the holding unit 5. The image correction unit 9 includes the LUT holding unit 5
The correction is performed on the image data stored in the image buffer 3 based on the LUT stored in the image buffer 3 and the weight data stored in the image buffer 3, and the image data is stored in the image buffer 3 again. The image output unit 10 reads image data from the image buffer 3 and writes the read image data to the output image holding unit 11.

Reference numeral 16 denotes a controller for controlling the entire apparatus, which includes a CPU 13, a ROM 14, and a RAM 15. The CPU 13 reads out the control program stored in the ROM 14 onto the RAM 15 and executes the control program.
Control specific to the present embodiment, which is shown in a flowchart described later, is realized.

[Detailed Configuration of UI Section] FIG. 2 shows an example of a display screen in the UI section 12, and a user interface which is a feature of the present embodiment will be described.

According to FIG. 2, the UI screen is displayed on the image display unit 16.
1, an end button 162, a cancel button 163, and a target luminance value setting unit 165. Note that the cursor 164 can be arbitrarily moved on the UI screen.

The image display unit 161 displays the contents of the image buffer 3, that is, the stored image data, and when a mask area is designated by a cursor, the boundary is displayed in an overlay manner. In the present embodiment, a method for specifying a mask area is not particularly defined. For example,
A diagonal line of a circle or a rectangle may be specified, or a center and an end point may be specified. Also, an arbitrary shape may be specified by tracing the contour.

The target luminance value setting section 165 sets a corrected target luminance value in the mask area designated by the cursor 164.

The operation when the end button 162 and the cancel button 163 are pressed will be described later.

[Explanation of Mask Data] FIG. 3 shows an example of mask data in the present embodiment. The mask data corresponds to each pixel of the image data, and holds information (mask information) as to whether or not the pixel is masked. When creating the mask data, as described above, the UI unit 1
In step 2, an area in the image data is designated, and mask data is created using the designated area as a mask area. For example, when the face portion of a person is designated as a mask area by the user in the image data shown in FIG. 3A, the mask information is created as shown in FIG. 3B. That is, the white area shown in FIG. 3B is a mask area, and the black area is an unmasked area that is not masked. In this embodiment, it is assumed that at least one closed area is specified as a mask area.

FIG. 3 shows the data structure of the mask data.
It is shown in (c). As shown in the figure, the mask data has not only the mask information but also a target luminance value specified by the UI unit 12 in its header part.

[Outline of Image Processing] FIG. 4 shows an outline flowchart of image processing by the configuration shown in FIG. The processing shown by the flowcharts in the present embodiment, including the flowchart, is realized by the CPU 13 executing a control program stored in the ROM 14.

In step S 51, the image input unit 2 reads image data from the input image holding unit 1 and stores it in the image buffer 3.

In step S52, the UI unit 12 displays the image data stored in the image buffer 3. In step S53, the process waits until a user instruction is generated on the display screen of the UI unit 12, and if the end button 162 is pressed, the process proceeds to step S57. If the cancel button 163 is pressed, the process ends. On the other hand, if an area in the image data is designated in the image display unit 161, the process proceeds to step S <b> 54, and an image correction process described later is started. That is, in step S53, the user selects the image in the image displayed on the image display
An area in which optimal correction is most desired, that is, an area including the main object is selected. Hereinafter, the area specified by the user is referred to as a “mask area”.

In step S54, the mask data stored in the histogram holding unit 6 and the image buffer 3 is initialized, and in step S55, the process proceeds to step S53.
Creates mask data based on the mask area specified by. Further, the target luminance value set by the target luminance value setting unit 165 of the UI unit 12 is stored in the header of the mask data. In addition, about the area other than the mask area,
In particular, the mask data is not changed.

In step S56, the image is corrected. The details will be described later. Thereafter, in step S57, the image output unit 10 writes the image data held in the image buffer 3 into the output image holding unit 11 to output the image data.

[Outline of Image Correction Processing] FIG. 5 is a schematic flowchart of the image correction processing shown in step S56 in FIG. 4 described above.

In step S 2, a histogram is created in the histogram creating section 6 based on the image data and the mask data stored in the image buffer 3, and the result is stored in the histogram holding section 4. FIG. 6 shows the details of the operation of the histogram creation unit 6.

In step S3, the LUT creation unit 7
Then, based on the histogram stored in the histogram holding unit 4, parameters necessary for correction are calculated to create an LUT, and the result is stored in the LUT holding unit 5. FIG. 7 shows details of the operation of the LUT creation unit 7.

In step S4, the image correcting section 9
The image data stored in the image buffer 3 is corrected based on the LUT stored in the LUT holding unit 5 and stored in the image buffer 3 again.

[Histogram Creation Processing] FIG. 6 shows a flowchart of the histogram creation processing in the histogram creation section 6. That is, step S2 in FIG. 5 is shown in detail.

In step S11, the image buffer 3
, The image data and the mask data are extracted for each pixel. Note that luminance data (R, G, B) of each color of RGB is stored as image data, and data indicating presence / absence of a mask is stored as mask data.

In step S12, the image data R
The luminance L of the pixel is obtained from the GB value according to the following equation.

L = (3R + 6G + B) / 10 In this embodiment, as shown in the above equation, an example in which the luminance value is weighted and averaged with a weight of R: G: B = 3: 6: 1 will be described. Weighting other than this may be performed. For example, the weight may be obtained based on the average value of the maximum and minimum values of RGB.

In step S13, the histogram stored in the histogram storage 4 is updated. The histogram holding unit 4 stores a histogram HistL of the luminance L calculated based on the above equation and HistR, HistG, and HistR that store the accumulated luminance value of each color of RGB for each value of the luminance L of the pixel.
HistB and a histogram HistLMsk of the luminance L in the mask area indicated by the mask data are held. Note that the initial state of each histogram is all zero. The updating of each histogram is performed according to the following equation.

HistR [L] = HistR [L] + R HistG [L] = HistG [L] + G HistB [L] = HistB [L] + B HistL [L] = HistL [L] +1 In step S14 It is checked whether the pixel is masked by referring to the mask data.
If masked, the process proceeds to step S15, and if not masked, the process proceeds to step S16.

In step S15, the histogram of the mask area stored in the histogram holding unit 4 is updated according to the following equation.

HistLMsk [L] = HistLMsk [L] +1 In step S16, it is determined whether or not the histogram creation processing has been completed for all pixels. If not completed, the flow returns to step S11.

FIG. 9 shows an example of the histogram ListL of the luminance L created in this embodiment.

[LUT Creation Processing] FIG. 7 shows a flowchart of the LUT creation processing in the LUT creation section 7. That is, step S3 in FIG. 5 is shown in detail.

In step S21, the maximum luminance of the image is obtained from the histogram HistL stored in the histogram holding unit 4. In the histogram illustrated in FIG. 9, the maximum luminance is “252”.

In step S22, a predetermined amount is sequentially reduced from 255, which is the logical maximum luminance, and each time, the luminance is compared with the maximum luminance obtained in step S21, and the luminance at the time when the maximum luminance becomes larger is calculated. Find LH '. Then, a luminance region (highlight region) including a predetermined ratio of the total number of pixels in the luminance region equal to or less than the LH ′ is obtained. For example, assuming that the predetermined amount sequentially reduced in the histogram shown in FIG. 9 is “10”, the maximum luminance is “25”.
.. Are sequentially compared with 255, 245, 235..., LH ′ becomes “245”. Then, when a luminance region including 1% of the total number of pixels in the luminance region equal to or less than LH ′ is obtained as a highlight region, the highlight point LH that is the minimum luminance value of the highlight region is “2”
34 ".

Then, the average luminance (RH, GH, BH) for each of the RGB in the highlight area (the area where the luminance is not less than LH and not more than LH ') is calculated according to the following equation.

[0044]

(Equation 1)

Next, in step S23, from the histogram HistL stored in the histogram holding unit 4,
Find the minimum brightness of the image. In the histogram illustrated in FIG. 9, the minimum luminance is “5”.

In step S24, a predetermined amount is sequentially added from 0, which is the logical minimum luminance, and each time, the value is compared with the minimum luminance obtained in step S23. The luminance LS 'is obtained. Then, a luminance region (shadow region) including a predetermined ratio of the total number of pixels in the luminance region equal to or larger than LS ′ is obtained.
For example, assuming that the predetermined amount to be sequentially added in the histogram shown in FIG. 9 is “10”, the minimum brightness “5” is sequentially compared with 0, 10, 20,.
LS ′ becomes “10”. Then, when a brightness area including 1% of the total number of pixels is obtained as a shadow area in the brightness area equal to or larger than LS ′, the shadow point LS which is the maximum brightness value of the shadow area is “22”.

Then, the average luminance (RS, GS, BS) for each of the RGB in the shadow area (the area whose luminance is not less than LS ′ and not more than LS) is calculated according to the following equation.

[0048]

(Equation 2)

Next, in step S25, based on the RH, GH, BH, RS, GS, and BS obtained above, the LUTs LUT R, LUT G,
An LUTB is created and stored in the LUT holding unit 5. FIG. 10A shows an example of the created LUT.

LHTmp = (3RH + 6GH + BH) / 10 LSTmp = (3RS + 6GS + BS) / 10 Based on the luminance LHTmp and LSTmp obtained by
An LUT Tmp, which is an LUT for L correction, is created.

In this embodiment, as shown in the above equation, an example in which the luminance values are weighted and averaged with weights of R: G: B = 3: 6: 1 will be described. Of course, other weights are applied. For example, the weight may be calculated based on the average of the maximum and minimum values of RGB.

In step S26, HistLMsk is corrected with reference to LUTTmp. Note that this HistLMsk is used when calculating the exposure correction amount later.

In step S27, the corrected HistLM
The average luminance of the mask area is obtained from sk, and an LUT LMsk, which is an LUT for exposure correction, is created based on the target luminance value held in the header of the mask data.
To be stored. FIG. 10B shows an example of LUTLMsk.
The LUT LMsk shown in FIG. 10B is an LUT for optimally correcting exposure in the mask area.

The LUTR and LUT shown in FIG.
G and LUTB are LUTs for correcting contrast and color cast. Here, the highlight gamma is set in the order of G, B, and R. In this way, by increasing G and B with respect to R, it is possible to correct, for example, the color cast of a bluish image. At the same time, the contrast can be corrected.

[Image Correction Processing] FIG. 8 shows a flowchart of the image correction processing in the image correction section 9. this is,
FIG. 6 shows step S4 of FIG. 5 in detail.

In step S41, the image buffer 3
Is extracted for one pixel. Here, the image data stores the luminance (R, G, B) of each of the RGB colors.

In step S42, LUTLMsk and LUTR, LU for optimally correcting the exposure of the mask area
Based on TG and LUTB, the image data taken out of the image buffer 3 is corrected according to the following equation, and the result is overwritten on the image buffer 3.

R = LUTLMsk [LUTR [R]] G = LUTLMsk [LUTG [G]] B = LUTLMsk [LUTB [B]] In step S43, it is determined whether or not the correction processing has been completed for all pixels, and the processing is not completed. If so, the process returns to step S41.

As described above, according to the present embodiment,
By using a dedicated UI to set a mask area in the image data, create an LUT for performing optimal exposure correction on the mask area, and correct the entire image,
Optimal exposure correction can be performed on an arbitrary mask area.

Therefore, for example, in a photographic image obtained by photographing a person, mask data for the person part is created,
Since an image can be corrected by creating an LUT that optimally corrects the exposure of a person, even in a photographic image in which a person image is crushed due to a white wall as a background, the exposure of the person portion can be prevented. Can be optimally corrected.

<Modification> A modification of this embodiment will be described below.

FIG. 11 shows a schematic flowchart of the image processing in this modification. In the flowchart, steps for performing the same processing as in FIG. 4 described above are denoted by the same reference numerals, and description thereof will be omitted.

In FIG. 11, the timing of initializing the mask data in the histogram holding unit 4 and the image buffer 3 shown in step S54 is different from that in FIG. That is, in FIG. 4 described above, an example has been described in which there is only one region for which a histogram is to be created. In this modified example, as shown in FIG. 11, the timing is such that the initialization of the mask data (S54) is not performed after the image correction processing (S56).

As a result, the user can specify a plurality of regions on the UI unit 12, so that an arbitrary plurality of regions can be subjected to exposure correction in consideration of the image characteristics.

Other Embodiments In the present embodiment, an example has been described in which the luminance data is a digital value from 0 to 255, but the present invention is limited to these values, for example, the maximum value of data is 255. It is not something to be done. Further, the present invention can be realized not only by creating a histogram based on luminance but also by halftone dot density or the like.

Even if the present invention is applied to a system including a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.), an apparatus including one device (for example, a copier, a facsimile) Device).

Further, an object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and a computer (a computer) of the system or the apparatus. It is needless to say that the present invention can also be achieved by a CPU or an MPU) reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention. Also,
When the computer executes the readout program code, not only the functions of the above-described embodiments are realized, but also the operating system (OS) running on the computer based on the instructions of the program code.
It goes without saying that a case where the functions of the above-described embodiments are implemented by performing some or all of the actual processing, and the processing performs the functions of the above-described embodiments.

Further, after the program code read from the storage medium is written in the memory provided in the function expansion card inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the program code. Needless to say, the CPU included in the function expansion card or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

When the present invention is applied to the storage medium, the storage medium is provided with the above-described flowchart (FIG. 5).
The program codes corresponding to FIG. 6, FIG. 7, FIG. 8, FIG. 7, FIG. 8, FIG.

[0069]

As described above, according to the present invention, it is possible to perform appropriate exposure correction on an arbitrary region in a target image.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration for performing image correction according to an embodiment of the present invention;

FIG. 2 is a diagram showing an example of a display screen in a UI unit.

FIG. 3 is a diagram showing an example of mask data;

FIG. 4 is a flowchart showing an outline of image processing;

FIG. 5 is a flowchart showing an image correction process;

FIG. 6 is a flowchart showing a histogram creation process,

FIG. 7 is a flowchart showing a lookup table creation process;

FIG. 8 is a flowchart illustrating an image correction process;

FIG. 9 is a diagram showing an example of a histogram.

FIG. 10 is a diagram showing an example of a lookup table;

FIG. 11 is a flowchart illustrating an outline of image processing in a modification of the present embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Input image holding part 2 Image input part 3 Image buffer 4 Histogram holding part 5 Lookup table holding part 6 Histogram creation part 7 Lookup table creation part 9 Image correction part 10 Image output part 11 Output image holding part 12 UI part

Claims (13)

[Claims] A setting unit for setting a mask region in an image; a luminance distribution obtaining unit for obtaining a first luminance distribution in the entire image and a second luminance distribution in the mask region; An image processing apparatus, comprising: correction information creating means for creating correction information based on the luminance distribution of the correction information; and correction means for correcting the image based on the correction information. 2. The image processing apparatus according to claim 1, wherein the correction information creating unit creates first correction information for the entire image and second correction information for the mask area. 3. The image processing apparatus according to claim 2, wherein the setting unit includes: a display unit that displays the image; and a designating unit that designates an arbitrary area on the displayed image as a mask area. Image processing device. 4. The image processing apparatus according to claim 3, wherein the specifying unit specifies at least one area on the image. 5. The setting unit further sets a corrected target luminance value for the set mask area, and the correction information generating unit generates the second correction information based on the target luminance value. 5. The method according to claim 4, wherein
An image processing apparatus as described in the above. 6. The image processing apparatus according to claim 4, wherein the first correction information is correction information for each color component. 7. The image processing apparatus according to claim 4, wherein said first and second correction information is a look-up table. 8. A setting step of setting a mask area in an image; a luminance distribution obtaining step of obtaining a first luminance distribution in the entire image and a second luminance distribution in the mask area; An image processing method, comprising: a correction information generating step of generating correction information based on the luminance distribution of the correction information; and a correction step of correcting the image based on the correction information. 9. The correction information creating step includes: a first creation step of creating first correction information for the entire image; a second creation step of creating second correction information for the mask area; The image processing method according to claim 8, comprising: 10. The method according to claim 9, wherein the setting step includes a displaying step of displaying the image, and a specifying step of specifying an arbitrary area on the displayed image as a mask area. Image processing method. 11. The setting step further includes a target value setting step of setting a corrected target luminance value for the mask area specified in the specifying step. 11. The image processing method according to claim 10, wherein the second correction information is created based on the second correction information. 12. A recording medium on which an image processing program is recorded, the program comprising at least a code of a setting step of setting a mask area in an image, a first luminance distribution in the entire image and the mask area A code of a luminance distribution obtaining step for obtaining a second luminance distribution in the above, a code of a correction information generating step of generating correction information based on the first and second luminance distributions, and the image based on the correction information. And a code for a correction step for correcting. 13. A recording medium on which a program for realizing the image processing method according to claim 8 is recorded.