JP5022811B2 - Image processing apparatus, image processing method, and imaging apparatus - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method, and more particularly to an image processing apparatus and an image processing method for correcting a red-eye area included in an image captured by an imaging apparatus.
The present invention further relates to an imaging apparatus having a function of correcting a red-eye area in a captured image.

2. Description of the Related Art Conventionally, a digital camera that captures a subject optical image using a photoelectric conversion element and records the captured image as digital data on a recording medium represented by a memory card is known.
There is also known a red-eye phenomenon in which a person's eyes are photographed in red when a person is photographed with a flash (photographed with the flash turned on). The red-eye phenomenon occurs when a blood vessel of the retina is imaged, and is particularly likely to occur when flash photography is performed with the pupil wide open, such as in a dark place.

  In order to alleviate the occurrence of such a red-eye phenomenon, an imaging device having a red-eye alleviation function is known in which a lamp or a flash is lit once (pre-emission) immediately before flash photography and the pupil is contracted to perform flash photography. ing. However, there is a problem that the red-eye mitigation effect is small when the subject is not gazing at the camera during the pre-flash.

  The red-eye phenomenon occurs in the same manner regardless of whether it is a digital camera or a film camera (silver salt camera). However, in the case of a digital camera, image processing can be easily applied to a captured image. Therefore, a red-eye correction technique has been proposed in which a human face or eye included in a captured image is detected and a red-eye region is corrected automatically or semi-automatically when it is determined that a red-eye phenomenon has occurred. Japanese Patent Application Laid-Open No. H10-228667 discloses that a region regarded as a skin color in a captured image is detected as a face region, and a red-eye region is detected in the detected face region. Patent Document 2 discloses that a camera having a red-eye correction function detects a face area in a captured image by using a combination of an algorithm that compares a detected face shape model with a face probability and pattern matching. .

Japanese Patent Laid-Open No. 10-0233929 JP 2001-309225 A

  In these conventional technologies, a candidate for a red eye region (red eye candidate region) is detected using the degree of redness of a pixel as an evaluation value, and a final red eye region is identified from the size and shape of the red eye candidate region. Thus, the pixels in the red-eye area are corrected.

  When red-eye correction is performed by such a method, a corrected area and an uncorrected area can be distinguished, and an unnatural correction result may occur. In addition, because the red eye candidate area is detected based on the red color of the pixel, the red eye candidate area may be erroneously detected as a congested part such as the head of the eye, a red part of the skin, or a skin color part of a person photographed under a red light source. There were many problems, and the probability of erroneous correction was high.

  The present invention has been made in view of such a problem of the prior art, and an object of the present invention is to realize an image processing apparatus and an image processing method capable of correcting a red-eye region more effectively.

The above-described object is to detect, for each pixel included in a block, a detecting unit that detects an eye area determined to be an eye from the image, a dividing unit that divides the eye area into a plurality of blocks, and a plurality of blocks. an evaluation value calculating means for calculating an evaluation value that indicates the red of the pixel, an index indicating the degree of variation of the evaluation value, before and index calculation means for calculating for each Chivu lock, correction values for the individual pixels included in the eye area Correction value calculation means for calculating the correction value based on the evaluation value calculated for the pixel and the index, and correction means for applying the correction value to the pixel in the eye area. This is achieved by an image processing apparatus that calculates a correction value by correcting an evaluation value based on an index so that the correction value becomes smaller as a pixel in a smaller block .

  The above-described object is also achieved by an imaging apparatus having the image processing apparatus of the present invention, wherein the captured image is corrected by the image processing apparatus.

Further, the above object is, from within images, a detection step of detecting an eye region is determined to eye, a dividing step of dividing the eye region into a plurality of blocks, for each of the multiple blocks, included in the block for each pixel, the evaluation value calculation step of calculating an evaluation value that indicates the red of the pixel, an index indicating the degree of variation of criticism value, the index calculation step of obtaining for each block, correction for each of pixels included in the eye area values, and the calculated evaluation value for the pixel has a correction value calculation step of calculating, based on the index, and a correction step of applying the compensation value to the pixels in the eye region, the compensation values calculating step, also it affects the image processing method characterized by calculating a correction value by correcting on the basis of the index evaluation value such as the pixels in a small block variation degree correction value decreases are achieved.

  With such a configuration, according to the image processing apparatus and the image processing method of the present invention, the red-eye area can be effectively corrected.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings.
(Configuration of imaging device)
FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus 100 as an example of an image processing apparatus according to an embodiment of the present invention.
101 is a lens, 12 is a shutter having a diaphragm function, 14 is an image sensor such as a CCD or CMOS sensor that converts an optical image into an electrical signal, and 16 is an A / D that converts an analog signal output of the image sensor 14 into a digital signal. It is a converter.

  The timing generator 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory controller 22 and the system controller 50.

  The image processing unit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control unit 22.

  Further, the image processing unit 20 performs predetermined calculation processing using the captured image data. Then, based on the obtained calculation result, the system control unit 50 controls the exposure control unit 40 and the distance measurement control unit 42, and TTL (through the lens) AF (autofocus), AE (automatic exposure). , EF (flash pre-emission) function is realized.

  Further, the image processing unit 20 performs predetermined calculation processing using the captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained calculation result.

  The memory control unit 22 controls the A / D converter 16, the timing generation unit 18, the image processing unit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression unit 32.

  The output data of the A / D converter 16 is sent to the image display memory 24 or the memory 30 via the image processing unit 20 and the memory control unit 22 or the output data of the A / D converter 16 is directly sent to the memory control unit 22. Is written to.

  Display image data written in the image display memory 24 is displayed by an image display unit 28 such as an LCD or an organic EL display via a D / A converter 26. An electronic viewfinder function can be realized by sequentially displaying the captured image data on the image display unit 28.

  The image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control unit 50. When the display is turned off, the power consumption of the imaging apparatus 100 can be significantly reduced. it can.

The memory 30 is a storage device for storing captured still images and moving images, and has a storage capacity sufficient to store a predetermined number of still images and a predetermined time of moving images. Therefore, even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed.
The memory 30 can also be used as a work area for the system control unit 50.

The compression / decompression unit 32 reads an image stored in the memory 30, performs well-known data compression processing or decompression processing using adaptive discrete cosine transform (ADCT), wavelet transform, etc., and stores the processed data in the memory 30. Write to.
The exposure control unit 40 controls the shutter 12 having a diaphragm function, and also has a flash light control function in cooperation with the flash 48.

  The distance measurement control unit 42 controls the focusing of the lens 10, and the zoom control unit 44 controls the zooming of the lens 10. The barrier control unit 46 controls the operation of the protection unit 102 that is a lens barrier for protecting the lens 10.

The flash 48 functions as an auxiliary light source at the time of photographing and also has a light control function. It also has a function of projecting AF auxiliary light.
The red-eye reduction lamp 49 is a light source for reducing the pupil of a person who is a subject by emitting light for about 1 second before photographing using the flash 48. As described above, by reducing the pupil immediately before shooting, the red-eye phenomenon at the time of flash shooting can be reduced.

  The exposure control unit 40 and the distance measurement control unit 42 are controlled using the TTL method. Based on the calculation result obtained by calculating the captured image data by the image processing unit 20, the system control unit 50 performs the exposure control unit 40 and the distance measurement. The controller 42 is controlled.

  The system control unit 50 is a CPU, for example, and controls the entire imaging apparatus 100 by executing a program stored in the memory 52. The memory 52 stores constants, variables, programs, etc. for operating the system control unit 50.

  The display unit 54 is configured by a combination of output devices such as an LCD, an LED, and a speaker, for example, and outputs an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control unit 50. . One or a plurality of display units 54 are installed at a position in the vicinity of the operation unit 70 of the imaging apparatus 100 that is easily visible. A part of the display unit 54 is installed in the optical viewfinder 104.

  Examples of the display contents of the display unit 54 include the following. Single shot / continuous shooting display, self-timer display, compression rate display, number of recorded pixels, number of recorded images, number of remaining shots displayed, shutter speed display, aperture value display, exposure compensation display, flash display, red-eye reduction display, Macro shooting display and buzzer setting display. Clock battery level display, battery level display, error display, multi-digit number information display, recording medium 200 and 210 attachment / detachment status display, communication I / F operation display, date / time display, external computer Display showing connection status. Focus display, shooting preparation completion display, camera shake warning display, flash charge display, recording medium writing operation display, etc. A part of this is displayed in the optical viewfinder 104.

  Further, among the display contents of the display unit 54, examples of what is displayed by an LED or the like include the following. Focus display, shooting preparation completion display, camera shake warning display, camera shake warning display, flash charge display, flash charge completion display, recording medium writing operation display, macro shooting setting notification display, secondary battery charge status display, etc.

Of the display contents of the display unit 54, what is displayed by a lamp or the like includes, for example, a self-timer notification lamp. This self-timer notification lamp may be used in common with AF auxiliary light.
The nonvolatile memory 56 is an electrically erasable / recordable memory, and for example, an EEPROM or the like is used.

  The mode dial 60, the first shutter switch (SW1) 62, the second shutter switch (SW2) 64, the power switch 66, the wireless connection button 68, and the operation unit 70 allow the system control unit 50 to start and stop a predetermined operation. It is an operation member for instructing. These operation members are composed of buttons, switches, dials, touch panels, line-of-sight detection devices, voice recognition devices, or combinations thereof.

Here, a specific description of these operating means will be given.
The mode dial 60 is a switch for switching and setting each function mode such as power-off, automatic shooting mode, program shooting mode, panoramic shooting mode, playback mode, multi-screen playback / erase mode, PC connection mode, and the like.

  The first shutter switch (SW1) 62 is turned on by a first stroke (for example, half-pressed) of a shutter button (not shown) provided in the digital camera 100. When the first shutter switch (SW1) 62 is turned on, AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, EF processing, and the like are started.

  The second shutter switch (SW2) 64 is turned on by a second stroke (for example, full press) of a shutter button provided in the digital camera 100, and instructs to start a series of processes including exposure processing, development processing, and recording processing. . First, in the exposure process, a signal read from the image sensor 14 is written into the memory 30 via the A / D converter 16 and the memory control unit 22, and further, an operation in the image processing unit 20 and the memory control unit 22 is performed. Development processing using is performed. Further, a recording process is performed in which the image data is read from the memory 30, compressed by the compression / decompression unit 32, and written to the recording medium 200 or 210.

  The image display ON / OFF switch 66 is a display ON / OFF setting switch of the image display unit 28. When shooting using the optical viewfinder 104, it is possible to save power by turning off the display of the image display unit 28 made of, for example, a TFT LCD or the like to cut off the current supply.

  The flash setting button 68 is a button for setting / changing the operation mode of the flash. Modes that can be set in the present embodiment include auto, constant light emission, red-eye reduction auto, and constant light emission (red-eye reduction). Auto is a mode in which a flash is automatically emitted according to the brightness of the subject. The constant light emission is a mode in which a flash is always emitted to take a picture. The red-eye reduction auto is a mode in which a flash is automatically emitted in accordance with the brightness of the subject and the red-eye reduction lamp is always emitted when the flash is emitted. The constant light emission (red-eye reduction) is a mode in which a red-eye reduction lamp and a strobe are always emitted.

  The operation unit 70 includes various buttons and a touch panel, and includes, for example, the following buttons and switches. Menu button, set button, macro button, multi-screen playback page break button, single-shot / continuous-shot / self-timer switching button, menu move + (plus) button, menu move-(minus) button. Play image move + (plus) button, play image move-(minus) button, shooting image quality selection button, exposure compensation button, date / time setting button, compression mode switch, etc.

  The compression mode switch is a switch for selecting a compression rate of JPEG (Joint Photographic Expert Group) compression, or for selecting a RAW mode in which a signal of an image sensor is directly digitized and recorded on a recording medium.

  In the present embodiment, for example, a normal mode and a fine mode are prepared as JPEG compression modes. The user of the imaging apparatus 100 can perform shooting by selecting the normal mode when emphasizing the data size of the captured image and selecting the fine mode when emphasizing the image quality of the captured image.

  In the JPEG compression mode, the compression / decompression unit 32 reads the image data written in the memory 30, compresses it to the set compression rate, and then records it on the recording medium 200, for example.

  In the RAW mode, the image data is read as it is for each line according to the pixel arrangement of the color filter of the image sensor 14, and the image data written in the memory 30 is read via the A / D converter 16 and the memory control unit 22. Read and record on the recording medium 200.

  The power supply control unit 80 includes a battery detection unit, a DC-DC converter, a switch unit that switches a block to be energized, and the like, detects whether or not a battery is installed, the type of battery, and the remaining battery level. The DC-DC converter is controlled based on an instruction from the system control unit 50, and a necessary voltage is supplied to each unit including the recording medium for a necessary period.

  The power source 86 includes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, an AC adapter, or the like, and is attached to the imaging apparatus 100 by connectors 82 and 84.

  Recording media 200 and 210 such as a memory card and a hard disk have recording units 202 and 212 constituted by a semiconductor memory and a magnetic disk, interfaces 204 and 214 and connectors 206 and 216 with the imaging apparatus 100. The recording media 200 and 210 are attached to the imaging apparatus 100 via medium-side connectors 206 and 216 and connectors 92 and 96 on the imaging apparatus 100 side. Interfaces 90 and 94 are connected to the connectors 92 and 96. Whether or not the recording media 200 and 210 are attached is detected by the recording medium attachment / detachment detection unit 98.

  In the present embodiment, the imaging apparatus 100 is described as having two interfaces and connectors for attaching a recording medium. However, any number of interfaces and connectors for attaching a recording medium can be provided. Different standard interfaces and connectors may be used for each system.

By connecting various communication cards to the connectors 92 and 96, image data and management information attached to the image data can be transferred to and from peripheral devices such as other computers and printers.
The barrier 102 covers the imaging unit including the lens 10 of the imaging device 100 to prevent the imaging unit from being dirty or damaged.

  The optical finder 104 is, for example, a TTL finder, and forms an image of a light beam that has passed through the lens 10 using a prism or a mirror. By using the optical viewfinder 104, it is possible to take a picture without using the electronic viewfinder function of the image display unit 28. Further, as described above, some functions of the display unit 54 such as an in-focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, an exposure correction display, etc. are provided in the optical viewfinder 104. Information is displayed.

The communication unit 110 performs communication processing conforming to various standards such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, wireless communication such as IEEE802.11x.
A connector (antenna in the case of wireless communication) 112 connects the imaging apparatus 100 to another device via the communication unit 110.

FIG. 2 is a perspective view showing an example of the appearance of the imaging apparatus 100 having the configuration of FIG. 1, and the same reference numerals are assigned to the configurations shown in FIG.
A power button 201 is a button for turning on / off the power of the imaging apparatus 100. The MENU button 205 is a button for displaying a menu screen for changing shooting parameters and camera settings and for canceling the display. The SET button 203 is used to determine setting values and menu items. The delete button 207 is used when designating deletion of a captured image. The DISP button 208 is a button for switching a combination of information displayed on the image display unit 28. The cross button 209 is composed of up / down / left / right buttons, and is used to change a menu screen, move a selection item, and switch a display image in a reproduction mode. These buttons are included in the operation unit 70.

(Imaging and recording processing)
FIG. 3 is a flowchart for explaining the overall flow of the imaging / recording process in the imaging apparatus according to the present embodiment.
In this embodiment, it is assumed that red-eye correction processing is performed at the time of imaging, and the corrected image (corrected image) is stored in a recording medium. However, it is needless to say that the red-eye correction process can be executed even when shooting is not performed.

  When the shutter button is fully pressed and SW2 is turned on, shooting processing is performed (S302). The subject image converted by the image sensor 14 into an analog electrical signal is converted into a digital signal by the A / D converter 16. Then, the image processing unit 20 performs photographed image processing on the digital signal output from the A / D converter 16 (S303).

  Although details are omitted, in this captured image processing, the signal read from the image sensor 14 is converted into pixel-unit image data represented by, for example, a luminance component Y and a color difference component UV based on an array of color filters, for example. Processing to convert to is included.

Next, the image processing unit 20 performs face detection processing for detecting an area (face area) determined to be a face in the captured image using the image data (S304).
The face detection process may be performed by any method, but can be performed by using pattern matching using a face shape model as described in, for example, Japanese Patent Laid-Open No. 2001-309225. Further, it is possible to obtain a face reliability, which is a scale indicating the likelihood of a face, based on the degree of coincidence (similarity) between the model and the image area at the time of pattern matching.

  In the present embodiment, it is assumed that the image processing unit 20 obtains 10-level face reliability of 1 to 10. Here, reliability 1 indicates that the degree of coincidence at the time of pattern matching is high and that the possibility of being a face is high, and reliability 10 indicates that the degree of coincidence at the time of pattern matching is low and that the possibility of being a face is low. Shall.

  The image processing unit 20 also obtains the size of the detected face area (face size). In the present embodiment, it is assumed that the image processing unit 20 obtains the face having the larger maximum number of pixels in the horizontal direction and the vertical direction of the face area as the face size.

  The image processing unit 20 has a reliability equal to or less than a preset threshold FR (3 in the present embodiment) for the detected face area, and the face size is a threshold FS (200 in the present embodiment). The above face area is set as a finally detected face area.

  When the face area is detected by the face detection process, the image processing unit 20 performs a red-eye correction process (S305). Specifically, an area determined to be an eye (eye area) is detected from the face areas detected by the face detection process, and the red-eye correction process is applied to the eye area. At this time, it is desirable that the eye area is set so as to completely include the area where the eyes actually exist. Therefore, although it depends on the accuracy of the face detection and eye area detection algorithms, if it is determined that the detection accuracy is low, a relatively large area centering on the area where the eyes are supposed to exist is used as the eye area. Is set. In the present embodiment, when the width of the face is F, the width of the eye area for one eye is set to F × 0.4.

  The image processing unit 20 outputs the image data subjected to the red-eye correction process to the memory 30 through the memory control unit 22. The memory control unit 22 writes the image data from the image processing unit 20 in the image display memory 24 in accordance with the resolution of the image display unit 28. As a result, the corrected image is displayed on the image display unit 28 (quick review operation). On the other hand, the image data written in the memory 30 is encoded, for example, in the JPEG format by the compression / decompression unit 32. The encoded image data is recorded on the recording medium 200 or 210 according to the control of the system control unit 50, for example, according to the DCF standard (S307).

  If no face area is detected in S304, the red-eye correction process in S305 is skipped, and an uncorrected image is displayed and recorded in S306.

(Red-eye correction processing)
Next, details of the red-eye correction process performed by the image processing unit 20 in S305 of FIG. 3 will be described.
FIG. 4 is a diagram schematically illustrating internal processing and signal flow that constitute red-eye correction processing in the imaging apparatus according to the present embodiment. The functions of the processing units in FIG. 4 are actually realized by the image processing unit 20 in terms of software, for example.

  As described above, the image processing unit 20 detects an eye region from the face region detected in S304 of FIG. Then, the pixel data (Yuv format) 401 included in the eye area is supplied to the evaluation value calculation processing unit 402. Here, it is assumed that the eye region has a rectangular shape with a horizontal width pixel number W and a vertical width pixel number H.

The evaluation value calculation processing unit 402 converts each pixel of the pixel data 401 into the RGB format using the following formula and obtains an evaluation value E.
R = Y + 1.402 * V Formula (1)
G = Y−0.344 * U−0.714 * V Formula (2)
B = Y + 1.772 * U Formula (3)
E = (2 * R−G−B) / 2 Formula (4)

  As is clear from the equation (4), the evaluation value E increases as the red color of the pixel increases. The evaluation value calculation processing unit 402 outputs the obtained evaluation value E to the block division variance calculation processing unit 403 and the correction coefficient calculation processing unit 406.

  A block division variance calculation processing unit 403 as a division unit and an index calculation unit divides the eye area into a plurality of areas (blocks), and calculates the variance D of the evaluation value E of the pixels included in each block. In this embodiment, as shown in FIG. 5, the eye area 500 is divided into a total of 25 blocks of 5 in the horizontal direction and 5 in the vertical direction, and the variance D of the evaluation value E for each block is calculated. The variance D is calculated using the following equations (5) and (6).

In the equation, E (I) is an individual evaluation value, and N is the number of pixels included in the block.
The block division variance calculation processing unit 403 outputs the obtained variance D for each block to the enlargement / resize processing unit 404.

  The enlargement / resize processing unit 404 enlarges and resizes the 25 (5 × 5) variances D obtained for each block by, for example, linearly interpolating W × H pieces of data having a width W and a height H. The variance data resized to W × H data is output to the maximum value coordinate detection processing unit 405 and the correction coefficient calculation processing unit 406.

  A maximum value coordinate detection processing unit 405 serving as a position detecting unit detects a pixel position (MaxX, MaxY) of data having the maximum value among the enlarged and resized distributed data. Then, the detected pixel position is output to the correction coefficient calculation processing unit 406.

FIG. 6 is a diagram illustrating an eye region where a red-eye phenomenon occurs and an example of a distribution of dispersion.
In FIG. 6, reference numeral 601 denotes a distribution of dispersion in the X-axis direction when Y = MaxY, where the horizontal axis represents the X coordinate and the vertical axis represents the magnitude of the dispersion. Reference numeral 602 denotes a distribution of dispersion in the Y-axis direction when X = MaxX, where the horizontal axis indicates the Y coordinate and the vertical axis indicates the magnitude of the dispersion.

  A correction coefficient calculation processing unit 406 serving as a correction value calculation unit determines a weighting coefficient corresponding to the distance from the pixel position (MaxX, MaxY) detected by the maximum value coordinate detection processing unit 405. Specifically, as shown in FIG. 7, a weighting factor that is larger as it is closer to the pixel position (MaxX, MaxY) is determined.

  In FIG. 6, reference numeral 603 denotes a distribution of weighting coefficients in the X-axis direction when Y = MaxY, where the horizontal axis indicates the X-coordinate and the vertical axis indicates the size of the weighting coefficient. Reference numeral 604 denotes a distribution of weighting coefficients in the Y-axis direction when X = MaxX, where the horizontal axis indicates the Y coordinate and the vertical axis indicates the magnitude of the weighting coefficient. In the example shown in FIG. 6, a weighting factor that decreases linearly in proportion to the distance from the pixel position (MaxX, MaxY) is determined.

Then, the correction coefficient calculation processing unit 406 receives the evaluation value E (x, y) from the evaluation value calculation processing unit 402, the data D (x, y) obtained by performing the resizing process on the variance, and the weight coefficient W (x, y). Then, the correction coefficient H (x, y) is calculated by the following equation (7).
H (x, y) = E (x, y) × D (x, y) × W (x, y) Equation (7)
Here, (x, y) is a coordinate in the eye region, and in this example, 1 ≦ x ≦ W and 1 ≦ y ≦ H.
Thus, the correction coefficient H increases as the evaluation value E increases, the variance D increases, and the weighting coefficient increases.

The correction coefficient H obtained by the correction coefficient calculation processing unit 406 is output to the correction processing unit 407. The correction processing unit 407 executes red-eye correction processing by applying the correction coefficient H to the pixel data 401 in the eye region. Specifically, the luminance component of the Yuv format pixel data 401 is Y (x, y), the color components are U (x, y) and V (x, y), and the corrected color component of the pixel data is U. Assuming that '(x, y) and V' (x, y), correction is performed by the following equations (8) and (9).
U ′ (x, y) = U (x, y) × (1−H (x, y)) Equation (8)
V ′ (x, y) = V (x, y) × (1−H (x, y)) Equation (9)

The correction processing unit 407 overwrites the corrected pixel data 408 on the corresponding eye area data in the captured image data.
When the above process is executed for all other eye regions, the red-eye correction process is terminated.

  In this embodiment, each eye area is divided into 5 × 5 blocks. However, the number of divisions is not limited to 5 × 5, and may be subdivided into 9 × 9 or 9 × 7. It is also possible to set different numbers of divisions in the horizontal direction and the vertical direction as shown in FIG. The number of divisions may be changed according to the accuracy of the eye area detection algorithm. For example, if a detection method that can detect the position of the pupil with high accuracy is used, the processing time is reduced by setting the eye area relatively smaller than the area where the eyes are considered to exist and reducing the number of divisions. Shortening can be achieved. Alternatively, even if the eye area detection algorithm is the same, the higher the reliability of the obtained eye area detection result, the smaller the eye area size may be set.

In the present embodiment, the correction processing unit 407 reduces the redness of the red-eye region by greatly reducing the color component of the pixel data as the correction value increases. However, for example, it is also possible to set the corrected target color components Ut and Vt and correct them by the following equation.
U ′ (x, y) = U (x, y) × (1−H (x, y)) + Ut × H (x, y)
Formula (10)
V ′ (x, y) = V (x, y) × (1−H (x, y)) + Vt × H (x, y)
Formula (11)

  In this embodiment, the variance of the evaluation value E is used as an example of an index of the degree of variation in the evaluation value E. However, any value that can be used as an indicator of the degree of variation in the evaluation value E for the pixels included in the block, including the standard deviation, can be used instead.

  In the present embodiment, the imaging apparatus has been described as an example of the image processing apparatus. However, the present invention can be applied to any apparatus that can use the red-eye correction process, such as a printer or an information processing apparatus.

  Furthermore, in this embodiment, although it demonstrated as what weighted according to the distance from the pixel position where variation degree becomes the maximum, weighting is not essential. The magnitude of the evaluation value E is corrected in accordance with the degree of variation in the evaluation value E obtained for the block including the pixel so that the correction amount for the pixels in the block with the small degree of variation is reduced. Only the effect of the present invention can be obtained. Further, in the present embodiment, when the face width is F, the width of the eye region is set to F × 0.4, but the present invention is not limited to this. Any setting may be used as long as it is set so that the red region of the red eye (that is, the pupil region) is included in one block when the dispersion is obtained. In other words, the eye area size setting may be changed according to the number of block divisions.

  As described above, according to the present embodiment, the eye area is divided into a plurality of blocks, and the degree of variation in evaluation values representing the degree of redness of pixels included in the blocks is obtained for each block. Then, by correcting the evaluation value with the degree of variation, the correction value for a pixel included in a block with a small degree of variation is reduced. That is, the correction value is corrected so that the correction value becomes smaller as the pixel having a smaller variation degree of the evaluation value.

  For example, in FIG. 5, the pixel of the eye area 501 in the congested portion of the eye has a strong reddish color, so the evaluation value E increases, but the degree of variation in the evaluation value E regarding the block 502 including that portion is small. . Therefore, the evaluation value for the eye area 501 is reduced due to the small degree of variation, and as a result, the correction coefficient becomes small. Further, even for a relatively large area composed of pixels of substantially the same color, such as the skin area 503, the degree of variation in the evaluation value in the block is small, so that the correction coefficient is small.

  That is, the magnitude of the evaluation value E is corrected according to the degree of variation in the evaluation value E obtained for the block including the pixel, and is calculated as a correction coefficient. Thereby, it can suppress that the correction amount (correction coefficient) with respect to the red pixel of the area | region where the degree of dispersion | variation is small, such as an eye area | region and a skin area | region, becomes large.

  Furthermore, the correction coefficient for the eye area 501 and the skin area 503 is further reduced by correcting the evaluation value by weighting (weighting coefficient) that becomes smaller as the distance from the coordinate (pixel position) at which the degree of variation becomes maximum. Become. Further, by gradually changing the weighting, the boundary portion between the area where correction is performed and the area where correction is not performed becomes inconspicuous, and unnaturalness can be reduced.

  Conversely, in blocks that include the red-eye area and the iris area that surrounds it, the degree of variation in the evaluation value increases, so that a sufficient amount of correction coefficient can be obtained to achieve effective correction. it can. Furthermore, if the coordinates that maximize the degree of variation are regarded as the center of the red-eye area and weighted so that the correction coefficient decreases as the distance from the center increases, the correction coefficient for the red-eye area increases and the surrounding iris area increases. In, the correction coefficient becomes small. Therefore, erroneous correction can be more effectively suppressed.

(Other embodiments)
The above-described embodiment can also be realized in software by a computer of a system or apparatus (or CPU, MPU, etc.).
Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  A computer program for realizing the above-described embodiment is supplied to a computer via a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which the computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the user's computer Installation may be allowed. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.
Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

1 is a diagram illustrating a configuration example of an imaging apparatus as an example of an image processing apparatus according to an embodiment of the present invention. It is a perspective view which shows the example of an external appearance of the imaging device 100 which has a structure of FIG. It is a flowchart explaining the whole flow of the imaging and recording process in the imaging device which concerns on embodiment of this invention. FIG. 5 is a diagram schematically illustrating internal processing and signal flow that constitute red-eye correction processing in the imaging apparatus according to the embodiment of the present invention. It is a figure which shows the example of a division | segmentation of the eye area | region in the imaging device which concerns on embodiment of this invention. It is a figure which shows the distribution example of the eye area | region where the red-eye phenomenon has generate | occur | produced, and the dispersion | distribution calculated | required by the imaging device concerning embodiment of this invention. It is a figure which shows the example of distribution of the magnitude | size of the weighting coefficient which the imaging device which concerns on embodiment of this invention determines.

Claims (8)

Detecting means for detecting an eye area determined to be an eye from the image;
Dividing means for dividing the eye area into a plurality of blocks;
For each of the plurality of blocks, for each pixel included in the block, evaluation value calculation means for obtaining an evaluation value indicating the redness of the pixel;
And index calculating means for calculating an index indicating the degree of variation of the evaluation value, for each front Chivu lock,
Correction value calculation means for calculating a correction value for each pixel included in the eye area based on the evaluation value calculated for the pixel and the index;
Correction means for applying the correction value to the pixels of the eye area,
Image the correction value calculating means, characterized by calculating the correction value by correcting on the basis of the evaluation value as the correction value as the pixel is reduced in a small block of the degree of variation in the index Processing equipment. Resize means for calculating the index for each pixel position of the eye area from the index determined for each block;
The image processing apparatus according to claim 1, wherein the correction value calculation unit corrects the correction value using the index obtained by the resizing unit for a corresponding pixel position. Position detecting means for detecting a pixel position where the degree of variation is maximum in the eye area from the index for each pixel position obtained by the resizing means;
3. The image according to claim 2, wherein the correction value calculating unit further corrects the correction value so that the correction value becomes smaller as the distance from the pixel position where the degree of variation is maximum in the eye region is larger. Processing equipment.   The image processing apparatus according to claim 1, wherein the index is a variance or a standard deviation of the evaluation values.   5. An imaging apparatus comprising the image processing apparatus according to claim 1, wherein the captured image is corrected by the image processing apparatus. From within images, a detection step of detecting an eye region is determined as the eye,
A dividing step of dividing the pre-Symbol eye region into a plurality of blocks,
For each of the previous SL plurality of blocks, for each pixel included in the block, the evaluation value calculation step of calculating an evaluation value that indicates the red of the pixel,
An index indicating the degree of variation of the previous SL evaluation value, and the index calculation step of calculating for each said block,
The correction values for the individual pixels included before Symbol eye region, and the evaluation value calculated for the pixel, the correction value calculation step of calculating, based on said index,
And a correction step of applying a pre-Symbol correction value to the pixel of the eye region,
Prior SL correction value calculating step, characterized by calculating the correction value by correcting on the basis of the evaluation value as the correction value as the pixel is reduced in a small block of the degree of variation in the index Image processing method.   A program for causing a computer to execute each step of the image processing method according to claim 6.   A computer-readable recording medium on which the program according to claim 7 is recorded.

Images