JP2003299115A - Image signal processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain an image signal processor applicable to an electronic camera, a printer, and the like, in which a preferable complexion color can be reproduced without preparing any special color chart for color regulation. <P>SOLUTION: A plurality of pieces of data of complexion color reproduction target values are stored, while sorted by the race, sex, and the like, in the nonvolatile memory 44 of an electronic camera 10 to which the invention is applied. When a user selects a 'custom complexion color balance mode' and performs operation for selecting the race, sex, and the like, of an objective person, a frame line indicative of a complexion color specification area is displayed on a display 42. The user photographs the person such that the complexion color part of the object falls within the complexion color specification area. From photograph data (e.g. mean chromaticity in the complexion color specification area) thus attained and the complexion color reproduction target value, coefficients of color regulation operation at a linear matrix processing section 26 and a C-matrix processing section 32 are calculated at a control section 22 and the coefficients thus determined are set. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal processing device applied to an electronic camera, a printer or the like, and more particularly to a color correction technique for performing good skin color reproduction.

[0002]

2. Description of the Related Art In the field of electronic cameras, there is known a technique for taking a full-screen image of a white subject and automatically adjusting the white balance inside the camera based on the photographed data in order to obtain the optimum white balance under the photographing light source. Has been. However, with this method, the white balance is adjusted, but the content of the color correction calculation cannot be adjusted. For example, under a light source such as a fluorescent lamp having a poor color rendering property, there is a drawback in that an image with good color reproduction cannot be obtained simply by taking a white balance.

On the other hand, in Japanese Patent Laid-Open No. 10-75458, in order to match the colors of the output images obtained from a plurality of video cameras for broadcasting stations, a color chart is photographed under a photographing light source and the color chart inside the camera is photographed. A color correction device that determines the contents of color correction calculation is disclosed. According to this device, a marker is displayed on the screen of the viewfinder at the time of photographing the color chart. The user moves the marker on the image of the color chart, selects the color to be weighted, and inputs the weighting level.

[0004]

However, in the technique disclosed in the above publication, the user must always bring a dedicated color chart for color adjustment. In addition, the operation of aligning the chart and specifying the color is complicated. Further, according to the publication, since the color target value is the image pickup signal value of one of the reference video cameras, the preference of the color is not reflected.

The present invention has been made in view of such circumstances, and image signal processing capable of automatically setting a color adjustment calculation coefficient for achieving preferable color reproduction (especially skin color reproduction) without preparing a special chart for exclusive use. The purpose is to provide a device.

[0006]

In order to achieve the above-mentioned object, an image signal processing apparatus according to the present invention comprises a storage unit in which a target value for skin color reproduction is stored, and a region of a skin color portion of an input image. From the skin color area setting means to be set, the image information included in the area set by the skin color area setting means, and the target value read from the storage means, color adjustment is performed to achieve skin color reproduction close to the target value. It is characterized in that it is provided with a calculation means for calculating a calculation coefficient, and a color correction means for performing a color adjustment calculation on an image signal using the color adjustment calculation coefficient obtained by the calculation means.

According to the present invention, preferable skin color reproduction target value data is stored in the storage means in the apparatus, and image information obtained from the skin color area set for all or part of the input image is obtained. Based on the data of the target value, the color adjustment calculation coefficient is calculated by the calculation means inside the apparatus, and the obtained color calculation coefficient is applied to the color correction means to perform the color adjustment of the image signal. For example, the color correction means is a linear matrix arithmetic processing unit that performs linear conversion on primary color signals such as R, G, and B, and a C-matrix that performs linear conversion on color difference signals (Cr, Cb) after conversion into luminance / color difference signals. At least one of the arithmetic processing units is included.

As a result, the color of the flesh color area relating to the setting is corrected to a color substantially equal to the target value, and a preferable flesh color reproduction is possible. According to the present invention, it is possible to reproduce a flesh color portion related to setting with a color close to a target value without using a special chart.

According to one aspect of the present invention, the target value is
A plurality of data is stored for at least one of the following categories: race type, country / region, gender, and age, and a selection for the user to select a desired target value from these multiple target values. It is characterized by having an operating means.

The skin color varies depending on the race (white race / yellow race / black race), sex, age, etc., and the favorite color tone also differs depending on the country or region. Therefore, in order to reflect the characteristics of the person who is the subject and the intention of the photographer, the skin color reproduction target value is classified into person type, country / region, gender, age, etc., and a plurality of data are stored. It is preferable to set.

An image signal processing apparatus according to another aspect of the present invention is provided with an image observing means capable of observing a target image in addition to the above-mentioned configuration, and the image observing means is set as an area of the skin color portion. It is characterized in that information indicating the area is displayed together with the image content.

As the image observing means, a display means (for example, an electronic image display device such as a liquid crystal display or an organic EL) for displaying image contents based on an input image signal, a real image type finder or the like can be applied.

According to this aspect, the user can easily grasp the set area by displaying the information (frame line, mark, etc.) clearly indicating the area set as the area of the skin color portion on the image. .

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of an image signal processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings.

[First Embodiment] FIG. 1 is a block diagram of an electronic camera to which the present invention is applied. This electronic camera 1
Reference numeral 0 denotes a digital camera which converts an optical image of a subject into digital image data and records it on a recording medium, and the image signal processing device of the present invention is part of a signal processing means for processing an image signal obtained by photographing. Applied.

The optical system 12 of the electronic camera 10 includes a taking lens 13, a diaphragm 14, and an optical filter (infrared cut filter or low-pass filter) 15, and the optical system 12
The light that has passed through is incident on a CCD solid-state image sensor (hereinafter referred to as CCD) 16. A large number of photosensors are two-dimensionally arrayed on the light receiving surface of the CCD 16, and red (R), green (G), and blue (B) primary color filters are arranged in a predetermined array structure corresponding to each sensor (Bayer, G). Stripes, honeycombs, etc.). Instead of the CCD 16, C
Other image pickup devices such as a MOS image sensor may be used.

The subject image formed on the light receiving surface of the CCD 16 is converted by each photosensor into a signal charge of an amount corresponding to the amount of incident light. The CCD 16 has an electronic shutter function that controls the charge storage time (shutter speed) of each photo sensor according to the timing of a shutter gate pulse.

The signal charge accumulated in each photosensor of the CCD 16 is a voltage signal (image signal) corresponding to the signal charge based on a pulse given from a CCD driver (not shown).
Are sequentially read. The image signal output from the CCD 16 is amplified by the amplifier 18, and then R, G,
The color signals of B are color-separated and sent to the white balance adjustment circuit 20.

The white balance adjusting circuit 20 has R,
A gain variable amplifier for increasing or decreasing the levels of G and B color signals is included, and the gain of each color signal is adjusted based on a command from the control unit 22. The R, G, and B image signals whose gains have been adjusted by the white balance adjustment circuit 20 are A / D.
It is applied to the converter 24 and converted into a digital signal.

R, G, output from the A / D converter 24
The digital image signal of B is supplied to the linear matrix processing unit 2
6 and the correction is performed in consideration of the spectral transmittance of the color filter of the CCD 16.

That is, the linear matrix processing unit 26
Then, calculation is performed to obtain new primary color data R ', G', B'from the input R, G, B data according to the following equation (1).

[0022]

[Equation 1]

However, A _ij (i, j = 1,2,3) is a constant, and these constants are variably set based on a command from the control unit 22. The linear matrix processing unit 26
Is selected such that the white balance-adjusted gray color is invariant.

The signal generated by the linear matrix processing unit 26 is sent to the gamma conversion unit 28. The gamma conversion unit 28 converts the gradation characteristic by using the non-linear amplification means and converts the input / output characteristic so as to obtain a desired gamma characteristic.

The gamma-converted signal is sent to the Y / C converter 3
0, and the RGB primary color data is converted into a luminance signal Y and color difference signals Cr and Cb in the Y / C conversion section 30.

The luminance / color difference signals (Y, Cr, Cb) generated by the Y / C conversion unit 30 are transferred to the C-matrix processing unit 3.
Sent to 2. The C-matrix processing unit 32 carries out an operation for obtaining new color difference signals Cr 'and Cb' from the input data of Cr and Cb according to the following equation (2).

[0027]

[Equation 2]

However, B _ij (i, j = 1,2) is a constant,
These constants are variably set based on commands from the control unit 22.

The image data color-corrected in the C-matrix processing unit 32 is processed by the compression circuit 34 to JP
After being compressed according to a predetermined format such as EG, it is recorded on the recording medium 36 such as a memory card. The recording medium 36 is not limited to a removable medium, but may be a recording medium (internal memory) built in the electronic camera 10.

The image data processed by the C-matrix processor 32 is sent to the encoder 38. The encoder 38 converts the input signal into a video signal of a predetermined system for display (for example, a color video signal of NTSC system or PAL system). A signal output from the encoder 38 is converted into an analog signal via a D / A converter 40, and a display device (for example, a liquid crystal monitor) included in the electronic camera 10 is provided.
42 is output.

The display device 42 displays a real-time video image (through image) being picked up by the CCD 16 and a reproduced image of the image recorded on the recording medium 36, and information on the currently set mode and image compression rate. Information, date and time information,
Frame numbers are also displayed. The display device 42 is also used as a user interface display screen when the user performs various setting operations and the like, and also displays menu information such as setting items as necessary.

The control unit 22 is, for example, a central processing unit (C
PU), and functions as a calculation unit that performs optimization calculations such as a white balance coefficient and a color adjustment coefficient according to a predetermined program. The electronic camera 10 of this example has a mode (hereinafter, referred to as “custom skin color balance mode”) in which color adjustment calculation coefficients are automatically set in order to properly reproduce the skin color of the person who is the subject, and the electronic camera is used. A non-volatile memory (for example, ROM) 44 in 10 stores data of skin color reproduction target values (for example, coordinate values of a predetermined color space). A plurality of pieces of skin color reproduction target value data are categorized by person type, country (region), gender, age group, and the like, and are stored.

The user can appropriately operate the operation unit 46 of the electronic camera 10 to set the electronic camera 10 to the "custom skin color balance mode" and select a desired skin color reproduction target value according to the subject. it can. The control unit 22 reads the data of the corresponding skin color reproduction target value from the non-volatile memory 44 based on the signal from the operation unit 46, and calculates a color adjustment calculation coefficient that achieves a desirable skin color reproduction.

The operation unit 46 is means for inputting various instructions to the electronic camera 10, and includes a mode selection switch for selecting an operation mode of the electronic camera 10, a menu key for instructing display of a menu screen, and a menu item. Select operation (cursor movement operation) and frame-by-frame playback /
Cross key for inputting instructions such as frame return, OK button for instructing (registration) of selected items and execution of operations, deletion of desired objects such as selected items and cancellation of instruction contents, or returning to the previous screen It includes various operation means such as a BACK button, a power switch, a zoom switch, a shutter button (release switch), a display switch for switching on / off the display device 42 and a display mode, which are used at times.

Next, the electronic camera 1 constructed as described above
The operation of 0 will be described.

FIG. 2 is a flow chart showing the operation procedure of the custom skin color balance mode in the electronic camera 10 of this example. Turn on the camera power as shown in the figure.
After that (step S110), the user selects the "custom skin color balance mode" according to a predetermined operation procedure (step S112). Next, the race, gender, etc. of the person as the subject are set using a predetermined user interface such as a menu screen (step S114). Thus
According to the set conditions, the data of the skin color reproduction target value related to the setting is read from the non-volatile memory 44.

Thereafter, the image input waiting state in the custom skin tone balance mode is set (step S116), and the "custom skin tone balance photographing screen" as shown in FIG. 3 is displayed on the display device 42.

That is, a through image is displayed on the display device 42, and a frame line indicating the flesh color designation area 50 is overlaid and displayed in the central portion of the screen. It should be noted that mode information indicating the "custom skin color balance" mode is displayed at the upper left of the screen, and an operation guide is displayed at the bottom of the screen.

While looking at the screen of the display device 42, the user performs an operation such as a magnification change operation with a zoom switch or a change in the direction of the photographic optical axis so that the flesh-colored area of the subject is displayed in the entire flesh-colored designation area 50. The corner is set (step S116 in FIG. 2).

The shutter button is pressed at the timing when an angle of view in which the skin color designation area 50 is filled with the skin color portion of the subject is obtained (step S1).
18). The control unit 22 detects the brightness and the like of the subject from the input image signal and determines whether or not the brightness and the like are appropriate (step S120).

If it is determined that the adjustment of the custom skin color bunlas is impossible due to the excessive or insufficient brightness, or the deviation of the correction range, a warning is displayed on the display device 42 (step S122), and it is appropriate for the user. Prompt for re-selection or exposure correction of the skin-colored part with a good brightness. The user takes measures such as reselecting a skin color portion having a different brightness or performing exposure correction (step S124). Then, the process returns to step S116.

If it is determined in step S120 that the brightness of the image is appropriate, then step S1
Proceed to 26. In step S126, the control unit 22
White balance adjustment value (gain value) using the image signals of all areas inside and outside the skin color specification area 50 (entire shooting screen)
To obtain the white balance.

Then, the photographed data (for example, the average chromaticity value in the skin color region) obtained from the skin color designation area 50 is compared with the skin color reproduction target value, and color correction is realized so as to approach the skin color reproduction target value. A color adjustment calculation coefficient for performing the calculation is calculated inside the camera, and the calculated coefficient is set in the linear matrix processing unit 26 and the C-matrix processing unit 32 (step S128).

In this way, after the color adjustment calculation coefficient is set, the actual photographing operation is performed under the same light source, so that the linear matrix processing unit 26 and the C-matrix processing unit 32 can obtain the required image for the input image. Color correction processing is performed, and an image of skin color reproduction close to the skin color reproduction target value is obtained.

Although it is possible to use only one of the linear matrix processing unit 26 and the C-matrix processing unit 32 to perform flesh color correction, in order to perform color correction closer to the target value, the above-mentioned method is used. As described above, it is preferable that the skin color correction is performed using both the linear matrix processing unit 26 and the C-matrix processing unit 32.

According to the electronic camera 10 of the present embodiment, a dedicated color chart, which has been conventionally required, is unnecessary, and it is possible to take a photograph with good skin color reproduction even under a light source with a low color rendering property such as a fluorescent lamp. Become.

In FIG. 3, the center portion of the photographing screen is set as the skin color designation area 50, but the position and shape of the skin color designation area 50 are not limited to the example of FIG. Skin color designation area 50
It is also possible to configure the position and shape of the image so that the user can select the position and shape.
It is also possible to obtain the custom flesh color balance photographing information by performing full-screen photographing of the flesh color portion of the subject by setting to.

In the case of the electronic camera 10 including the display device 42 for displaying the image content based on the input image signal like a liquid crystal monitor, the frame line of the skin color designation area 50 is selected according to the mode selection of the custom skin color balance mode. It is possible to switch the display / non-display of etc. In carrying out the present invention, the present invention is not limited to such an electronic display device, and it is also possible to apply an optical image observing means such as a real image finder. When displaying a frame that indicates the skin color designation area in the viewfinder field in the optical viewfinder, a method of forming a frame on the prism surface may be applied as with the AF target mark, or a method similar to the switching field frame is used. The skin color designation area may be displayed.

Information regarding the presence / absence of custom skin color balance adjustment in the above-described custom skin color balance mode, information about the adjustment value (color adjustment calculation coefficient), and the like are recorded in the recording medium 36 as additional information of the image at the time of image recording. Is also preferable. By using the custom flesh color balance information recorded as the additional information at the time of printing, it is possible to perform printing with preferable flesh color reproduction.

[Second Embodiment] Next, an example of a printer to which the present invention is applied will be described.

FIG. 4 shows a printer 6 according to the embodiment of the present invention.
It is a block diagram of 0. As shown in FIG.
0 is a removable recording medium (removable medium),
Alternatively, wired or wireless data communication means (for example, US
B, IEEE 1394, RS232C, etc.), an image input unit 62 for receiving image data, a JPEG compressed image decompression processing unit 64, a skin color portion designation function unit 66 for designating a skin color portion in the decompressed image, and a skin color. A non-volatile memory 68 in which a reproduction target value is stored, a color adjustment coefficient optimization calculation unit 70, a YCr Cb matrix correction unit 72, a YCr Cb → RGB conversion circuit (hereinafter referred to as an RGB conversion circuit) 74, Look-up table (LU
T) correction circuit 76 and RGB → YMC conversion circuit (hereinafter,
It is called a YMC conversion circuit. ) 78 and print engine 8
It has 0.

The compressed image data read from the image input means 62 is decompressed (decompressed) by the JPEG compressed image decompression processing unit 64. The expanded image is sent to the YCr Cb matrix correction unit 72. The image input as the print target is displayed on an image display device (not shown) provided in the printer 60 or an image display device attached to a printer control device such as a personal computer or a mobile terminal.

The printer 60 of this example has a "custom flesh color balance mode" in which the color adjustment calculation coefficient is automatically set in order to properly reproduce the flesh color of the person included in the print image. When a print that emphasizes skin color reproduction is desired, the user operates a predetermined operation unit to select the "custom skin color balance mode", and at the same time, designates a skin color portion in the print image displayed on the image display device, Select information such as race and gender of a person.

When the custom skin color balance mode is selected, as shown in FIG. 5, a frame line indicating the skin color designation area 90 is overlaid on the reproduction screen of the print image. The user operates the cross key provided on the printer 60 or other predetermined operation means to move the skin color designation area 90 to a desired position, and designates a skin color portion in the image. At this time, the size of the skin color designation area 90 may be scaled up or down as necessary.

In this way, from the image information of the designated skin color portion (for example, the average chromaticity value in the skin color designation area 90) and the skin color reproduction target value read from the non-volatile memory 68 shown in FIG. The color adjustment coefficient optimization calculator 70 calculates the color adjustment coefficient.

The obtained color adjustment calculation coefficient is set as the matrix coefficient of the YCr Cb matrix correction unit 72. The YCr Cb matrix correction unit 72 performs a calculation to obtain new image data Y ', Cr', Cb 'from the input Y, Cr, Cb data according to the following equation (3).

[0057]

[Equation 3]

However, C _ij (i, j = 1,2,3) is a constant, and these constants are variably set based on a command from the color adjustment coefficient optimizing calculation unit 70.

The data generated by the YCr Cb matrix correction unit 72 is sent to the RGB conversion circuit 74 and converted from the YC signal to the RGB signal. The RGB signal generated by the RGB conversion circuit 74 is subjected to gamma correction processing in the LUT correction circuit 76 and then sent to the YMC conversion circuit 78. The YMC conversion circuit 78 converts the received RGB data into Y (yellow), magenta (M), and cyan (C) color signals. The image data converted into the YMC signal is sent to the print engine 80, and the image content indicated by the image data is printed on a printing paper or other printing medium. In this way, the printed matter 82 on which the image content is printed
Is obtained. As a result, it is possible to obtain a print with a preferable skin color reproduction.

In the above-described embodiment, the color correction means 3 is used.
Although the matrix of × 3 is illustrated, the color correction means is not limited to this, and a mode such as a three-dimensional LUT is also possible.

[0061]

As described above, according to the image signal processing device of the present invention, the data of the skin color reproduction target value is built in the storage means in the device, and the image obtained from the skin color region of the input image is obtained. Since the color adjustment calculation coefficient is calculated from the information and the skin color reproduction target value and is set in the color correction means, it is possible to reproduce the skin color substantially equal to the target value. According to the invention,
No special chart is required, and it is possible to easily obtain an image of preferable skin color reproduction suitable for the subject.

Further, according to the present invention, a plurality of skin color reproduction target values are prepared for at least one of the types of people, countries / regions, sexes, and ages. Since the user can select a desired target value, it is possible to deal with various shooting scenes.

Further, according to the present invention, the image observing means for observing the target image is added, and the information indicating the area set as the area of the skin color portion is presented together with the image contents. The skin color part of can be specified easily.

[Brief description of drawings]

FIG. 1 is a block diagram of an electronic camera to which the present invention is applied.

FIG. 2 is a flowchart showing an operation procedure of a custom skin color balance mode in the electronic camera of this example.

FIG. 3 is a diagram showing an example of a shooting screen in a custom skin color balance mode.

FIG. 4 is a block diagram of a printer to which the present invention is applied.

FIG. 5 is a diagram showing an example of a user interface screen for designating a skin color portion of a print image.

[Explanation of symbols]

10 ... Electronic camera, 16 ... CCD, 20 ... White balance adjustment circuit, 22 ... Control section, 26 ... Linear matrix processing section, 32 ... C-matrix processing section, 42 ... Display device, 44 ... Non-volatile memory, 46 ... Operation Part, 50 ... skin color designation area, 60 ... printer, 62 ... image input means,
66 ... Skin color part designation function part, 68 ... Non-volatile memory, 7
0 ... Color adjustment coefficient optimization calculation unit, 72 ... YCr Cb matrix correction unit, 80 ... Print engine

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 9/04 H04N 1/46 Z 5C079 9/79 9/79 HF term (reference) 5B057 BA02 CA01 CA08 CA12 CA16 CB01 CB08 CB12 CB16 CC01 CE17 CE18 CH07 CH08 5C055 AA14 BA05 BA07 BA08 DA01 DA02 EA02 EA04 EA05 HA35 HA37 5C065 AA01 BB16 CC02 CC03 CC04 CC05 DD02 DD17 EE12 GG18 GG44 GG49 5C066 AA01 AA05 AA11 BA01 CA08 EB03 EC01 GA01 GA02 GA04 GA05 GA31 GB01 KE01 KM02 KM13 LA02 5C077 LL19 MP08 PP32 PP34 PP37 PQ08 PQ12 PQ23 SS05 SS06 TT09 5C079 HB01 HB04 HB11 LA02 LB01 MA04 MA11 NA03 PA05

Claims (3)

[Claims] 1. A storage unit that stores a target value of skin color reproduction, a skin color region setting unit that sets a region of a skin color portion of an input image, and a storage unit included in the region set by the skin color region setting unit. Calculation means for calculating a color adjustment calculation coefficient so as to achieve skin color reproduction close to the target value from the image information to be read and the target value read out from the storage means; and the color adjustment calculation coefficient obtained by the calculation means. An image signal processing device comprising: a color correction unit that performs a color adjustment calculation on the image signal. 2. The target value stores a plurality of data for at least one of the following categories: race type, country / region, gender, and age. The image signal processing apparatus according to claim 1, further comprising a selection operation unit for selecting a target value. 3. An image observing unit capable of observing a target image, wherein the image observing unit displays information indicating an area set as a region of the flesh color portion together with image contents. The image signal processing device according to 1 or 2.