JP2006217277A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of obtaining an optimum image by recognizing a main object and performing DRC processing regardless of the size within an image of the main object. <P>SOLUTION: There is dynamic range compression (DRC) for adjusting gray level distribution and making images beautiful for digital images photographed by a digital camera or the like. In this invention, the DRC processing is operated by an appropriate parameter for turning the main object to the optimum exposure. For the detection of the main object, a person detected by face recognition or an area selected by scene assist is used for instance. The face recognition and area detection are performed in a main object detection part 106 inside an image processing part 105. A DRC processing part 107 performs gray level correction preferentially to the area. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention performs gradation distribution correction (dynamic range compression: hereinafter referred to as DRC processing) for adjusting the gradation distribution and making the image beautiful with respect to digital image data photographed using a digital camera or the like. The present invention relates to an imaging apparatus suitable for performing the DRC processing with a DRC parameter at which a photographed main subject is optimally exposed.

First, dynamic range compression (DRC) will be described.
The dynamic range (the width of brightness (gradation) that can be expressed at one time) detected by an image sensor such as a CCD is generally narrower than the human eye and narrower than the camera film. For this reason, even in a scene where there is a difference in brightness, even if the change in gradation can be perceived by the human eye, when shooting in the automatic exposure control mode of a digital camera, the high-brightness part will “overshoot” and the low-brightness part will It may “black out”.

In addition, if exposure control does not work as intended, the image may not be effectively used for DRC processing.
FIG. 13 is a diagram illustrating an example of a luminance histogram of an image in which DRC processing is almost effectively used. In the figure, the vertical axis represents frequency and the horizontal axis represents luminance level. In the figure, the horizontal axis is black on the left side and white on the right side. In other words, the horizontal axis represents the gradation from black to white (for example, the horizontal axis represents a maximum of 4095 bits). The vertical axis represents frequency (for example, the vertical axis is a maximum of 255 bits). FIG. 13 shows an image in which the DRC process is almost effectively used because there are few blackouts and whiteouts because the frequency at both ends of the luminance histogram, that is, black and white is low.

FIG. 14 is a diagram illustrating an example of a luminance histogram when both overexposure and underexposure occur. In FIG. 14, the relationship between the vertical axis and the horizontal axis is the same as that in FIG. 13 (horizontal axis: frequency, vertical axis: luminance level).
In the DRC process, by adjusting the gradation distribution as shown in FIG. 14, a wide gradation area is assigned to the gradation in which blackout and overexposure occur, thereby clarifying the degree of luminance change. This is an image processing technique that makes it appear as if the dynamic range has expanded. Therefore, even if the image of the luminance histogram shown in FIG. 14 is performed, the person recognizes it as an image with no overexposure or underexposure as shown in FIG.

  FIG. 15 shows the gradation distribution shown in FIG. 14 that has been subjected to DRC processing. The castle that was crushed before the DRC process is expressed using a certain level of gradation area after the DRC process, so the castle that was originally crushed is raised and the degree of brightness change is clear. And visibility can be improved. Similarly, it is possible to make the whitened area stand out, to clarify the degree of luminance change, and to improve visibility.

  Note that the DRC process is not simply a process of raising the overall luminance level simply (linearly). In the luminance histogram shown in FIG. 14 already described, peaks are formed on the high luminance side and the low luminance side, respectively, and the intermediate gradation has a relatively flat shape. In such a luminance histogram, the DRC process acts so that the slopes of the two peaks become gentle and distributed on an average. The DRC process can clean an image even if it is not blacked out or overexposed by adjusting the tone distribution. In other words, the DRC process detects the unused gradation area by analyzing the luminance histogram of the image, adjusts the entire gradation area to be used, recreates the image, and clearly changes the gradation of details. To obtain a processed image.

  That is, in this specification, DRC processing means a technique for appropriately adjusting the gradation distribution. In the DRC process, when blackout or overexposure occurs, the dynamic range of the image is processed in the direction of compression. However, for an image that uses only a part of the dynamic range, the dynamic range is reduced. This is a process that makes it possible to correct an image so that it can be used effectively. In other words, the DRC process is a process of dynamically adjusting the gradation distribution by analyzing the luminance histogram.

Next, a specific example of DRC processing will be described. Here, as an example, a method for averaging the level distribution in the luminance histogram will be described.
First, the luminance histogram is analyzed, and a correction function is created based on the analysis result. The level is corrected by the correction function, and the level distribution is averaged. A method for creating a correction function will be described below with reference to FIGS.

Step 1: In FIG. 16, the horizontal axis of the luminance histogram is divided into equal intervals. The finer the section width, the better the accuracy. However, since it takes a lot of time for processing, it is appropriately determined. In FIG. 16, the horizontal axis is divided into four as an example. However, the greater the number of divisions, the higher the accuracy.
Step 2: In each section divided into four in FIG. 16, a ratio between the number of pixels belonging to each section (total frequency) and the number of pixels of the entire image is obtained. In FIG. 16, as shown on the horizontal axis, numerical values of 60% (first section), 10% (second section), 20% (third section), and 10% (fourth section) are obtained from the left side. It was. That is, as shown on the vertical axis in FIG. 17, 60% of the total number of pixels exists in the first section, 10% of the total number of pixels exists in the second section, and 20% of the total number of pixels in the third section. And 10% of the total number of pixels exists in the fourth section.

  Step 3: Next, as shown in FIG. 18, in each section, point a where the horizontal axis and the vertical axis intersect 60%, point b where the horizontal axis and the vertical axis intersect 10%, A point c where 20% intersects the vertical axis and a point (end point) e where 10% intersects the horizontal axis and the vertical axis are obtained. Next, as shown in FIG. 18, a polygonal line f passing through each point obtained by the above method is drawn with the origin 0 of the luminance histogram as the starting point. Next, in FIG. 18, paying attention to the straight line L connecting the origin 0 and the end point e of the luminance histogram and the broken line f, the luminance histogram is calculated based on the difference (gradation difference) between the straight line L and the broken line f in the horizontal axis direction. The luminance histogram shown in FIG. 19 is obtained. The obtained luminance histogram is averaged in distribution compared to the original luminance histogram shown in FIG.

As a prior art related to the present invention, an object having a specific color, for example, a specific area occupied by an important color such as a human skin color is accurately extracted from an original color image, and image processing corresponding to the specific area is performed. There is a technique for performing (image processing such as dynamic range compression / decompression processing) (see Patent Document 1).
JP-A-11-266358

  As described above, by analyzing the image by DRC processing and correcting the gradation distribution of the luminance histogram, the gradation of the unused portion can be effectively used. However, when DRC processing is performed using the entire luminance histogram, there is a tendency that more gradations are given to a subject with a large area. For this reason, when a main subject such as a person is photographed in a small size on the screen, the gradation of the main subject is crushed and there is a problem that an optimum image cannot be obtained.

In addition, a method using a luminance histogram of a high-frequency part on the assumption that the main subject has many high-frequency components has been proposed. However, when the main subject is a human face, for example, the high-frequency part is contrary to the above assumption. Therefore, there is a problem that it is difficult to obtain a remarkable improvement effect.
In addition, there is a method for detecting the main subject (person) with an important color such as skin color, but it is highly probable that an erroneous determination will occur if only the skin color is determined, and it is difficult to specify the size of the main subject (person). There is a problem.

  The present invention has been made in view of the above-described problems of the prior art. An optimal image can be obtained by performing a DRC process by accurately recognizing the main subject regardless of the size of the main subject in the screen. An object is to provide an imaging device that can be obtained.

  An imaging apparatus according to claim 1, wherein an exposure control unit that controls exposure, a main subject detection unit that detects a main subject area, and a main subject used for dynamic range compression from the main subject area detected by the main subject detection unit Main subject information setting means for setting information on the brightness and size of the camera, and parameter setting means for setting parameters used for dynamic range compression based on the information on the brightness and size set by the main subject information setting means. It is characterized by having.

According to the first aspect of the present invention, it is possible to perform dynamic range compression so that the main subject becomes an optimum image based on information on the luminance and size of the main subject.
According to a second aspect of the present invention, in the imaging apparatus of the first aspect, the main subject detection means is a recognition means for recognizing the main subject from the image signal.
According to a third aspect of the present invention, in the imaging apparatus of the second aspect, the recognition unit is a face recognition unit that performs face recognition.

According to a fourth aspect of the present invention, there is provided the imaging apparatus according to the third aspect, further comprising a parameter correcting unit that corrects a parameter used for dynamic range compression based on a face size detected by the face recognition unit. Features.
According to a fifth aspect of the present invention, in the imaging apparatus according to the first aspect, the main subject detecting means is auxiliary frame display information recognizing means for recognizing photographing auxiliary frame information for determining a composition. To do.

  According to the present invention, it is possible to provide an imaging device capable of obtaining an optimal image by accurately recognizing a main subject and performing DRC processing regardless of the size of the main subject in the screen.

Hereinafter, an embodiment of the present invention will be described.
FIG. 1 is a block diagram showing an embodiment of the present invention. This embodiment corresponds to all the claims.
In FIG. 1, reference numeral 100 denotes an exposure control unit including a lens optical system 101 and an image sensor 102, 103 denotes an A / D converter that converts an analog image signal output from the exposure control unit 100 into a digital image signal, and 104 denotes a digital image signal. This is a memory for storing a digital image signal output from the A / D converter 103 or a digital image signal output from the image processing unit 105.

The image processing unit 105 includes a main subject detection unit 106 and a DRC processing unit 107.
First, the main subject detection unit 106 detects a main subject based on the digital image signal obtained from the A / D converter 103, and the DRC processing unit 107 performs DRC processing based on the detection result.

  Secondly, the main subject detection unit 106, when the user designates photographing auxiliary frame information (so-called scene assist information) using the photographing setting switching device 108, the designated photographing auxiliary frame information (so-called scene assist information). The DRC processing unit 107 performs DRC processing on the main subject area obtained by reading out (information) from the imaging setting unit 109. Here, the shooting setting unit 109 stores a plurality of pieces of shooting auxiliary frame information (coordinates and sizes) with respect to the main subject area (for example, human face, human face and shoulder). The designated photographing auxiliary frame information is given to the main subject detection unit 106.

The DRC parameters used when the DRC processing unit 107 performs the DRC processing are determined based on the main subject area detected by the main subject detection unit 106. Further, when the user designates shooting assistance frame information (so-called scene assistance information), it is determined based on the main subject area determined from the designated shooting assistance frame information.
The digital image information after the DRC process is recorded on the image recording element 110 and displayed on the display unit 111. The display unit 111 can also display an image before recording on the image recording element 110 to show to the user. The CPU 112 performs overall operation control.

FIG. 2 is a block diagram showing details of the image processing unit 105 shown in FIG. As described above, the operation of the present embodiment is performed when the main subject detection unit 106 detects the main subject based on the digital image signal obtained from the A / D converter 103 and performs the DRC process, and when the user assists in shooting. This is divided into cases in which frame information (so-called scene assist information) is designated, and DRC processing is performed based on the main subject area determined from the designated shooting assistance frame information (see the description about the first and second). .
(1) First, the case where the main subject detection unit 106 detects the main subject based on the digital image signal obtained from the A / D converter 103 and performs DRC processing will be described.

As shown in FIG. 2, a digital image signal obtained by imaging is input to the image processing unit 105.
The main subject detection unit 106 includes a face recognition unit 1061 and a main subject area detection unit 1062. For example, the face recognition unit 1061 detects features such as the distance between the left eye and the right eye, the lines between the eyes and the shoulder, and determines a face area, a face and shoulder area, and the brightness of the area. FIG. 3 is an example of face recognition when the face area is small, and FIG. 4 is an example of face recognition when the face area is large.

The main subject area detection unit 1062 obtains information on the main subject area based on the face position and size (or the position and size of the face and shoulders) output from the face recognition unit 1061 and luminance information, and the DRC processing unit. It outputs to 107. The DRC processing unit 107 includes a DRC parameter correction unit 1071, a DRC parameter setting unit 1072, and a gradation correction unit 1073.
Based on the information (position, size, and luminance) of the main subject area output from the main subject area detection unit 1062, the DRC parameter correction unit 1071 adjusts the DRC parameter so that the gradation of the detected main subject is appropriate. The correction value of is obtained.

  The DRC parameter setting unit 1072 obtains a DRC parameter with which the entire screen obtained from the digital image signal is an optimal image. Next, the DRC parameter setting unit 1072 corrects the DRC parameter obtained by the DRC parameter correction unit 1071 for the DRC parameter for which the entire screen is an optimal image. That is, the DRC parameter setting unit 1072 corrects the DRC parameter so that the detected gradation of the main subject is appropriate for the DRC parameter that optimizes the entire screen.

  Specifically, the correction is performed as follows. That is, if the DRC parameter correction unit 1071 determines that the luminance of the area detected by the main subject area detection unit 1062 is under (dark), it outputs a correction value in the over (bright) direction and is determined to be over. Then, an under direction correction value is output. Accordingly, the DRC parameter setting unit 1072 corrects the DRC parameter so that the luminance level of the main subject area becomes an appropriate value.

The tone correction unit 1073 performs DRC processing using the DRC parameters set by the DRC parameter setting unit 1072.
The DRC-processed image is subjected to interpolation processing by the interpolation unit 1051, color reproduction processing is performed by the color correction unit 1052, and contour enhancement is performed by the contour enhancement unit 1053.
(2) Next, a case will be described in which the user designates shooting assistance frame information (so-called scene assist information) and performs DRC processing based on the main subject area determined from the designated shooting assistance frame information.

  Differences from the above (1) are as follows. That is, in the case of (1), the main subject area detection unit 1062 calculates the main subject area based on the face position and size (or face and shoulder position and size) output from the face recognition unit 1061. , Output to the DRC processing unit 107. However, in (2), shooting auxiliary frame information (scene assist information) read from the shooting setting unit 109 is output to the main subject area detection unit 1062. The main subject area detection unit 1062 calculates a main subject area where the face position and size (or the position and size of the face and shoulders) and the like exist based on the input auxiliary shooting frame information, and outputs the main subject area to the DRC processing unit 107. To do. FIG. 5 shows an example of face recognition using shooting assistance frame information (scene assist information). The other operations are the same as those in the case (1), and the description thereof will be omitted.

  In the above description, the correspondence with the claims is as follows. That is, the exposure control means described in the claims corresponds to the exposure control section 100, the main subject detection means corresponds to the main subject detection section 106, the main subject information setting means corresponds to the main subject area detection section 1062, and the parameter The setting means corresponds to the DRC parameter correction unit 1071 and the DRC parameter setting unit 1072. Further, the parameter correction means described in the claims corresponds to the DRC parameter correction unit 1071, and the auxiliary frame display information recognition means corresponds to the imaging setting switching device 108, the main subject area detection unit 1062, and the like.

Next, changes in the luminance histogram based on the difference in the DRC parameters will be described using the luminance histograms shown in FIGS.
FIG. 6 is a luminance histogram that is the basis of the luminance histograms shown in FIGS.
FIG. 7 is a luminance histogram when DRC processing (gradation correction) is not performed (denoted as OFF in the figure). Therefore, it is the same as the luminance histogram shown in FIG.

  FIG. 8 is a luminance histogram when DRC processing (gradation correction) is automatically performed (denoted as standard in the figure). In this case, gradations are assigned so that empty gradations are reduced. When the dynamic range of the image is narrow, gradation is assigned so that the use range of the dynamic range is widened. FIG. 8 is a luminance histogram obtained when DRC parameter correction is not performed in the DRC parameter setting unit 1072 (see FIG. 2). In the luminance histogram of FIG. 8, it can be seen that the gradations are dispersed on average compared to the luminance histograms shown in FIGS.

FIG. 9 is a luminance histogram (referred to as “under” in the figure) in which gradations are assigned with priority on the gradations in the dark part. Compared to FIG. 7, it can be seen that in FIG. 9, the gradation of the dark portion is assigned near the center of the horizontal axis (luminance level).
FIG. 10 is a luminance histogram (referred to as “over” in the figure) in which gradations are assigned with priority given to the gradations in the bright part. Compared with FIG. 7, it can be seen that in FIG. 10, the bright gradation is assigned near the center of the horizontal axis (luminance level).

An instruction not to perform the DRC process (tone correction) shown in FIG. 7, an instruction to automatically perform the DRC process shown in FIG. 8, and the like can be designated by the shooting setting switching device 108 shown in FIG. .
Next, the DRC processing result when the subject is large and the DRC processing result when the subject is small in the embodiment shown in FIGS. 1 and 2 will be described using the luminance histograms shown in FIGS. 11 and 12. FIG. 11 shows an example in which the main subject is low brightness and large, and FIG. 12 shows an example in which the main subject is low brightness and small.

FIG. 11A is a luminance histogram when DRC processing (gradation correction) is not performed. As shown in the figure, the main subject corresponds to a gradation area surrounded by an ellipse.
FIG. 11B is a luminance histogram when a main subject is not detected and DRC processing (gradation correction) is automatically performed (denoted as standard in the drawing). In this case, as described with reference to FIG. 8, gradations are assigned so that empty gradations are reduced, and the whole is averaged.

FIG. 11C is a luminance histogram when a main subject is detected and DRC processing (gradation correction) is automatically performed. As shown in the figure, gradation is assigned with priority given to the gradation of the main subject portion. Since the main subject has a low luminance, the gradation in the dark part is also prioritized and the gradation is assigned.
FIG. 12A is a luminance histogram when DRC processing (gradation correction) is not performed. As shown in the figure, the main subject corresponds to a gradation area surrounded by an ellipse. Since the main subject is small, the number of pixels occupied by the main subject is small. Furthermore, since the main subject is small, it tends to be under (dark) when backlit.

FIG. 12B is a luminance histogram when the main subject is not detected and when DRC processing (gradation correction) is automatically performed (denoted as standard in the drawing). In this case, as described with reference to FIG. 8, gradations are assigned so that empty gradations are reduced, and the whole is averaged.
FIG. 12C is a luminance histogram when DRC processing (gradation correction) is performed when the main subject is detected and the main subject is small. Since the main subject is small, the number of pixels occupied by the main subject is small, and low-brightness priority processing is not performed in the standard DRC processing. However, by detecting the main subject, the area where the main subject exists is weighted (depending on the DRC parameter), and the gradation can be assigned with priority given to the area where the main subject exists and the gradation of the dark part.

If the main subject is large, the number of pixels occupied by the main subject is large, and weighting (by the DRC parameter) is easy to perform. However, according to the present embodiment, since the main subject is detected, the gradation of the main subject can be preferentially assigned regardless of the size of the main subject.
Note that each luminance histogram shown in FIG. 12 can be obtained by the same processing as when the main subject has a low luminance even when the main subject has a high luminance.

  As is clear from the above description, according to the present embodiment, the main subject can be detected (or the position and size of the main subject can be obtained from the photographing auxiliary frame information), so that the DRC process is executed. The main subject is optimally exposed and a beautiful image can be obtained.

  INDUSTRIAL APPLICABILITY The present invention can be greatly utilized industrially in the field using an imaging device such as a digital camera.

It is a block diagram which shows one Embodiment of this invention. It is a block diagram which shows the detail of the image process part 105 shown in FIG. It is an example of face recognition when a face area is small. It is an example of face recognition when a face area is large. It is an example of face recognition using photographing auxiliary frame information (scene assist information). 10 is a luminance histogram that is the basis of the luminance histograms shown in FIGS. 7 to 9. It is a brightness | luminance histogram in case DRC processing is not performed. FIG. 8 is a luminance histogram when the DRC process is automatically performed (described as “standard” in the figure). It is a luminance histogram (referred to as “under” in the figure) in which gradations are assigned with priority given to the gradations of dark parts. It is a luminance histogram (referred to as “over” in the figure) in which gradation is assigned with priority on the gradation of the bright part. In the embodiment shown in FIG. 1, it is a brightness | luminance histogram for demonstrating the DRC process when a to-be-photographed object is large. In the embodiment shown in FIG. 1, it is a brightness | luminance histogram for demonstrating the DRC process when a to-be-photographed object is small. It is a figure which shows an example of the brightness | luminance histogram of the image in which DRC processing is utilized substantially effectively. It is a figure which shows an example of the brightness | luminance histogram of the image in which the overexposure and the underexposure have generate | occur | produced. It is a figure which shows the brightness | luminance histogram after performing the DRC process on the brightness | luminance histogram (gradation distribution) shown in FIG. It is a figure which shows the procedure which averages the level distribution in a brightness | luminance histogram by DRC process. It is a figure which shows the procedure which averages the level distribution in a brightness | luminance histogram by DRC process. It is a figure which shows the procedure which averages the level distribution in a brightness | luminance histogram by DRC process. It is a figure which shows the procedure which averages the level distribution in a brightness | luminance histogram by DRC process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Exposure control part 101 Lens optical system 102 Image pick-up element 103 A / D converter 104 Memory 105 Image processing part 106 Main subject detection part 107 DRC processing part 108 Shooting setting switching device 109 Shooting setting part 110 Image recording element 111 Display part 112 CPU
1051 Interpolation unit 1052 Color correction unit 1053 Outline enhancement unit 1061 Face recognition unit 1062 Main subject area detection unit 1071 DRC parameter correction unit 1072 DRC parameter setting unit 1073 Tone correction unit

Claims (5)

Exposure control means for controlling exposure; and
Main subject detection means for detecting a main subject area;
Main subject information setting means for setting information on luminance and size of the main subject used for dynamic range compression from the main subject area detected by the main subject detection means;
An image pickup apparatus comprising: parameter setting means for setting parameters used for dynamic range compression based on the luminance and size information set by the main subject information setting means. The imaging device according to claim 1,
The imaging apparatus according to claim 1, wherein the main subject detection means is a recognition means for recognizing the main subject from an image signal. The imaging apparatus according to claim 2, wherein
The imaging apparatus according to claim 1, wherein the recognition means is a face recognition means for performing face recognition. The imaging device according to claim 3.
An image pickup apparatus comprising: a parameter correction unit that corrects a parameter used for dynamic range compression based on a face size detected by the face recognition unit. The imaging device according to claim 1,
The imaging apparatus according to claim 1, wherein the main subject detection means is auxiliary frame display information recognition means for recognizing shooting auxiliary frame information for determining a composition.

Images