JP2008271227A - Image sensing device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image sensing device capable of separating a subject with a high accuracy. <P>SOLUTION: The image sensing devices (11 to 26) obtain the variations of brightnesses among a plurality of photographing images acquired by continuously conducting a plurality of photographies while changing the conditions of a flash for separating the subject regions and non-subject regions of the images and loading a specified photographing mode. The image sensing devices have a control means (20). The control means (20) controls the variations of the brightnesses of the subject regions among a plurality of the photographing images and the conditions of an exposure in the specified photographing mode so that separation quantities with the variations of the brightnesses in the non-subject regions are made larger than the photography is conducted under the same exposure conditions as an automatic exposure photographing mode. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus having a function for separating a subject and a non-subject from an image acquired by photographing. Hereinafter, separating a subject and a non-subject from an image is referred to as “subject separation”.

Many techniques for subject separation have been proposed. As an example of the prior art, for example, Patent Document 1 discloses a technique in which strobe flash photography and strobe non-flash photography are continuously performed, and a region in which luminance changes from two acquired images is extracted as a subject region. Has been.
JP-A-10-210340

  However, in the prior art, when the subject and the non-subject (such as the background) are relatively close to each other, the change in luminance between the two images described above is small, and the subject region may not be extracted correctly. .

  The present invention is to solve the above-mentioned problems of the prior art. An object of the present invention is to provide an imaging apparatus that enables subject separation with high accuracy.

  In order to separate the subject area and the non-subject area of the image, the imaging device according to the first aspect of the present invention continuously performs a plurality of shootings while changing the flash condition, and between the plurality of shooting images obtained thereby. An imaging apparatus equipped with a specific photographing mode for obtaining the amount of change in luminance includes a control unit. This control means is such that the amount of deviation between the change amount of the luminance of the subject area and the change amount of the luminance of the non-subject area between a plurality of captured images is larger than when shooting under the same exposure conditions as in the automatic exposure shooting mode. Control exposure conditions for specific shooting modes. Note that the exposure condition in this specification indicates a combination of an aperture, a shutter speed, and shooting sensitivity.

  In a second aspect based on the first aspect, the control means obtains the change amount by a difference or ratio of luminance between a plurality of captured images.

  According to a third invention, in the first or second invention, the control means forcibly sets the aperture of the specific photographing mode to the maximum.

  In a fourth aspect based on the first or second aspect, the control means forcibly sets the aperture in the specific shooting mode to be larger than the aperture in the automatic exposure shooting mode.

  According to a fifth aspect of the present invention, in the third or fourth aspect of the present invention, the control means is configured so that the brightness of an image shot in the specific shooting mode is equal to the brightness of the image shot in the automatic exposure shooting mode. Controls the shutter speed in shooting mode along with the aperture.

  In a sixth aspect based on any one of the first to fifth aspects, the control means forcibly sets the photographing sensitivity in the specific photographing mode to be higher than the photographing sensitivity in the automatic exposure photographing mode.

  According to a seventh aspect, in the sixth aspect, the control means is configured to control the specific shooting mode so that the brightness of the image shot in the specific shooting mode is equal to the brightness of the image shot in the automatic exposure shooting mode. Control the shutter speed along with the shooting sensitivity.

  In an eighth invention according to any one of the first to seventh inventions, the control means limits the object of the control to photographing with a maximum intensity flash among a plurality of photographing.

  According to a ninth invention, in any one of the first to eighth inventions, an image processing means is further provided. The image processing means separates the subject area and the non-subject area from the image based on information acquired in a specific shooting mode, and performs different image processing on the separated subject area and the non-subject area. .

  In a tenth aspect based on the ninth aspect, the image processing means performs blur processing on the separated non-subject area.

  In the present invention, when subject separation is performed, the amount of divergence between the amount of change in the luminance of the subject region and the amount of change in the luminance of the non-subject region between a plurality of captured images taken while changing the flash condition is the automatic exposure shooting mode. Shooting is performed by controlling the exposure condition so that it is larger than when shooting under the same exposure condition. Therefore, by using the present invention, it is possible to increase subject separation accuracy even when a subject and a non-subject (such as a background) are relatively close to each other, and subject separation with high accuracy is possible.

  FIG. 1 is a block diagram of an electronic camera to which an imaging apparatus of the present invention is applied.

  The electronic camera includes an imaging lens 11 and a lens driving unit 12, an aperture 13, an imaging device 14, a signal processing unit 15, a timing generator (TG) 16, a buffer memory 17, a data processing unit 18, a compression / The decoding unit 19, the control unit 20, the photometry unit 21, the monitor 22, the recording medium 23, the light emitting unit 24, the operation unit 25, and the bus 26 are included. Here, the buffer memory 17, the data processing unit 18, the compression / decoding unit 19, the control unit 20, the photometry unit 21, the monitor 22, and the recording medium 23 are connected via a bus 26. Further, the lens driving unit 12, the diaphragm 13, the signal processing unit 15, the TG 16, the light emitting unit 24, and the operation unit 25 are connected to the control unit 20, respectively.

  The imaging lens 11 includes a plurality of lens groups including a focus lens and a zoom lens. For simplicity, the imaging lens 11 is illustrated as a single lens in FIG.

  The lens driving unit 12 generates a lens driving signal in accordance with an instruction from the control unit 20, performs focus adjustment and zoom adjustment by moving the imaging lens 11 in the optical axis direction, and subjects the light flux that has passed through the imaging lens 11 to the subject. An image is formed on the light receiving surface of the image sensor 14.

  The diaphragm 13 adjusts the amount of the light beam reaching the light receiving surface of the image sensor 14 by adjusting the opening amount in accordance with an instruction from the control unit 20.

  The imaging element 14 is a CCD type or CMOS type imaging element, and is arranged on the image space side of the imaging lens 11. The image sensor 14 photoelectrically converts a subject image formed on the light receiving surface to generate an analog image signal. The output of the image sensor 14 is connected to the signal processing unit 15.

  The signal processing unit 15 performs CDS (correlated double sampling), gain adjustment, A / D conversion, white balance adjustment, color separation on the analog image signal output from the image sensor 14 in accordance with an instruction from the control unit 20. Signal processing such as (interpolation) and gamma conversion is performed, and the processed image signal is output. Further, the signal processing unit 15 sets an adjustment amount for gain adjustment based on an instruction from the control unit 20, and thereby adjusts the photographing sensitivity corresponding to the ISO sensitivity. The output of the signal processing unit 15 is connected to the buffer memory 17.

  The TG 16 supplies timing pulses to the image sensor 14 and the signal processing unit 15 based on instructions from the control unit 20. The drive timing of the image sensor 14 and the signal processor 15 is controlled by the timing pulse.

  The buffer memory 17 temporarily stores an image signal output from the signal processing unit 15 at the time of shooting as image data. The buffer memory 17 temporarily stores image data created in the course of processing by the control unit 20.

  The data processing unit 18 performs image processing such as blur processing on the image data in accordance with an instruction from the control unit 20.

  The compression / decoding unit 19 performs a compression process on the image data in accordance with an instruction from the control unit 20. The compression process is performed in JPEG (Joint Photographic Experts Group) format or the like.

  In accordance with an instruction from the control unit 20, the photometry unit 21 uses a known photometry method such as multi-division photometry (multi-pattern photometry), center-weighted photometry, or spot photometry based on the image data recorded in the buffer memory 17. And an evaluation value indicating the brightness of the entire shooting scene and the brightness distribution of the shooting scene (an EV value described later).

  The monitor 22 is an LCD monitor provided on the back surface of the electronic camera housing or the like, an electronic viewfinder having an eyepiece, and the like, and displays various images according to instructions from the control unit 20.

  Image data after compression processing is recorded on the recording medium 23 by the control unit 20. The recording medium 23 is a memory card incorporating a semiconductor memory, a small hard disk, or the like.

  The light emitting unit 24 drives an electronic flash or the like provided in the electronic camera housing based on an instruction from the control unit 20 to emit strobe light for illuminating the subject toward the object scene.

  The operation unit 25 includes operation members such as a release button and a shooting mode switching button, and sends an operation signal according to the content of the member operation by the user to the control unit 20.

  The control unit 20 performs overall control of each unit of the electronic camera according to the operation signal sent from the operation unit 25. For example, when the electronic camera is set to the shooting mode, the control unit 20 drives the lens driving unit 12, the diaphragm 13, and the TG 16 to start shooting a through image. At this time, the image sensor 14 is driven in the draft mode, and the image data of the through image is sequentially recorded in the buffer memory 17 via the signal processing unit 15. The control unit 20 performs focus adjustment control (AF) of the imaging lens 11 in cooperation with the lens driving unit 12 based on the image data of the through image. Further, when the control unit 20 drives the photometry unit 21 to calculate the evaluation value of the shooting scene based on the image data of the through image in the buffer memory 17, the setting contents of the signal processing unit 15 and the like are set based on the evaluation value. Adjust.

  At the time of shooting, the control unit 20 determines the presence / absence of strobe light emission and the exposure condition based on the evaluation value calculated by the photometry unit 21, and under the determined conditions, the lens driving unit 12, the diaphragm 13, the signal processing unit 15, The TG 16 and the light emitting unit 24 are driven to perform photographing. The shutter speed is set by setting at least one of an opening time of a mechanical shutter (not shown) and a charge accumulation time of the image sensor 14. At this time, the image sensor 14 is driven in the frame mode, and the image data of the captured image is recorded in the buffer memory 17 via the signal processing unit 15. The control unit 20 drives the data processing unit 18 and the compression / decoding unit 19 to perform image processing and compression processing on the image data of the image recorded in the buffer memory 17 and record the processed image data. Recording on the medium 23.

  The subject separation operation of the electronic camera of this embodiment will be described below with reference to the flowchart of FIG. 2 and FIG. The flow in FIG. 2 is executed, for example, when the operation mode of the electronic camera is a subject separation shooting mode (hereinafter referred to as “subject separation mode”). Note that the “subject” here is an object that the user wants to shoot close to the camera, and the “background” is located farther from the camera than the subject. It is an object that is difficult for light to reach.

(Explanation of flow for shooting scenes that do not require flash firing)
First, the operation when the shooting scene is a scene that does not require strobe light emission (for example, a bright scene) will be described. In FIG. 2, the flow from step 101 (S101) to step 110 (S110) corresponds to this.

  Step 101: When the user half-presses the release button of the operation unit 25, the operation unit 25 sends a half-press operation signal to the control unit 20. Upon receiving this half-press operation signal, the control unit 20 determines whether or not the shooting scene does not require strobe light emission based on the evaluation value calculated by the photometry unit 21 at that time prior to shooting. If it is a shooting scene that does not require strobe light emission (Yes side), the control unit 20 proceeds to step 102. On the other hand, if it is a shooting scene that requires strobe light emission (No side), the control unit 20 The process proceeds to step 111.

  Step 102: The control unit 20 determines an appropriate exposure condition on the assumption that the current shooting scene is a scene that does not require strobe light emission. Specifically, based on a program diagram prepared in advance for the automatic exposure shooting mode, an exposure condition (aperture value and shutter speed) that approaches the appropriate exposure is determined. For example, when the shooting scene is outdoors in the daytime and the shooting sensitivity of the electronic camera is set to ISO 100, the EV value calculated by the photometry unit 21 from the program diagram corresponding to the shooting sensitivity “ISO 100” ( For example, the exposure condition “aperture value“ F5.6 ”, shutter speed“ 1/125 seconds ”” ”corresponding to“ 12 ”is found and set as an appropriate exposure condition (in FIG. 3, this exposure condition is indicated by ●). Show).

  Step 103: Based on the exposure condition determined in Step 102, the control unit 20 sets the exposure condition for the subject separation mode as the exposure condition for the subject separation mode (main exposure condition) and the exposure condition for preliminary shooting (preliminary exposure condition). And decide. The main shooting is non-flash shooting, and the preliminary shooting is flash emission shooting. Both flash conditions are different, but the exposure conditions are basically the same.

  Here, the principle of determining the exposure condition for the subject separation mode will be described with reference to FIG. Here, for the sake of simplicity, it is assumed that the reflectance of the subject and the background is equal, and the ambient light illuminates the subject and the background with equal intensity.

  FIG. 4A shows the exposure amount of each light incident on the image sensor 14 when the exposure condition determined in step 102 is directly adopted as the exposure condition for the subject separation mode. Here, the light quantity indicated by the vertical axis is the brightness, that is, the light quantity per unit time. In FIG. 4A, (1) shows the amount of light Ps0 of the strobe light reflected by the subject and the exposure amount Ss0 obtained by integrating this Ps0 over the light emission time Ts0 of the strobe light. (2) shows the light amount Psb0 of the strobe light reflected from the background and the exposure amount Ssb0 obtained by integrating this over Ts0. (3) shows the amount of ambient light Pe0 reflected by the subject or the background and the exposure amount Se0 obtained by integrating this over the shutter speed Te0.

  Here, as described above, main shooting in the subject separation mode is flash non-flash shooting, and preliminary shooting is flash emission shooting. In this case, the brightness of the subject in the image obtained by actual photographing (main photographed image) is represented by the exposure amount Se0 (FIG. 4 [a] (3)) by ambient light, and the image obtained by preliminary photographing (preliminary photographing). The brightness of the subject in the captured image) is the exposure amount Se0 (FIG. 4 [a] (3)) due to the ambient light and the exposure amount Ss0 (FIG. 4 [a] (1)) due to the strobe light reflected by the subject. It is represented by the sum (Se0 + Ss0). Accordingly, the luminance difference of the subject between the main photographed image and the preliminary photographed image is (Se0 + Ss0) −Se0 = Ss0. Here, the luminance indicates the brightness of the image.

  Further, the background brightness in the actual captured image is represented by the exposure amount Se0 (FIG. 4 [a] (3)) by the ambient light, and the background brightness in the preliminary captured image is the exposure amount Se0 (by the ambient light). 4 [a] (3)) and the sum (Se0 + Ssb0) of the exposure amount Ssb0 (FIG. 4 [a] (2)) reflected by the strobe light reflected in the background. Therefore, the luminance difference of the background between the main photographed image and the preliminary photographed image is (Se0 + Ssb0) −Se0 = Ssb0.

  From the above, the amount of divergence between the subject luminance difference Ss0 and the background luminance difference Ssb0 in the main photographed image and the preliminary photographed image is expressed by (Ss0−Ssb0), and the larger the divergence amount, the higher the accuracy of subject separation.

  FIG. 4B shows the amount of light incident on the image sensor 14 when the exposure condition determined in step 102 with the aperture changed to the maximum is used as the exposure condition for the subject separation mode. ing. By changing the aperture to the maximum, the amount of light per unit time incident on the image sensor 14 increases at the same ratio α (α: constant, α> 1) for both strobe light and ambient light (Ps1 = αPs0). , Psb1 = αPsb0, Pe1 = αPe0). For this reason, the exposure amount (FIG. 4 [b] (1)) by the strobe light reflected from the subject increases α times to αSs0. Similarly, the exposure amount by the stroboscopic light reflected in the background (FIG. 4 [b] (2)) and the exposure amount by the ambient light (FIG. 4 [b] (3)) are also calculated from the amount before the aperture change. Doubled to αSsb0 and αSe0.

  Therefore, the luminance difference of the subject and the luminance difference of the background between the main photographic image and the preliminary photographic image are represented by αSs0 and αSsb0, respectively.

  From the above, the difference between the subject luminance difference αSs0 and the background luminance difference αSsb0 between the main photographed image and the preliminary photographed image is α (Ss0−Ssb0) = α {(Ss0−Ssb0)} = α × (aperture change). It is expressed by the previous deviation amount). As described above, since α> 1, the deviation amount after the aperture change is larger than the deviation amount before the aperture change. Therefore, when the aperture is increased, the accuracy of subject separation increases.

  However, if the aperture is increased, the amount of exposure increases as a whole, so there is a possibility that the number of pixels saturated due to overexposure will increase. Therefore, as shown in FIG. 4C, the shutter speed of the exposure condition in the subject separation mode is increased to Te1 so as to be equal to the luminance of the actual captured image in the exposure condition determined in step 102 above. In the present embodiment, an example in which the exposure amount due to ambient light (FIG. 4 [c] (3)) becomes equal to the exposure amount Se0 in FIGS. 4 [a] (3) by increasing the shutter speed is shown. On the other hand, the exposure amount by the strobe light (FIG. 4 [c] (1), (2)) does not decrease because the strobe light emission time Ts0 is shorter than the time of the shutter speed Te1, and remains as αSs0 and αSsb0. . Therefore, even if the shutter speed is increased, the luminance difference of the subject and the luminance difference of the background between the actual captured image and the preliminary captured image do not change, and the divergence between them does not change. Therefore, even if the shutter speed is controlled together with the aperture, the accuracy of subject separation can be increased.

  In summary, the amount of light incident on the image sensor 14 per unit time can be increased by greatly changing the aperture of the exposure condition determined in step 102. In this case, the exposure amount by the strobe light reflected by the subject and the exposure amount by the strobe light reflected by the background increase at the same ratio, but the difference between the two increases. As a result, the amount of divergence between the luminance difference of the subject between the two captured images and the luminance difference of the background between the two captured images increases, and the accuracy of subject separation increases. Further, even if the aperture is increased and the shutter speed is increased so as to be equivalent to the luminance of the actual captured image under the exposure condition determined in step 102, the accuracy of subject separation can be increased.

  In accordance with the above principle, in this step, the exposure condition “aperture value“ F5.6 ”, shutter speed“ 1/125 seconds ”” determined in step 102 is changed, and the main exposure condition for the subject separation mode is changed. And the preliminary exposure condition are determined to be “aperture value“ maximum (F1.4) ”and shutter speed“ 1/2000 seconds ”(in FIG. 3, this exposure condition is indicated by ◯).

  Step 104: When the user fully presses the release button of the operation unit 25 following the half-press in the above step 101, the operation unit 25 sends a full-press operation signal to the control unit 20. When receiving the full-press operation signal, the control unit 20 performs preliminary photographing under the preliminary exposure condition determined in step 103. However, in this preliminary shooting, the light emitting unit 24 is driven to forcibly emit strobe light. The image data of the pre-photographed image acquired by this pre-photographing is recorded in the buffer memory 17.

  Step 105: The control unit 20 performs the main photographing under the main exposure condition determined in the above step 103. The flash is not emitted during this shooting. The image data of the actual captured image acquired by the actual capture is recorded in a different area from the preliminary captured image in the buffer memory 17.

  Step 106: The control unit 20 compares the brightness of the preliminary captured image recorded in the buffer memory 17 and the actual captured image, extracts an area where the difference in brightness value between the images exceeds a predetermined threshold as a subject area, Separation information for separating the subject and the background is created.

  Step 107: The control unit 20 separates the subject and the background from the actual captured image in the buffer memory 17 based on the separation information created in Step 106.

  Step 108: The data processing unit 18 performs blur processing on the background according to an instruction from the control unit 20. Here, the blur process can be performed by, for example, a filter operation (convolution operation) of a point spread function, or by using various digital filters such as an averaging filter and a Gaussian filter. When the blur process is completed, the data processing unit 18 sends an interrupt signal to the control unit 20 to notify that the blur process has been completed.

  Step 109: After receiving the blur processing end notification from the data processing unit 18, the control unit 20 displays the image after the blur processing on the monitor 22.

  Step 110: The control unit 20 drives the compression / decoding unit 19 to perform compression processing on the image after blur processing and records the compressed image on the recording medium 23.

(Explanation of flow for shooting scenes that require flash firing)
Next, the operation in the case where the shooting scene is a scene that requires strobe light emission (for example, a dark scene) will be described. In FIG. 2, the flow of step 101, step 111 to step 114, and step 106 to step 110 corresponds to this. Since the flow other than steps 111 to 114 is the same as the flow in the case of a shooting scene that does not require strobe light emission, a description thereof will be omitted.

  Step 111: The control unit 20 determines an exposure condition for strobe shooting on the assumption that the current shooting scene is a scene that requires strobe light emission. For example, in the case of shooting in a dark room, if the shooting sensitivity of the electronic camera is ISO100, the control unit 20 determines the exposure condition as “aperture value“ F2.8 ”, shutter speed“ 1/30 second ””. To do.

  Step 112: The control unit 20 determines the exposure condition for the subject separation mode based on the exposure condition determined in Step 111 above. The main shooting is flash photography and the preliminary photography is non-flash photography, and the exposure conditions of both are basically the same. The principle of the determination is the same as that in step 103 above. As a result, both the main exposure condition and the preliminary exposure condition are determined as “aperture value” maximum (F1.4) ”and shutter speed“ 1/125 seconds ”.

  Step 113: When the user fully presses the release button of the operation unit 25 following the half-press in the above step 101, the operation unit 25 sends a full-press operation signal to the control unit 20. When the control unit 20 receives this full-press operation signal, it performs preliminary photographing under the preliminary exposure conditions determined in step 112. In this preliminary shooting, no strobe light is emitted.

  Step 114: The control unit 20 performs the main photographing under the main exposure conditions determined in the above step 112. However, in this actual photographing, the light emitting unit 24 is driven to emit strobe light. The image data of the actual captured image acquired by the actual capture is recorded in a different area from the preliminary captured image in the buffer memory 17.

  Heretofore, the subject separation operation of the electronic camera of the present embodiment has been described.

  Note that the imaging sensitivity may be increased instead of increasing the aperture of the subject separation mode. When the shooting sensitivity is increased, the amplification factor of the image signal is increased, and the luminance of the image with respect to the same exposure amount is increased. Therefore, when the exposure condition determined in step 102 that has been changed to increase the shooting sensitivity is adopted as the exposure condition for the subject separation mode, the luminance of the subject in the image and the background has the same ratio β ( β: constant, β> 1).

  That is, the luminance Se0 of the subject in the actual captured image and the luminance (Se0 + Ss0) of the subject in the pre-captured image under the exposure conditions determined in step 102 described above are respectively increased by βSe0 and β (Se0 + Ss0) by increasing the imaging sensitivity. ) Similarly, the background brightness Se0 in the actual captured image and the background brightness (Se0 + Ssb0) in the pre-captured image under the exposure conditions determined in step 102 described above are set to βSe0 and β ( Se0 + Ssb0). Therefore, the luminance difference of the subject between the actual captured image after the change of the imaging sensitivity and the preliminary captured image is β (Se0 + Ss0) −βSe0 = βSs0, and the difference between the actual captured image and the preliminary captured image after the change of the imaging sensitivity is obtained. The luminance difference of the background is β (Se0 + Ssb0) −βSe0 = βSsb0.

  From the above, the divergence amount between the subject luminance difference and the background luminance difference in the main photographic image and the pre-photographed image after the photographic sensitivity change is β (Ss0−Ssb0) = β × (the divergence amount before the photographic sensitivity change). It is represented by As described above, since β> 1, the divergence amount after changing the shooting sensitivity is larger than the divergence amount before changing the shooting sensitivity. Therefore, when the shooting sensitivity is increased, the accuracy of subject separation increases. However, as in the case where the aperture is increased, the actual captured image may be overexposed even if the shooting sensitivity is increased. Therefore, as in the case where the aperture is increased, the accuracy of subject separation can be improved even if the shutter speed is increased so as to be equivalent to the luminance of the actual captured image under the exposure conditions determined in step 102 above.

  Each of the method for increasing the photographing sensitivity and the method for increasing the aperture may be used singly or in combination. Further, when a method for increasing the photographing sensitivity and a method for increasing the aperture are used in combination, a method for further increasing the shutter speed may be used in combination. Ordinarily, a method of enlarging the aperture may be used, and a method of increasing the photographing sensitivity may be used in combination when the subject separation accuracy cannot be increased only by enlarging the aperture.

  In the above description, the method of increasing the aperture or the method of increasing the shooting sensitivity has been described in both the preliminary shooting and the main shooting. However, these methods are implemented in two shooting modes. Of these, it may be limited to only the flash photography.

  In the above description, the amount of light incident on the image sensor 14 is adjusted by adjusting the opening amount of the diaphragm 13, but instead of adjusting the opening amount of the diaphragm 13, the ND filter may be inserted into and removed from the optical path. . In such a case, the ND filter may be removed or an ND filter having a high transmittance may be inserted to increase the amount of light incident on the image sensor 14, and an ND filter having a low transmittance may be inserted to reduce the aperture.

  In the above description, in the description of the principle for determining the exposure condition for the subject separation mode, the luminance comparison between the strobe light emission image and the strobe non-light emission image is performed based on the luminance difference. It can also be done using a ratio. When subject separation is performed based on the difference value of luminance, separation accuracy may be lowered for a subject in which a portion with a high reflectance and a portion with a low reflectance are mixed. In such a case, if subject separation is performed based on the ratio of luminance, the reflectance of the subject is canceled in the comparison calculation process, so that high-precision separation that does not depend on the reflectance of the subject is possible (the luminance Japanese Patent Laid-Open No. 2005-130268 has proposed a technique for extracting a subject by luminance comparison based on a ratio).

  The following explains the principle of determining the exposure conditions for the subject separation mode when performing luminance comparison based on the luminance ratio. For bright scenes where the main shooting is a non-flash image and the preliminary shooting is a flash image. This will be described with reference to FIG.

  As described above, when the exposure condition determined in step 102 is used as it is as the exposure condition in the subject separation mode (see FIG. 4A), the luminance of the subject in the actual captured image is Se0, The luminance of the subject can be expressed as Se0 + Ss0, the luminance of the background in the actual captured image is Se0, and the luminance of the background in the preliminary captured image can be expressed as Se0 + Ssb0. Therefore, the luminance ratio of the subject between the main captured image and the preliminary captured image is (Se0 + Ss0) / Se0, and the luminance ratio of the background between the main captured image and the preliminary captured image is (Se0 + Ssb0) / Se0. Therefore, the difference between the luminance ratio of the subject (Se0 + Ss0) / Se0 and the luminance ratio of the subject (Se0 + Ssb0) / Se0 in the main captured image and the preliminary captured image is {(Se0 + Ss0) / Se0} − {(Se0 + Ssb0) / Se0} = It is expressed by (Ss0−Ssb0) / Se0, and the accuracy of subject separation increases as the deviation amount increases.

  As described above, when the aperture is changed to the maximum among the exposure conditions determined in step 102, the exposure amount by the strobe light reflected by the subject, the exposure amount by the strobe light reflected by the background, and the exposure amount by the ambient light Are multiplied by α (α: constant, α> 1) (see FIG. 4B), the luminance of the subject in the main captured image is αSe0, and the luminance of the subject in the preliminary captured image is α (Se0 + Ss0). , The background brightness in the main captured image can be expressed as αSe0, and the background brightness in the preliminary captured image can be expressed as α (Se0 + Ssb0). However, as described above, if the aperture is increased, the actual captured image may be overexposed. Therefore, the shutter in the subject separation mode is set so as to be equivalent to the luminance of the actual captured image under the exposure condition determined in step 102 above. The speed is increased (FIG. 4 [c] (3) shows an example in which the exposure amount by the ambient light becomes equal to the exposure amount Se0 in FIGS. 4 [a] and (3) by increasing the shutter speed from Te0 to Te1. ). In this case, the exposure amount by the ambient light decreases to Te1 / Te0 (= 1 / α) by increasing the shutter speed, but the exposure amount by the strobe light (refer to FIGS. 4C, 1C, and 2B). Since the strobe light emission time Ts0 is shorter than the shutter speed Te1, it does not change from FIG. 4 [b] (1), (2).

  That is, in the situation shown in FIG. 4C, the luminance of the subject in the actual captured image is Se0, the luminance of the subject in the preliminary captured image is Se0 + αSs0, the luminance of the background in the actual captured image is Se0, and the background in the preliminary captured image. Can be expressed as Se0 + αSsb0. Therefore, the luminance ratio of the subject between the main captured image and the preliminary captured image is (Se0 + αSs0) / Se0, and the luminance ratio of the background between the main captured image and the preliminary captured image is (Se0 + αSsb0) / Se0.

  From the above, the difference between the subject luminance ratio (Se0 + αSs0) / Se0 and the background luminance ratio (Se0 + αSsb0) / Se0 in the main photographed image and the preliminary photographed image is {(Se0 + αSs0) / Se0} − {(Se0 + αSsb0) / Se0} = α × {(Ss0−Ssb0) / Se0} = α × (deviation amount under the exposure conditions in FIG. 4A). Here, since α> 1 as described above, the amount of deviation after the aperture and shutter speed change is larger than the amount of deviation before the aperture and shutter speed is changed, and the accuracy of subject separation can be improved.

(Operational effect of this embodiment)
As described above, in the electronic camera of the present embodiment, when subject separation is performed, the aperture of the shooting condition is changed to be larger than the aperture of the exposure condition in the automatic exposure shooting mode. The amount of strobe light to be increased increases, and the difference between the exposure amount due to the strobe light reflected by the subject and the exposure amount due to the strobe light reflected from the background increases. As a result, the amount of divergence between the luminance difference of the subject area and the luminance difference of the background area between the strobe light emission image and the strobe non-light emission image is increased, so that the accuracy of subject separation is improved. The accuracy of subject separation is highest when the aperture after the change is maximum (open), but an improvement effect can be obtained by making it at least larger than the aperture of the exposure condition in the automatic exposure shooting mode.

  Further, in the electronic camera of this embodiment, since the shooting sensitivity at the time of subject separation is changed to be higher than the shooting sensitivity of the exposure condition in the automatic exposure shooting mode, the image signal is amplified, and as a result, the flash The amount of divergence between the luminance difference in the subject area and the luminance difference in the background area between the light emission image and the non-flash image is increased, and the accuracy of subject separation is increased.

  In the electronic camera according to the present embodiment, the shutter speed is controlled in accordance with the change of the aperture and / or the change of the photographing sensitivity so as to be equivalent to the luminance of the image photographed in the automatic exposure photographing mode. As a result, overexposure does not occur and saturated pixels do not increase, so that the accuracy of subject separation is improved.

  Further, in the electronic camera of the present embodiment, the luminance comparison between the strobe light emission image and the strobe non-light emission image is performed based on the difference in luminance, and not only when the subject is separated, but also from the strobe light emission image based on the luminance ratio. Even when the luminance is compared with the non-flash image and the subject is separated, the separation accuracy can be improved.

  Therefore, in the electronic camera of this embodiment, subject separation is performed with high accuracy.

(Other)
Note that it is not necessary to change the shutter speed for the pre-exposure condition for the subject separation mode determined in step 103 above. This is because the pre-photographed image is different from the ornamental main-photographed image and is not used other than the luminance comparison in the subject separation, and may be an overexposed image.

  In the above description, an example is described in which preliminary shooting is first performed in step 104 and step 113, and then main shooting is performed in step 105 and step 114. However, the execution order of these shootings may be reversed.

  In addition, the preliminary photographing in step 104 may be performed a plurality of times while changing the intensity of the strobe light. If preliminary shooting is performed a plurality of times, the luminance comparison in step 106 may be performed using only a plurality of preliminary captured images with different strobe light intensities without using the actual captured image. .

  Further, the luminance comparison between the preliminary photographed image and the main photographed image in step 106 may be performed in units of pixels, or may be performed in units of areas composed of a plurality of pixels.

  In step 107, after the subject and the background are separated, a process of displaying the separation state on the monitor 22 and allowing the user to confirm the separation state may be executed. If the user instructs the re-execution of the separation process when confirming the separation state, the threshold for extracting the subject area is changed to another value, and the subject separation process in steps 106 to 107 is performed again. May be executed. Further, these series of processes may be repeatedly executed until a result desired by the user is obtained.

  In step 106, the subject / background separation information obtained by the luminance comparison is recorded on the recording medium 23 together with the actual photographic image, and the processing in steps 107 to 110 may be completed without being executed. Good. In such a case, when the operation mode of the electronic camera is a non-photographing mode such as an image playback mode, the processing from step 107 to step 110 is performed using the separation information of the recording medium 23 and the actual captured image. You may make it perform.

  In addition, some or all of the processing in steps 106 to 110 may be executed by an external processing device such as a computer instead of being executed by the electronic camera.

  Further, in the above step 108, an example of performing blur processing on the background has been shown as an example of image processing, but the image processing is not limited to this, and brightness correction processing, contrast enhancement processing, sharpening processing other than blur processing is performed. Various processes can be applied.

  The image processing described above may be performed only on the subject, may be performed only on the background, or may be performed on both the subject and the background. When performing image processing on both the subject and the background, it is desirable to perform different image processing on the subject and the background, respectively. For example, it is desirable that the subject is sharpened, the background is blurred, the subject is weaker than the background, or the subject is subjected to contrast enhancement processing or sharpening stronger than the background. .

  In step 109, after the image processed image is displayed on an LCD monitor or the like, an instruction to redo the image processing may be received from the user. When the user instructs to redo, the conditions of the image processing (for example, in the case of the blur processing, the strength of the blur processing, etc.) are changed, and the processing from step 108 to step 109 is executed again. Also good. Further, these series of processes may be repeatedly executed until a result desired by the user is obtained.

  In the above example, the subject and the background are separated from the photographed image. However, the subject and the foreground may be separated, or the subject, the background, and the foreground may be separated at the same time.

It is a block diagram of the electronic camera to which the imaging device of the present invention is applied. It is a flowchart which shows the object separation operation | movement of the electronic camera of this embodiment. It is a figure explaining an example of the relationship between the appropriate exposure conditions of automatic exposure photography mode, and the exposure conditions of subject separation mode in a daytime outdoor scene. It is a figure explaining the principle which determines the exposure conditions for object separation modes.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Imaging lens, 12 ... Lens drive part, 13 ... Aperture, 14 ... Imaging element, 15 ... Signal processing part, 16 ... Timing generator (TG), 17 ... Buffer memory, 18 ... Data processing part, 19 ... Compression / decoding , 20 ... control unit, 21 ... photometry unit, 22 ... monitor, 23 ... recording medium, 24 ... light emitting unit, 25 ... operation unit, 26 ... bus

Claims (10)

Specific shooting that separates the subject area and non-subject area of the image, and continuously takes multiple shots while changing the flash conditions, and determines the amount of change in brightness between the multiple shot images In an imaging device equipped with a mode,
Control the exposure condition of the specific shooting mode so that the difference between the change amount of the subject area and the change amount of the non-subject area is larger than when shooting under the same exposure conditions as in the automatic exposure shooting mode. An image pickup apparatus comprising a control unit. The imaging apparatus according to claim 1,
The control unit obtains the amount of change by a luminance difference or ratio between the plurality of captured images. In the imaging device according to claim 1 or 2,
The control unit forcibly sets the aperture of the specific shooting mode to the maximum. In the imaging device according to claim 1 or 2,
The control device forcibly sets an aperture in the specific shooting mode to be larger than an aperture in the automatic exposure shooting mode. In the imaging device according to claim 3 or 4,
The control means combines the shutter speed of the specific shooting mode with the aperture so that the luminance of the image shot in the specific shooting mode is equal to the luminance of the image shot in the automatic exposure shooting mode. And an imaging device. In the imaging device according to any one of claims 1 to 5,
The image pickup apparatus, wherein the control means forcibly sets the shooting sensitivity of the specific shooting mode to be higher than the shooting sensitivity of the automatic exposure shooting mode. The imaging device according to claim 6,
The control means combines the shutter speed of the specific shooting mode with the shooting sensitivity so that the luminance of the image shot in the specific shooting mode is equal to the luminance of the image shot in the automatic exposure shooting mode. And an image pickup apparatus characterized by being controlled. In the imaging device according to any one of claims 1 to 7,
The control unit limits an object of the control to photographing with a flash of maximum intensity among the plurality of photographing. In the imaging device according to any one of claims 1 to 8,
An image processing unit that separates the subject region and the non-subject region from the image and performs different image processing on the subject region and the non-subject region based on information acquired in the specific shooting mode. An image pickup apparatus further comprising: The imaging device according to claim 9,
The imaging apparatus, wherein the image processing means performs blur processing on the non-subject area.

Images