JP4956403B2 - Imaging apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to a technique for correcting camera shake by combining a plurality of images in an imaging apparatus such as a digital camera.

  2. Description of the Related Art Conventionally, various techniques for correcting camera shake occurring during hand-held shooting such as a silver salt camera, a video camera, and an electronic still camera have been proposed and put into practical use.

  For example, a technology for suppressing camera shake by detecting camera shake using an angular velocity sensor such as a vibration gyro and swinging the optical system or driving a correction optical system in a direction perpendicular to the optical axis is used for single-lens reflex cameras. Already used in interchangeable lenses, electronic still cameras, and the like. The anti-vibration function makes it possible to shoot without being affected by camera shake in more situations, and the effect is great. However, with the addition of the angular velocity sensor and the correction optical system, there is a problem that an increase in size and cost are inevitable as compared with an interchangeable lens and an electronic still camera that do not have a camera shake correction function.

  On the other hand, video cameras shoot at 1/60 second or 1/50 second intervals according to television systems such as NTSC and PAL. Therefore, the motion vector between the captured images is detected, and the angular velocity sensor Some use a technique to measure the amount of camera shake instead. In addition, with regard to camera shake correction, a camera shake suppression function is realized without having a movable part like a correction optical system by changing the cutout position for extracting an image from the image sensor according to the detected camera shake amount. There are also things. These technologies can avoid an increase in size and cost.

  However, in an electronic still camera, it is difficult to apply such a camera shake correction technique as it is because of the difference between a moving image and a still image. This is because the electronic still camera continuously accumulates in the image sensor during the exposure time determined by the photometric result, like the silver salt camera. Since it is difficult to read out image data in parallel with accumulation, a motion vector cannot be obtained from an image in real time during exposure with an electronic still camera. As for camera shake correction, since an image on which camera shake is superimposed is already accumulated on the image sensor during exposure, the camera shake cannot be corrected even if the cutout position of the image is changed.

  Thus, the following techniques have been proposed as camera shake correction techniques suitable for electronic still cameras. In other words, multiple under-exposed images are continuously shot at a fast shutter speed that does not make camera shake noticeable, and combined while aligning the multiple images in the post-shooting process. An image of exposure is obtained.

Patent Document 1 already filed by the applicant of the present application discloses a technique for synthesizing sequentially captured images after performing coordinate conversion corresponding to the positional displacement over time.
Japanese Patent No. 3110797

  By the way, in the electronic still camera, it is possible to set a sensitivity corresponding to the ISO sensitivity of the silver salt film for each photographing frame. A camera having an automatic exposure function for automatically setting an aperture value and a shutter speed including sensitivity has been put into practical use. However, when the sensitivity is increased, there is a problem that noise increases and image quality is lowered.

  Here, the technique for correcting camera shake by combining a plurality of images described above basically does not suppress camera shake by increasing the sensitivity and increasing the shutter speed, but without increasing the sensitivity. This is a composite of a plurality of under-exposed images continuously photographed at a high shutter speed.

  However, even with camera shake correction by image synthesis, if the subject is dark, there is a problem that if the sensitivity is not increased, the number of images to be continuously shot increases, and it takes much time for the shooting operation and image synthesis processing. On the other hand, if the sensitivity is increased, the number of images to be continuously shot can be reduced, but there is a problem that the noise is increased as described above and the image quality of the combined image is lowered.

  Accordingly, the present invention has been made in view of the above-described problems, and its purpose is to determine the relationship between the sensitivity and the number of images to be continuously shot when correcting a camera shake by combining a plurality of continuously shot images. It is to be able to control appropriately.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention obtains a target exposure image by synthesizing a plurality of underexposed images that are continuously shot. An imaging unit that generates an image signal from a subject image; a setting unit that sets an upper limit value of sensitivity of the imaging unit; and the continuous value based on the upper limit value of the sensitivity set by the setting unit. Control means for controlling the shooting conditions for each single image to be shot and the number of images to be shot continuously, and the control means continuously controls the luminance of the subject as the luminance decreases. Control is performed to increase the number of images to be shot, and when the target exposure cannot be obtained even when the predetermined number of images is reached, the predetermined number is maintained and the sensitivity of each one is increased. this control The features.

In addition, a control method for an image pickup apparatus according to the present invention includes an image pickup unit that generates an image signal from a subject image, and combines a plurality of consecutively shot underexposed images to obtain a target exposure. A method for controlling an imaging device that obtains an image, wherein a setting step of setting an upper limit value of sensitivity of the imaging unit and the continuous shooting are performed based on the upper limit value of the sensitivity set in the setting step A control step for controlling the shooting conditions for each single image and the number of images to be continuously shot. In the control step, the images to be continuously shot as the luminance of the subject decreases. this controlled to increase the number, when the well is no longer exposed to obtain a target reaches the number of sheets predetermined for controlling so as to raise the respective one of the sensitivity while maintaining the number of the predetermined of The features.

  According to the present invention, it is possible to appropriately control the relationship between the sensitivity and the number of images to be continuously shot when correcting a camera shake by combining a plurality of continuously shot images.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus (a lens interchangeable single-lens reflex digital camera) according to an embodiment of the present invention.

  In FIG. 1, a photographing lens 100 is detachably attached to an imaging device 200 via a lens mounting mechanism of a mount unit (not shown). An electrical contact unit 107 is provided in the mount portion. The imaging apparatus 200 communicates with the photographing lens 100 via the electrical contact unit 107 to control the driving of the focus lens 101 in the photographing lens 100 and the diaphragm 102 that adjusts the amount of light. In FIG. 1, only the focus lens 101 is shown as a lens in the photographic lens 100. However, a variable power lens and a fixed lens are provided in addition to this, and a lens unit is configured including these.

  A light beam from a subject (not shown) is guided to a quick return mirror 203 in the imaging apparatus 200 via a lens unit including a focus lens 101 in the photographing lens 100 and a diaphragm 102. The quick return mirror 203 is disposed obliquely with respect to the optical axis in the photographing optical path, and retracts to the first position (illustrated position) for guiding the light beam from the subject to the upper finder optical system and out of the photographing optical path. Movement to the second position is possible.

  The central portion of the quick return mirror 203 is a half mirror. When the quick return mirror 203 is down to the first position, a part of the light beam from the subject passes through the half mirror portion. The transmitted light beam is reflected by a sub mirror 204 provided on the back side of the quick return mirror 203 and guided to a phase difference AF sensor 205 that constitutes an automatic focus adjustment unit together with a focus detection circuit 206. The focus detection circuit 206 detects the focus state (focus detection) of the taking lens 100 from the output signal of the phase difference AF sensor 205 using a known phase difference AF technique.

  On the other hand, the light beam reflected by the quick return mirror 203 reaches the eyes of the photographer through a finder optical system including a finder screen 202 existing on the focus surface, a pentaprism 201, and an eyepiece lens 207.

  When the quick return mirror 203 is raised to the second position, the light beam from the photographing lens 100 reaches the image sensor 212 via the focal plane shutter 210 and the optical filter 211 that are mechanical shutters. The image sensor 212 is an image sensor that photoelectrically converts a subject image represented by a CCD or CMOS sensor. The optical filter 211 has a function of cutting infrared rays and guiding only visible light to the image sensor 212 and a function as an optical low-pass filter.

  The focal plane shutter 210 has a front curtain and a rear curtain, and controls transmission and blocking of a light beam from the photographing lens 100.

  When the quick return mirror 203 is raised to the second position, the sub mirror 204 is also folded with respect to the quick return mirror 203 and retracts out of the photographing optical path.

  Further, the imaging apparatus 200 of the present embodiment includes a system controller 230 that controls the entire imaging apparatus. The system controller 230 is configured by a CPU, an MPU, and the like, and controls operations of respective circuits described later.

  The system controller 230 communicates with the lens control circuit 104 and the aperture control drive circuit 106 in the photographing lens 100 via the electrical contact unit 107. The lens control circuit 104 controls the lens driving mechanism 103 that performs focusing by driving the focus lens 101 along the optical axis in accordance with a signal from the system controller 230. The lens driving mechanism 103 has a stepping motor or a DC motor as a driving source.

  The diaphragm control drive circuit 106 controls the diaphragm drive mechanism 105 that drives the diaphragm 102 in accordance with a signal from the system controller 230.

  When the taking lens 100 is a zoom lens, a zoom mechanism 108 that can manually change the focal length of the lens is incorporated.

  Further, a change in the focal length of the lens by the zoom mechanism 108 is detected by the focal length detection unit 109 and communicated as focal length information to the system controller 230 via the lens control circuit 104 and the electrical contact unit 107.

  The system controller 230 is also connected to the shutter control circuit 215 and the photometry circuit 209. The shutter control circuit 215 controls the driving of the front and rear curtains of the focal plane shutter 210 in accordance with a signal from the system controller 230.

  An EEPROM (storage means) 223 is also connected to the system controller 230. The EEPROM 223 stores parameters that need to be adjusted to control the imaging apparatus 200 and camera ID (identification) information that is unique information for identifying the individual imaging apparatus. In addition, adjustment values and the like of parameters relating to shooting, which are adjusted using a reference lens (a shooting lens used when the imaging apparatus 200 is adjusted in a factory), are stored.

  The photometric circuit 209 is connected to a photometric sensor 208 disposed in the vicinity of the eyepiece lens 207, and measures the luminance of the subject through the photometric sensor 208. The measurement result of the photometry circuit 209 is sent to the system controller 230.

  Further, the system controller 230 controls the lens driving mechanism 103 via the lens control circuit 104 to move the focus lens 101 and form a subject image on the image sensor 212. The system controller 230 controls the aperture driving mechanism 105 via the aperture control drive circuit 106 based on the set Av value (aperture value), and further, based on the set Tv value (shutter speed). A control signal is output to the shutter control circuit 215.

  The drive source of the front curtain and rear curtain of the focal plane shutter 210 is constituted by a spring, and after the shutter travels, a spring charge is required for the next operation. The shutter charge mechanism 214 performs this spring charge. Also, the click return mirror 203 is driven up and down by the mirror drive mechanism 213.

  In addition, a camera DSP 227 is connected to the system controller 230. The camera DSP 227 is a correction data sample circuit and a correction circuit configured by a DSP (digital signal processor). Then, control of the image sensor 212 and correction and processing of image data input from the image sensor 212 are executed based on a command from the system controller 230. The items of image data correction and processing include auto white balance.

  The auto white balance is a function for correcting the maximum luminance portion in the photographed image to a predetermined color (white). For auto white balance, the correction amount can be changed by a command from the system controller 230.

  An A / D converter 217, a video memory 221, and a work memory 226 are connected to the camera DSP 227 via a timing generator 219 and a selector 222.

  The electrical signal from the image sensor 212 is supplied to the CDS / AGC circuit 216 that removes reset noise and the like included in the output of the image sensor 212 by a known method such as correlated double sampling and amplifies the output to a predetermined signal level. Entered. The output signal from the CDS / AGC circuit 216 is converted into a predetermined digital signal by the A / D converter 217 in order for each pixel.

  Here, the image sensor 212 is driven by the output from the driver circuit 218 for horizontal driving and vertical driving for each pixel based on the signal from the timing generator 219 that determines the overall driving timing. Generate and output an image signal. Similarly, the CDS / AGC circuit 216 and the A / D converter 217 that process the image signal output from the image sensor 212 in an analog manner and convert it into a predetermined signal level are also based on the timing signal from the timing generator 219. Operate.

  The output from the A / D converter 217 is input to the memory controller 228 via the selector 222 that selects a signal based on the signal from the system controller 230, and is all transferred to the DRAM 229 that is a frame memory.

  After the shooting operation is completed, the contents of the DRAM 229 storing the shooting data are transferred to the camera DSP 227 via the selector 222 under the control of the memory controller 228. The camera DSP 227 generates RGB color signals based on the pixel data of the shooting data stored in the DRAM 229.

  In a video camera or a compact digital camera, the result is periodically transferred (for each frame) to the video memory 221 in the pre-shooting state, so that a finder display (live view) is performed by the monitor display unit 220. . In a single-lens reflex digital camera, since the image pickup device 212 is normally shielded by the quick return mirror 203 and the focal plane shutter 210 before shooting, live view cannot be performed. However, the live view operation can be performed by raising the quick return mirror 203 and retracting it from the photographing optical path and then opening the focal plane shutter 210. In addition, the camera DSP 227 or the system controller 230 processes the image signal from the image sensor during live view, so that a contrast evaluation value can be obtained. Using this evaluation value, contrast AF can be performed.

  At the time of shooting, each pixel data for one frame is read from the DRAM 229 according to a control signal from the system controller 230, subjected to image processing by the camera DSP 227, and temporarily stored in the work memory 226. Then, the data in the work memory 226 is compressed by the compression / decompression circuit 225 based on a predetermined compression format, and the result is stored in an external nonvolatile memory (external memory) 224. As the nonvolatile memory 224, a nonvolatile memory such as a flash memory is usually used. Further, it may be a hard disk, a magnetic disk, or the like.

  The camera DSP 227 also has a function of detecting a motion vector for image blur correction, which will be described later. When motion vector detection is performed, pixel data for two frames stored in the DRAM 229 is read, and correlation calculation is performed by the camera DSP 227 to obtain a motion vector. Details of how to determine the motion vector will be described later.

  When observing image data that has already been taken, the data compressed and stored in the non-volatile memory 224 is expanded into data for each normal captured pixel through the compression / decompression circuit 225, and the result is connected to the camera DSP 227. The video memory 221 is transferred. Thereby, display can be performed through the monitor display circuit 220.

  The system controller 230 is further connected to a display circuit 231, a release switch SW1 (233), a release switch SW2 (234), and an image stabilization mode switch 235 for switching the image stabilization mode ON / OFF state. A mode dial 236 for switching the shooting mode and operation switches 232 for performing other settings and various selections are also connected. The mode dial 236 has positions indicating various shooting modes such as “P (program mode)”, “Tv (shutter speed priority mode)”, “Av (aperture priority mode)”, “M (manual mode)”. In addition, there are shooting mode positions that are preset according to the subject, such as sports for landscapes, macro photography, and moving body photography. In addition, there is a position indicating power OFF, which also serves as a power switch.

  In addition to the above, a sensitivity upper limit setting unit 237 that is a switch for setting an upper limit of sensitivity at the time of shooting is also connected to the system controller 230.

  The display circuit 231 displays the operation state of the camera set or selected by the above switches using a display element such as a liquid crystal element, LED (light emitting diode), or organic EL.

  The release switch SW1 (233) is a switch for starting photographing preparation operations such as photometry and focus detection. The release switch SW2 (234) is a switch for starting a photographing operation (mirror up for acquiring a still image, diaphragm drive, shutter drive, charge accumulation and charge read-out operation, etc.).

  On the other hand, in the photographing lens 100, the lens control circuit 104 has performance information such as an open aperture value of the photographing lens 100, lens ID (identification) information that is unique information for identifying the photographing lens 100, and a system controller 230. A memory (not shown) is provided for storing information received by communication from. Note that the performance information and the lens ID information are transmitted to the system controller 230 by initial communication at the time of mounting on the imaging apparatus 200, and the system controller 230 stores them in the EEPROM 223.

  Next, the operation of the imaging apparatus 200 according to an embodiment will be described with reference to FIGS.

  2 and 3 are flowcharts showing a main routine of the operation of the imaging apparatus 200 of the present embodiment.

  The system controller 230 initializes flags, control variables, and the like by turning on the power supply such as battery replacement (step S101).

  The system controller 230 determines the setting position of the mode dial 236, and if the mode dial 236 is set to power OFF (step S102), the process proceeds to step S104. Then, the display of each display unit is changed to the end state, and necessary parameters, setting values, and setting modes including flags and control variables are recorded in the EEPROM 223. In addition, after performing a predetermined end process such as shutting off unnecessary power sources of each part of the imaging apparatus 200 (step S104), the process returns to step S102.

  If the mode dial 236 is set to a photographing mode such as P, Tv, Av, M, etc. (step S102), the process proceeds to step S105.

  The system controller 230 determines whether there is a problem in the operation of the imaging apparatus 200 due to the remaining capacity of the power source (not shown) and the operation status that are configured by a battery or the like (step S105). If there is a problem, the display circuit 231 is used to display a predetermined warning by image or sound (step S107), and then the process returns to step S102.

  If there is no problem with the power supply (step S105), the system controller 230 determines whether the operation state of the nonvolatile memory 224 that is a recording medium is a problem with the operation of the imaging apparatus 200, particularly with respect to the recording / reproducing operation of the image data with respect to the recording medium. Is determined (step S106). If there is a problem, the display circuit 231 is used to display a predetermined warning by image or sound (step S107), and then the process returns to step S102.

  If there is no problem in the operation state of the non-volatile memory 224 (step S106), the display circuit 231 is used to display various setting states of the imaging device 200 using images and sounds (step S108).

  The system controller 230 checks the setting state of the image stabilization mode switch 235 (step S109), and sets the image stabilization function flag if the image stabilization function is ON (step S110). If the image stabilization function is OFF, the image stabilization function flag is canceled (step S111).

  The state of the image stabilization function flag is stored in the internal memory of the system controller 230 or the DRAM 229.

  Next, the sensitivity upper limit setting unit 237 is operated, and it is determined whether or not the user has set the upper limit of the photographing sensitivity that can be accepted by the user (step S121). If set, the set upper limit sensitivity is stored. The upper limit sensitivity is stored in the internal memory of the system controller 230 or the DRAM 229. In the present embodiment, the settable upper limit sensitivity can be selected from three types: ISO400, ISO800, and ISO1600.

  If release switch SW1 (233) has not been pressed (step S112), the process returns to step S102.

  If the release switch SW1 (233) is pressed (step S112), the system controller 230 performs phase difference distance measurement processing by the phase difference AF sensor 205 and the focus detection circuit 206. Then, communication is performed on the photographing lens 100 side according to the result, and the focus of the focus lens 101 is focused on the subject by the lens control circuit 104 and the lens driving circuit 103. Next, photometric processing is performed to determine the aperture value and shutter time (imaging conditions) (step S113).

  In the photometric processing, although not shown or described in the present embodiment, a flash built in or externally attached to the imaging apparatus 200 is also set if necessary.

  Details of the distance measurement / photometry processing routine S113 will be described later with reference to FIG.

  When the distance measurement / photometry processing step S113 is completed, the process proceeds to the determination (step S114) as to whether or not the release switch SW2 (234) has been pressed.

  If release switch SW2 (234) is not pressed (step S114) and release switch SW1 (233) is also released (step S115), the process returns to step S102.

  If release switch SW2 (234) is pressed (step S114), the process proceeds to step S116.

  The system controller 230 includes an exposure process for writing captured image data in the DRAM 229 via the image sensor 212, the A / D converter 217, the camera DSP 227, and the memory controller 228, and the memory controller 228 and, if necessary, the camera DSP 227. Then, a photographing process including a developing process for reading out the image data written in the DRAM 229 and performing various processes is executed (step S116).

  Details of the photographing processing routine S116 will be described later with reference to FIG.

  The system controller 230 reads the captured image data written in the DRAM 229 and performs various image processing using the memory controller 228 and, if necessary, the camera DSP 227. Further, after performing an image compression process according to the mode set using the compression / decompression circuit 225, a recording process for writing image data to the nonvolatile memory 224 as a recording medium is executed (step S117).

  Details of the recording processing routine S117 will be described later with reference to FIG.

  If the release switch SW2 (234) has been pressed when the recording processing step S117 is completed (step S118), the system controller 230 will store the continuous shooting flag stored in the internal memory of the system controller 230 or the DRAM 229. Is determined (step S119). If the continuous shooting flag has been set, the flow returns to step S116 to perform continuous shooting, and the next shooting is performed.

  If the continuous shooting flag is not set (step S119), the current process is repeated until the release switch SW2 (234) is released (step S118).

  If the release switch SW1 (233) has been pressed (step S120), the system controller 230 returns to step S114 to prepare for the next shooting.

  If the release switch SW1 (233) has been released (step S120), the system controller 230 finishes a series of shooting operations and returns to step S102.

  FIG. 4 is a flowchart showing details of the distance measurement / photometry processing in step S113 of FIG.

  First, in the distance measurement / photometry processing, the system controller 230 causes the focus detection circuit 206 to transfer a part of the light beam from the subject that is transmitted through the half mirror portion of the quick return mirror 203 and reflected by the sub mirror 204 to the phase difference AF sensor 205. Read out as the top image. Then, the distance to the subject is detected by a known phase difference AF technique (step S201).

  Then, the system controller 230 communicates a lens drive command to the lens control circuit 104 via the electrical contact unit 107. Then, the lens control circuit 104 drives the focus lens 101 via the lens driving mechanism 103 based on the command value (step S202).

  After driving, the system controller 230 performs distance measurement again using the focus detection circuit 206 and the phase difference AF sensor 205 (step S203).

  If it is determined that the in-focus state is obtained as a result of the distance measurement (step S204), the process proceeds to photometry processing. If it is determined that it is not in focus, the sequence from step S201 to step S204 is repeated until it is in focus.

  In the photometric process (step S205), the system controller 230 measures the subject brightness by the photometric sensor 208 via the photometric circuit 209.

  Next, the system controller 230 determines the state of the image stabilization function flag stored in the internal memory of the system controller 230 or the DRAM 229 (step S206). When the image stabilization function is ON, image stabilization AE control is performed (step S207), and when the image stabilization function is OFF, normal AE control is performed (step S208).

  When the image stabilization function is ON, a plurality of images are continuously photographed, and thereafter, addition and synthesis are performed while correcting the positional deviation of the image by the image processing apparatus. The shutter speed at this time needs to be controlled at a high shutter speed at which camera shake is unlikely to occur according to the number of images to be combined, compared to the normal AE control. Therefore, each of the plurality of images becomes an underexposed image, and by combining them, a target exposure image is obtained as a result.

  The AE control (step S207) when the image stabilization function is ON will be described later with reference to FIG.

  Thus, the distance measurement / photometry process ends.

  FIG. 5 is a flowchart showing details of the photographing process in step S116 of FIG.

  When the photographing process is started, first, the system controller 230 mirrors the quick return mirror 203 via the mirror driving mechanism 213 and moves it to the second position retracted out of the photographing optical path. At this time, the sub mirror 204 is also folded with respect to the quick return mirror 203 and retracted out of the photographing optical path (step S301).

  Next, the system controller 230 communicates the aperture drive command to the aperture control drive circuit 106 via the electrical contact unit 107 based on the Av value obtained during the AE calculation. Based on the command value, the diaphragm control drive circuit 106 drives the diaphragm 102 to narrow down via the diaphragm drive mechanism 105 (step S302).

  Next, the system controller 230 outputs a control signal to the shutter control circuit 215, causes the shutter front curtain to travel, and opens the shutter (step S303). By this operation, exposure of the image sensor 212 is started (step S304).

  Next, the system controller 230 determines whether or not a time corresponding to the Tv value set at the time of photometry (step S205 to step S208) has elapsed by using an internal timer or the like from the start of exposure (step S305). If the set time has not elapsed, the process waits by looping step S305 until it elapses. If it has elapsed, it is determined that the exposure is completed and the process proceeds to the next step S306.

  In step S306, the system controller 230 outputs a control signal to the shutter control circuit 215, runs the shutter rear curtain, closes the shutter, and ends the exposure of the image sensor 212.

  Next, the system controller 230 communicates an aperture drive command for opening the aperture to the aperture control drive circuit 106 via the electrical contact unit 107. Based on the command value, the aperture control drive circuit 106 drives the aperture 102 to the open state via the aperture drive mechanism 105 (step S307).

  Next, the system controller 230 lowers the quick return mirror 203 via the mirror drive mechanism 213 and moves it to the first position for guiding the light beam from the subject to the viewfinder optical system (step S308).

  Next, the system controller 230 performs spring charging by the shutter charge mechanism 214 in preparation for the next shooting (step S309).

  Next, the system controller 230 gives a command to the camera DSP 227 and reads out an image signal formed on the image sensor 212 during exposure (step S310). The read signal is amplified by the CDS / AGC circuit 216 via the timing generator 219 and the selector 222 connected to the camera DSP 227, and converted into a digital signal by the A / D converter 217 for each pixel. The image data converted into the digital signal is input to the memory controller 228 via the selector 222, transferred to and saved in the DRAM 229 which is a frame memory (step S311).

  Next, the system controller 230 determines the state of the image stabilization function flag stored in the internal memory of the system controller 230 or the DRAM 229 (step S312). ) If the image stabilization function is OFF in step S312, the shooting operation is terminated.

  If the image stabilization function flag has been set (step S312), it is determined whether image stabilization continuous shooting has been completed (step S313). Repeat the above steps until copying is complete.

  After the image stabilization continuous shooting is completed, image stabilization processing is performed next. First, the system controller 230 initializes to 1 a variable n indicating what number is processed with the variables used in the image stabilization image processing in the DRAM 229 (step S314).

  Next, the system controller 230 uses the motion vector detection function of the camera DSP 227 to calculate a motion vector between the n-th frame image and the n + 1-th frame image in the image stabilization continuous shooting (step S315).

  Various methods are known as motion vector detection methods, and any of these methods can be used. An example will be described with reference to FIG. FIG. 6 is a diagram for explaining an example of a motion vector detection method.

  FIG. 6A is a diagram showing how the screen is divided into small blocks. Reference numeral 300 denotes a photographed image. Here, it is assumed that an image having 3072 horizontal pixels, 2048 vertical pixels and a total number of pixels of 6,291,456 pixels is captured. In order to obtain the motion vector, the entire screen is divided into small blocks 301 each having 256 pixels horizontally and 256 pixels vertically. In this way, as shown in FIG. 6A, it is divided into 12 horizontal blocks and 8 vertical blocks.

  Next, for each small block, a two-dimensional correlation value with another image for obtaining a motion vector is obtained. The correlation value is obtained by successively calculating the sum of absolute values of differences between corresponding pixels while shifting the small block in units of pixels, and the shift amount that minimizes the sum is used as the motion vector of the small block. .

  In this way, as shown in FIG. 6B, the motion vector 302 is obtained for all the small blocks.

  Next, the frequency of the motion vector thus obtained is examined. FIG. 6C shows a histogram in which the frequencies are plotted in order from the mode value. A motion vector between two images is obtained from the histogram thus obtained. For example, in the case of FIG. 6C, (2, 1) is the mode value, which is a motion vector between the two images.

  Next, in order to perform image composition, the system controller 230 uses the camera DSP 227 to perform coordinate conversion using the motion vector value obtained in step S315 so that the n + 1 frame image overlaps the n frame image. This is performed (step S316).

  Next, the coordinate-converted n + 1 frame image is added and combined with the n frame image (step S317). The synthesized image is recorded in the DRAM 229 (step S318). Then, n = n + 1 is set (step S319), and it is determined whether or not the synthesis of all the images subjected to the vibration proof continuous shooting is finished (step S320). If not finished, the steps from step S315 to step S320 are repeated. . If it is determined in step S320 that the compositing process has been completed to the end of the continuously shot images, the sequence of the photographing process is ended.

  Next, the operation of the recording process will be described with reference to FIG. FIG. 7 is a flowchart showing details of the recording process in step S117 of FIG.

  The system controller 230 reads out the captured image data written in the DRAM 229 using the memory controller 228 and, if necessary, the camera DSP 227, and performs pixel square processing for interpolating the vertical / horizontal pixel ratio of the image sensor to 1: 1 ( Step S401). Thereafter, the processed image data is written into the work memory 226 using the camera DSP 227.

  Then, the image data written in the work memory 226 is read out, and image compression processing according to the set mode is performed by the compression / decompression circuit 225 (step S402). Thereafter, the compressed image data is written into the non-volatile memory 224 such as a memory card or a compact flash (registered trademark) card (step S403).

  When the writing to the recording medium is finished, the recording processing routine S117 is ended.

  Next, the AE control at the time of image stabilization in step S207 of FIG. 4 will be described using FIG. FIG. 8 is a flowchart showing the operation of the anti-vibration AE control.

  First, the photometric value measured in step S205 and stored in the memory in the system controller 230 or the DRAM 229 is read (step S501).

  Next, the system controller 230 reads the focal length information from the focal length detection unit 109 of the photographic lens 100 from the lens control circuit 104 via the electrical contact unit 107 by communication (step S502).

  Next, a lower limit shutter speed at which the camera shake is difficult at the focal length is calculated from the lens focal length information obtained in step S502 (step S503). Such a shutter speed can be obtained as follows. In a camera using a 35 mm film from the past, assuming that the focal length of the taking lens is fmm, camera shake occurs at a shutter speed of 1 / f (sec) or higher (1 / f (sec) or faster shutter speed). It is said that it does not affect the image (image quality). In general, an image pickup device of a digital camera is smaller than a 35 mm film, but a focal length equivalent to a 35 mm film camera is obtained from a ratio of the size of the image pickup device to the 35 mm film and an actual focal length of the photographing lens, and is obtained as f ′. And If a shutter speed equal to or greater than the reciprocal (1 / f '(sec)) is used, camera shake can be made less likely to affect the image. Alternatively, a higher shutter speed may be set in consideration of an enlarged display on a PC monitor or the like.

  In a full-size image sensor having an imaging area equivalent to that of a 35 mm film, as in the case of a 35 mm film, a photographer with a standard amount of camera shake may regard 1 / f (sec) as the lower limit shutter speed. For example, in the case of a lens having a focal length of 135 mm, 1 / f (sec) is 1/135 sec. However, considering a standard shutter speed series, 1/125 sec may be regarded as the lower limit shutter speed.

  Next, the upper limit sensitivity setting which is set by the sensitivity upper limit setting unit 237 and is acceptable to the photographer stored in the internal memory of the system controller 230 or the DRAM 229 is read (step S504). Here, when the sensitivity is increased, noise is increased and the image quality is deteriorated. However, the photographer sets an upper limit value of sensitivity (limit of image quality deterioration) that can be permitted by the sensitivity upper limit setting unit 237 in advance.

  Next, from the photometric value read in step S501, the camera shake limit shutter speed calculated in step S503, the upper limit sensitivity setting value read in step S504, and the program diagram set in the system controller 230 The shutter speed Tv, the aperture value Av, the sensitivity, and the number of shots for composition are calculated (step S505).

  Comparing the case where camera shake is prevented by combining multiple images and the case where shooting is performed simply by increasing the ISO sensitivity. Is advantageous.

  Therefore, in the case of low brightness, an exposure program that gives priority to a camera shake prevention effect by image composition is given priority over increasing sensitivity. In addition, in the anti-shake effect by combining a plurality of images, it is possible to obtain a 2-stage effect by combining 4 images and a 3-stage effect by combining 8 images. Will increase. Since the continuous shooting speed with a single-lens reflex camera of the popular class is around 3 frames per second, the time required for continuous shooting is about 1.3 seconds until 4 frames are combined, so the usability is not so bad. Therefore, after the shutter speed reaches the camera shake limit second on the low luminance side, image stabilization is first performed by combining images, and images are combined (continuously shot) until the number of continuous shots reaches four. This can be done by increasing the number. On the lower luminance side, this can be dealt with by increasing the sensitivity while fixing the number of images to be combined (continuously shot) to four. When the brightness further decreases, the number of images to be combined (continuous shooting) is allowed up to eight to prevent camera shake by image combining. Further, since it becomes impossible to interlock when the brightness becomes lower, the shutter speed is set to a low speed side to cope with appropriate exposure.

  To obtain a one-step effect from 8 images, 16 continuous shots are required. However, if it increases to this point, it takes more than 5 seconds to shoot only with the current popular single-lens reflex camera (about 3 frames per second). It will be. However, when the upper limit of sensitivity is set low, up to 16 continuous shots may be allowed.

  How to determine the shutter speed, aperture value, sensitivity, and number of shots, the program line when using a lens with a focal length of 135 mm and an open aperture value of F2.0 in a camera using a full-size image sensor. The figure will be specifically described with reference to FIG. As described above, with a full-size image sensor and a focal length of 135 mm, 1/125 sec may be regarded as the lower limit shutter speed at which camera shake does not occur.

  9A to 9C are program diagrams showing an exposure program under the above conditions, and FIG. 9A is a program diagram when the above intention is realized and the upper limit of sensitivity is ISO1600.

  FIG. 9B shows a program diagram when the upper limit of sensitivity is ISO400.

  For further reference, FIG. 9C shows a program diagram in the case where camera shake prevention by image composition is not performed and the upper sensitivity limit is set to ISO1600.

  First, FIG. 9A will be described.

  In FIG. 9A, EV23 to EV9 are sufficiently bright, so the sensitivity remains ISO100, the shutter speed gradually decreases, and EV9 becomes 1/125 seconds. When the subject becomes darker than EV9, the shutter speed becomes slower than 1/125 seconds, and camera shake affects the image quality. Therefore, when the subject brightness is EV8, the shutter speed is 1/125 seconds and two images are shot. Camera shake is suppressed by continuously shooting images and synthesizing these images. In EV7, four images are continuously shot at a shutter speed of 1/125 seconds, and these images are combined. In EV7 to EV3, if the sensitivity is 100, the number of continuous shots is more than four. Therefore, the sensitivity is gradually increased while the number of continuous shots is kept at four. In EV3, the sensitivity is ISO 1600 set as the upper limit. When the subject becomes darker than EV3, the sensitivity has already reached the set upper limit value, so this is dealt with by increasing the number of continuous shots (composite number) again, and EV2 sets the number of continuous shots to eight. . If the brightness of the subject becomes even darker than EV2, the sensitivity and the number of continuous shots have already reached the upper limit values, so this is dealt with by gradually decreasing the shutter speed.

  The above is an example of a program diagram when the sensitivity upper limit is ISO1600.

  FIG. 9B shows an example in which the upper limit of ISO sensitivity is set to 400, and FIG. 9C shows an example in which camera shake correction by image synthesis is not performed. When the upper limit of sensitivity is set to ISO400 (FIG. 9B), compared to the case where ISO1600 is set (FIG. 9A), the upper limit sensitivity is ISO400, so the number of composites is increased to 8 or 16 at EV4 or lower. . When camera shake correction by image composition is not performed (FIG. 9C), the sensitivity is gradually increased to EV1600 between EV9 and EV5, and the shutter speed is gradually decreased from 1/125 seconds in a darker range.

  As described above, the photographer can set the upper limit of sensitivity, and using the corresponding exposure program to perform camera shake correction by image synthesis, the noise is as low as possible and the effects of camera shake are within the allowable range. An image with less can be obtained.

  At present, about 2-8ch A / D converters are used for reading image information with a digital camera. However, a higher number of A / D converters, for example, one A / D converter per line is provided for high-speed reading. Techniques to perform are also proposed. In the future, the upper limit of the number of continuous shots may be increased if image reading is performed at a speed higher than that at present.

  Also, in the above description, the example in which the alignment reference is the image that has been taken first of the two images has been described. However, as long as alignment is possible as a result, where is the alignment reference placed? Also good. For example, on the basis of the first image, the motion vectors of the first image, the second image, the first image, the third image, the first image and the fourth image are obtained, and the second image to the fourth image are converted into the first image. It does not matter if it is processed to overlap.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to said embodiment, A various deformation | transformation is possible.

  In the above description, the example in which the present invention is applied to an interchangeable lens single-lens reflex type imaging apparatus has been described. However, the present invention may be applied to a so-called compact type digital camera in which a lens and a lens barrel are integrated with a main body. Absent. Needless to say, the present invention can also be applied to a video camera having a still image shooting function.

  Further, the operation system may be configured not to be a mode dial but to be able to switch modes by a combination of push switches.

  Further, in the above-described embodiment, the image alignment is described using the parallel movement (motion vector) of the image. However, the image may be combined in consideration of image rotation and deformation such as affine transformation.

  The number of divided pixels of the small block is not limited to the number of pixels described in the present embodiment, and may be a block having an arbitrary size such as 128 pixels × 128 pixels. Also, the motion vector for the entire screen may be determined by selecting, for example, two to three motion vector candidate values in order of frequency and obtaining them by interpolation. Also, other known methods may be used. For example, a motion vector may be obtained by extracting a feature point from the screen and searching for a corresponding point. Further, instead of examining the entire screen, some small blocks may be selected and obtained from motion vectors for reducing the amount of calculation. As described above, various methods are known, and the motion vector may be obtained by any of these methods.

  Further, in the present embodiment, the case of performing alignment and composition processing in the camera has been described. However, only continuous shooting is performed on the camera side, and a plurality of captured images are taken into a PC, a printer, or the like, and are post-processed. You may make it perform combining and a synthetic | combination process. Also, like a silver salt film lab, users bring images stored in a storage medium to the lab, send images to the lab by the communication function such as the Internet, and perform image composition processing on the lab side It doesn't matter in the form.

  The nonvolatile memory 224 is not only a memory card such as a PCMCIA card or a compact flash (registered trademark), a hard disk, but also a micro DAT, a magneto-optical disk, an optical disk such as a CD-R or CR-WR, a DVD, or the like. It may be composed of a changeable optical disk or the like.

  Alternatively, a composite medium in which the nonvolatile memory 224 and a hard disk or the like are integrated may be used. Further, a part of the composite medium may be detachable.

  In the above description of the embodiment, the nonvolatile memory 224 is described as being separated from the imaging device 200 and can be arbitrarily connected, but may be fixed to the imaging device 200.

  The non-volatile memory 224 may be configured to be connected to an arbitrary number of single or plural non-volatile memories 224.

  Image processing of the image processing apparatus that performs image composition may be realized by software or hardware.

  In addition to the above, various modifications may be made without departing from the scope of the present invention.

(Other embodiments)
In addition, an object of each embodiment is to supply a storage medium (or recording medium) on which a program code of software that realizes the functions of the above-described embodiments is recorded to a system or apparatus, and a computer (or CPU) of the system or apparatus Needless to say, this can also be achieved by reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an operating system (OS) running on the computer based on the instruction of the program code. It goes without saying that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function is determined based on the instruction of the program code. It goes without saying that the CPU or the like provided in the expansion card or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is a block diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention. It is a flowchart which shows the main routine of the imaging device concerning one Embodiment of this invention. It is a flowchart which shows the main routine of the imaging device concerning one Embodiment of this invention. It is a flowchart which shows the distance measurement and photometry routine of the imaging device concerning one Embodiment of this invention. 4 is a flowchart illustrating a shooting routine of the imaging apparatus according to the embodiment of the present invention. It is a figure explaining the motion vector detection method in one Embodiment of this invention. 4 is a flowchart illustrating a recording routine of the imaging apparatus according to the embodiment of the present invention. It is a flowchart which shows the photometry calculation routine of the imaging device concerning one Embodiment of this invention. It is a figure explaining the program diagram of the exposure program in one Embodiment of this invention. It is a figure explaining the program diagram of the exposure program in one Embodiment of this invention. It is a figure explaining the reference example of the program diagram of an exposure program.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Shooting lens 101 Focus lens 104 Lens control circuit 107 Electrical contact unit 109 Focal length detection part 200 Imaging device 208 Photometric sensor 209 Photometric circuit 212 Imaging element 227 Camera DSP
230 System Controller 237 Sensitivity Upper Limit Setting Unit

Claims (5)

An imaging device that obtains an image of a target exposure by combining a plurality of underexposed images that are continuously captured,
Imaging means for generating an image signal from the subject image;
Setting means for setting an upper limit value of sensitivity of the imaging means;
On the basis of the upper limit value of the sensitivity set by the setting means, said comprising: a photographing condition of the single image that is continuously captured, and control means for controlling the number of images to be photographed in succession ,
The control means controls to increase the number of images to be continuously taken as the luminance of the subject decreases, and when the target exposure cannot be obtained even when the predetermined number is reached, the control means An image pickup apparatus that controls to increase the sensitivity of each one while maintaining a predetermined number of sheets . When the control means performs control so as to increase the sensitivity of each one while maintaining the predetermined number, when the target exposure cannot be obtained even when the predetermined sensitivity is reached, the control means the imaging apparatus according to claim 1 which exceeds the number defined and controls to increase the number of images to be captured by the succession. A method for controlling an imaging apparatus that includes an imaging unit that generates an image signal from a subject image and obtains a target exposure image by combining a plurality of underexposed images that are continuously captured.
A setting step for setting an upper limit value of sensitivity of the imaging means;
On the basis of the upper limit value of the sensitivity set at the setting step, the comprising: a photographing condition of the single image that is continuously photographed, and a control step of controlling the number of images to be photographed in succession ,
In the control step, control is performed to increase the number of images to be continuously captured as the brightness of the subject decreases, and when the target exposure cannot be obtained even when the predetermined number is reached, A control method for an imaging apparatus, wherein control is performed so as to increase the sensitivity of each sheet while maintaining the predetermined number . In the control step, when controlling to increase the sensitivity of each one while maintaining the predetermined number of sheets, when the target exposure cannot be obtained even when the predetermined sensitivity is reached, 4. The control method for an imaging apparatus according to claim 3, wherein control is performed so that the number of images to be continuously captured exceeds a predetermined number. A program for causing a computer to execute the control method according to claim 3 or 4.

Images