JP5445956B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus such as a so-called digital camera that captures a subject image, and more particularly to an imaging apparatus that effectively reduces the influence of blooming on an imaging element.

An imaging device such as a so-called digital camera that captures a subject image by using an image sensor that converts an optical image into electronic information and obtains digital image data of the subject is widely used. In general, solid-state imaging devices such as CMOS (complementary metal oxide semiconductor) imaging devices and CCD (charge coupled device) imaging devices are used for this type of imaging device, but high sensitivity and high resolution are required. Therefore, for example, various devices have been made as disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2005-130045), Patent Document 2 (Japanese Patent Laid-Open No. 2009-218895), and the like.
That is, Patent Document 1 (Japanese Patent Application Laid-Open No. 2005-130045) discloses a technique for inferring the internal temperature of a solid-state image sensor from the dark current of the solid-state image sensor and appropriately changing image processing according to the temperature of the solid-state image sensor. It is disclosed.
Japanese Patent Application Laid-Open No. 2009-218895 discloses a technique for expanding a dynamic range by combining two or more images having different exposure amounts in an imaging apparatus. .
Furthermore, in recent years, various technologies have been developed for solid-state imaging devices in response to the demand for higher sensitivity. For example, as an example of high sensitivity, there is a backside illumination technique in a CMOS image sensor used as a CMOS image sensor.

That is, FIG. 7 shows a schematic cross-sectional structure of a front-illuminated CMOS image sensor and a back-illuminated CMOS image sensor, (a) is a schematic cross-sectional view of the front-illuminated CMOS image sensor, and (b) ) Is a schematic cross-sectional view of a backside illuminated CMOS image sensor.
In the CMOS image sensor structure based on the conventional surface irradiation technique shown in FIG. 7A, since the wiring layer CL is provided on the photodiode layer PD, the photodiode layer PD is located in the deep layer of the device. This is not only because the incident angle of light is restricted and the light collection efficiency is deteriorated, but also the incident light IL is reflected on the wiring layer CL and enters the adjacent pixels to cause color mixing. .
With respect to these problems, the CMOS image sensor structure based on the backside illumination technique shown in FIG. 7B solves the problem by positioning the photodiode layer PD above the wiring layer CL. On the other hand, the backside illumination type CMOS image sensor has a new problem due to the backside illumination technology.

One of them is blooming. Blooming is a phenomenon in which an adjacent pixel overflows when the charge of one pixel is saturated, and has also occurred in a conventional surface irradiation type CMOS image sensor. In the case of a conventional front-illuminated CMOS image sensor, the occurrence of leakage to adjacent pixels is reduced by discharging saturated charges to a semiconductor substrate (not shown) under the photodiode PD. . However, in the backside-illuminated CMOS image sensor, since the lower layer of the photodiode PD is the wiring layer CL, the phenomenon is worsened because there is no place for overflowing charges and leaks into adjacent pixels.
For example, in high-temperature shooting or long-time shooting, a defective pixel of a CMOS image sensor, which is an image sensor, grows due to a dark current based on heat mainly due to the environmental temperature or the length of a shutter-second. The charge of the pixel overflows to the adjacent pixel, and the charge of the adjacent pixel also overflows to the further adjacent pixel, resulting in a large defect over a plurality of pixels centering on the defective pixel and losing subject information. .
A normal defective pixel can be corrected by a correction process such as replacement with peripheral pixels. However, a large defect over a plurality of pixels can be corrected to some extent, but since subject information is lost over a wide range, an unnatural image is generated in the corrected image.

As described above, in long-time shooting in an imaging apparatus using, for example, a back-illuminated CMOS image sensor or the like as an imaging element, the imaging element generates heat mainly based on the length of the shutter speed, and this is caused by this heat. This phenomenon is further promoted when blooming based on charge accumulation due to dark current is likely to occur and the environmental temperature is high. This phenomenon is mainly caused by a dark current based on the heat of the image sensor, and is considered to strongly depend on the shutter speed.
The present invention has been made in view of the above-described circumstances, and provides an imaging apparatus that limits the occurrence of defects due to blooming due to a rise in the temperature of the imaging element and can perform imaging without generating large noise. It is intended to provide.
According to the first object of the present invention, in particular, it is possible to limit the imaging operation so that noise depending on the temperature of the imaging element is not generated, and to effectively suppress the generation of noise . The second object is to provide an imaging apparatus that can suppress the generation of noise even when long exposure is required and can capture images with appropriate brightness even when there is a sudden temperature fluctuation. is there.
An object of claim 2 of the present invention is to provide an image pickup apparatus capable of suppressing the generation of noise by effectively limiting the generation of defective pixels and blooming.

A third object of the present invention is to provide an imaging apparatus that is particularly sensitive and that can effectively prevent defective pixels and blooming.

In order to achieve the above-described object, an imaging apparatus according to the present invention described in claim 1
An image pickup unit that has an image pickup element that converts an optical image into electronic information, and picks up a subject image using the image pickup element;
Temperature measuring means for measuring the temperature of the image sensor in the imaging means;
A table storage means for storing a shutter time limit table that defines the temperature of the image sensor and the shutter speed limit value corresponding thereto;
At the time of shooting, based on the temperature of the image sensor measured by the temperature measuring unit, the shutter time limit value corresponding to the temperature of the image sensor is obtained by referring to the shutter time limit table of the table storage unit And a second control means for limiting the shutter speed ,
Adding means for adding images sequentially taken by the imaging means;
The second time control means is configured to capture images sequentially captured by the imaging means based on the shutter time limit value when shooting at a second time exceeding a shutter time limit value based on the shutter second time limit table. A means for generating a photographed image by adding by an adding means, and for capturing images sequentially based on a limit value for the shutter speed, temperature information of the image sensor is acquired for each imaging frame, and the corresponding shutter time limit And means for taking an image using a value and using it for addition by the addition means .
An imaging device according to the present invention described in claim 2 is the imaging device according to claim 1,
The shutter time limit table of the table storage means is a table of shutter time limit values based on defective pixels of the image sensor corresponding to the temperature of the image sensor and blooming occurrence data. It is characterized by.

Imaging device according to the present invention described in 請 Motomeko 3 is an imaging apparatus according to claim 1 or claim 2,
The image sensor is
A backside illumination type CMOS image sensor is included.

According to the present invention, it is possible to provide an imaging apparatus that can limit the occurrence of defects due to blooming due to the temperature rise of the imaging element and can perform imaging without generating large noise.
That is, according to the imaging device of claim 1 of the present invention,
An image pickup unit that has an image pickup element that converts an optical image into electronic information, and picks up a subject image using the image pickup element;
Temperature measuring means for measuring the temperature of the image sensor in the imaging means;
A table storage means for storing a shutter time limit table that defines the temperature of the image sensor and the shutter speed limit value corresponding thereto;
At the time of shooting, based on the temperature of the image sensor measured by the temperature measuring unit, the shutter time limit value corresponding to the temperature of the image sensor is obtained by referring to the shutter time limit table of the table storage unit And a second control means for limiting the shutter speed ,
Adding means for adding images sequentially taken by the imaging means;
The second time control means is configured to capture images sequentially captured by the imaging means based on the shutter time limit value when shooting at a second time exceeding a shutter time limit value based on the shutter second time limit table. A means for generating a photographed image by adding by an adding means, and for capturing images sequentially based on a limit value for the shutter speed, temperature information of the image sensor is acquired for each imaging frame, and the corresponding shutter time limit Including means for imaging using the value and subjecting to addition by the addition means ,
In particular, it terminates the imaging operation before generating the noise of an image sensor noise which depends on the temperature of the imaging device is dependent on the temperature so as not to generate, it becomes possible to effectively suppress generation of noise Even when long exposure is required, it is possible to suppress the generation of noise and to capture an image with appropriate brightness even if there is a sudden temperature fluctuation.

According to the imaging device of claim 2 of the present invention, in the imaging device of claim 1,
The shutter time limit table of the table storage means is a table of shutter time limit values based on defective pixels of the image sensor corresponding to the temperature of the image sensor and blooming occurrence data. By
In particular, the occurrence of defective pixels and blooming is effectively limited, and the occurrence of noise can be suppressed, thereby improving the shooting image quality .

According to the imaging device of claim 3 of the present invention, in the imaging device of claim 1 or claim 2 ,
The image sensor is
By including a backside illuminated CMOS image sensor,
In particular, it is possible to effectively prevent the occurrence of defective pixels and blooming with high sensitivity.

1 is a block diagram schematically showing a system configuration of a digital camera as an imaging apparatus according to an embodiment of the present invention. It is the top view which looked at the typical external appearance structure of the digital camera of FIG. 1 from upper direction. It is the front view which looked at the typical external appearance structure of the digital camera of FIG. 1 from the front subject side. It is the rear view which looked at the typical external appearance structure of the digital camera of FIG. 1 from the back photographer side. 3 is a flowchart showing a flow of imaging operation of the digital camera of FIG. 1. FIG. 2 is a diagram illustrating an example of a shutter time limit table used in the digital camera of FIG. 1, where (a) is a chart illustrating the contents of the table, and (b) is a graph illustrating the table values. 1 schematically shows a cross-sectional structure of a CMOS image sensor, where (a) is a schematic cross-sectional view showing a conventional surface-irradiation type CMOS image sensor, and (b) is a back-illumination type that has recently been used. It is typical sectional drawing which shows this CMOS image sensor.

Hereinafter, based on the embodiment concerning the present invention, the imaging device of the present invention is explained in detail with reference to drawings.
In the following embodiment, an embodiment of a digital camera as an imaging device will be described. However, the present invention is not limited to this, and an image related to an electronic device having a camera function or an imaging device is described. It is possible to apply to general image processing such as an image processing IC (integrated circuit) and image processing software.
1 to 4 show a schematic configuration of a digital camera as an imaging apparatus according to an embodiment of the present invention. FIG. 1 is a block diagram schematically showing a system configuration of a digital camera as an imaging apparatus according to an embodiment of the present invention. 2 is a plan view of the schematic external configuration of the digital camera of FIG. 1 viewed from above, FIG. 3 is a front view of the schematic external configuration of the digital camera of FIG. 1 viewed from the front subject side, and FIG. 4 is a rear view of the schematic external configuration of the digital camera of FIG. 1 as viewed from the rear photographer side.

The external appearance of the digital camera shown in FIGS. 1 to 4 is as shown in FIGS. 2 to 4. The sub liquid crystal display (sub LCD) 1, the release button 2, the strobe light emitting unit 3, the mode switching dial 4, the distance measurement Unit 5, remote control light receiving unit (remote control light receiving unit) 6, lens barrel unit 7, autofocus display light emitting diode (AF display LED) 8, strobe display light emitting diode (strobe display LED) 9, liquid crystal display monitor (LCD monitor) 10 The camera body 15 mainly includes an optical viewfinder 11, a zoom button 12, a power switch 13, an operation button group 14, and a memory card slot 121.
Further, the digital camera shown in FIGS. 1 to 4 includes an operation unit 101, a CMOS (complementary metal oxide semiconductor) image sensor 102, an SDRAM (synchronizer) as shown in FIG. 1 mainly showing a system configuration related to electronic control. Eggplant dynamic random access memory) 103, camera processor 104, RAM (random access memory) 107, ROM (read only memory) 108, sub processor 109, sub LCD driver 111, buzzer 113, strobe circuit 114, audio recording unit 115, audio Playback unit 116, LCD monitor driver 117, video amplifier (video amplifier) 118, video jack 119, built-in memory 120, memory card slot 121, USB (Universal Serial Bus) connector 122, serial driver Circuit 123-1, which includes the RS-232C connector 123-2 and the temperature sensor 124,. The lens barrel unit 7 includes a zoom mechanism 7-1, a focus mechanism 7-2, a diaphragm mechanism 7-3, a mechanical shutter mechanism 7-4, and a motor driver 7-5.

The zoom mechanism 7-1 includes a zoom lens system 7-1a and a zoom drive motor 7-1b, and the focus mechanism 7-2 includes a focus lens system 7-2a and a focus drive motor 7-2b. 7-3 includes an aperture unit 7-3a and an aperture motor 7-3b, and the mechanical shutter mechanism 7-4 includes a mechanical shutter unit 7-4a and a shutter motor 7-4b.
The operation unit 101 includes operation input means such as various buttons, keys, switches, dials or levers including a release button 2, a mode switching dial 4, a zoom button 12, a power switch 13, and an operation button group 14.
The CMOS image sensor 102 includes a sensor pixel unit 102-1, a CDS (correlated double sampling) unit 102-2, an AGC (automatic gain control) unit 102-3, an A / D (analog-digital) conversion unit 102-4, and It has a TG (timing generation) section 102-5.
The camera processor 104 includes a first signal processing block 104-1, a second signal processing block 104-2, a CPU (central processing unit) block 104-3, a local SRAM (static random access memory) 104-4, and a USB block. 104-5, serial block 104-6, JPEG codec (CODEC) block 104-7, resizing block 104-8, video signal display block 104-9, memory card controller block 104-10, and pixel addition processing block 104 -11.

The audio recording unit 115 includes a microphone (microphone) 115-1, a microphone amplifier (microphone amplifier) 115-2, and an audio recording circuit 115-3. The audio reproduction unit 116 includes an audio reproduction circuit 116-1, an audio amplifier (audio). Amplifier) 116-2 and speaker 116-3.
As shown in FIGS. 2 to 4, a sub LCD (sub liquid crystal display) 1, a release button 2, and a mode switching dial 4 are arranged on the top surface (FIG. 2) of the digital camera according to this embodiment. Has been. For example, the sub LCD 1 is a display unit for displaying the number of shootable images and the like, and the mode switching dial 4 is an operation unit for switching an operation mode such as a shooting mode / reproduction mode. In addition, a strobe light emitting unit 3, a distance measuring unit 5, a remote control light receiving unit (remote control light receiving unit) 6, and a lens barrel unit 7 are arranged on the front of the body (FIG. 3). The optical viewfinder 11 is provided in the body of the digital camera so that the object plane can be seen from the front of the body and the eyepiece can be seen from the back of the body. The memory card slot 121 is a slot for inserting and connecting the memory card MC, and is provided on the side of the body so as to be covered with a lid that can be opened and closed. Further, on the back of the body, an AF display LED (autofocus display light emitting diode) 8, a strobe display LED (strobe display light emitting diode) 9, an LCD monitor (liquid crystal display monitor) 10, a zoom button 12, and a power switch 13 are provided. And an operation button group 14 are arranged.

Next, the basic control configuration and operation of this digital camera will be described with reference to FIG. 1 and with reference to FIGS.
The strobe light emitting unit 3 and the strobe circuit 114 are used to supplement the amount of light when the amount of natural light or the like for the subject is insufficient. That is, when shooting in a dark place or when the subject is dark, a strobe emission signal is given to the strobe circuit 114 from the camera processor 104, which will be described in detail later, and the strobe circuit 114 causes the strobe light emitting unit 3 to emit light to the subject. Irradiate.
The distance measuring unit 5 is substantially provided for measuring the distance between the digital camera and the subject. In general, in a digital camera, a contrast-AF method is used in which contrast information of an image formed on an image sensor is detected, and a lens is moved to a position with the highest contrast to focus. However, this contrast-AF method has a problem that the focusing operation is slow because the lens is moved little by little to obtain the contrast. Therefore, in the digital camera according to this embodiment, the measurement is not performed. By using the distance unit 5, subject distance information is always acquired, and the lens is moved to the vicinity of the in-focus point based on the subject distance information, thereby speeding up the focusing operation.

The temperature sensor 124 measures the environmental temperature. The temperature sensor 124 measures the temperature inside and outside the body. If the temperature is abnormally high, the camera itself is turned off or the temperature sensor 124 is measured. The control content of the camera is changed with reference to the measurement data. The restriction on the shutter speed corresponding to the temperature according to the present invention is also performed based on the measured value of the temperature sensor 124.
The lens barrel unit 7 includes a zoom mechanism 7-1 including a zoom lens system 7-1a and a zoom drive motor 7-1b, a focus mechanism 7-2 including a focus lens system 7-2a and a focus drive motor 7-2b, An aperture mechanism 7-3 including an aperture unit 7-3a and an aperture motor 7-3b, a mechanical shutter mechanism 7-4 including a mechanical shutter unit 7-4a and a shutter motor 7-4b, and a motor driver 7-5 are provided. An optical image of a subject is formed by the zoom lens system 7-1a and the focus lens system 7-2a, and the zoom drive motor 7-1b and the focus drive motor 7-2b are formed by the motor driver 7-5. The aperture motor 7-3b and the shutter motor 7-4b are driven. The aperture unit 7-3a and the mechanical shutter unit 7-4a may be combined and integrated.

The motor driver 7-5 receives inputs from the remote control light receiving unit 6, as well as various buttons and keys including the release button 2, the mode switching dial 4, the zoom button 12, the power switch 13, and the operation button group 14 of the operation unit 101. Based on an operation input by an operation input means such as a switch, dial or lever, the drive is controlled by a drive command given from a CPU block 104-3 in the camera processor 104 described later.
The ROM 108 stores a control program, control parameters, and the like that are described in codes readable by the CPU block 104-3. For example, when the power of the digital camera is turned on by turning on the power switch 13 of the operation button group 14, the control program is loaded into the SDRAM 103, and the CPU block 104-3 follows the program loaded into the SDRAM 103. While controlling the operation of each part in the camera, data necessary for the control and the like are temporarily stored in the RAM 107 and the local SRAM 104-4 in the camera processor 104. By using a rewritable flash ROM as the ROM 108, it is possible to change the control program and parameters for control, and the function can be easily upgraded.

In this case, a CMOS image sensor 102 configured as a back-illuminated CMOS image sensor is used as a solid-state image sensor for optical / electrical conversion of an optical image. The CMOS image sensor 102 includes a sensor pixel unit 102-1 including a photodiode, a CDS unit 102-2 for performing correlated double sampling (CDS) to remove image noise, and gain by automatic gain control (AGC). An AGC unit 102-3 for adjusting, an A / D conversion unit 102-4 for performing signal conversion (A / D conversion) from an analog signal to a digital signal, and a drive timing signal for the CMOS image sensor 102 are generated. And a TG unit 102-5. The TG unit 102-5 is supplied with a synchronization signal from the first signal processing block 104-1 in the camera processor 104, and is controlled by the CPU block 104-3 to generate a drive timing signal for the CMOS image sensor 102. To do.
The first signal processing block 104-1 of the camera processor 104 performs white balance adjustment and gamma adjustment on the output data of the CMOS image sensor 102, and supplies a synchronization signal to the CMOS image sensor 102 as described above. The second signal processing block 104-2 of the camera processor 104 converts the image data into luminance data / color difference data by filtering processing. The CPU block 104-3 of the camera processor 104 controls the operation of each part of the apparatus as described above.

  The local SRAM 104-4 of the camera processor 104 temporarily stores data necessary for control as described above. The USB block 104-5 of the camera processor 104 performs signal processing for communication with an external device such as a PC (personal computer to so-called “personal computer”) according to the USB standard. The serial block 104-6 of the camera processor 104 performs signal processing for communicating with an external device such as a PC in accordance with a serial communication standard, for example, the RS-232C standard. The JPEG codec block 104-7 of the camera processor 104 performs compression / decompression processing according to the JPEG standard for image data. The resize block 104-8 of the camera processor 104 enlarges / reduces the size of the image data by interpolation / compression processing. A video signal display block 104-9 of the camera processor 104 converts the image data into a video signal to be displayed on an external display device such as the LCD monitor 10, an external display device, or a TV (television) receiver. The memory card controller block 104-10 of the camera processor 104 performs write / read control of the memory card MC that records captured image data. The pixel addition processing block 104-11 of the camera processor 104 performs addition synthesis processing for each pixel when shooting and synthesizing a plurality of frames.

The SDRAM 103 temporarily stores image data before or during processing when the camera processor 104 performs various processing on the image data. The stored image data is acquired by, for example, the CMOS image sensor 102, the RAW-RGB image data in a state where the white balance adjustment and the gamma adjustment are performed in the first signal processing block 104-1, and the second image data. In the signal processing block 104-2, the luminance data / color difference data converted YUV image data, and in the JPEG codec block 104-7, JPEG compressed JPEG image data and the like. The memory card slot 121 is a slot for mounting a removable memory card MC. The built-in memory 120 is a memory that allows captured image data to be retained and recorded even when the memory card MC is not inserted into the memory card slot 121.
The LCD monitor driver 117 is a circuit that drives the LCD monitor 10, and also has a function of converting the video signal output from the video signal display block 104-9 into a signal for display on the LCD monitor 10. The LCD monitor 10 displays the state of the subject before photographing for observing the state of the subject before photographing and determining the framing and release timing, the display for confirming the photographed image, and the memory card MC or the built-in memory 120. Display to refer to the recorded image data.

The video amplifier 118 is an amplifier for converting the video signal output from the video signal display block 104-9 into an impedance of 75Ω, and the video jack 119 is an external display device or TV for external display. This is a video jack as a connector for connecting to an external display device such as a receiver and supplying a video signal from the video amplifier 118.
The USB connector 122 is a connector for making a connection according to the USB standard so as to communicate with an external device such as a PC according to the USB standard. The serial driver circuit 123-1 converts the output signal of the serial block 104-6 into an appropriate voltage according to the RS-232C standard in order to perform serial communication with an external device such as a PC according to the RS-232C standard, for example. The RS-232C connector is a connector for performing serial connection with an external device such as a PC in accordance with the RS-232C standard.
The sub-processor 109 is a CPU (Central Processing Unit) in which ROM (Read Only Memory) and RAM (Random Access Memory) are built on a common one chip, and responds to a detection signal from the operation unit 101 or the remote control light receiving unit 6. The user operation information is supplied to the CPU block 104-3, and based on the camera state information output from the CPU block 104-3, the display control signal corresponding to the state information is sent to the sub LCD 1, the AF display LED 8, Supply to the corresponding ones of the strobe LED 9 and the buzzer 113.

The sub LCD driver 111 drives the sub LCD 1 in response to the output of the sub processor 109 and causes the sub LCD 1 to display, for example, the number of shootable images. The AF display LED 8 lights up in a predetermined color at the time of in-focus, for example, to display the in-focus state during imaging, and the strobe display LED 9 blinks in the middle of charging, for example, and continuously lights up in a fully charged state. The strobe circuit 114 is used to display the charge state of the power supply capacitor for strobe light emission. The AF display LED 8 and the strobe display LED 9 may be used for another display application such as a display during access to a memory card. The remote control light receiving unit 6 receives a remote control signal such as infrared rays or radio waves from a remote control transmitter (remote controller) operated by the user.
In the audio recording unit 115, the audio signal input to the microphone 115-3 is amplified by the microphone amplifier 115-2, and the amplified audio signal is recorded as audio information by the audio recording circuit 115-3. In the audio reproduction unit 116, the recorded audio information is converted into an audio signal for driving the speaker 116-3 by the audio reproduction circuit 116-1, and the converted audio signal is amplified by the audio amplifier 116-2. Then, the speaker 116-3 is driven to reproduce sound.
The features of the present invention in the digital camera configured as described above will be specifically described.
FIG. 5 is a flowchart showing the flow of the imaging process related to the operation characteristic of the present invention in the digital camera described above. The processing according to the flowchart of FIG. 5 is mainly executed by the camera processor 104. Hereinafter, the operation of the digital camera according to the present invention will be described with reference to the flowchart of FIG.

[Acquire temperature of image sensor]
When the release button 2 is pressed by the user and the release operation is started (step S11), immediately after the temperature information of the CMOS image sensor 102 as the image sensor is acquired from the temperature sensor 124 (step S12). Here, the temperature sensor 124 is assumed to be installed as close to the image sensor (CMOS image sensor 102) as possible in order to accurately measure the temperature of the image sensor (CMOS image sensor 102). The reason why the temperature information of the image sensor (CMOS image sensor 102) is acquired is that the defective pixel of the image sensor such as the backside-illuminated CMOS image sensor 102 is the temperature of the image sensor (machine by the mechanical shutter mechanism 7-4). This is because the exposure time is not limited by an electrical shutter time limit (to be described later), and the temperature information of the image sensor is required here. It becomes. In this case, the acquisition of the temperature information has been described as after the release operation, but the present invention is not limited to this. For example, during the observation of a subject using a so-called through image as an electronic viewfinder, the temperature information is continuously acquired. Temperature information may be acquired from the temperature sensor 124, and the temperature information acquired last after the release operation may be used.

[Refer to the shutter speed limit table]
After acquiring the temperature information in step S12, the shutter time limit is set with reference to the shutter time limit table (step S13). The shutter time limit table is a table that stores the temperature of the image sensor (CMOS image sensor 102) and the shutter time limit corresponding thereto, that is, the limit value of the electrical shutter time, for example, FIG. ). FIG. 6B is a graph of the table of FIG. FIG. 6 shows the shutter time limit corresponding to the temperature of the image sensor (CMOS image sensor 102) every 5 ° C., and when temperature information of the image sensor having no temperature corresponding to the table is acquired (that is, When temperature information intermediate between the temperatures stored in the table is acquired), an approximate calculation is performed from a section including the temperature. That is, T is the temperature at which the shutter time limit is to be calculated, T1 and T2 are the temperatures at both ends of the section including the temperature T, and t1 and t2 are the shutter time limits corresponding to them. If the second time limit is t, the approximate expression is as follows.

t = ((t2-t1) T + (t1T2-t2T1)) / (T2-T1) (1)
For example, when the detected temperature, that is, the acquired temperature information is 32 ° C., referring to the shutter time limit table of FIG. 6, the shutter time limit of 30 ° C. is 0.96 sec, 35 ° Since the shutter speed limit of C is 0.85 sec, t = ((0.85−0.96) × 32 + (0.96 × 35−0.85 × 30) in the approximate calculation by the equation (1). ) / (35-30) ≈0.92 sec.
As another method for obtaining the shutter time limit corresponding to the temperature information, the temperature information and the number of data corresponding to the shutter time limit are matched with the resolution of the temperature sensor 124 without using the approximate expression (1). May be increased.

[How to create a shutter speed limit table]
Here, an example of a specific method for creating the shutter time limit table will be described.
First, the ambient temperature including the digital camera is changed using a thermostatic chamber or the like, the temperature of the image sensor is sequentially changed, and light is shielded while the electrical shutter time is sequentially changed for each temperature state (for example, mechanical Take an image with the shutter closed. That is, obtaining a light-shielded image for each shutter speed at a certain temperature is repeated for each temperature to obtain a large number of light-shielded images. Then, the defective pixel and the occurrence level of blooming are counted from the obtained light-shielded image. The defective pixel and blooming occurrence level are counted as follows.
That is, first, the brightness value of the entire image of the shaded image is acquired. As a method for acquiring the brightness value, a histogram is created from the brightness values of all the pixels of the image, and the peak value of the histogram is used as the brightness value of the entire image. Alternatively, instead of this method, in order to eliminate the influence of defective pixels and blooming, an average of luminance values of the entire image calculated without adding a predetermined signal level or higher may be used.

In determining the occurrence of defective pixels, a threshold value for defective pixel determination is set in advance, the luminance values of the target pixel and all pixels of the image are compared, and the target pixel is defective with respect to the luminance values of all pixels of the image. If the difference is greater than or equal to the pixel determination threshold, it is determined as a defective pixel. The same determination is performed on the entire image to count the number of defective pixels.
In determining the occurrence of blooming, a threshold for determining blooming is set in advance, and the average of the target pixel portion (for example, 3 × 3 pixels) is compared with the luminance value of all the pixels of the image. If the luminance value of all pixels in the image shows a difference greater than or equal to the blooming determination threshold value, it is determined that blooming has occurred. The same determination is performed on the entire image, and the number of blooming is counted.
The above-mentioned defective pixel and blooming threshold are set to a level that can be corrected by image processing, and the pixel range of the central pixel portion used for blooming determination is set to an area that is one size larger than the pixel range that can be corrected by image processing. To do. For example, if the pixel range that can be corrected by image processing is 2 × 2 pixels, blooming determination is performed with 3 × 3 pixels. This is because if it is determined in the correctable pixel range, it is not known even if blooming larger than the correctable pixel range occurs.
Defective pixels for each temperature and shutter time with 0 blooming count number are defective pixels and blooming at a level that can be corrected by image processing, so the longest shutter time with count number 0 is the shutter time limit And set as a table of shutter speed limit for temperature.

[Shooting and saving]
In step S13, the shutter time limit corresponding to the temperature information of the image sensor is acquired with reference to the shutter time limit table created as described above. This shutter time limit is the second that is the longest exposure time when shooting, and in shooting within the shutter time limit, shooting is performed with a shooting time set exactly the same as normal shooting (step S14). First image processing (image processing 1) by the first signal processing block 104-1 (step S15), and second image processing (image processing 2) by the second signal processing block 104-2 (step S20). Are sequentially performed and the image is stored (at this time, it passes through step S16 and step S17, and it is determined in step S18 that it is within the shutter time limit, and the process reaches step S20).

[When exposure is longer than the shutter speed limit]
On the other hand, when exposure longer than the shutter time limit is required, for example, at a fixed time, if the shutter time limit is activated as it is, shooting is performed in a time shorter than the intended time. It becomes a dark image. In order to prevent such a situation, it is determined whether or not the shooting is longer than the shutter time limit (step S18). In shooting at a time longer than the shutter time limit, shooting is performed continuously after the first image is shot (multiple shots). Frame shooting).

[From acquisition of temperature information in multiple frames to shutter limit setting]
Two methods are conceivable for setting the shutter speed limit when photographing multiple frames. In the first method, as shown in FIG. 5, the temperature information of the image sensor is acquired for each shooting (step S12), and the shutter time limit table is referenced from the acquired temperature information each time by referring to the shutter time limit table. This is a method of setting a limit (step S13). Photographing is performed according to the shutter time set in step S13 (step S14), and the first image processing (image processing 1) by the first signal processing block 104-1 is performed (step S15). Then, it is determined whether or not it is the first frame (step S16). If it is the first frame, it is stored in the SDRAM 103 as it is (step S17), and it is determined whether or not the set shooting time is sufficient (step S18). ). If it is determined in step S18 that the shooting time is not sufficient, the process returns to step S12 to acquire the temperature information of the image sensor, and in step S13, the shutter time limit table is referred to by referring to the shutter time limit table. Set. In step S14, the remaining amount of the set shooting time is compared with the shutter time limit, and shooting is performed in the smaller shooting time of the two. After performing the first image processing (image processing 1) by the first signal processing block 104-1 in step S15, it is determined in step S16 whether or not it is the first frame, and it is not the first frame. The added image and the image previously stored in the SDRAM 103 are added (step S19), and stored in the SDRAM 103 in step S17.

If it is determined in step S18 that the shooting time is sufficient, the second image processing (image processing 2) by the second signal processing block 104-2 is performed (step S20), and the processing is performed. finish.
In this method, the shutter time limit may fluctuate every time an image is repeatedly shot. The advantage is that the shutter speed is appropriately adjusted when the image sensor temperature changes with the shooting time in long-time shooting. That is, the shooting time can be limited by the second time limit.
The second method is a method for setting the shutter speed limit using only the temperature information of the image sensor acquired at the time of shooting the first shooting frame. In this method, all frames are shot with the same shutter speed limit, and the advantage is that the process of acquiring temperature information, referring to the shutter time limit table, and setting the shutter time limit is omitted. The processing time can be shortened.
Of the two methods described above, the first method is effective for photographing with a relatively long exposure time in which the change in the image sensor temperature is likely to occur, and the second method is a comparison in which the change in the image sensor temperature is small. This is effective for shooting with a short exposure time.

  As described above, the shutter-second limit is set for the second and subsequent frames and shooting is performed. When shooting a plurality of frames, in step S15, the first signal processing block 104-1 performs dark signal noise removal and defective pixel correction processing for each frame until the state of the RAW-RGB image data. In step S19, the image is added to the previous image frame in step S19, and the added image is stored in the SDRAM 103 in step S17. After all the frames have been added, normal image processing (image processing 2) after the second signal processing block 104-2, JPEG compression, and the like are performed in step S20 and recorded in the memory card MC. In this way, the brightness of the image can be set to an appropriate level by adding the images of a plurality of frames. Also, in practice, the conditions for multiple frame shooting are limited to long-time shooting, so there is no effect on normal shooting, and a good image without defective pixels and blooming does not occur in long-time shooting image quality. Can be obtained.

The imaging device according to the present invention described above has the following characteristics.
The temperature of the image sensor is acquired by the temperature measurement means, the shutter time limit for the image sensor temperature is set with reference to the shutter time limit table, and the image sensor noise depends on the temperature by limiting the shooting time. Shooting can be terminated before the occurrence of.
By creating the shutter time limit table from the defective pixel and blooming occurrence data, it is possible to set a shutter time limit at which the defective pixel and blooming do not occur, and the shooting image quality is improved.
In shooting exceeding the shutter speed limit, one frame image shot with the shutter speed limit is a dark image, but images are repeatedly captured continuously within the shutter speed limit range, and an image is created by addition processing. As a result, an image with appropriate brightness can be obtained.
Appropriate shutter even when temperature changes occur in the image sensor during continuous shooting by acquiring temperature information of the image sensor for each shooting frame and shooting by setting the shutter time limit for each shooting frame when shooting multiple frames A second time limit can be set.
When shooting multiple frames, acquire the temperature information of the image sensor at the first shooting frame, and apply the same limits on all frames, so that the process of temperature information acquisition, shutter time limit table reference and shutter time limit setting This can be omitted, and the processing time can be shortened.
In the above description, the case where a back-illuminated CMOS image sensor is used as an image sensor has been described. However, other image sensors are mainly caused by dark current due to defective pixels or blooming depending on the shutter speed. When noise occurs, it can be carried out in the same manner as described above.

1 Sub liquid crystal display (sub LCD)
2 Release button 3 Strobe light emitting unit 4 Mode switching dial 5 Ranging unit 6 Remote control light receiving unit 7 Lens barrel unit 7-1 Zoom mechanism 7-1a Zoom lens system 7-1b Zoom drive motor 7-2 Focus mechanism 7-2a Focus lens System 7-2b Focus drive motor 7-3 Aperture mechanism 7-3a Aperture unit 7-3b Aperture motor 7-4 Mechanical shutter mechanism 7-4a Mechanical shutter unit 7-4b Shutter motor 7-5 Motor driver 8 Autofocus display light emitting diode (AF display LED)
9 Strobe display light emitting diode (strobe display LED)
10 Liquid crystal display monitor (LCD monitor)
DESCRIPTION OF SYMBOLS 11 Optical finder 12 Zoom button 13 Power switch 14 Operation button group 15 Camera body 121 Memory card slot 101 Operation part 102 CMOS (complementary metal oxide semiconductor) image sensor 102-1 Sensor pixel part 102-2 CDS (correlation double sampling) ) Unit 102-3 AGC (automatic gain control) unit 102-4 A / D (analog-digital) conversion unit 102-5 TG (timing generation) unit 103 SDRAM (synchronous dynamic random access memory)
104 Camera Processor 104-1 First Signal Processing Block 104-2 Second Signal Processing Block 104-3 CPU (Central Processing Unit) Block 104-4 Local SRAM (Static Random Access Memory)
104-5 USB block 104-6 Serial block 104-7 JPEG codec (CODEC) block 104-8 Resize block 104-9 Video signal display block 104-10 Memory card controller block 104-11 Pixel addition processing block 107 RAM (Random access memory)
108 ROM (Read Only Memory)
109 Sub Processor 111 Sub LCD Driver 113 Buzzer 114 Strobe Circuit 115 Audio Recording Unit 115-1 Microphone (Microphone)
115-2 Microphone Amplifier (Microphone Amplifier)
115-3 Audio Recording Circuit 116 Audio Reproduction Unit 116-1 Audio Reproduction Circuit 116-2 Audio Amplifier (Audio Amplifier)
116-3 Speaker 117 LCD Monitor Driver 118 Video Amplifier (Video Amplifier)
119 Video Jack 120 Internal Memory 121 Memory Card Slot 122 USB (Universal Serial Bus) Connector 123-1 Serial Driver Circuit 123-2 RS-232C Connector 124 Temperature Sensor 124

JP 2005-130045 A JP 2009-218895 A

Claims (3)

An image pickup unit that has an image pickup element that converts an optical image into electronic information, and picks up a subject image using the image pickup element;
Temperature measuring means for measuring the temperature of the image sensor in the imaging means;
A table storage means for storing a shutter time limit table that defines the temperature of the image sensor and the shutter speed limit value corresponding thereto;
At the time of shooting, based on the temperature of the image sensor measured by the temperature measuring unit, the shutter time limit value corresponding to the temperature of the image sensor is obtained by referring to the shutter time limit table of the table storage unit And a second control means for limiting the shutter speed ,
Adding means for adding images sequentially taken by the imaging means;
The second time control means is configured to capture images sequentially captured by the imaging means based on the shutter time limit value when shooting at a second time exceeding a shutter time limit value based on the shutter second time limit table. A means for generating a photographed image by adding by an adding means, and for capturing images sequentially based on a limit value for the shutter speed, temperature information of the image sensor is acquired for each imaging frame, and the corresponding shutter time limit Means for picking up an image using a value and using it for addition by the adding means .   The shutter time limit table of the table storage means is a table of shutter time limit values based on defective pixels of the image sensor corresponding to the temperature of the image sensor and blooming occurrence data. The imaging apparatus according to claim 1. The image sensor is
The imaging apparatus according to claim 1 or claim 2, characterized in that it comprises a back-illuminated CMOS image sensor.

Images