JP2014068253A - Imaging device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately control processing for reducing a noise component for image data also in consideration of the intensity of subject light.SOLUTION: An imaging device comprises: a light measuring unit that measures subject light; a noise reduction processing unit that performs noise reduction processing for reducing a noise component due to noise generated in an image sensor on image data obtained by exposing the image sensor; and a control unit that controls the strength of the noise reduction processing on the basis of an exposure time in which the image sensor is exposed and the intensity of the subject light measured by the light measuring unit.

Description

  The present invention relates to an imaging apparatus and a program.

An imaging apparatus that removes fixed pattern noise from an image signal generated by an imaging body such as an imaging element is known (see, for example, Patent Document 1).
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2008-048277

  Although the noise intensity increases by taking an image with a long exposure time, the intensity of the image component based on the subject light flux with respect to the noise intensity increases as the intensity of the subject light increases. For this reason, as the intensity of the subject light increases, the necessity of performing the process of reducing the noise component decreases. Conventionally, there has been a problem that processing for reducing noise components cannot be appropriately controlled in consideration of the intensity of subject light.

  In the first aspect of the present invention, the imaging device includes a photometric unit that measures subject light, and noise components caused by noise generated by the imaging device with respect to image data obtained by exposure with the imaging device. A noise reduction processing unit that performs noise reduction processing to reduce noise, and a control unit that controls the intensity of the noise reduction processing based on the exposure time exposed by the image sensor and the intensity of subject light measured by the photometry unit .

  In the second aspect of the present invention, the program measures a noise component caused by noise generated in the image sensor with respect to the photometric step for metering the subject light and image data obtained by exposure with the image sensor. The computer executes a step of performing a noise reduction process to be reduced and a control step of controlling the intensity of the noise reduction process based on the exposure time exposed by the image sensor and the intensity of the subject light measured in the photometry step.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows an example of a system configuration of a camera 10. An example of the outline | summary of operation | movement in the camera 10 is shown typically. An example of the amount of charge accumulated in the image sensor 132 according to the exposure time is schematically shown. Another example of the charge amount accumulated in the image sensor 132 according to the exposure time is schematically shown. Another example of the charge amount accumulated in the image sensor 132 according to the exposure time is schematically shown. A processing flow from the start to the end of the camera 10 is shown. An example of the processing flow in the case of performing an imaging operation is shown.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of the system configuration of the camera 10. The camera 10 is an interchangeable lens camera as an example. This figure shows a block configuration of the camera 10 with the lens unit 120 mounted.

  The lens unit 120 includes a lens mount having a lens mount contact 121. The camera body 130 includes a camera mount having a camera mount contact 131. When the lens mount and the camera mount are engaged and the lens unit 120 and the camera body 130 are integrated, the lens mount contact 121 and the camera mount contact 131 are connected. The lens MPU 123 is connected to the camera MPU 140 via the lens mount contact 121 and the camera mount contact 131, and controls the lens unit 120 in cooperation with each other while communicating with each other.

  The lens unit 120 includes a lens group 122, a lens driving unit 124, and a lens MPU 123. The subject light passes through the lens group 122 as an optical system of the lens unit 120 along the optical axis and enters the camera body 130. The main mirror 145 can take an oblique state where the main mirror 145 is obliquely provided in the subject light flux centered on the optical axis of the lens group 122 and a retreat state where the main mirror 145 is retracted from the subject light flux.

  When the main mirror 145 is in an oblique state, the main mirror 145 reflects a part of the subject light flux that has passed through the lens group 122. Specifically, the region near the optical axis of the main mirror 145 in the oblique state is formed as a half mirror. Part of the subject luminous flux incident on the region near the optical axis is transmitted and the other part is reflected. The subject luminous flux reflected by the main mirror 145 is guided to the finder unit 147 and observed by the user. The user can check the composition and the like through the finder unit 147. The finder unit 147 may include a display device that presents various types of information including information indicating the setting state of the imaging operation to the user, along with the subject image based on the subject luminous flux.

  Part of the subject luminous flux that has passed through the region near the optical axis of the main mirror 145 is reflected by the sub mirror 146 and guided to the AF unit 142. The AF unit 142 includes a plurality of photoelectric conversion element arrays that receive a subject light beam. The photoelectric conversion element array outputs a signal with a phase match when in a focused state, and outputs a signal with a phase shift when in a front pin state or a rear pin state. The amount of phase shift corresponds to the amount of shift from the focus state. The AF unit 142 detects a phase difference by performing a correlation operation on the output of the photoelectric conversion element array, and outputs a phase difference signal indicating the phase difference to the camera MPU 140.

  The focus state of the lens group 122 is adjusted using the phase difference signal from the AF unit 142 under the control of the camera MPU 140 and the like. For example, the target position of the focus lens included in the lens group 122 is determined by the camera MPU 140 based on the focus state detected from the phase difference signal, and the position of the focus lens is controlled by the control of the lens MPU 123 toward the determined target position. Is done. Specifically, the lens MPU 123 controls the lens driving unit 124 including a focus lens motor as an example, and moves the focus lens constituting the lens group 122. As described above, when the main mirror 145 is down and in the inclined state, the focus state of the lens group 122 is detected by the phase difference detection method, and the focus adjustment is performed. The AF unit 142 is provided with photoelectric conversion element arrays at a plurality of positions respectively corresponding to the plurality of regions in order to adjust the focus state in each of the plurality of regions in the subject image.

  The photometric element 144 is an example of a photometric unit that measures subject light. The photometric element 144 has a photoelectric conversion element that receives a part of the light beam guided to the finder unit 147. The luminance information of the subject detected by the photoelectric conversion element included in the photometric element 144 is output to the camera MPU 140 as a photometric value. The camera MPU 140 controls each unit based on the luminance information acquired from the photometric element 144. For example, the camera MPU 140 calculates an AE evaluation value based on the luminance information, and performs exposure control based on the AE evaluation value. The camera MPU 140 functions as an exposure time determination unit that determines the exposure time based on the intensity of subject light.

  When the main mirror 145 retracts from the subject light beam, the sub mirror 146 retracts from the subject light beam in conjunction with the main mirror 145. Is provided on the lens group 122 side of the image sensor 132. The focal plane shutter 143 is a mechanical shutter as an example. When the main mirror 145 is in the retracted state and the focal plane shutter 143 is in the open state, the subject light flux that has passed through the lens group 122 is incident on the light receiving surface of the image sensor 132. The focal plane shutter 143 controls exposure by opening and closing an optical path of subject light incident on the image sensor 132.

  The imaging element 132 functions as an imaging unit. The image sensor 132 captures an image of the subject using the subject light flux that has passed through the lens group 122. The image sensor 132 includes a solid-state image sensor such as a CMOS sensor or a CCD sensor. The imaging element 132 has a plurality of photoelectric conversion elements that receive the subject light flux, and outputs an analog signal corresponding to the amount of accumulated charge generated by each of the plurality of photoelectric conversion elements to the analog processing unit 133. The analog processing unit 133 performs analog processing such as amplification processing and OB clamping processing on the analog signal output from the image sensor 132 and outputs the analog signal to the A / D converter 134. The A / D converter 134 converts the analog signal output from the analog processing unit 133 into a digital signal representing image data and outputs the digital signal. The image sensor 132, the analog processing unit 133, and the A / D converter 134 are driven by a driving unit 148 that receives an instruction from the camera MPU 140.

  The digital signal output as a digital signal from the A / D converter 134 is input to the ASIC 135 as image data. The ASIC 135 is an integrated circuit in which circuits related to the image processing function are integrated into one. The ASIC 135 uses at least a part of the memory area of the RAM 136 as an example of a volatile memory as a buffer area for temporarily storing image data, and performs various image processing on the image data stored in the RAM 136. Apply. Examples of the image processing by the ASIC 135 include noise reduction processing, defective pixel correction, white balance correction, color interpolation processing, color correction, gamma correction, contour enhancement processing, image data compression processing, and the like. The ASIC 135 functions as a noise reduction processing unit that performs noise reduction processing for reducing noise components caused by noise generated in the image sensor 132 on image data obtained by exposure with the image sensor 132. Examples of the noise generated in the image sensor 132 include noise caused by dark current generated in the image sensor 132 and noise generated when the accumulated charge of each photoelectric conversion element of the image sensor 132 is converted into a voltage. . Noise that occurs when accumulated charge is converted to voltage may be referred to as high sensitivity noise. When the image sensor 132 continuously captures images, sequentially output image data is sequentially stored in the buffer area. A plurality of image data obtained by continuously capturing images by the image sensor 132 is sequentially stored in the buffer area as image data of continuous still images or image data of each image constituting the moving image. The RAM 136 also functions as a frame memory that temporarily stores frames when the ASIC 135 processes moving image data.

  Examples of image processing in the ASIC 135 include processing for generating image data for recording, processing for generating image data for display, and image data processing for automatic focus adjustment (AF). Further, the image processing in the ASIC 135 includes processing for detecting the contrast amount for AF processing and the like. Specifically, the ASIC 135 detects the contrast amount from the image data and supplies it to the camera MPU 140. For example, the ASIC 135 detects the contrast amount from each of a plurality of image data obtained by imaging with the focus lens positioned at different positions in the optical axis direction. The camera MPU 140 adjusts the focus state of the lens group 122 based on the detected contrast amount and the position of the focus lens. For example, the camera MPU 140 determines the target position of the focus lens so as to increase the amount of contrast, and causes the lens MPU 123 to control the position of the focus lens toward the determined target position. As described above, when the main mirror 145 is up and in the retracted state, the focus state of the lens group 122 is detected by the contrast detection method, and the focus adjustment is performed. As described above, the camera MPU 140 performs the focus adjustment of the lens group 122 in cooperation with the ASIC 135 and the lens MPU 123.

  When recording the image data output from the A / D converter 134, the ASIC 135 converts the image data into a standardized image format. For example, the ASIC 135 performs a compression process for generating still image data obtained by encoding still image data in an encoding format compliant with a standard such as JPEG. Further, the ASIC 135 converts a plurality of frames into QuickTime, H.264, and the like. H.264, MPEG2, Motion JPEG, etc. A compression process for generating moving image data encoded by an encoding method compliant with a standard such as JPEG is performed. The ASIC 135 outputs and records the generated image data such as still image data and moving image data to an external memory 180 as an example of a nonvolatile recording medium. For example, the ASIC 135 records in the external memory 180 as a still image file and a moving image file. An example of the external memory 180 is a semiconductor memory such as a flash memory. Specifically, as the external memory 180, various memory cards such as an SD memory card, a CF storage card, and an XQD memory card can be exemplified. The image data stored in the RAM 136 is transferred to the external memory 180 through the recording medium IF 150. The image data recorded in the external memory 180 is transferred to the RAM 136 through the recording medium IF 150 and stored in the RAM 136. Examples of the recording medium IF 150 include a card controller that controls access to the above-described memory card.

  The ASIC 135 generates display image data in parallel with the generation of recording image data. For example, the ASIC 135 generates display image data to be displayed on the display unit 138 during a so-called live view operation. Further, at the time of image reproduction, the ASIC 135 generates image data for display from the image data read from the external memory 180. The generated image data for display is converted into an analog signal under the control of the display control unit 137 and displayed on the display unit 138 such as a liquid crystal display. In addition to the image display based on the image data obtained by imaging, various menu items relating to various settings of the camera 10 are also displayed under the control of the ASIC 135 and the display control unit 137 without displaying the image based on the image data. 138.

  The external device IF 152 is responsible for communication with the external device. The image data recorded in the external memory 180 is transferred to the external device through the external device IF 152. In addition, image data acquired by communication from an external device through the external device IF 152 is recorded in the external memory 180. The external device IF 152 may communicate with the external device by USB communication.

  The operation input unit 141 receives an operation from the user. The operation input unit 141 includes various operation members such as a release button, an imaging mode dial, a playback button, a live view switch, a moving image button, and a power switch. Further, the operation input unit 141 may include an input member that is integrated with the display unit 138 as a touch panel or the like. The camera MPU 140 detects that the operation input unit 141 has been operated, and executes an operation corresponding to the operation. For example, the camera MPU 140 controls each unit of the camera 10 so as to execute an imaging operation when a release button is pressed. In addition, the camera MPU 140 controls each unit of the camera 10 so as to perform an operation according to the menu item displayed on the display unit 138 and the operation content when an input member mounted as a touch panel is operated.

  The camera MPU 140 controls the intensity of the noise reduction processing in the ASIC 135 based on the exposure time exposed by the image sensor 132 and the intensity of the subject light measured by the photometry element 144. For example, the camera MPU 140 decreases the intensity of the noise reduction process as the subject light intensity increases. More specifically, the camera MPU 140 decreases the intensity of the noise reduction process as the ratio of the intensity of the subject light to the exposure time increases. Note that the image data to be subjected to noise reduction processing is image data obtained by performing exposure with the image sensor 132 during the exposure time determined by the camera MPU 140. The camera MPU 140 uses the exposure time and subject determined by the camera MPU 140. The intensity of the noise reduction process is controlled based on the light intensity.

  The temperature sensor 154 is an example of a temperature acquisition unit that acquires the temperature of the image sensor 132. The temperature sensor 154 measures the temperature of the image sensor 132 and outputs a temperature measurement value indicating the temperature to the camera MPU 140. The camera MPU 140 controls the noise reduction process in the ASIC 135 based on the temperature measurement value and the motion detection value. For example, the camera MPU 140 controls the intensity of the noise reduction process based on the exposure time, the intensity of the subject light measured by the photometric element 144, and the temperature measurement value. For example, the intensity of the noise reduction process is increased as the camera MPU 140 and the temperature measurement value are increased.

  The motion sensor 156 is an example of a detection unit that detects a time change of the subject. The motion sensor 156 detects the temporal movement of the subject as an example of the temporal change of the subject. The motion sensor 156 detects the motion of the camera 10 and outputs a motion detection value indicating the motion to the camera MPU 140. The motion sensor 156 detects the movement of the camera 10 as a relative movement of the image sensor 132 with respect to the subject. Examples of the motion sensor 156 include an acceleration sensor and a gyro sensor. The camera MPU 140 determines that noise reduction processing is not performed on the image data when the temporal motion amount of the subject detected by the motion sensor 156 is greater than a predetermined value. For example, when the time change amount of the subject detected by the motion sensor 156 is greater than a predetermined value, the camera MPU 140 performs noise reduction processing based on the exposure time and the subject light intensity on the image data. Judge that there is no.

  The camera 10 is controlled directly or indirectly by the camera MPU 140 and the ASIC 135 including the control described above. Constants, parameters such as variables, programs, and the like necessary for the operation of the camera 10 are stored in the system memory 139. The system memory 139 is an electrically erasable / storable nonvolatile memory, and is configured by, for example, a flash ROM, an EEPROM, or the like. The system memory 139 stores parameters, programs, and the like so that they are not lost even when the camera 10 is not operating. Parameters, programs, and the like stored in the system memory 139 are expanded in the RAM 136 and used for controlling the camera 10. The ASIC 135, the RAM 136, the system memory 139, the display control unit 137, the camera MPU 140, and the external device IF 152 in the camera body 130 are connected to each other via a connection interface 149 such as a bus and exchange various data.

  Each part of the camera body 130, each part of the lens unit 120, and the external memory 180 are supplied with power from the power supply 190 via the power supply circuit 192. Examples of the power source 190 include a secondary battery such as a lithium ion battery that can be attached to and detached from the camera body 130, a system power source, and the like. The secondary battery is an example of a battery, and the battery includes a non-rechargeable battery that cannot be substantially charged. The camera MPU 140 controls power supply from the power supply 190 to each unit of the camera 10 by controlling the power supply circuit 192.

  FIG. 2 schematically shows an example of the outline of the operation in the camera 10. In this figure, a case where three shootings are performed will be described.

  In the first shooting, when the camera MPU 140 detects pressing of the release button (t1), the camera MPU 140 performs photometry on the photometry element 144 (t2) and acquires a photometric value. The camera MPU 140 determines at least the exposure time based on the acquired photometric value. Here, it is assumed that the exposure time of time length T is determined. The camera MPU 140 controls the drive unit 148 to start the exposure with the subject light beam by the image sensor 132 from t3, and ends the exposure when the exposure time T has elapsed. Exposure by the image sensor 132 with the subject light beam is called main exposure. When the main exposure is completed, reading from the image sensor 132 is performed and the image data 201 is output to the ASIC 135. The image data 201 is stored in the RAM 136.

  The camera MPU 140 determines whether dark exposure needs to be performed for noise reduction processing on the image data 201. It should be noted that blocking the subject light and accumulating charges in the image sensor 132 is called dark exposure. When making this determination, the camera MPU 140 calculates an exposure amount indicating the amount of charge generated in the image sensor 132 due to subject light. Specifically, the camera MPU 140 calculates an exposure amount indicating the amount of charge generated in the image sensor 132 based on the photometric value and the exposure time T. The camera MPU 140 calculates the amount of noise generated in the image sensor 132 due to dark current. Specifically, the camera MPU 140 calculates the amount of noise generated in the image sensor 132 due to dark current based on the exposure time T. Then, in order to determine whether it is necessary to perform dark exposure, the camera MPU 140 compares the ratio of the exposure amount with respect to the noise amount with a predetermined reference value. Here, it is assumed that the ratio of the exposure amount to the noise amount is smaller than a predetermined reference value. Further, the camera MPU 140 compares the motion detection value detected by the motion sensor 156 with a predetermined motion reference value. Here, it is assumed that the motion detection value is smaller than a predetermined motion reference value. In this case, the camera MPU 140 determines to perform dark exposure.

  Following the main exposure, the camera MPU 140 closes the focal plane shutter 143 to block the subject light incident on the image sensor 132, and starts to accumulate charges in the image sensor 132 from t4, and the time T has elapsed. At the time, the exposure is terminated. The focal plane shutter 143 is exposed while the focal plane shutter 143 is closed for the same time as the exposure time T exposed by the image sensor 132 when generating the image data 201.

  The ASIC 135 stores the image data 202 output from the A / D converter 134 by dark exposure in the RAM 136. The ASIC 135 performs a difference calculation between the image data 201 and the image data 201. Specifically, the ASIC 135 performs processing for subtracting the image data 202 from the image data 201 for each pixel. The ASIC 135 performs image processing on the image data 203 obtained by the difference and records the image data 203 in the external memory 180.

  Next, in the second shooting, when the camera MPU 140 detects pressing of the release button (t5), the photometry element 144 performs photometry (t6), and acquires a photometric value. The camera MPU 140 determines at least the exposure time based on the acquired photometric value. Here, it is assumed that the exposure time having the same length as the exposure time T at the time of shooting when the image data 201 is generated is determined. The camera MPU 140 controls the driving unit 148 to start the exposure with the subject light flux by the image sensor 132 from t5, and ends the exposure when the exposure time T has elapsed. When the main exposure is completed, reading from the image sensor 132 is performed and output to the ASIC 135 as image data 211. The image data 211 is stored in the RAM 136.

  The camera MPU 140 determines whether it is necessary to perform dark exposure for noise reduction processing on the image data 211. The camera MPU 140 determines whether or not dark exposure is necessary as in the first main exposure. In the second exposure, it is assumed that the ratio of the exposure amount to the noise amount calculated from the photometric value and the exposure time T is a value greater than or equal to a predetermined reference value. In this case, the camera MPU 140 determines that dark exposure is not performed, causes the ASIC 135 to perform image processing on the image data 211, and records the image data 211 in the external memory 180.

  Next, in the third shooting, when the camera MPU 140 detects pressing of the release button (t8), the photometry element 144 performs photometry (t9), and acquires a photometric value. The camera MPU 140 determines at least the exposure time based on the acquired photometric value. Here, it is assumed that the exposure time having the same length as the exposure time T at the time of shooting when the image data 201 is generated is determined. The camera MPU 140 controls the drive unit 148 to start the exposure with the subject light flux from the image sensor 132 from t10, and ends the exposure when the exposure time T has elapsed. When the main exposure is completed, reading from the image sensor 132 is performed and output to the ASIC 135 as image data 221. The image data 221 is stored in the RAM 136.

  The camera MPU 140 determines whether or not it is necessary to perform dark exposure for noise reduction processing on the image data 221. The camera MPU 140 determines whether or not dark exposure is necessary as in the first main exposure. In the third exposure, it is assumed that the ratio of the exposure amount to the noise amount calculated from the photometric value and the exposure time T is smaller than a predetermined reference value. On the other hand, it is assumed that the motion detection value detected by the motion sensor 156 is greater than or equal to a predetermined motion reference value. In this case, the camera MPU 140 determines that dark exposure is not performed, causes the ASIC 135 to perform image processing on the image data 211, and records the image data 211 in the external memory 180. Thus, according to the camera 10, it is possible to determine whether or not dark exposure is necessary in consideration of the movement of the camera 10. For this reason, when it can be determined that the camera 10 has moved during exposure, such as in a panning shot, or when the subject has moved relative to the image sensor 132, the prediction accuracy of the exposure amount based on the photometric result may be low. There is sex. In such a case, the camera 10 can suppress control of noise reduction processing according to the photometric result.

  Note that the camera MPU 140 may determine whether or not dark exposure is necessary at any timing when the photometric value and the exposure time are fixed. For example, the camera MPU 140 may determine whether or not dark exposure is necessary at any timing before the start of the main exposure, during the main exposure, and after the main exposure.

  As described above, the camera MPU 140 determines whether or not to perform the noise reduction process based on the exposure time exposed by the imaging device 132 and the intensity of the subject light measured by the photometry device 144. Specifically, the camera MPU 140 determines whether to perform noise reduction processing on the image data based on the exposure time and the intensity of the subject light. Specifically, when it is determined that noise reduction processing is to be performed, the camera MPU 140 blocks the subject light and causes the image sensor 132 to accumulate charges, thereby causing the image sensor 132 to perform a dark exposure operation. The ASIC 135 performs dark current correction on the image data using dark exposure image data that is image data obtained by the dark exposure operation. Thereby, the noise resulting from dark current can be reduced. Note that controlling the intensity of the noise reduction process is a concept including whether or not to perform the noise reduction process. That is, the camera MPU 140 can control the intensity of the noise reduction process based on the exposure amount and the noise amount.

  FIG. 3 schematically shows an example of the amount of charge accumulated in the image sensor 132 according to the exposure time. Here, a case where the temperature of the image sensor 132 is constant will be described. Each photoelectric conversion element included in the imaging element 132 accumulates not only charges generated by receiving subject light but also charges generated by dark current due to thermal noise. A line 301 in this figure indicates the amount of charge generated by the dark current. A line 300 in this figure indicates a total value of charges generated by subject light and charges generated by dark current.

  As exemplified by the line 301, when the temperature of the image sensor 132 is constant, the charge generated by the dark current of the image sensor 132 is substantially proportional to the exposure time. The camera MPU 140 calculates the amount of charge generated by the dark current based on the amount of charge generated by the dark current per time and the exposure time. The amount of charge generated by the dark current is calculated as the amount of noise. Note that the amount of charge generated by dark current per hour may be stored in the system memory 139. Assuming that the intensity of the subject light is constant over the exposure time, the exposure amount indicating the amount of charge generated by the photoelectric conversion element of the image sensor 132 receiving the subject light is proportional to the intensity of the subject light and the exposure time. To do. The camera MPU 140 calculates the exposure amount using at least two of the photometric value and the exposure time. The camera MPU 140 may calculate the exposure amount by further using other information such as an aperture value. The camera MPU 140 determines whether or not dark exposure is necessary based on the ratio of the exposure amount to the calculated noise amount. In this figure, the case where the noise amount is the same as the exposure amount is illustrated. As shown in this example, when the ratio of the exposure amount to the noise amount is small, it is predicted that the noise component caused by the dark current is large. For this reason, the camera MPU 140 determines to perform dark exposure.

  FIG. 4 schematically shows another example of the amount of charge accumulated in the image sensor 132 according to the exposure time. Here, a case where the temperature of the image sensor 132 is constant will be described. A line 401 in this figure indicates the amount of charge generated by dark current. A line 400 in the figure shows a total value of charges generated by subject light and charges generated by dark current. In this figure, the case where the noise amount is 15% of the exposure amount is illustrated. In this way, when the ratio of the exposure amount to the noise amount is large, it is predicted that the influence of the noise component caused by the dark current is small. For this reason, the camera MPU 140 determines to perform dark exposure. Note that an arbitrary value such as 5 may be applied as a predetermined reference value to be compared with the ratio of the exposure amount to the noise amount when the camera MPU 140 determines whether dark exposure is necessary. As the reference value, any value can be applied as long as it can be determined whether or not the influence of the noise component can be ignored. The reference value may be set based on a user instruction.

  As described above, according to the camera 10, when it is determined that the subject is dark and the exposure amount is not sufficient for the noise generated by exposure, the image data with reduced noise is obtained by the dark current correction function based on the dark exposure. Can be provided. On the other hand, when the camera 10 determines that the exposure amount is sufficiently large with respect to noise, such as when the subject is bright, the dark current correction based on the dark exposure is not applied, so that the image is deteriorated due to excessive correction processing. The possibility of being lost can be reduced. Thus, according to the camera 10, the noise reduction process can be controlled in consideration of the actual exposure amount, instead of simply controlling the noise reduction process according to the exposure time.

  FIG. 5 schematically shows another example of the amount of charge accumulated in the image sensor 132 according to the exposure time. Here, a case where the temperature of the image sensor 132 is considered will be described. A line 501 in the figure indicates the amount of charge generated by the dark current. A line 500 in this figure indicates the total value of the charge generated by the subject light and the charge generated by the dark current.

  The amount of charge generated by the dark current in the image sensor 132 increases as the temperature of the image sensor 132 increases. Therefore, the longer the exposure time, the higher the temperature of the image sensor 132 and the larger the amount of charge generated by the dark current. For this reason, the rate of increase in the amount of charge per hour increases as the exposure time increases. The camera MPU 140 calculates the amount of noise based on the temperature measurement value detected by the temperature sensor 154, the exposure time, and the temperature dependence of the amount of charge generated by dark current per unit time. Information indicating the temperature dependence of the amount of charge generated per unit time may be stored in advance in the system memory 139 or the like.

  According to the camera 10, the amount of noise can be appropriately determined even when the noise occurrence rate changes with time during exposure due to the temperature rise of the image sensor 132. When calculating the amount of noise in consideration of the temperature of the image sensor 132, the camera MPU 140 may calculate the amount of noise based on the history of temperature change over the exposure period and the exposure time. Further, the amount of noise may be calculated based on at least one of the temperature immediately before starting the exposure and after the exposure, and the exposure time. Also, if the exposure time is not fixed at the start of exposure, such as when shooting in long exposure mode such as bulb shooting mode or time shooting mode, the noise amount and exposure amount at least after the end of exposure, not before the start of exposure. May be calculated to determine whether or not dark exposure is necessary. Even when imaging is performed in the non-long-second exposure mode, the calculation of the noise amount and the exposure amount and the determination of the necessity of dark exposure may be performed after the exposure is completed, not before the exposure is started.

  FIG. 6 shows a processing flow from activation to termination of the camera 10. This flow is started, for example, when a power switch as a part of the operation input unit 141 is switched to the ON position. This flow is executed by controlling each part of the camera 10 mainly by the camera MPU 140.

  In step S600, the camera MPU 140 starts an initial setting. For example, the camera MPU 140 expands various parameters for controlling the camera 10 from the system memory 139 to the RAM 136. Examples of the various parameters include the amount of charge generated by the dark current per time described above, information indicating its temperature dependence, various reference values used for determining whether or not dark exposure is necessary, and the like. In addition, the camera MPU 140 sets operating conditions of each unit of the camera 10 based on, for example, the state of an imaging mode dial as a part of the operation input unit 141 and various developed parameters. Examples of the operating conditions include a shooting mode, an imaging condition, a recording condition, and the like. Examples of the shooting mode include a continuous shooting mode, a single shooting mode, a bulb shooting mode, and a time shooting mode. Examples of imaging conditions include exposure time, aperture value, imaging sensitivity, and the like. Examples of the recording condition include a file format of image data, the number of recording pixels, and the like.

  Subsequently, in step S602, the camera MPU 140 causes the display unit 138 to display the content set in the initial setting. For example, the camera MPU 140 causes the display unit 138 to display information such as an imaging mode, imaging conditions, and recording conditions in various formats such as icon display.

  Subsequently, in step S604, the camera MPU 140 specifies a user instruction. The camera MPU 140 specifies a user instruction based on an operation on the operation input unit 141. If the user instruction is an instruction to execute various settings, the instructed setting process is performed (step S606). Examples of the setting process include a process for setting an imaging mode, an imaging condition, a compression condition, and a recording condition. Further, as the setting process, a setting for turning on or off the noise reduction processing function can be exemplified. When the noise reduction process is set to OFF, the noise reduction process including dark exposure and dark current correction based on dark exposure is not executed. When the noise reduction process is set to ON, it is permitted to perform a noise reduction process including dark current correction based on dark exposure, for example, at the time of long exposure. When the operation setting is changed in this step, the parameter variable is changed according to the changed operation setting.

  If it is determined in step S604 that the user instruction is an instruction relating to imaging execution, processing relating to imaging execution is performed (step S612). Examples of instructions related to shooting execution include operations on a live view button, a moving image recording button, and a release button. If it is determined in step S604 that the user instruction is an instruction to perform image reproduction, reproduction processing is performed (step S622). Examples of playback processing include processing for displaying thumbnails based on image data such as still images and moving images recorded in the external memory 180, processing for displaying images based on image data selected by the user, and the like. Can do.

  If it is determined in step S604 that there is no user instruction, the process proceeds to step S608. Also when the processes of step S606, step S612, and step S622 are completed, the process proceeds to step S608. In step S608, it is determined whether to turn off the power. For example, it is determined to turn off the power when the power switch is switched to the OFF position, or when there is no user instruction for a predetermined period after the camera 10 starts operating. If it is determined that the power is to be turned off, this flow is terminated. If it is determined that the power is not to be turned off, the process proceeds to step S604.

  FIG. 7 shows an example of a processing flow when performing an imaging operation. This flow can be applied to a part of the processing in step S612. This flow is started when pressing of the release button is detected. For example, this flow is an example of an operation when a still image is captured. This flow is executed by controlling each part of the camera 10 mainly by the camera MPU 140.

  When this flow starts, the camera MPU 140 causes the photometric element 144 to perform photometry and obtains a photometric value (step S700). Subsequently, it is determined whether or not the shooting mode is the long exposure mode (step S702). The long exposure mode is a photographing mode in which exposure is started by exposure control in which exposure is started in response to a first user operation and exposure is ended in response to a second user operation. Examples of the long exposure mode include imaging modes such as a bulb photography mode and a time photography mode.

  If it is determined that the exposure mode is not the long second exposure mode, the camera MPU 140 determines an imaging condition (step S704). Examples of imaging conditions include aperture value, exposure time, imaging sensitivity, and the like. The camera MPU 140 may determine the imaging condition based on the photometric value acquired in step S700. At this time, the camera MPU 140 may determine the imaging condition according to the program diagram.

  Subsequently, the camera MPU 140 starts storing the temperature measurement value acquired from the temperature sensor 154 and the motion detection value acquired from the motion sensor 156 (step S706). Then, the camera MPU 140 resets the accumulated charge of each photoelectric conversion element included in the imaging element 132 (step S708). When the accumulated charge is reset, the focal plane shutter 143 is opened (step S710), the exposure time determined in step S704 is waited (step S712), and the focal plane shutter 143 is closed (step S714). . Subsequently, the ASIC 135 acquires image data read from the image sensor 132 (step S716).

  In step S718, the camera MPU 140 calculates a noise amount and an exposure amount. Specifically, the camera MPU 140 calculates the amount of noise based on the temperature measurement value measured over the exposure time. Further, the camera MPU 140 calculates an exposure amount based on the photometric value acquired in step S700, the aperture value determined in step S704, and the exposure time. Subsequently, in step S720, the camera MPU 140 determines whether or not dark exposure is necessary. Specifically, camera MPU 140 determines that dark exposure is necessary when the ratio of the exposure amount to the noise amount is smaller than a predetermined reference value. Note that the camera MPU 140 determines that dark exposure is not necessary when the noise reduction processing function is set to OFF. Even when the noise reduction processing function is set to ON, it is determined that dark exposure is not necessary if the ratio of the exposure amount to the noise amount is equal to or greater than a predetermined reference value.

  If it is determined in step S720 that dark exposure is necessary, the camera MPU 140 determines whether there is any motion (step S722). Specifically, the camera MPU 140 determines whether or not a motion is detected based on the motion detection value acquired from the motion sensor 156. For example, the camera MPU 140 may determine that there is no motion when the motion detection value acquired from the motion sensor 156 is smaller than a predetermined motion reference value. For example, the camera MPU 140 may determine that no motion has occurred when the motion detection value acquired from the motion sensor 156 is smaller than a predetermined motion reference value over the exposure period. In addition, the camera MPU 140 may determine that there is no motion when the time average value of the motion detection values acquired from the motion sensor 156 is smaller than a predetermined motion reference value.

  If it is determined in step S722 that there is no movement, in step S724, the camera MPU 140 performs dark exposure (step S724). Specifically, the camera MPU 140 resets the accumulated charge of each photoelectric conversion element included in the imaging element 132 while keeping the focal plane shutter 143 in the closed state, and performs reading after the exposure time determined in step S704. . Subsequently, the ASIC 135 performs dark current correction processing on the image data acquired in step S716 using the image data obtained by the dark exposure in step S724 (step S726). The ASIC 135 performs predetermined image processing on the image data that has undergone dark current correction (step S728), and records the image data in the external memory 180 (step S730). Examples of the image processing in step S728 include defective pixel correction, white balance correction, color interpolation processing, color correction, gamma correction, contour enhancement processing, and image data compression processing.

  If it is determined in step S720 that the ratio of the exposure amount to the noise amount is equal to or greater than a predetermined reference value, the process proceeds to step S728. If it is determined in step S722 that there is a movement, the process proceeds to step S728. In step S728, the ASIC 135 performs predetermined image processing on the image data acquired in step S716.

  If it is determined in step S702 that the long exposure mode is set, the camera MPU 140 determines an imaging condition (step S740). Examples of imaging conditions determined here include an aperture value, imaging sensitivity, and the like. Subsequently, the camera MPU 140 starts storing the temperature measurement value acquired from the temperature sensor 154 and the motion detection value acquired from the motion sensor 156 (step S742). Subsequently, in step S744, the camera MPU 140 resets the accumulated charge of each photoelectric conversion element included in the imaging element 132 (step S744). When the accumulated charge is reset, the camera MPU 140 opens the focal plane shutter 143 (step S746). It waits for a user operation to instruct the end of exposure (step S748). If a user operation to end exposure is detected, the process proceeds to step S714.

  As described above, the camera MPU 140 can calculate an estimated value of a value obtained by removing the charge amount generated by noise from the charge amount generated in the image sensor 132 by imaging as the exposure amount. Then, the calculated exposure amount is compared with the noise amount generated during imaging to determine whether or not dark exposure needs to be executed. For this reason, when it is determined that the subject is bright enough to ignore noise, dark exposure and dark current correction can be skipped. Therefore, it is possible to reduce the possibility of performing dark exposure and dark current correction when dark current correction is not actually required. For this reason, since the user does not need to wait for dark exposure, the next photographing can be instructed relatively quickly. Further, the image data can be quickly recorded in the external memory 180. Further, for example, it is possible to reduce the possibility that the dark current correction is performed with an intensity higher than necessary and the image is deteriorated. Thus, according to the camera 10, it is possible to appropriately determine when dark current correction is necessary and apply dark current correction to provide an image with reduced dark current noise.

  In the above description, the processing for reducing so-called long-time noise has been mainly described as the noise reduction processing. However, a process for reducing so-called high-sensitivity noise may be applied as the noise reduction process. For example, the camera MPU 140 may control the intensity of the noise reduction process based further on the imaging sensitivity at the time of imaging. For example, the camera MPU 140 calculates a noise amount related to the imaging sensitivity based on the imaging sensitivity, and when the ratio of the exposure amount to the noise amount is smaller than a predetermined reference value, a noise reduction process for reducing high sensitivity noise You may decide to apply. When it is determined that the noise reduction process is to be performed, the ASIC 135 may apply a process of reducing a value corresponding to the imaging sensitivity from each pixel value of the image data. In addition, the ASIC 135 may apply various image processing for reducing high sensitivity noise as noise reduction processing based on the characteristics of high sensitivity noise. In addition, the camera MPU 140 performs noise reduction when the ratio of the exposure amount to the total noise amount of the noise amount related to the imaging sensitivity calculated based on the imaging sensitivity and the noise amount due to the dark current is smaller than a predetermined reference value. It may be determined that processing is to be performed.

  2 to 7, the process in the case of determining whether or not to perform the noise reduction process on the image data based on at least the photometric value and the exposure time has been described. However, the ASIC 135 may control the intensity of the noise reduction process. For example, when subtracting dark exposure image data obtained by dark exposure from the image data, the ASIC 135 may subtract the dark exposure image data with a smaller weight as the subject light intensity increases. In addition, when applying noise reduction processing that reduces high-sensitivity noise, the ASIC 135 may decrease the value to be subtracted from each pixel value of the image data as the subject light intensity increases.

  In the above example, it has been described that the time change of the subject is mainly due to the movement of the camera 10. However, the camera 10 may detect the movement of the subject relative to the stationary camera 10. For example, when the imaging element 132 includes at least one second photoelectric conversion element that can be driven independently of the plurality of first photoelectric conversion elements that output image data to be subjected to noise reduction processing, the camera MPU 140 includes a plurality of During the exposure period of the first photoelectric conversion element, the second photoelectric conversion element may be exposed and read out a plurality of times. The camera MPU 140 may detect the time change of the output of the second photoelectric conversion element as the time change of the subject based on the data obtained by the plurality of exposure and reading operations. In addition, when the camera 10 includes a light receiving unit that receives a subject light flux different from the subject light flux incident on the image sensor 132, the camera MPU 140 is based on a temporal change in the intensity of the subject light received by the light receiving unit. You may detect the time change of a to-be-photographed object.

  In addition, with reference to FIGS. 2 to 7, an example of control of noise reduction processing has been described mainly taking still image shooting. However, the same control may be applied to each frame of the moving image. In the case where similar control is applied to a moving image, it is preferable to have means for detecting subject light independently of the image sensor 132. For example, when the imaging device 132 includes at least one second photoelectric conversion element that can be driven independently of the plurality of first photoelectric conversion elements that output frame data, the camera MPU 140 generates a plurality of frame data to generate frame data. Prior to the exposure period of the first photoelectric conversion element, exposure and readout may be performed by the second photoelectric conversion element, and data obtained by the exposure and readout operation may be applied as photometric value data. When the camera 10 has a light receiving unit that receives a subject light beam different from the subject light beam incident on the image sensor 132, the camera MPU 140 may apply the intensity of the subject light received by the light receiving unit as a photometric value. . Note that if the intensity of noise reduction processing is greatly different for each frame of a moving image, the brightness and the like may vary greatly from frame to frame. Therefore, when the ASIC 135 detects a scene change in a moving image, the camera MPU 140 detects noise that should be applied to a plurality of frames obtained before the next scene change when the scene change is detected. The intensity of the reduction process may be determined, and the noise reduction process may be applied collectively to the plurality of frames with the determined intensity. The ASIC 135 may apply the change in the average value of the brightness in the frame as an indicator of the change in the scene.

  In the above description, the processing described as the operation of the camera MPU 140 is realized by the camera MPU 140 controlling each hardware of the camera 10 according to a program. In the above description, the processing realized by the ASIC 135 can be realized by a processor. For example, the processing described as the operation of the ASIC 135 is realized by the processor controlling each hardware included in the camera 10 according to the program. That is, the processing described in relation to the camera 10 of the present embodiment is performed by the processor operating in accordance with the program to control each hardware, so that each hardware including the processor, the memory, and the like cooperates with the program. It can be realized by operating. That is, the process can be realized by a so-called computer device. The computer device may load a program for controlling the execution of the above-described process, operate according to the read program, and execute the process. The computer device can load the program from a computer-readable recording medium storing the program.

  In the present embodiment, the camera 10 with the lens unit 120 attached is taken as an example of an imaging apparatus. However, the imaging device is a concept including the camera body 130 to which the lens unit 120 is not attached. As an imaging device, in addition to a single-lens reflex camera that is an example of an interchangeable lens camera, a compact digital camera that is an example of a non-interchangeable camera, a mirrorless camera, a video camera, a mobile phone with an imaging function, imaging Various electronic devices having an imaging function, such as a portable information terminal with a function and an entertainment device such as a game machine with an imaging function, can be applied.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

  DESCRIPTION OF SYMBOLS 10 Camera, 120 Lens unit, 121 Lens mount contact, 122 Lens group, 123 Lens MPU, 124 Lens drive part, 130 Camera body, 131 Camera mount contact, 132 Image sensor, 133 Analog processing part, 134 A / D converter, 135 ASIC, 136 RAM, 137 Display control unit, 138 Display unit, 139 System memory, 140 Camera MPU, 141 Operation input unit, 142 AF unit, 143 Focal plane shutter, 144 Photometric element, 145 Main mirror, 146 Sub mirror, 147 Finder Unit, 148 drive unit, 149 connection interface, 150 recording medium IF, 152 external device IF, 154 temperature sensor, 156 motion sensor, 180 external memory, 190 power supply, 192 power Circuit, 201,202,203,211,221 image data

Claims (13)

A metering unit for metering subject light;
A noise reduction processing unit that performs noise reduction processing to reduce noise components caused by noise generated in the image sensor on the image data obtained by exposure with the image sensor;
An imaging apparatus comprising: a control unit that controls the intensity of the noise reduction process based on an exposure time exposed by the imaging element and an intensity of subject light measured by the photometry unit. The imaging apparatus according to claim 1, wherein the control unit decreases the intensity of the noise reduction process as the intensity of the subject light increases. The imaging apparatus according to claim 1, wherein the control unit reduces the intensity of noise reduction processing as the ratio of the intensity of the subject light to the exposure time increases. An exposure time determining unit that determines the exposure time based on the intensity of the subject light;
The image data is image data obtained by exposing with the image sensor at an exposure time determined by the exposure time determination unit,
4. The imaging according to claim 1, wherein the control unit controls the intensity of the noise reduction process based on the exposure time determined by the exposure time determination unit and the intensity of the subject light. 5. apparatus. A temperature acquisition unit that acquires the temperature of the image sensor;
5. The control unit controls the intensity of the noise reduction process based on the exposure time, the intensity of subject light measured by the photometry unit, and the temperature acquired by the temperature acquisition unit. The imaging device according to any one of the above. The imaging apparatus according to claim 5, wherein the control unit increases the intensity of the noise reduction process as the temperature acquired by the temperature acquisition unit increases. An exposure amount calculation unit for calculating an exposure amount indicating the amount of charge generated in the image sensor by subject light;
A noise calculating unit that calculates a noise amount indicating a charge amount generated in the image sensor due to a dark current based on the exposure time;
The imaging device according to claim 1, wherein the control unit controls the intensity of the noise reduction process based on the exposure amount and the noise amount. The control unit determines whether or not to perform the noise reduction process based on the exposure time and the intensity of the subject light on the image data based on the exposure time and the intensity of the subject light. The imaging device according to any one of claims 1 to 7. It further includes a detection unit that detects a time change of the subject,
The control unit may reduce the noise reduction based on the exposure time and the intensity of the subject light with respect to the image data when a time change amount of the subject detected by the detection unit is larger than a predetermined value. The imaging apparatus according to claim 8, wherein it is determined that no processing is performed. The detection unit detects temporal movement of the subject,
The control unit determines that the noise reduction processing is not performed on the image data when a temporal movement amount of the subject detected by the detection unit is larger than a predetermined value. 9. The imaging device according to 9. When the control unit determines that the noise reduction process is to be performed, the control unit causes the image sensor to perform a dark exposure operation by blocking subject light and causing the image sensor to accumulate electric charge.
11. The imaging according to claim 8, wherein the noise reduction processing unit performs dark current correction on the image data using dark exposure image data that is image data obtained by the dark exposure operation. apparatus. The imaging device according to any one of claims 1 to 7, wherein the control unit controls the intensity of the noise reduction processing based further on imaging sensitivity at the time of imaging. A metering step for metering subject light;
Applying noise reduction processing to reduce noise components caused by noise generated in the image sensor on the image data obtained by exposure with the image sensor;
A program for causing a computer to execute a control step of controlling the intensity of the noise reduction processing based on the exposure time exposed by the image sensor and the intensity of the subject light measured in the photometry step.

Images