JP2016100879A - Imaging apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device that makes available a configuration to correct an existing dark current and can reduce an influence of a difference in dark current occurring due to a difference in pixel structure.SOLUTION: An imaging device includes: an imaging element that has an effective pixel area where function pixels used for a purpose other than image formation are arranged in an area where image pixels are arranged in matrix, and a light-shielded pixel area where image pixels and function pixels having the same structure as that in the effective pixel area are arranged, while shielded from light, with a rule different from that for the effective pixel area; and a dark current detection part 41 that detects a dark current of the image pixels in the light-shielded area and a dark current of the function pixels in the light-shielded area.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus and an image processing method for detecting dark current of an imaging element in which image pixels and functional pixels are arranged.

  In an imaging apparatus that converts an optical image formed by an optical system into an electric signal, a technique for correcting a dark current of an effective pixel portion formed of regions having different dark current characteristics has been conventionally proposed.

  For example, Japanese Unexamined Patent Application Publication No. 2009-284424 describes a technique for correcting a difference in dark current characteristics for each divided exposure region of an image sensor manufactured by divided exposure. That is, when the mask pattern of a semiconductor integrated circuit is reduced using a stepper and printed on a silicon substrate, the exposure range of the stepper is limited, so the image sensor is divided into a plurality of regions and each divided region unit Thus, the exposure process is performed. At this time, characteristics including dark current characteristics may be different for each divided region. Therefore, in the technique described in the publication, the correlation between the dark current characteristics of each area and the dark current characteristics of the optical black (OB) area is stored as table data in advance, and the pixel output of the OB area and the table data thereof are stored. Is used to calculate and correct the difference in dark current characteristics between the OB area and each area.

  In recent years, there has been proposed an imaging device in which functional pixels having pixel characteristics different from those of image pixels for acquiring an image are arranged. As a specific example, a first phase difference pixel that images and photoelectrically converts a light beam that has passed through a part of the pupil, and a first phase difference pixel that photoelectrically converts a light beam that has passed through another part of the pupil and formed an image. An image plane phase difference AF image pickup device in which two phase difference pixels are discretely arranged in the image pickup region can be used. Such an image plane phase difference AF imaging element performs phase difference detection based on the pixel signal obtained from the first phase difference pixel and the pixel signal obtained from the second phase difference pixel, and the position is detected. It is used for phase difference AF.

  Even in such a configuration, a difference in dark current characteristics occurs between the image pixel and the phase difference pixel. This point will be described with reference to FIG. 7 according to the embodiment of the present invention.

  When comparing the functional pixel PxA such as a phase difference pixel and the image pixel PxP, the functional pixel PxA has an optical path from the imaging optical system (not shown in FIG. 7) to the photodiode 53 via the microlens 52. A light shielding portion 54 that shields a part of the light beam is provided. Such a light-shielding portion 54 is formed of, for example, a metal film (for example, an aluminum film) when forming wirings, that is, is formed of a material having charging properties. For this reason, for example, the light shielding portion 54 may be charged by the electric charge 59 generated in the photodiode 53. Therefore, the functional pixel PxA and the image pixel PxP have different capacitances and different characteristics of the generated dark current.

  As described above, the functional pixel includes a circuit for generating a function, a processing circuit that performs processing dedicated to the functional pixel, a member, or the like as compared with the image pixel. Even in the state, the dark current tends to increase due to a minute charge from the leak current, and an amount of dark current different from that of the image pixel is generated. Therefore, when image processing is performed without considering the dark current difference between the image pixel and the functional pixel, the pixel value increases only at the functional pixel position, for example, and a false color is generated.

  As a technique for coping with such a difference in dark current characteristics, for example, Japanese Patent Application Laid-Open No. 2012-95061 discloses a part of an OB region in an image plane phase difference AF imaging device including a light receiving pixel region and an OB region. A phase difference detection pixel dark current measurement region in which a phase difference detection pixel for measuring dark current is disposed, an image generation pixel dark current measurement region in which an image generation pixel for measuring dark current is disposed, It is described to provide. Here, the signals detected in the phase difference detection pixel dark current measurement region are averaged and used to remove dark current noise from the signal of the phase difference detection pixel. Further, the signals detected in the image generation pixel dark current measurement region are averaged for each of the pixels R, G, and B, and are used to remove dark current noise from the pixels R, G, and B. .

Japanese Unexamined Patent Publication No. 2009-284424 Japanese Unexamined Patent Publication No. 2012-95061

  The technology described in the above Japanese Patent Application Laid-Open No. 2009-284424 is not assumed to be a dark current difference due to an imaging device in which pixels having different structures are mixed. It is impossible to effectively suppress image quality deterioration of an image obtained from an imaging device in which pixels having a structure are mixed.

  Further, the technique described in Japanese Patent Application Laid-Open No. 2012-95061 requires dark current noise correction processing separately for functional pixels and image pixels, and corrects the conventional dark current. It is necessary to use a circuit or the like different from the configuration, and the existing configuration cannot be used.

  The present invention has been made in view of the above circumstances, and an imaging device and an image that can reduce the influence of a dark current difference caused by a difference in pixel structure while making it possible to use a configuration for correcting an existing dark current. It aims to provide a processing method.

  An imaging apparatus according to an aspect of the present invention includes an effective pixel region in which functional pixels used for other than image formation are arranged in a region in which image pixels for image formation are arranged in a matrix in the horizontal direction and the vertical direction, Arranged around the effective pixel area, the image pixels having the same structure as the image pixels in the effective pixel area and the functional pixels having the same structure as the functional pixels in the effective pixel area are arranged in a light-shielded manner. An imaging device having a light-shielding pixel region, and a dark current detection unit that detects a dark current of an image pixel in the light-shielding pixel region and a dark current of a functional pixel in the light-shielding pixel region.

  An image processing method according to an aspect of the present invention includes an effective pixel area in which functional pixels used for other than image formation are arranged in an area in which image pixels for image formation are arranged in a matrix in the horizontal and vertical directions; An image pixel having the same structure as the image pixel in the effective pixel region and a functional pixel having the same structure as the functional pixel in the effective pixel region are disposed in the vicinity of the effective pixel region and shielded from light. An image processing method for processing an image signal output from an image sensor having a light-shielded pixel region, wherein the dark current of the image pixel in the light-shielded pixel region and the function of the light-shielded pixel region are obtained from the image signal. A dark current detection step for detecting a dark current of a pixel; a dark current of a functional pixel in the light-shielded pixel region detected by the dark current detection step; a dark current of an image pixel in the light-shielded pixel region; A dark current difference calculating step for calculating a dark current difference that is a difference between the dark current difference and a dark current difference correcting step for correcting a pixel signal read from the functional pixel in the effective pixel region based on the dark current difference. Have.

  According to the imaging apparatus and the image processing method of the present invention, it is possible to reduce the influence of a dark current difference generated due to a difference in pixel structure while making it possible to use a configuration for correcting an existing dark current.

1 is a block diagram illustrating a configuration of an imaging apparatus according to Embodiment 1 of the present invention. FIG. 2 is a block diagram showing a configuration of an image processing unit in the first embodiment. The block diagram which shows the structure of the pre dark current correction | amendment part in the said Embodiment 1. FIG. FIG. 3 is a diagram illustrating a configuration of a pixel in the image sensor of the first embodiment. FIG. 3 is a diagram illustrating a pixel configuration of an effective pixel area in the first embodiment. FIG. 4 is a diagram illustrating an arrangement example of image pixels and functional pixels in a calculation region determined in the horizontal light-shielding pixel region of the first embodiment. FIG. 3 is a diagram illustrating a cross-sectional structure of functional pixels and image pixels in a light-shielding pixel region according to the first embodiment. 3 is a flowchart showing still image shooting processing in the imaging apparatus according to the first embodiment. 9 is a flowchart showing details of image processing in step S8 of FIG. 8 in the first embodiment. The flowchart which shows the detail of the pre dark current correction process in FIG.9 S21 of the said Embodiment 1. FIG. FIG. 6 is a diagram illustrating an example of 2 × 2 pixel addition with respect to a pixel Gb in an effective pixel region in the first embodiment. The figure which shows the example of the addition signal obtained by the said Embodiment 1 by 2x2 pixel addition. FIG. 4 is a diagram showing an example of 3 × 3 pixel addition with respect to a pixel Gb in an effective pixel region in the first embodiment. FIG. 4 is a diagram illustrating an example of an addition signal obtained by 3 × 3 pixel addition in the first embodiment. The chart which shows how many functional pixels A are mixed with the addition signal obtained by 3x3 pixel addition in the said Embodiment 1. FIG. The chart which shows how many functional pixels B are mixed with the addition signal obtained by 3x3 pixel addition in the said Embodiment 1. FIG.

Embodiments of the present invention will be described below with reference to the drawings.
[Embodiment 1]

  1 to 16 show the first embodiment of the present invention, and FIG. 1 is a block diagram showing the configuration of the imaging apparatus 1.

  The imaging device 1 includes a lens 11, a diaphragm 12, a mechanical shutter 13, a drive unit 14, an imaging element 15, an A-AMP 16, an ADC 17, a bus 18, a DRAM 19, an image processing unit 21, A focus detection unit 22, an imaging control unit 23, a video encoder 24, a ROM 25, an operation unit 26, and a CPU 27 are provided.

  The lens 11 includes a focus lens and is configured to be able to adjust the focus position, and is a photographing optical system for forming an optical image of the subject OBJ on the image sensor 15.

  The diaphragm 12 controls the brightness, the amount of blur, and the like of the subject image formed by the lens 11 by defining the passing range of the photographing light flux.

  The mechanical shutter 13 is disposed between the lens 11 and the image sensor 15 and controls exposure by regulating the passage time of the light beam reaching the image sensor 15.

  The drive unit 14 drives and controls the lens 11, the diaphragm 12, and the mechanical shutter 13. That is, the drive unit 14 drives the focus lens of the lens 11 to adjust the focus, drives the diaphragm 12 to adjust the aperture diameter of the diaphragm, and drives the mechanical shutter 13 to perform an opening operation and a closing operation.

  The imaging element 15 is an imaging unit that photoelectrically converts an optical image formed through the lens 11 to generate an electrical image signal. For example, the image sensor 15 is configured as a single-plate color image sensor including a color filter with a primary color Bayer arrangement, and an image in which only one color component of RGB color components is obtained per pixel is obtained. Generated.

  Furthermore, the image sensor 15 of the present embodiment forms an image in an area in which image pixels for image formation (pixels R, Gr, Gb, and B shown in FIG. 5) are arranged in a matrix in the horizontal direction and the vertical direction. The functional pixels (functional pixel A and functional pixel B shown in FIG. 5) to be used are arranged. In this embodiment, it is assumed that phase difference pixels for image plane phase difference AF are arranged as functional pixels A and B. However, the functional pixel is not limited to the phase difference pixel, and a pixel having any other function can be applied.

  Furthermore, the image sensor 15 of the present embodiment is configured to be operable in a plurality of drive modes including a drive mode in which at least one of pixel addition (also referred to as binning) and pixel thinning is performed.

  The A-AMP 16 is an analog amplifier that amplifies an analog image signal output from the image sensor 15.

  The ADC 17 converts the analog image signal amplified by the A-AMP 16 into a digital image signal (hereinafter referred to as image data as appropriate). The converted image data (referred to as RAW image data) is stored in the DRAM 19.

  The bus 18 is a transfer path that transfers commands, data, and the like from one part in the imaging apparatus 1 to another part. In the example shown in FIG. 1, the ADC 17, the DRAM 19, the image processing unit 21, the focus detection unit 22, the imaging control unit 23, the video encoder 24, the ROM 25, the CPU 27, and the flash memory are connected to the bus 18. 3 are connected.

  The DRAM 19 is a storage unit that temporarily stores image data being processed, various data, and the like.

  The image processing unit 21 reads out image data from the DRAM 19 and also reads out processing parameters and other various data as necessary, and performs various image processing as described later with reference to FIG. (For example, image processing for display or image processing for recording) is performed. Here, the display image data or the recording image data image-processed by the image processing unit 21 is stored again in the DRAM 19. Here, the display image data is processed by a video encoder 24 described later and displayed on the LCD / TV 2. The LCD / TV 2 is configured, for example, as a liquid crystal display element, a television monitor, or the like, and may be either built in the imaging device 1 or connected to the outside of the imaging device 1. The recording image data is recorded on a recording medium such as the flash memory 3. The flash memory 3 may be either a built-in flash memory 3 or a memory card that can be attached to and detached from the imaging device 1.

  The focus detection unit 22 reads out the pixel signal of the phase difference pixel included in the image data from the DRAM 19, performs phase difference detection, and generates focus information indicating the focus position corresponding to the distance to the subject. The focus information generated here is stored in the DRAM 19. Here, the focus detection unit 22 performs phase difference detection based on the pixel signal of the phase difference pixel, but in addition to this, contrast AF or the like may be performed based on the pixel signal of the image pixel. Absent.

  The imaging control unit 23 controls a driving mode (for example, a readout method) of the imaging element 15 based on a command from the CPU 27. Here, examples of the drive mode of the image sensor 15 include an all-pixel readout mode, a 2 × 2 pixel addition readout mode, and a 3 × 3 pixel addition readout mode.

  The video encoder 24 reads display image data from the DRAM 19 and generates and outputs a display image signal. The image signal output from the video encoder 24 is displayed on the LCD / TV 2 as described above.

  The ROM 25 stores various processing programs, specified setting values, processing parameters, and the like necessary for the operation of the imaging apparatus 1 in a nonvolatile manner, and is configured as a memory that can be additionally written or rewritten, for example. The information stored in the ROM 25 includes functional pixel arrangement information and drive mode information of the image sensor 15. Further, in the ROM 25, for example, mixing ratio information as shown in FIGS. 15 and 16 to be described later may be stored in advance as table data.

  The operation unit 26 is a user interface for performing an operation input to the imaging apparatus 1, and includes a power button for turning on / off the imaging apparatus 1, a shooting button for instructing shooting start, a shooting mode, and the like. A mode setting button for setting and various other setting buttons are included.

  The CPU 27 is a control unit that controls each unit in the imaging apparatus 1 in an integrated manner. For example, the CPU 27 transmits a command to the drive unit 14 to drive the lens 11 to the focus position detected by the focus detection unit 22. The CPU 27 reads out image data from the DRAM 19 and calculates appropriate exposure conditions, that is, a shutter speed, an aperture value (so-called F value), ISO sensitivity, and the like. The calculated values are used as the drive unit 14, the imaging control unit 23, Each drive control is performed by setting the A-AMP 16 or the like. Then, the CPU 27 causes the image sensor 15 to perform an imaging operation via the imaging control unit 23, causes the image processing unit 21 to process the acquired image data, and displays the acquired image data on the LCD / TV 2 via the video encoder 24. Record in the flash memory 3.

  Next, FIG. 2 is a block diagram showing a configuration of the image processing unit 21.

  The image processing unit 21 includes a pre-dark current correction unit 31, a dark current correction unit 32, a white balance (WB) correction processing unit 33, a synchronization processing unit 34, a luminance characteristic conversion unit 35, and an edge enhancement processing unit. 36, a noise reduction (NR) processing unit 37, and a color reproduction processing unit 38.

  The pre dark current correction unit 31 calculates the dark current difference between the image pixel and the functional pixel based on the pixel signals of the light shielding pixel regions 15b and 15c (see FIG. 4). Based on the calculated dark current difference, the pre dark current correction unit 31 then outputs the pixel signals of the image pixels R, Gr, Gb, and B and the functional pixels A and B of the effective pixel region 15a (see FIG. 4). Correction of the dark current difference from the signal is performed on the functional pixels A and B.

  The dark current correction unit 32 outputs the pixel signals read from the image pixels R, Gr, Gb, and B to the image data of the effective pixel region 15a corrected by the pre-dark current correction unit 31, as in the conventional case. And the process of reducing the dark current noise detected as the dark current component of the image pixel without distinguishing between the pixel signals read from the functional pixels A and B.

  That is, the dark current correction unit 32 corrects the dark current of the image pixels in the effective pixel region 15a based on the dark current of the image pixels in the light-shielded pixel regions 15b and 15c detected by the dark current detection unit 41 described later. At the same time, the dark current of the functional pixels in the effective pixel region 15a after correction by the dark current difference correction unit 42 described later is further corrected.

  The WB correction processing unit 33 corrects the color balance of each RGB color component of the image so that a white subject appears white.

  The synchronization processing unit 34 obtains a color component that does not exist in the pixel of interest by interpolating from the surrounding pixels with respect to the Bayer array RAW image data in which only one color component of RGB components is obtained for each pixel. Thus, processing (also referred to as demosaicing processing) is performed to generate image data in which RGB three color components are aligned per pixel.

  The luminance characteristic conversion unit 35 performs gradation conversion of the luminance component of the image.

  The edge enhancement processing unit 36 performs edge enhancement by extracting an edge component in the image, multiplying it by an edge enhancement coefficient, and adding it to the original image.

  The NR processing unit 37 performs noise reduction processing for reducing random noise, periodic noise, and the like by performing coring processing or the like corresponding to the frequency on the image data.

  The color reproduction processing unit 38 performs color reproduction of the image.

  Before describing the configuration of the pre-dark current correction unit 31 with reference to FIG. 3, the configuration of the image sensor 15 will be described in more detail with reference to FIGS. 4 to 7. First, FIG. 4 is a diagram showing the configuration of the image sensor 15.

  The image sensor 15 has an effective pixel area 15a and light-shielding pixel areas 15b and 15c arranged around the effective pixel area 15a.

  FIG. 5 is a diagram illustrating a pixel configuration of the effective pixel region 15a.

  As shown in FIG. 5, in the effective pixel region 15a, image pixels R, Gr, Gb, and B for image formation are arranged in a matrix in the horizontal direction and the vertical direction. Here, the image pixel R is a pixel provided with a red light transmission color filter, the image pixel Gr is provided on the same horizontal line as the image pixel R, and a green light transmission color filter is provided. The pixel Gb is a pixel in which a green light transmission color filter is disposed on the same horizontal line as the image pixel B, and the image pixel B is a pixel in which a blue light transmission color filter is disposed.

  The effective pixel region 15a includes functional pixels A and B used for other than image formation within a region where image pixels R, Gr, Gb, and B are arranged in a matrix. Are arranged in a discrete manner.

  Specifically, in the example shown in FIG. 5, both the functional pixel A and the functional pixel B are periodically arranged at a ratio of 1 pixel per 4 pixels in the horizontal direction, and 1 in 4 pixels also in the vertical direction. The rule is that the pixels are arranged periodically at a ratio of pixels.

  Here, an example is shown in which the functional pixels A and B are discretely arranged according to a regular cycle rule. However, the present invention is not limited to this. For example, the functional pixels A and B may be of a type arranged in a part of the effective pixel area 15a. It doesn't matter.

  In addition, in the configuration example illustrated in FIG. 4, as the light-shielding pixel region, the horizontal light-shielding pixel region 15 b disposed in the horizontal direction (near the horizontal edge) of the effective pixel region 15 a and the vertical direction of the effective pixel region 15 a. And a vertical light-shielding pixel region 15c disposed in the vicinity of (a vicinity of the edge in the vertical direction). However, for example, only one of the horizontal light-shielding pixel region 15b and the vertical light-shielding pixel region 15c may be provided in the image sensor 15.

  For example, as shown in FIG. 6, the light-shielding pixel regions 15b and 15c include image pixels having the same structure as the image pixels R, Gr, Gb, and B in the effective pixel region 15a and functional pixels in the effective pixel region 15a. Functional pixels having the same structure as A and B are arranged.

  FIG. 6 is a diagram showing an arrangement example of image pixels and functional pixels in the calculation area 15d determined in the horizontal light-shielding pixel area 15b. In the calculation region 15d of the 3 × n pixel array [(m, i) to (m + 2, i + n−1)] shown in FIG. 6, the functional pixel A is on the (m + 1) line and the functional pixel A is on the (m + 2) line. Functional pixels B are respectively arranged, and for example, image pixels Gb and B are alternately arranged on the m line. Further, although not shown, for example, image pixels R and Gr are alternately arranged on the (m−1) line.

  As shown in the figure, the functional pixels A and B in the light-shielded pixel regions 15b and 15c are arranged according to a different rule from the functional pixels A and B in the effective pixel region 15a. In the specific example shown in FIG. The image pixels and the functional pixels arranged in the area 15b are continuously arranged in the arrangement range without any gap in the horizontal direction (so that there is no unplaced pixel position) (FIG. 6 shows the arrangement range). The calculation area 15d set as a part is shown). Similarly, the image pixels and the functional pixels arranged in the vertical light-shielding pixel region 15c are continuously arranged in the arrangement range without a gap in the vertical direction (so that there is no unplaced pixel position).

  More specifically, the image pixels R, Gr, Gb, and B in the light-shielded pixel regions 15b and 15c are set so that the image pixel R and the image pixel Gr are on the same line, and the image pixel Gb and the image pixel are used. Similar to the arrangement of the image pixels R, Gr, Gb, and B in the effective pixel region 15a, except that the pixels B are arranged on the same line and the functional pixels A and B do not exist on the same line. is there.

  On the other hand, in the functional pixels A and B of the light-shielding pixel regions 15b and 15c, only the functional pixel A is arranged in a horizontal line without a gap in the horizontal direction, and only the functional pixel B is arranged in the other horizontal line without a gap in the horizontal direction. ing. With such a configuration, image pixels and functional pixels in the horizontal light-shielding pixel region 15b can be read out independently.

  In the case of the horizontal light-shielding pixel region 15b, the horizontal arrangement range of the image pixels and the horizontal arrangement range of the functional pixels are made to be the same range in the horizontal direction. In this case, the vertical arrangement range of the image pixels and the vertical arrangement range of the functional pixels are set to be the same range in the vertical direction.

  Next, FIG. 7 is a diagram illustrating a cross-sectional structure of the functional pixel PxA and the image pixel PxP in the light-shielding pixel regions 15b and 15c.

  The light-shielding pixel regions 15b and 15c are provided with a light-shielding film 51 on the lens 11 side, thereby shielding the subject light flux.

  The image pixel PxP includes a micro lens 52 for increasing the aperture ratio by condensing a subject light beam, a color filter 56p of any one of an R filter, a G filter, and a B filter, and a photodiode 53. Yes. Whether the image pixel PxP corresponds to the image pixel R, G (Gr or Gb) or B described above is determined by the transmission spectrum band of the color filter 56p.

  In the functional pixel PxA, the color filter 56p of the image pixel PxP is replaced with a transparent filter (white filter) 56a, and a photodiode is used to limit a part of the pupil of the light beam that passes through the pupil of the lens 11 and enters. A light shielding portion 54 that shields a part of the light 53 is provided in the wiring layer 57, for example. Whether the functional pixel PxA corresponds to the functional pixel A or B described above is determined by the arrangement of the light shielding portion 54 in the pixel opening (which portion of the pixel opening the light shielding portion 54 shields light).

  In the wiring layer 57, for example, a circuit 55 for identifying whether the pixel is the functional pixel PxA or the image pixel PxP (or the type thereof) is formed.

  In such a configuration, as can be seen from the fact that the light-shielding portion 54 is provided in the wiring layer 57, it is formed of, for example, a metal film (for example, an aluminum film) at the time of forming the wiring. It is made of a certain material. For this reason, the electric charge 59 generated by the photodiode 53 is charged, and the functional pixel PxA and the image pixel PxP have different capacitances. Thus, the characteristics of the generated dark current differ between the functional pixel PxA and the image pixel PxP.

  The pre-dark current correction unit 31 described above is for correcting such a difference in dark current characteristics between the functional pixel PxA and the image pixel PxP.

  FIG. 3 is a block diagram showing the configuration of the pre-dark current correction unit 31. As shown in FIG.

  The pre-dark current correction unit 31 is an image pixel (for example, a pixel arranged in the m line in FIG. 6) in the light-shielded pixel region (at least one of the horizontal light-shielded pixel region 15b and the vertical light-shielded pixel region 15c shown in FIG. 4). And the dark current of the functional pixel in the light-shielding pixel region (for example, the functional pixel A arranged in the (m + 1) line in FIG. 6 or the functional pixel B arranged in the (m + 2) line) in FIG. A dark current detection unit 41 that calculates a dark current difference; a dark current difference correction unit that corrects pixel signals read from the functional pixels A and B in the effective pixel region 15a based on the calculated dark current difference; It is equipped with.

  Here, the dark current detection unit 41 includes a calculation region determination unit 43 and a dark current difference calculation unit 44, and the dark current difference correction unit 42 includes a dark current period calculation unit 45 and a dark current difference subtraction unit 46. .

  The calculation area determination unit 43 determines which coordinate position of the pixel in the light-shielding pixel areas 15b and 15c is used for dark current calculation.

  First, the calculation area determination unit 43 determines which of the horizontal light-shielding pixel area 15b and the vertical light-shielding pixel area 15c is to be set as the calculation area 15d.

  Then, the calculation area determination unit 43 detects the dark currents of the image pixels and the functional pixels for the same area in the horizontal direction for the horizontal light-shielding pixel area 15b, and for the vertical light-shielding pixel area 15c in the vertical direction. The calculation area 15d is determined so that each dark current of the image pixel and the functional pixel is detected for the same area.

  In this way, the calculation area determination unit 43 matches the pixel position of the area for detecting the dark current of the image pixel with the pixel position of the area for detecting the dark current of the functional pixel in the horizontal direction or the vertical direction. The shading characteristic in the horizontal direction or the vertical direction can be canceled.

  At this time, the calculation area determination unit 43 may further divide the horizontal light-shielding pixel area 15b or the vertical light-shielding pixel area 15c into a plurality of areas and set the calculation area 15d for each divided area. Specifically, for example, when dark current is detected from the horizontal light-shielding pixel region 15b, the horizontal light-shielding pixel region 15b is divided into a plurality of partial regions having different horizontal positions (different column numbers), and divided partial regions. The calculation area 15d is determined for each of the above, and the dark current difference calculation unit 44 is caused to calculate the dark current difference of each calculation area 15d. Then, when the dark current difference correction unit 42 corrects the dark current difference in the effective pixel region 15a, the dark current difference calculated based on the calculation region 15d corresponding to the horizontal position may be used. The same applies to the detection of dark current from the vertical light-shielding pixel region 15c, except that the horizontal is replaced with the vertical.

  Further, the calculation area determination unit 43 obtains the calculation area 15d from the ROM 25, as described above, the position of the heat source with respect to the image sensor 15, the temperature of the heat source, and the drive mode of the image sensor 15. It may be determined dynamically according to at least one of them.

  For example, when there is a heat source in the vicinity of the image sensor 15 and the temperature of the heat source rises according to the usage time of the imaging device 1, the position and temperature of the heat source are set so as not to use a portion close to the heat source. The calculation area 15d may be changed accordingly. This is because the generation amount of the dark current varies depending on the temperature, so that it is not appropriate to use the dark current detected in the pixel region at a certain temperature to remove the dark current in the pixel region at another temperature.

  At this time, the calculation area determination unit 43 may also determine which of the horizontal light-shielding pixel area 15b and the vertical light-shielding pixel area 15c is used to determine the calculation area 15d according to the position and temperature of the heat source.

  In addition, examples of the drive mode of the image sensor 15 that is a factor for the calculation region determination unit 43 to dynamically determine the calculation region 15d include the magnitude of the curtain speed of the electronic shutter that may change depending on the drive mode. Specifically, for example, in the all-pixel readout mode, the curtain speed is slowed down to read out pixel signals from all horizontal lines, but in the addition readout mode or the thinning-out readout mode (or in the mode in which addition and thinning are combined). Since the number of lines output from the image sensor 15 decreases, the curtain speed increases. When reading out the pixel signal, voltage and current for reading are sequentially applied to each line. Therefore, when the curtain speed is low, a difference occurs in the dark current characteristics between the first reading line and the last reading line. there's a possibility that. Accordingly, when the curtain speed is fast, either the horizontal light-shielded pixel area 15b or the vertical light-shielded pixel area 15c may be used for determining the calculation area 15d. However, when the curtain speed is slow, the vertical light-shielded pixels extending over a plurality of lines. The calculation area determination unit 43 may determine that only the horizontal light-shielded pixel area 15b is used for determining the calculation area 15d without using the area 15c.

  The dark current difference calculation unit 44 uses the pixel value of the calculation region 15d determined by the calculation region determination unit 43, and the dark current of the functional pixels of the light shielding pixel regions 15b and 15c and the image for the light shielding pixel regions 15b and 15c. A dark current difference that is a difference between the dark current of the pixel is calculated.

Specifically, the dark current difference calculation unit 44, for example, as shown in Equation 1 below, the pixel signal Ia (k) in the k-th column of the functional pixel A in the light-shielding pixel regions 15b and 15c and the k columns of the image pixels. The dark current difference DIa between the functional pixel A and the image pixel is calculated using the pixel signal Ip (k) at the eye.
[Equation 1]

Similarly, the dark current difference calculation unit 44, for example, as shown in Equation 2 below, the pixel signal Ib (k) in the k-th column of the functional pixel B in the light-shielding pixel regions 15b and 15c and the k-th column of the image pixel. The dark current difference DIb between the functional pixel B and the image pixel is calculated using the pixel signal Ip (k) at.
[Equation 2]

  In these formulas 1 and 2, the dark current of the image pixel is calculated without distinguishing the color of the pixel. However, in consideration of the use of the dark current correction unit 32 in the subsequent stage, first, each color is calculated. It may be calculated every time. In this case, the calculated dark current of the image pixel for each color is temporarily stored for processing by the dark current correction unit 32, and then the average value of each color (weighted average value corresponding to the number of samples) Etc.) to calculate the dark current of the image pixel.

  Also, here, image pixels and functional pixels A and B for one line, that is, image pixels i to (i + n−1) for one line continuous in the horizontal direction as shown in FIG. The dark current difference was calculated using the functional pixels Ai to A (i + n−1) and the functional pixels Bi to B (i + n−1). As described later, the image pixels and the functional pixels A and B for a plurality of lines are used. May be used, the number of samples can be increased by using data for a plurality of lines to further reduce random noise.

  Alternatively, instead of using pixel signals that are continuous in the horizontal direction, the dark current difference may be calculated using pixels having the following arrangement.

  That is, as shown in FIG. 5, the functional pixels A in the effective pixel area 15a are arranged at a ratio of one pixel to, for example, four pixels in the horizontal direction. Similarly, the functional pixels B in the effective pixel area 15a are also arranged in, for example, four pixels in the horizontal direction. They are arranged at a rate of one pixel. The functional pixels A in the effective pixel area 15a and the functional pixels A in the horizontal light-shielding pixel area 15b are functional pixels A arranged in the same column, but between the functional pixels A arranged in different columns. It is thought that the correlation of dark current is higher than that. Similarly, regarding the functional pixel B, it is considered that the dark current has a higher correlation when the effective pixel region 15a and the horizontal light-shielding pixel region 15b are arranged in the same column.

  Therefore, for the horizontal light-shielding pixel region 15b, the dark current detection unit 41 detects each dark current of the functional pixels A and B at the same pixel position as the horizontal pixel position of the functional pixels A and B in the effective pixel region 15a. Then, the dark current difference calculation unit 44 may calculate each dark current difference from the dark current of the image pixel.

  Similarly, with regard to the effective pixel region 15a and the vertical light-shielding pixel region 15c, it is considered that the pixels arranged in the same row have higher dark current correlation. Therefore, for the vertical light-shielding pixel region 15c, the dark current detection unit 41 detects each dark current of the functional pixels A and B at the same pixel position as the vertical pixel position of the functional pixels A and B in the effective pixel region 15a. Then, the dark current difference calculation unit 44 may calculate each dark current difference from the dark current of the image pixel.

  In this case, the functional pixels in the light-shielded pixel regions 15b and 15c are arranged at least at the same pixel position as the horizontal pixel position of the functional pixels in the effective pixel region 15a when arranged in the horizontal light-shielded pixel region 15b. In the case of being arranged in the vertical light-shielding pixel region 15c, it is sufficient that it is arranged at least at the same pixel position as the vertical pixel position of the functional pixel in the effective pixel region 15a (therefore, as described above, It does not have to be a continuous arrangement, but it does not preclude a continuous arrangement).

  The dark current cycle calculation unit 45 acquires the functional pixel arrangement information and the drive mode information of the image sensor 15 from the ROM 25, and is in the addition readout mode or the thinning readout mode (or in the mode in which addition and thinning are combined). In the pixel signal read out after addition or after thinning, the dark current component of the functional pixel is included in what period and at what ratio, and the dark current difference calculated by the dark current difference calculation unit 44 is determined. It is calculated whether the correction should be performed at such a ratio.

  The dark current difference subtracting unit 46 calculates the dark current difference of the ratio for each pixel calculated by the dark current period calculating unit 45 as a pixel signal (a pixel signal of a functional pixel or a pixel of a functional pixel) read from the effective pixel region 15a. Subtraction from an addition signal mixed with pixel signals).

  The processes of the dark current cycle calculation unit 45 and the dark current difference subtraction unit 46 will be described later in detail with reference to FIGS.

  FIG. 8 is a flowchart showing still image shooting processing in the imaging apparatus 1.

  For example, when the still image shooting process is started from the main process (not shown) of the image pickup apparatus 1 when the shooting button is pressed while the mechanical shutter 13 is opened and the live view is performed, first, the CPU 27 is as described above. The exposure condition is calculated and set in each part (step S1).

  Next, the drive unit 14 drives the focus lens in the lens 11 to adjust the focus so that the focus position detected by the focus detection unit 22 is reached (step S2).

  Subsequently, exposure of the image is started by closing and opening the mechanical shutter 13 once or by completing the pixel reset of the image sensor 15 (step S3).

  When the exposure time determined by the shutter speed has elapsed from the start of exposure in step S3, the mechanical shutter 13 is closed or the charge of each pixel of the image sensor 15 is transferred from the photodiode to the memory in the image sensor 15. This completes the exposure (step S4).

  When the exposure is completed, the pixel signals of the image pixels in the horizontal light-shielding pixel region 15b and the vertical light-shielding pixel region 15c are read from the image sensor 15, processed by the A-AMP 16 and ADC 17, and stored in the DRAM 19 (step S5).

  Similarly, the pixel signals of the functional pixels in the horizontal light-shielding pixel region 15b and the vertical light-shielding pixel region 15c are read from the image sensor 15, processed by the A-AMP 16 and ADC 17, and stored in the DRAM 19 (step S6). As described above, the pixel signals of the functional pixels in the light-shielding pixel regions 15b and 15c and the pixel signals of the image pixels are acquired independently.

  Further, the pixel signal of each pixel in the effective pixel area 15a is read from the image sensor 15, processed by the A-AMP 16 and ADC 17, and then stored in the DRAM 19 (step S7).

  Note that the processing of steps S5 to S7 is not limited to the order described above, and may be performed in an appropriate order.

  Then, the image processing unit 21 performs image processing as will be described later with reference to FIG. 9 (step S8).

  Thereafter, the CPU 27 creates a file header including the dark current information of the image pixels and the dark current information of the functional pixels as necessary, and further creates an image file including the created file header and image data ( Step S9).

  Here, the image file to be recorded includes, for example, a JPEG file subjected to image processing for recording by the image processing unit 21 and subjected to, for example, JPEG compression, and a RAW image file including RAW image data. is there.

  Of these, in the case of a JPEG file or the like, since it is an image file including image data from which dark current has already been removed, recording of dark current information is not essential (however, recording of dark current information is hindered). Not a thing).

  On the other hand, in the case of a RAW image file, since it is premised on that an application is executed later on a device such as a personal computer to perform image processing (so-called RAW development processing), dark current information is used as, for example, a file header. (However, it is not always necessary to include it in the file header, and other types of accompanying information may be used.)

  At this time, as the RAW image data to be recorded, for example, the following two types of RAW image data can be considered.

  First, the first RAW image data is RAW image data that has only been converted into a digital signal by the ADC 17. In this case, the pixel signal read from the effective pixel region 15a included in the RAW image data still has a difference in dark current characteristics between the image pixel and the functional pixel. Accordingly, at this time, information on dark currents of image pixels in the light-shielded pixel regions 15b and 15c (for example, an average value of dark current of image pixels) and information on dark currents of functional pixels in the light-shielded pixel regions 15b and 15c ( For example, the dark current average value of the functional pixels) is included in the file header in a format according to, for example, the Exif file format and recorded as a RAW image file. However, instead of recording the dark current information of the image pixel and the dark current information of the functional pixel, the dark current information of the image pixel and the dark current difference information of the image pixel and the functional pixel are recorded. You may make it.

  Next, the second RAW image data is RAW image data that has been pre-dark current corrected by the pre-dark current correcting unit 31 after being converted into a digital signal by the ADC 17. The RAW image data includes a pixel signal read from the image pixel in the effective pixel area 15a and a pixel signal of the functional pixel in the effective pixel area 15a corrected by the dark current difference correction unit 42. Therefore, the difference in dark current characteristics between the image pixel and the functional pixel has already been corrected. Therefore, at this time, it is sufficient to record the dark current information of the image pixels in the light-shielded pixel regions 15b and 15c (for example, the average value of the dark current of the image pixels) as a RAW image file by including it in the file header. .

  When the second RAW image data is adopted, not only the dark current information of the functional pixels and the dark current information of the image pixels need not be recorded in the RAW image file, but also the dark current of the functional pixels is not necessary. Since the conventional RAW development processing that does not distinguish between the image current and the dark current of the image pixel can be applied as it is, there is an advantage that the versatility of the image file is enhanced.

  Thereafter, after the created image file is recorded in the flash memory 3 (step S10), the process returns to the main process (not shown).

  FIG. 9 is a flowchart showing details of the image processing in step S8 of FIG.

  When this process is started, the pre-dark current correction unit 31 performs a pre-dark current correction process as will be described later with reference to FIG. 10 (step S21).

  Next, as described above, the dark current correction unit 32 applies the image pixels R, Gr, Gb, and the image data of the effective pixel region 15a (see FIG. 4) corrected by the pre dark current correction unit 31. Dark current correction for reducing dark current noise detected as a dark current component of an image pixel without distinguishing between a pixel signal read from B and a pixel signal read from functional pixels A and B Processing is performed (step S22).

  Subsequently, as described above, the WB correction processing unit 33 performs white balance correction processing (step S23), the synchronization processing unit 34 performs synchronization processing (step S24), and the luminance characteristic conversion unit 35 performs image luminance. Component gradation conversion processing is performed (step S25), the edge enhancement processing unit 36 performs edge enhancement processing (step S26), the NR processing unit 37 performs noise reduction processing (step S27), and the color reproduction processing unit 38 An image color reproduction process is performed (step S28), and the process returns to the process shown in FIG.

  FIG. 10 is a flowchart showing details of the pre-dark current correction process in step S21 of FIG.

  When the pre-dark current correction unit 31 starts this processing, first, the calculation region determination unit 43 first determines the calculation region 15d as described above, for example, in the light-shielding pixel region 15b, and the image pixel is determined from the determined calculation region 15d. The pixel value and the pixel value of the functional pixel are acquired (step S31).

  Next, the average pixel value of the image pixel line (for example, m line shown in FIG. 6) in the calculation area 15d and the average of the functional pixel A and B lines (for example, (m + 1), (m + 2) line shown in FIG. 6). The difference from the pixel value is calculated as shown in Equations 1 and 2 (step S32). At this time, as described above, pixel values of regions having the same horizontal pixel position or vertical pixel position are used.

  Subsequently, for example, the mixing ratio information is acquired from the DRAM 19 (step S33). This mixing ratio information is information indicating at what mixing ratio the pixel signal of the functional pixel A or the pixel signal of the functional pixel B is included in a signal that is output after pixel addition or pixel thinning. is there. The mixing ratio information may be information calculated at any time based on the functional pixel arrangement information stored in advance in the ROM 25 and the currently set addition reading mode and the like and stored in the DRAM 19. Information stored in advance as table data in the ROM 25 at the time of manufacture or the like may be used.

  Here, the mixing ratio information will be described with reference to FIGS.

  FIG. 11 is a diagram illustrating an example of 2 × 2 pixel addition to the pixel Gb in the effective pixel region 15a, and FIG. 12 is a diagram illustrating an example of an addition signal obtained by 2 × 2 pixel addition. 11 to 14, the pixel R is hatched in the vertical direction, the pixels Gr and Gb are hatched from the upper left to the lower right, and the pixel B is hatched in the horizontal direction. 11 to 14 show a row direction number and a column direction number in units of 2 × 2 pixel blocks in which a pixel R is arranged at the upper left, a pixel Gr at the upper right, a pixel Gb at the lower left, and a pixel B at the lower right. And attached.

  Since pixel addition is performed between pixels of the same color, in the case of 2 × 2 pixel addition, an addition signal of a 2 × 2 pixel block is obtained from a 4 × 4 pixel block. Specifically, in the example of 2 × 2 pixel addition for the pixel Gb as shown in FIG. 11, 2 × 2 = 4 pixels Gb in the addition area AA22 are added.

  That is, for example, (row direction number, column direction number) = (1,1) functional pixel A in the block, (1,2) image pixel Gb in the block, and (2,1) image in the block. Pixel addition of the pixel Gb and the image pixel Gb in the (2, 2) block is performed to generate an addition signal. Therefore, in this case, as shown in the block (1, 1) in FIG. 12, the pixel signal of the functional pixel A is mixed for one pixel in the addition signal. Here, in FIG. 12 and FIG. 14 to be described later, the numbers given after A or B indicate how many pixel signals of the functional pixel A or the pixel signal of the functional pixel B are mixed.

  In addition, functional pixels are included in the pixels R, Gr, and B in the (1,1) block, (1,2) block, (2,1) block, and (2,2) block shown in FIG. Therefore, there is no mixing of the pixel signals of the functional pixels as shown in the block (1, 1) in FIG.

  Similarly, when 2 × 2 pixel addition of each pixel shown in FIG. 11 is performed, an addition signal as shown in FIG. 12 is obtained. As shown in FIG. 12, information indicating how many functional pixels A and B are mixed at which pixel position in the addition signal is mixing ratio information. In addition, in the addition of 2 × 2 = 4 pixels, for example, as a mixing rate information when one functional pixel A is included, it is of course possible to use a mixing ratio of 1/4 instead of the number of mixing pixels of 1. .

  Next, FIG. 13 is a diagram showing an example of 3 × 3 pixel addition to the pixel Gb in the effective pixel region 15a, and FIG. 14 is a diagram showing an example of an addition signal obtained by 3 × 3 pixel addition. Note that FIG. 13 shows an example in which the block line of row direction number 1 is not subject to addition output due to the adjustment of the number of pixels of the output addition signal.

  In the case of this 3 × 3 pixel addition, a 2 × 2 pixel block addition signal is obtained from the 6 × 6 pixel block. Specifically, in the example of 3 × 3 pixel addition to the pixel Gb as shown in FIG. 13, 3 × 3 = 9 pixels Gb in the addition area AA33 are added.

  That is, for example, (2,1) block, (2,2) block, (2,3) block, (3,2) block, (4,1) block, (4,2) block, and (4,3) ) Pixel addition of the image pixel Gb in the block and the functional pixel B in the (3, 1) block and the (3, 3) block is performed to generate an addition signal. Therefore, in this case, as shown in the block (1, 1) in FIG. 14, the pixel signal of the functional pixel B is mixed for two pixels in the addition signal.

  Further, regarding the pixels R, Gr, and B in the same block as described above, since the functional pixels are not included, the pixel signals of the functional pixels are mixed as shown in the block (1, 1) in FIG. Absent.

  Similarly, when 3 × 3 pixel addition of each pixel is performed, an addition signal as shown in FIG. 14 is obtained. In the case of this 3 × 3 pixel addition, it can be seen that there are pixels in which both the functional pixel A and the functional pixel B are mixed in the addition signal.

  FIG. 15 is a chart showing how many functional pixels A are mixed in the addition signal obtained by 3 × 3 pixel addition, and FIG. 16 shows what functional pixel B is in the addition signal obtained by 3 × 3 pixel addition. It is a chart which shows whether it is mixed for pixels.

  The information as shown in FIGS. 15 and 16 is the mixing ratio information. Note that, for example, instead of the number of mixed pixels 1, 1/9 which is a mixing ratio may be used, as described above.

  When such pixel addition is performed in the image sensor 15 and an addition signal is output, pixel addition is performed on the light-shielded pixel regions 15b and 15c as well as the effective pixel region 15a, and an addition signal is output. become.

  Accordingly, FIG. 6 shows an example in which the image pixel, the functional pixel A, and the functional pixel B are provided for each line. However, the driving is performed not only in the all-pixel readout mode but also in the addition readout mode and the thinning readout mode. In the case of the image pickup device 15, the image pixels and the functional pixels in the light-shielding pixel regions 15 b and 15 c are image pixel signals in which the pixel signals of the functional pixels are not mixed in any of a plurality of operable drive modes. In addition, functional pixel signals that do not include pixel signals of image pixels are arranged so as to be output.

  Specifically, the 2 × 2 pixel addition readout mode in which three lines continuous for one color as shown in FIG. 11 are targeted, and the 3 × 3 in which five lines continuous for one color as shown in FIG. 13 are targeted. In order to be able to operate in both the pixel addition readout mode, since the larger number of lines is 5, five or more continuous lines in which only image pixels are arranged, and functional pixels For example, the horizontal light-shielding pixel region 15b is provided with five or more continuous lines in which only A is arranged and five or more lines in which only the functional pixel B is arranged at an arrangement interval corresponding to a plurality of driving modes. It will be.

  The dark current detection unit 41 detects the dark current of the image pixels in the light-shielded pixel regions 15b and 15c based on the image pixel signal into which the pixel signal of the functional pixel is not mixed, and does not mix the pixel signal of the image pixel. Based on the functional pixel signal, the dark current of the functional pixels in the light-shielding pixel regions 15b and 15c is detected, and the dark current difference calculation unit 44 determines the dark current based on the detected dark current of the image pixel and the dark current of the functional pixel. Calculate the current difference.

  Thereafter, the dark current difference correction unit 42 reads out the addition signal to be processed in the effective pixel region 15a from the DRAM 19 (however, the pixel signal to be processed in the all-pixel reading mode), and the mixture ratio information acquired in step S33. Based on the above, it is determined whether or not the read addition signal includes the pixel signal of the functional pixel (step S34).

  Here, when it is determined that the pixel signal of the functional pixel is included, the dark current cycle calculation unit 45, for example, from the mixing rate information that is table data as illustrated in FIG. 15 or FIG. The mixing ratio applied to the addition signal to be processed is acquired (step S35).

  Then, the dark current difference subtraction unit 46 calculates the dark current difference DI to be subtracted based on the mixing ratio, for example, as shown in the following Equation 3 (step S36).

In other words, the dark current difference subtraction unit 46 determines that the darkness between the functional pixel A and the image pixel described above when the number of mixed pixels is mm, the number of mixed functional pixels A is ma, and the number of mixed functional pixels B is mb. Using the current difference DIa and the dark current difference DIb between the functional pixel B and the image pixel, the dark current difference DI to be subtracted from the addition signal is calculated based on the following Equation 3.
[Equation 3]

  Here, for example, mm = 4 in the 2 × 2 pixel addition readout mode, and mm = 9 in the 3 × 3 pixel addition readout mode. In the all-pixel readout mode, mm = 1, and when the processing target is the functional pixel A, ma = 1 and mb = 0, and when the processing target is the functional pixel B, ma = 0 and mb = 1. It ’s fine.

  Then, the dark current difference subtraction unit 46 performs dark current difference correction by subtracting the dark current difference DI from the addition signal to be processed (or the pixel signal to be processed) (step S37).

  Specifically, when 2 × 2 pixel addition is performed, when, for example, one functional pixel A is mixed in the addition signal read from the effective pixel region 15a, the dark current difference DIa calculated by Equation 1 is calculated. For example, 1/4 is subtracted from the addition signal.

  Similarly, when 3 × 3 pixel addition is performed, when, for example, two functional pixels A and one functional pixel B are mixed in the addition signal read from the effective pixel region 15a, the calculation is performed using Equation 1. Then, 2/9 of the dark current difference DIa is subtracted from the addition signal, and 1/9 of the dark current difference DIb calculated by Equation 2 is subtracted from the addition signal.

  However, Equation 3 assumes that pixel addition is performed by addition averaging. When pixel addition is performed by simple addition, for example, division by the number of mixed pixels mm is not necessary.

  Subsequently, it is determined whether or not the processing of all the addition signals (or pixel signals) in the effective pixel region 15a has been completed (step S38), and if not completed yet, the process proceeds to step S34 to perform the next addition. The signal is processed as described above.

  Thus, when it is determined in step S38 that all the added signals have been processed, the process returns to the process shown in FIG.

  In the above description, the dark current difference is calculated in step S36 each time the loop of steps S34 to S38 is performed. However, the present invention is not limited to this. That is, the types of mixing ratios that can appear in the drive mode set for the image sensor 15 in operation are relatively limited, and are about several to several tens of types (for example, the drive mode in operation is shown in FIG. 12). In the case of the 2 × 2 pixel addition readout mode shown in FIG. 13, about two kinds of A1 and B1, and in the case of the 3 × 3 pixel addition readout mode shown in FIG. 13, A1, B1, A2, B2, A1B1, A1B2, and A2B1 , About 8 types of A2B2. Therefore, for example, the dark current difference subtracting unit 46 performs the calculation of the dark current difference DI to be subtracted as described above for all the mixture ratios that can appear in the driving mode during operation in advance. You may make it abbreviate | omit the calculation for every rotation. In this case, the dark current difference subtraction unit 46 selects the dark current difference DI corresponding to the mixing ratio of the addition signal to be processed from the plurality of calculated dark current differences DI and corrects the dark current difference. Will do.

  In the above description, in consideration of the utilization efficiency of the existing system configuration in the image pickup apparatus 1 and the versatility of the image file, after correcting the dark current difference of the functional pixels by the pre-dark current correction unit 31 in the previous stage, in the subsequent stage. The dark current correction unit 32 performs the same dark current correction as before without distinguishing between functional pixels and image pixels. However, for example, when correcting the dark current without dividing the former stage and the latter stage, it is determined whether the processing target pixel is a functional pixel or an image pixel, and dark current correction is performed based on the determination result. It is possible to correct the dark current appropriately.

  According to the first embodiment, in the light-shielding pixel regions 15b and 15c, the image pixel having the same structure as the image pixel in the effective pixel region 15a, the functional pixel having the same structure as the functional pixel in the effective pixel region 15a, And the dark current detection unit 41 detects the dark current of the image pixels in the light-shielded pixel regions 15b and 15c and the dark current of the functional pixels in the light-shielded pixel regions 15b and 15c. It becomes possible to detect each dark current of the image pixel and the functional pixel in the pixel region 15a with high accuracy.

  At this time, since the functional pixels in the light-shielding pixel regions 15b and 15c are arranged in a different rule from the functional pixels in the effective pixel region 15a, the dark current of the functional pixels is reduced without depending on the arrangement of the functional pixels in the effective pixel region 15a. It can be easily acquired and detected.

  In particular, when the image pixels and the functional pixels are continuously arranged without any gap in the horizontal direction within the arrangement range of the horizontal light-shielding pixel region 15b, the pixel signal of the image pixel and the pixel signal of the functional pixel are It can be acquired independently in a short time according to reading. At this time, the average value of the dark current can be easily calculated.

  In addition, when the image pixels and the functional pixels are continuously arranged without any gap in the vertical direction within the arrangement range of the vertical light-shielding pixel region 15c, for example, dark current is acquired without changing the vertical address. Is possible.

  The calculation region determination unit 43 of the dark current detection unit 41 detects each dark current of the image pixel and the functional pixel for the horizontal light-shielding pixel region 15b in the same region in the horizontal direction (positions in the vertical direction are different). In addition, for the vertical light-shielding pixel region 15c, in order to determine the calculation region so as to detect each dark current of the image pixel and the functional pixel with respect to the same region in the vertical direction (the position in the horizontal direction is different), The shading characteristics in the horizontal direction or the vertical direction can be canceled with high correlation and high accuracy.

  When the calculation area determination unit 43 determines the calculation area according to at least one of the position of the heat source with respect to the image sensor 15, the temperature of the heat source, and the drive mode of the image sensor 15, the dark current is determined. It is possible to accurately detect the dark current by removing the cause of the error in detection.

  The dark current detection unit 41 detects the dark current of the functional pixel at the same pixel position as the horizontal pixel position of the functional pixel in the effective pixel region 15a for the horizontal light-shielded pixel region 15b, and is effective for the vertical light-shielded pixel region 15c. Since the dark current of the functional pixel at the same pixel position as the vertical pixel position of the functional pixel in the pixel area 15a is detected, the dark current of the functional pixel in the effective pixel area 15a is detected by the functional pixel in the light-shielding pixel area. Based on the dark current, it is possible to detect with higher correlation.

  The dark current difference calculation unit 44 of the dark current detection unit 41 has a dark current difference that is a difference between the dark current of the functional pixels in the light shielding pixel regions 15b and 15c and the dark current of the image pixels in the light shielding pixel regions 15b and 15c. Therefore, the difference in dark current between the functional pixel and the image pixel can be easily corrected.

  The dark current difference correction unit 42 mixes the dark current difference calculated by the dark current difference calculation unit 44 and the image pixel and the functional pixel in the pixel signal read from the effective pixel region 15a according to the drive mode. Accordingly, the dark current difference to be subtracted from the pixel signal is calculated accordingly, so that even when the dark current difference component in the addition signal periodically changes in the addition readout mode, the thinning readout mode, etc. Regardless of the mode, it is possible to accurately subtract the component resulting from the dark current difference between the image pixel and the functional pixel from the addition signal.

  Since the dark current difference correction unit 42 corrects the dark current difference between the functional pixel and the image pixel, image quality deterioration due to the dark current difference can be reduced.

  The pixel signals read from the image pixels in the effective pixel area 15a, the pixel signals of the functional pixels in the effective pixel area 15a corrected by the dark current difference correction unit 42, and the image pixels in the light-shielded pixel areas 15b and 15c. When the dark current information is recorded as a RAW image file, there is no need to consider the dark current difference in the RAW development process executed by the image processing apparatus or the like, and there is an advantage that versatility as a RAW image file is increased. is there.

  The pixel signals read from the image pixels and functional pixels in the effective pixel area 15a, the dark current information of the image pixels in the light-shielded pixel areas 15b and 15c, and the darkness of the functional pixels in the light-shielded pixel areas 15b and 15c. When the current information is recorded as a RAW image file, it is possible to correct the dark current difference correctly in the RAW development process executed by the image processing apparatus or the like.

  Then, the dark current correction unit 32 corrects the dark current of the image pixels in the effective pixel region 15a based on the dark current of the image pixels, and also corrects the effective pixel region 15a corrected by the dark current difference correction unit 42. Since it suffices to correct the dark current of the functional pixel, it is not necessary to distinguish whether the pixel in the effective pixel region 15a is an image pixel or a functional pixel in the dark current correction process, and the same as in the conventional case. Dark current correction processing can be performed.

  In this way, according to the present embodiment, it is possible to reduce the influence of the dark current difference caused by the difference in pixel structure while making it possible to use a configuration for correcting the existing dark current.

  In the above description, the imaging apparatus is mainly described. However, an image processing method for performing image processing similar to that of the imaging apparatus may be used, and a processing program for causing a computer to execute the image processing method and the processing program are recorded. It may be a non-temporary recording medium readable by a computer.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope in the implementation stage. In addition, various aspects of the invention can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, you may delete some components from all the components shown by embodiment. Furthermore, the constituent elements over different embodiments may be appropriately combined. Thus, it goes without saying that various modifications and applications are possible without departing from the spirit of the invention.

1 ... Imaging device 2 ... LCD / TV
DESCRIPTION OF SYMBOLS 3 ... Flash memory 11 ... Lens 12 ... Diaphragm 13 ... Mechanical shutter 14 ... Drive part 15 ... Image pick-up element 15a ... Effective pixel area 15b ... Horizontal shading pixel area 15c ... Vertical shading pixel area 15d ... Calculation area 16 ... A-AMP
17 ... ADC
18 ... Bus 19 ... DRAM
DESCRIPTION OF SYMBOLS 21 ... Image processing part 22 ... Focus detection part 23 ... Imaging control part 24 ... Video encoder 25 ... ROM
26 ... Operation unit 27 ... CPU
DESCRIPTION OF SYMBOLS 31 ... Pre dark current correction part 32 ... Dark current correction part 33 ... WB correction process part 34 ... Synchronization process part 35 ... Luminance characteristic conversion part 36 ... Edge emphasis process part 37 ... NR process part 38 ... Color reproduction process part 41 ... Dark current detection unit 42 ... Dark current difference correction unit 43 ... Calculation area determination unit 44 ... Dark current difference calculation unit 45 ... Dark current period calculation unit 46 ... Dark current difference subtraction unit 51 ... Light shielding film 52 ... Micro lens 53 ... Photodiode 54 ... Light-shielding part 55 ... Circuit 56a ... Transparent filter (white filter)
56p ... color filter 57 ... wiring layer 59 ... charge

Claims (12)

An image pixel for image formation is arranged in an area where matrix pixels are arranged in a matrix in the horizontal direction and the vertical direction, and an effective pixel area in which functional pixels used other than image formation are arranged, and is arranged around the effective pixel area. An image sensor having an image pixel having the same structure as the image pixel in the effective pixel area, and a light-shielded pixel area in which a functional pixel having the same structure as the functional pixel in the effective pixel area is arranged to be shielded from light. ,
A dark current detector for detecting dark current of image pixels in the light-shielded pixel region and dark current of functional pixels in the light-shielded pixel region;
An imaging apparatus comprising: The functional pixels in the effective pixel region are discretely arranged in a regular cycle rule in the horizontal direction and the vertical direction,
The imaging device according to claim 1, wherein the functional pixels in the light-shielding pixel region are arranged according to a different rule from the functional pixels in the effective pixel region. The light-shielding pixel region includes at least one of a horizontal light-shielding pixel region disposed around the effective pixel region in the horizontal direction and a vertical light-shielding pixel region disposed around the effective pixel region in the vertical direction. And
When the image pixels and the functional pixels in the light-shielding pixel region are arranged in the horizontal light-shielding pixel region, they are continuously arranged without any gap in the horizontal direction within the arrangement range, and are arranged in the vertical light-shielding pixel region. 3. The imaging device according to claim 2, wherein the imaging device is continuously arranged in the vertical direction without a gap in the arrangement range.   The dark current detection unit detects the dark currents of the image pixels and the functional pixels for the same region in the horizontal direction for the horizontal light-shielded pixel region, and the same region in the vertical direction for the vertical light-shielded pixel region. The imaging apparatus according to claim 3, further comprising a calculation area determination unit that determines a calculation area so as to detect each dark current of the image pixel and the functional pixel.   The calculation area determination unit determines the calculation area according to at least one of a position of a heat source with respect to the image sensor, a temperature of the heat source, and a drive mode of the image sensor. The imaging device according to claim 4. The light-shielding pixel region includes at least one of a horizontal light-shielding pixel region disposed around the effective pixel region in the horizontal direction and a vertical light-shielding pixel region disposed around the effective pixel region in the vertical direction. And
When the functional pixels in the light-shielding pixel region are arranged in the horizontal light-shielding pixel region, the functional pixels in the effective pixel region are arranged at least at the same pixel position as the horizontal pixel position, and the vertical light-shielding pixel region Is disposed at least at the same pixel position as the pixel position in the vertical direction of the functional pixel in the effective pixel region,
The dark current detection unit detects a dark current of a functional pixel at the same pixel position as a horizontal pixel position of the functional pixel of the effective pixel region for the horizontal light-shielded pixel region, and for the vertical light-shielded pixel region, The imaging apparatus according to claim 2, wherein a dark current of a functional pixel at the same pixel position as a vertical pixel position of the functional pixel in the effective pixel region is detected. The dark current detection unit includes a dark current difference calculation unit that calculates a dark current difference that is a difference between a dark current of a functional pixel in the light-shielded pixel region and a dark current of an image pixel in the light-shielded pixel region. ,
The imaging apparatus according to claim 1, further comprising a dark current difference correction unit that corrects a pixel signal read from the functional pixel in the effective pixel region based on the dark current difference. The imaging element is configured to be operable in a plurality of drive modes including a drive mode for performing at least one of pixel addition and pixel thinning.
The image pixel and the functional pixel in the light-shielding pixel region are an image pixel signal in which the pixel signal of the functional pixel is not mixed and a functional pixel in which the pixel signal of the image pixel is not mixed in any of a plurality of operable drive modes. It is arranged so that the signal can be output,
The dark current detection unit detects a dark current of an image pixel in the light-shielded pixel region based on an image pixel signal in which a pixel signal of the functional pixel is not mixed, and a functional pixel in which the pixel signal of the image pixel is not mixed Based on the signal, the dark current of the functional pixel in the light-shielding pixel region is detected,
The dark current difference correction unit includes the dark current difference calculated by the dark current difference calculation unit, and an image pixel and a functional pixel in a pixel signal read from the effective pixel region according to the drive mode. The imaging apparatus according to claim 7, wherein a dark current difference to be subtracted from the pixel signal is calculated according to the mixing ratio.   The pixel signal read from the image pixel in the effective pixel area, the pixel signal of the functional pixel in the effective pixel area corrected by the dark current difference correction unit, and the dark current of the image pixel in the light-shielded pixel area The image pickup apparatus according to claim 7, wherein the information is recorded as a RAW image file.   Pixel signals read from the image pixels and functional pixels in the effective pixel area, information on dark current of image pixels in the light-shielded pixel area, and information on dark current of functional pixels in the light-shielded pixel area, The image pickup apparatus according to claim 1, wherein the image pickup apparatus is recorded as a RAW image file.   Based on the dark current of the image pixel in the light-shielded pixel region detected by the dark current detection unit, the dark current of the image pixel in the effective pixel region is corrected, and after correction by the dark current difference correction unit The imaging apparatus according to claim 7, further comprising a dark current correction unit that corrects a dark current of a functional pixel in the effective pixel region. An image pixel for image formation is arranged in an area where matrix pixels are arranged in a matrix in the horizontal direction and the vertical direction, and an effective pixel area in which functional pixels used other than image formation are arranged, and is arranged around the effective pixel area. An image sensor having an image pixel having the same structure as the image pixel in the effective pixel area, and a light-shielded pixel area in which a functional pixel having the same structure as the functional pixel in the effective pixel area is shielded from light. An image processing method for processing an output image signal,
A dark current detection step of detecting, from the image signal, a dark current of an image pixel in the light-shielded pixel region and a dark current of a functional pixel in the light-shielded pixel region;
Dark current difference calculation step for calculating a dark current difference which is a difference between the dark current of the functional pixel in the light-shielded pixel region detected by the dark current detection step and the dark current of the image pixel in the light-shielded pixel region. When,
A dark current difference correction step of correcting a pixel signal read from the functional pixel in the effective pixel region based on the dark current difference;
An image processing method comprising:

Images