JP6452354B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus.

  2. Description of the Related Art In recent years, imaging apparatuses such as electronic cameras that can acquire not only light intensity distribution but also light incident direction and distance information are known.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique capable of pupil focus type focus detection in an imaging apparatus. In Patent Document 1, one pixel has two photodiodes (hereinafter, these are referred to as divided pixels), and each photodiode receives light that has passed through different pupils of the photographing lens by one microlens. Accordingly, by comparing the output signal waveforms from the two photodiodes, it is possible to acquire the pupil division phase difference AF and the distance detection image. Moreover, a normal captured image can be obtained by adding the output signals from the two photodiodes.

  Patent Document 2 discloses a technique capable of detecting a focus by a so-called light travel time method or TOF (Time of Flight) method in an imaging apparatus. In Patent Document 2, one pixel has two floating diffusions and two transfer switches for one photodiode. Then, in synchronization with the pulse timing of the projection light, the two transfer switches are alternately opened and closed to distribute the charge generated by the reflected light from one photodiode to two floating diffusions. The distance to the subject can be estimated from the charge distribution ratio.

JP 2001-124984 A International Publication No. 2007/026777

  However, the focus detection of the pupil division method can speed up the focus detection compared to the contrast method using the contrast of the subject, but it is difficult to detect the optical field in situations where the depth of field is deep or the periphery of the image. It is difficult to obtain a pupil division phase difference depending on various conditions. For this reason, it may be difficult for pupil division type focus detection to perform good focus detection or acquisition of a distance detection image. On the other hand, in the light travel time method, distance information can be acquired in units of pixels, but measurement is difficult when the projection light does not reach a long-distance subject or a subject with low reflectivity.

  The objective of this invention is providing the imaging device which can perform the focus detection suitable according to a condition.

The imaging device of the present invention includes an imaging device having first and second unit pixels for converting light into electric charge, a light emitting device for projecting light onto a subject, and the first and second within one frame period. the performing accumulation of the unit pixels of the signal and a driving unit that drives the image pickup device so as to read, before SL focus detection of an object by the pupil division phase difference detection method using an output signal of the first unit pixel based on the time until receiving the reflected light by the second unit pixel from the projecting light to said object by said light emitting device by using the first focus detection unit, the pre-SL output signal of the second unit pixel A second focus detection unit that performs focus detection of a subject by a TOF (Time of Flight) method; a generation unit that generates a captured image using an output signal of the first unit pixel and the second unit pixel; Have The features.

  It is possible to perform appropriate focus detection according to various optical conditions.

It is a block diagram which shows the structural example of an imaging device. It is a pixel arrangement | positioning figure of the image pick-up element in 1st Embodiment. It is a conceptual diagram in which the light beam emitted from the exit pupil of the photographing lens enters the unit pixel. It is a schematic diagram which shows the structural example in the unit pixel in 1st Embodiment. It is a circuit diagram which shows the structural example in the unit pixel for phase difference detection. It is a circuit diagram which shows the structural example in the unit pixel for TOF focus detection. It is a figure explaining the read-out circuit of the image sensor in a 1st embodiment. It is a timing chart which shows the drive method of the pixel for phase difference detection. It is a timing chart of the 1st drive method of the pixel for a TOF focus detection. It is a timing chart of the 2nd drive method of the pixel for a TOF focus detection. It is a schematic diagram explaining the pulse timing which concerns on a TOF system. It is a flowchart explaining an imaging operation. It is a read timing chart explaining the first focus detection drive method. It is a read-out timing chart explaining the 2nd focus detection drive method. It is a pixel arrangement | positioning figure of the image pick-up element in 2nd Embodiment. It is a schematic diagram which shows the structural example in the unit pixel in 2nd Embodiment. It is a circuit diagram which shows the structural example in the normal pixel unit for imaging. It is a figure explaining the read-out circuit of the image pick-up element in 2nd Embodiment. It is a timing chart which shows the drive method of a normal imaging pixel. It is a read timing chart explaining the first focus detection drive method. It is a read-out timing chart explaining the 2nd focus detection drive method. It is a pixel arrangement | positioning figure of the image pick-up element in 3rd Embodiment. It is a schematic diagram which shows the structural example in the unit pixel in 3rd Embodiment. It is a circuit diagram which shows the structural example in the unit pixel in 3rd Embodiment. It is a figure explaining the read-out circuit of the image pick-up element in 3rd Embodiment. It is a timing chart which shows the drive method of the pixel for phase difference detection. It is a timing chart which shows the drive method of the pixel for a TOF focus detection. It is a timing chart which shows the drive method of a normal imaging pixel.

(First embodiment)
FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 100 according to the first embodiment of the present invention. The light that has passed through the photographing lens 101 forms an image in the vicinity of the focal position of the photographing lens 101 via the lens diaphragm 204. The microlens array 102 has a plurality of microlenses 1020 and is arranged in the vicinity of the focal position of the photographing lens 101, so that light that has passed through different pupil regions of the photographing lens 101 is divided and emitted for each pupil region. . The image sensor 103 is a solid-state image sensor such as a CMOS image sensor or a CCD image sensor, and is arranged so that a plurality of pixels correspond to one microlens 1020. The image sensor 103 receives the light emitted after being divided for each pupil region by the microlens 1020 while maintaining the division information, and converts the light into image signals that can be processed.

  An analog signal processing circuit (AFE) 104 performs correlated double sampling processing, signal amplification, reference level adjustment, analog / digital conversion processing, and the like on the image signal output from the image sensor 103. A digital signal processing circuit (DFE) 105 performs digital image processing such as reference level adjustment on the image signal output from the analog signal processing circuit 104.

  The image processing circuit 106 performs correlation calculation and focus detection of A and B images, which will be described later, and predetermined image processing and defect correction for the image signal output from the digital signal processing circuit 105. The memory circuit 107 and the recording circuit 108 are recording media such as a nonvolatile memory or a memory card that records and holds an image signal output from the image processing circuit 106.

  The control circuit 109 comprehensively drives and controls the entire imaging apparatus such as the photographing lens 101, the imaging element 103, the image processing circuit 106, the operation circuit 110, the display circuit 111, and the light emitting device 112. The operation circuit 110 inputs a signal from an operation member provided in the imaging apparatus 100 and outputs a user command to the control circuit 109. The display circuit 111 displays an image after shooting, a live view image, various setting screens, and the like. The light emitting device 112 projects light on the subject in accordance with a signal PLIGHT (FIGS. 8 to 11) from the control circuit 109.

  Next, the relationship between the photographing lens 101, the microlens array 102, and the image sensor 103, the definition of pixels, and the principle of focus detection by the pupil division method will be described.

  FIG. 2 is a diagram of the image sensor 103 and the microlens 1020 observed from the direction of the optical axis Z in FIG. In the present embodiment, one unit pixel 200 is provided for one microlens 1020. In FIG. 2, the image sensor 103 has 8 × 8 = 64 unit pixels 200. Each unit pixel 200 represents an address of the X coordinate and the Y coordinate as (X, Y). The unit pixel 200 converts light into electric charge. Here, the unit pixels 200 arranged in six rows with Y coordinates of 0, 1, 2, 4, 5, and 6 include two divided pixels 201A and 201B for one microlens 1020. The divided pixels 201A and 201B are arranged in the X-axis direction. Further, the unit pixels 200 arranged in two rows with Y coordinates of 3 and 6 have one pixel for one microlens 1020. Six rows of unit pixels 200 composed of two divided pixels 201A and 201B are used for focus detection of the pupil division phase difference detection method, and two rows of unit pixels 200 composed of one pixel are TOF-type focus points. Used for detection.

  FIG. 3 is a diagram in which the light emitted from the photographing lens 101 passes through one microlens 1020 and is received by the unit pixel 200 as observed from the direction perpendicular to the optical axis Z (Y-axis direction). . Pupil areas 202 and 203 represent exit pupil areas of the photographic lens 101. The light that has passed through the pupil regions 202 and 203 enters the unit pixel 200 with the optical axis Z as the center. The light beam passing through the pupil region 202 is received by the divided pixel 201A through the micro lens 1020, and the light beam passing through the pupil region 203 is received by the divided pixel 201B through the micro lens 1020. Accordingly, the divided pixels 201A and 201B receive light from different pupil regions 202 and 203 of the photographing lens 101, respectively.

  A subject image formed by the output signal group of the divided pixels 201A in the plurality of unit pixels 200 is defined as an A image. Similarly, a subject image formed by the output signal group of the divided pixels 201B in the plurality of unit pixels 200 is defined as a B image.

  The image processing circuit 106 performs a correlation operation on the A image and the B image, and detects an image shift amount (pupil division phase difference). Further, the image processing circuit 106 can calculate a focal position corresponding to an arbitrary subject position in the image by multiplying the image shift amount by a conversion coefficient determined by the focal position of the photographing lens 101 and the optical system. it can. The control circuit 109 can perform the pupil division phase difference AF by controlling the focus of the photographing lens 101 based on the focal position calculated here. Further, the image processing circuit 106 can use the A + B image signal for a normal captured image by adding the A image signal and the B image signal to the A + B image signal.

  FIG. 5 is a circuit diagram showing a configuration example of the unit pixel 200 having Y coordinates of 0, 1, 2, 4, 5, and 6 in FIG. 2 and is used for focus detection in the pupil division phase difference detection method. The unit pixel 200 is shown. The rows with Y coordinates of 0, 1, 2, 4, 5, and 6 are rows in which a plurality of first unit pixels 200 in FIG. 5 are arranged. The first unit pixel 200 includes a first photodiode 301A and a second photodiode 301B. The first photodiode 301A corresponds to the divided pixel 201A, and the second photodiode 301B corresponds to the divided pixel 201B. A transfer switch 302A is connected to the first photodiode 301A, and a transfer switch 302B is connected to the second photodiode 301B. A floating diffusion 303A is connected to the transfer switch 302A, and a floating diffusion 303B is connected to the transfer switch 302B. A reset switch 304A and a source follower amplifier 305A are connected to the floating diffusion 303A. A reset switch 304B and a source follower amplifier 305B are connected to the floating diffusion 303B. A select switch 306A is connected to the source follower amplifier 305A. A select switch 306B is connected to the source follower amplifier 305B. The drains of the reset switches 304A and 304B and the source follower amplifiers 305A and 305B are connected to a reference potential (VDD) node 308.

  The photodiodes 301 </ b> A and 301 </ b> B are photoelectric conversion units that receive light that has passed through the same microlens 1020 and generate electric charges according to the amount of received light. The transfer switch 302A transfers the charge generated in the first photodiode 301A to the floating diffusion 303A in response to the transfer pulse signal PTXA. The transfer switch 302B transfers the charge generated in the second photodiode 301B to the floating diffusion 303B in response to the transfer pulse signal PTXB. The floating diffusions 303 </ b> A and 303 </ b> B are charge-voltage conversion units that temporarily hold charges and convert the held charges into voltages.

  The reset switch 304A resets the potential of the floating diffusion 303A to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The reset switch 304B resets the potential of the floating diffusion 303B to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The source follower amplifier 305A is a source follower circuit including a MOS transistor and a reference potential node 308. The source follower amplifier 305A amplifies a voltage based on the electric charge held in the floating diffusion 303A and outputs it as a pixel signal. The source follower amplifier 305B is a source follower circuit including a MOS transistor and a reference potential node 308. The source follower amplifier 305B amplifies a voltage based on the electric charge held in the floating diffusion 303B and outputs it as a pixel signal.

  The select switch 306A outputs the pixel signal amplified by the source follower amplifier 305A to the vertical output line 307A according to the select pulse signal PSEL. The select switch 306B outputs the pixel signal amplified by the source follower amplifier 305B to the vertical output line 307B according to the select pulse signal PSEL. The vertical output lines 307A and 307B are shared by a plurality of unit pixels 200 in the same column as shown in FIG.

  FIG. 6 is a circuit diagram showing a configuration example of the unit pixel 200 with Y coordinates of 3 and 6 in FIG. 2, and shows a second unit pixel 200 used for TOF focus detection. The rows with Y coordinates 3 and 6 are rows in which a plurality of second unit pixels 200 in FIG. 6 are arranged. The second unit pixel 200 has one photodiode 301. The photodiode 301 corresponds to a pixel. Two transfer switches 302C and 302D are connected to the photodiode 301. A floating diffusion 303C is connected to the transfer switch 302C. A floating diffusion 303D is connected to the transfer switch 302D. A reset switch 304C and a source follower amplifier 305C are connected to the floating diffusion 303C. A reset switch 304D and a source follower amplifier 305D are connected to the floating diffusion 303D. A select switch 306C is connected to the source follower amplifier 305C. A select switch 306D is connected to the source follower amplifier 305D. The drains of the reset switches 304C and 304D and the source follower amplifiers 305C and 305D are connected to a reference potential (VDD) node 308.

  The photodiode 301 is a photoelectric conversion unit that receives light that has passed through the microlens 1020 and generates a charge corresponding to the amount of light received. The transfer switch 302C transfers charges generated in the photodiode 301 to the floating diffusion 303C in response to the transfer pulse signal PTXC. The transfer switch 302D transfers the charge generated in the photodiode 301 to the floating diffusion 303D in response to the transfer pulse signal PTXD. The floating diffusions 303 </ b> C and 303 </ b> D are charge-voltage conversion units that temporarily hold charges and convert the held charges into voltages.

  The reset switch 304C resets the potential of the floating diffusion 303C to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The reset switch 304D resets the potential of the floating diffusion 303D to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The source follower amplifier 305C is a source follower circuit including a MOS transistor and a reference potential node 308. The source follower amplifier 305C amplifies a voltage based on the electric charge held in the floating diffusion 303C and outputs it as a pixel signal. The source follower amplifier 305D is a source follower circuit including a MOS transistor and a reference potential node 308. The source follower amplifier 305D amplifies a voltage based on the charge held in the floating diffusion 303D and outputs it as a pixel signal.

  The select switch 306C outputs the pixel signal amplified by the source follower amplifier 305C to the vertical output line 307A according to the select pulse signal PSEL. The select switch 306D outputs the pixel signal amplified by the source follower amplifier 305D to the vertical output line 307B according to the select pulse signal PSEL. The vertical output lines 307A and 307B are shared by a plurality of unit pixels 200 in the same column as shown in FIG.

  FIG. 4A corresponds to FIG. 5 and is a layout diagram of the unit pixel 200 with the Y coordinate of 0, 1, 2, 4, 5, 6 in FIG. 2, and is used for focus detection in the pupil division phase difference detection method. The 1st unit pixel 200 used is shown. The first unit pixel 200 includes a first photodiode 301A and a second photodiode 301B that share a single microlens 1020. The first photodiode 301A is connected to the transfer switch 302A. The second photodiode 301B is connected to the transfer switch 302B. The transfer switch 302A is connected to the floating diffusion 303A. The transfer switch 302B is connected to the floating diffusion 303B.

  FIG. 4B is a layout diagram of the unit pixel 200 corresponding to FIG. 6 and having a Y coordinate of 3 and 6 in FIG. 2, and shows the unit pixel 200 used for TOF focus detection. A photodiode 301 is provided for one microlens 1020. Two transfer switches 302C and 302D are connected to both ends of the photodiode 301. The transfer switch 302C is connected to the floating diffusion 303C. The transfer switch 302D is connected to the floating diffusion 303D.

  FIG. 7 is a diagram illustrating a configuration example of the image sensor 103, the vertical shift register 401, the column circuit 403, the horizontal shift register 404, and the output amplifier 407. In the image sensor 103, a plurality of unit pixels 200 are arranged in a matrix. In FIG. 7, a total of 12 unit pixels 200 in 4 rows and 3 columns are illustrated, but in reality, the image sensor 103 has millions to tens of millions of unit pixels 200. The unit pixels 200 are arranged according to a Bayer array, and are provided with red (R), green (G), and blue (B) color filters, respectively. Here, for the purpose of increasing the light receiving efficiency of the infrared light used as the projection light of the light emitting device 112 and the reflected light, pixels formed with an infrared filter or a transparent filter may be arranged. The vertical shift register 401 selects and drives a row of unit pixels 200 via a signal line 402 connected to the unit pixels 200 of each row. Signals converted by the floating diffusions 303A to 303D of the unit pixel 200 are input to the column circuit 403 through the vertical output lines 307A and 307B, respectively. The signal processed by the column circuit 403 is transferred to the output amplifier 407 through the horizontal output lines 405 and 406 by the horizontal shift register 404.

  Next, the configuration of the column circuit 403 will be described. The clamp capacitor 408 is connected to the vertical output line 307A or 307B. The feedback capacitor 409 is connected between the output terminal and the input terminal of the operational amplifier 410. The reference voltage node 411 supplies a reference voltage to the operational amplifier 410. The switch 412 is a switch for short-circuiting both ends of the feedback capacitor 409 in accordance with the reset signal PC0R. Capacitors 413 and 414 are capacitors for holding voltage. The switch 415 writes the output voltage of the operational amplifier 410 to the capacitor 413 according to the signal PTS. The switch 416 writes the output voltage of the operational amplifier 410 to the capacitor 414 in accordance with the signal PTN. The switch 417 transfers the voltage of the capacitor 413 to the horizontal output line 405 in response to the signal PHS from the horizontal shift register 404. The switch 418 transfers the voltage of the capacitor 414 to the horizontal output line 406 in response to the signal PHN from the horizontal shift register 404. The output amplifier 407 outputs the voltage difference between the horizontal output lines 405 and 406.

  FIG. 8 is a timing chart illustrating a first driving method of the imaging apparatus. The first driving method is a driving method for performing pixel signal reading in normal imaging and focus detection using the pupil division phase difference detection method. In normal shooting, an image signal is generated. In focus detection using the pupil division phase difference detection method, focus detection using the pupil division phase difference detection method is performed. In the first driving method, the charges generated in the two photodiodes 301A and 301B are read out by the two floating diffusions 303A and 303B, respectively.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304A and 304B are turned on, and the floating diffusions 303A and 303B are reset. At time t1, the signals PTXA and PTXB become high level, the transfer switches 302A and 302B are turned on, and the photodiodes 301A and 301B are reset. At time t2, the signals PTXA and PTXB become low level, the transfer switches 302A and 302B are turned off, and the charge accumulation periods of the photodiodes 301A and 301B start. The photodiodes 301A and 301B generate charges by photoelectric conversion.

  Thereafter, at time t3, the signal PSEL becomes high level, the select switches 306A and 306B are turned on, and the source follower amplifiers 305A and 305B are in an operating state. At time t4, the signal PRES goes low, the reset switches 304A and 304B are turned off, and the reset of the floating diffusions 303A and 303B is released. The source follower amplifiers 305A and 305B output the voltages based on the charges of the floating diffusions 303A and 303B to the column circuit 403 via the vertical output lines 307A and 307B as reset signal levels (noise components), respectively. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signals PTXA and PTXB become high level, and the transfer switches 302A and 302B are turned on. Then, the electric charges accumulated in the photodiodes 301A and 301B are transferred to the floating diffusions 303A and 303B, respectively. At time t10, the signals PTXA and PTXB become low level, the transfer switches 302A and 302B are turned off, and the charge transfer from the photodiodes 301A and 301B to the floating diffusions 303A and 303B ends. That is, the charge accumulation period of the photodiodes 301A and 301B ends. The source follower amplifiers 305A and 305B output the voltages corresponding to the charges of the floating diffusions 303A and 303B as optical signal levels (light component + noise component) to the column circuit 403 via the vertical output lines 307A and 307B, respectively. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written in the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. After that, at time t15, the signal PRES goes high, the reset switches 304A and 304B are turned on, and the floating diffusions 303A and 303B are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, in the case of normal shooting, the image processing circuit 106 may add a signal based on the charges of the photodiodes 301A and 301B to obtain a shot image. On the other hand, in the case of focus detection using the pupil division phase difference detection method, the image processing circuit 106 performs correlation calculation on the above-described A image and B image, and acquires distance information. Also in this case, the image processing circuit 106 may add the signals of the A image and the B image after acquiring the distance information.

  FIG. 9 is a timing chart illustrating a second driving method of the imaging apparatus. The second driving method is a driving method for reading pixel signals for normal photographing without using light emission by the light emitting device 112 for the unit pixel 200 of FIG. In the second driving method, electric charges generated in one photodiode 301 are transferred to two floating diffusions 303C and 303D.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304C and 304D are turned on, and the floating diffusions 303C and 303D are reset. At time t1, the signals PTXC and PTXD become high level, the transfer switches 302C and 302D are turned on, and the photodiode 301 is reset. At time t2, the signals PTXC and PTXD become low level, the transfer switches 302C and 302D are turned off, and the charge accumulation of the photodiode 301 is started.

  After that, at time t3, the signal PSEL becomes high level, the select switches 306C and 306D are turned on, and the source follower amplifiers 305C and 305D are in the operating state. At time t4, the signal PRES goes low, the reset switches 304C and 304D are turned off, and the reset of the floating diffusions 303C and 303D is released. The source follower amplifiers 305C and 305D output the voltage based on the charges of the floating diffusions 303C and 303D to the column circuit 403 via the vertical output lines 307A and 307B as reset signal levels (noise components), respectively. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signals PTXC and PTXD become high level, the transfer switches 302C and 302D are turned on, and the charges accumulated in the photodiode 301 are transferred to the floating diffusions 303C and 303D. At time t10, the signals PTXC and PTXD become low level, the transfer switches 302C and 302D are turned off, and the charge transfer from the photodiode 301 to the floating diffusions 303C and 303D is completed. The source follower amplifiers 305C and 305D output voltages based on the charges of the floating diffusions 303C and 303D as optical signal levels (light component + noise component) to the column circuit 403 via the vertical output lines 307A and 307B, respectively. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. Thereafter, at time t15, the signal PRES goes high, the reset switches 304C and 304D are turned on, and the floating diffusions 303C and 303D are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, in the case of normal shooting, the image processing circuit 106 adds signals based on the charges of the floating diffusions 303C and 303D and uses them as a shot image.

  FIG. 10 is a timing chart illustrating a third driving method of the imaging apparatus. The third driving method is a driving method for performing pixel signal readout in TOF type focus detection. In the third driving method, electric charges generated in one photodiode 301 are transferred to two floating diffusions 303C and 303D.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304C and 304D are turned on, and the floating diffusions 303C and 303D are reset. At time t1, the signals PTXC and PTXD become high level, the transfer switches 302C and 302D are turned on, and the photodiode 301 is reset. At time t2, the signals PTXC and PTXD become low level, the transfer switches 302C and 302D are turned off, and charge accumulation in the photodiode 301 starts.

  After that, at time t3, the signal PSEL becomes high level, the select switches 306C and 306D are turned on, and the source follower amplifiers 305C and 305D are in the operating state. At time t4, the signal PRES goes low, the reset switches 304C and 304D are turned off, and the reset of the floating diffusions 303C and 303D is released. The source follower amplifiers 305C and 305D output the voltage based on the charges of the floating diffusions 303C and 303D to the column circuit 403 via the vertical output lines 307A and 307B as reset signal levels (noise components), respectively. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signal PTXC becomes high level, the transfer switch 302C is turned on, and the charge accumulated in the photodiode 301 starts to be transferred to the floating diffusion 303C. Thereafter, at time t9, the signal PLIGHT becomes high level, and the light emitting device 112 starts projecting pulsed light. At time t10, the signal PTXC becomes low level, and at the same time, the signal PTXD becomes high level. Then, the transfer switch 302C is turned off, the charge transfer from the photodiode 301 to the floating diffusion 303C is completed, the transfer switch 302D is turned on, and the charge transfer from the photodiode 301 to the floating diffusion 303D is started. At time t11, the signal PLIGHT becomes a low level, and the light emitting device 112 finishes projecting the pulsed light. At time t12, the signal PTXD becomes low level, the transfer switch 302D is turned off, and the charge transfer from the photodiode 301 to the floating diffusion 303D is completed.

  The source follower amplifiers 305C and 305D output voltages based on the charges of the floating diffusions 303C and 303D as optical signal levels (light component + noise component) to the column circuit 403 via the vertical output lines 307A and 307B, respectively. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. Thereafter, at time t15, the signal PRES goes high, the reset switches 304C and 304D are turned on, and the floating diffusions 303C and 303D are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, as will be described later, the image processing circuit 106 calculates a delay time of the reflected light with respect to the projection light based on a ratio of signals based on the charges of the floating diffusions 303C and 303D, and acquires a distance to the subject. Further, the image processing circuit 106 may add signals based on the charges of the floating diffusions 303C and 303D.

  By the way, in the pupil division phase difference method of FIG. 8, it may be difficult to detect the phase difference due to various optical conditions. Therefore, focus detection by the TOF method of FIG. 10 is performed. FIG. 11 shows a part of the timing chart of FIG. 10 and explains the principle of TOF focus detection.

  At time t8, the signal PTXC becomes high level, the transfer switch 302C is turned on, and the charge accumulated in the photodiode 301 starts to be transferred to the floating diffusion 303C. Thereafter, at time t9, the signal PLIGHT becomes high level, and the light emitting device 112 starts projecting pulsed light. At time t10, the signal PTXC becomes low level, and at the same time, the signal PTXD becomes high level. Then, the transfer switch 302C is turned off, the charge transfer from the photodiode 301 to the floating diffusion 303C is completed, the transfer switch 302D is turned on, and the charge transfer from the photodiode 301 to the floating diffusion 303D is started. At time t11, the signal PLIGHT becomes a low level, and the light emitting device 112 finishes projecting the pulsed light. At time t12, the signal PTXD becomes low level, the transfer switch 302D is turned off, and the charge transfer from the photodiode 301 to the floating diffusion 303D is completed.

  Here, the ON time of the transfer switch 302C by the signal PTXC is equal to the ON time of the transfer switch 302D by the signal PTXD. The pulsed light projected by the light emitting device 112 is reflected with respect to the subject. If the distance from the imaging device to the subject is zero, the reflected light is received in synchronization with the pulse light projection period of the signal PLIGHT, and the signals based on the charges of the floating diffusions 303C and 303D are the same.

  However, when the distance from the imaging device to the subject is not zero, the reflected light is received with a delay of (t9'-t9) with respect to the pulse light projection period by the signal PLIGHT as shown in FIG. As a result, the floating diffusion 303C accumulates charges corresponding to the reflected light received during the period (t10-t9 '), and the floating diffusion 303D stores charges corresponding to the reflected light received during the period (t11'-t10). Will accumulate. There is a bias in the amount of charge stored in the floating diffusions 303C and 303D. The image processing circuit 106 calculates the distance to the subject based on the phase difference between the light projected by the light emitting device 112 and the light received by the unit pixel 200 in FIG. That is, the image processing circuit 106 can estimate the delay time of the reflected light with respect to the projection light based on the ratio of the signals based on the charges of the floating diffusions 303C and 303D, and the subject based on the product of the delay time and the speed of light. Can be calculated.

  Based on the above, the imaging operation of the imaging apparatus will be described using the flowchart shown in FIG. In step S <b> 801, the control circuit 109 performs mode setting such as still image shooting and moving image shooting, and shooting condition settings such as sensitivity and aperture value, according to the operation of the operation circuit 110 by the user.

  In step S802, the control circuit 109 determines whether the luminance of the subject is higher than the first threshold value. If the luminance of the subject is higher than the first threshold value, the process proceeds to step S804. If the luminance of the subject is lower than the first threshold, the process proceeds to step S803.

  In step S803, the control circuit 109 determines whether the distance to the subject, which is the focus detection result in the previous frame, is closer than the second threshold value. If the distance to the subject is closer than the second threshold, the process proceeds to step S805. If the distance to the subject is greater than the second threshold, the process proceeds to step S804.

  In step S804, the unit pixel 200 in FIG. 5 reads out a signal by the first driving method (pupil division phase difference detection type focus detection) in FIG. 8 that is not accompanied by light emission by the light emitting device 112, and the unit pixel 200 in FIG. The signal is read out by the second driving method of FIG. Thereafter, the process proceeds to step S806.

  In step S805, the unit pixel 200 in FIG. 5 reads the signal by the first driving method in FIG. 8, and the unit pixel 200 in FIG. 6 performs the third driving method (TOF in FIG. The signal is read out by system focus detection. Thereafter, the process proceeds to step S806.

  In step S806, the image processing circuit 106 calculates the distance to the subject by the pupil division phase difference detection method based on the signal in step S804, and calculates the distance to the subject by the TOF method based on the signal in step S805. To do. Specifically, when passing through step S804, the first focus detection unit of the image processing circuit 106 detects the subject based on the output signal of the unit pixel 200 in FIG. 5 without the light emitting device 112 of step S804 projecting light. Get (calculate) information about the distance to. When passing through step S805, the second focus detection unit of the image processing circuit 106 provides information on the distance to the subject based on the output signal of the unit pixel 200 in FIG. 6 when the light emitting device 112 in step S805 projects light. Is obtained (calculated).

  Further, the image processing circuit 106 performs predetermined image processing on the image signal read in step S804 or S805, and outputs the image signal to the memory circuit 107, the recording circuit 108, and the display circuit 111.

  In step S807, the control circuit 109 determines whether or not shooting has ended. If continuing, the process proceeds to step S808, and if finished, the series of operations is terminated.

  In step S808, the control circuit 109 calculates a lens driving amount based on the distance to the subject obtained in step S806. In step S809, the control circuit 109 performs focus driving by driving the photographing lens 101 based on the calculated lens driving amount. Then, it returns to step S801 and repeats the said operation | movement.

  If the determinations in steps S802 and S803 cannot be made in the first frame, the process proceeds to step S804 after step S801. That is, in the first process of FIG. 12, the process proceeds from step S801 to step S804. In the second processing thereafter, the determinations in steps S802 and S803 are performed based on the first output of the unit pixel 200 in FIG. That is, in the second and subsequent processes, the determinations in steps S802 and S803 are performed based on the output signal of the unit pixel 200 of the previous frame. Note that the determination in step S802 may be performed based on the luminance measured by the photometry unit.

  Next, the readout timing of the image sensor 103 will be described with reference to FIGS. FIG. 13 is a read timing chart for explaining the first and second driving methods in step S804 of FIG. In FIG. 13, periods H00 to H15 divided by dotted lines represent the readout periods of the respective rows. In addition, the pixel address of the notation is the same as the address of the pixel arrangement described with reference to FIG. In the present embodiment, reading is performed by a rolling method in which each row is read sequentially. Regarding the order of the rows to be read, first, six rows of the unit pixel 200 in FIG. 5 capable of focus detection by the pupil division phase difference detection method are read out in ascending order of the Y coordinate address, and then the focus detection by the TOF method is possible. Two rows of unit pixels 200 in FIG. 6 are read in ascending order of Y coordinate address. At this time, for example, a period in which the unit pixel (0, 0) accumulates a signal is a period from H00 to H07, and a period in which the unit pixel (0, 3) accumulates a signal is a period from H06 to H13.

  FIG. 14 is a read timing chart for explaining the first and third driving methods in step S805 of FIG. Also in FIG. 14, as in FIG. 13, the order of the rows to be read is first to read out the rows of the unit pixel 200 in FIG. 5 capable of focus detection by the pupil division phase difference detection method in ascending order of the Y coordinate address. . Subsequently, two rows of the unit pixel 200 in FIG. 6 capable of TOF focus detection are read in ascending order of the Y coordinate address.

  Further, light emission by the light emitting device 112 used for the focus detection of the TOF method is performed in the periods of H05, H06, H13, and H14 in accordance with the signal PLIGHT. At this time, for example, the period in which the unit pixels (0, 0) in the row of the unit pixels 200 in FIG. 5 capable of focus detection by the pupil division phase difference detection method accumulate charges is a period from H00 to H07.

  On the other hand, the period in which the unit pixels (0, 3) in the row of the unit pixel 200 of FIG. 6 capable of detecting the focus of the TOF method accumulate charges is the period of H06 to H13 in FIG. In FIG. 14, in order to suppress as much as possible the influence of external light other than the reflected light from the subject of light emission by the light emitting device 112, by resetting the signal of the photodiode 301 in the period H12 immediately before the light emission in the period H13, The charge accumulation period is only H13.

  The charge accumulation period of the unit pixel 200 in FIG. 5 capable of focus detection by the pupil division phase difference detection method and the charge accumulation period of the unit pixel 200 in FIG. 6 capable of TOF type focus detection are different from each other. The charge accumulation period of the unit pixel 200 in FIG. 6 in which the TOF focus detection is possible includes a period during which the light emitting device 112 projects light.

  As described above, the image sensor 103 can perform both focus detection by the pupil division phase difference method and focus detection by the TOF method, and even when focus detection by one focus detection method is difficult, The focus detection result of the other focus detection method can be used. This makes it possible to perform appropriate focus detection regardless of the shooting environment and the state of the subject.

  In the above description, the pixels for calculating the distance are performed without distinction of the color filters. However, only the pixels of the infrared filter may be used according to the color of the projection light, for example, the infrared light, or more light may be used. In order to capture, transparent filter or G filter pixels may be used.

(Second Embodiment)
Next, an imaging apparatus according to the second embodiment of the present invention will be described. In the first embodiment, all unit pixels 200 are unit pixels capable of focus detection, whereas in the second embodiment, unit pixels dedicated for imaging are mixed.

  FIG. 15 is a view of the image sensor 103 and the microlens 1020 observed from the direction of the optical axis Z in FIG. The image sensor 103 has 8 × 8 = 64 unit pixels 200. Each unit pixel 200 is expressed as (X, Y) with addresses of X coordinate and Y coordinate. Here, in the present embodiment, the unit pixel 200 arranged in the two rows of Y coordinates 0 and 4 is provided with two divided pixels 201A and 201B for one microlens 1020. In addition, in the unit pixels 200 arranged in six rows of Y coordinates 1, 2, 3, 5, 6, and 7, one pixel is provided for one microlens 1020. A unit pixel 200 composed of two divided pixels 201A and 201B is used for focus detection in the pupil division phase difference detection method. Among the unit pixels 200 composed of one pixel, the unit pixels 200 with Y coordinates of 3 and 7 are used for TOF focus detection, and the unit pixels 200 with Y coordinates of 1, 2, 5, and 6 are It is not used for focus detection but only for imaging.

  The unit pixel 200 whose Y coordinate is 0 or 4 has the configuration shown in FIG. The unit pixel 200 having Y coordinates of 3 and 7 has the configuration shown in FIG.

  FIG. 17 is a circuit diagram illustrating a configuration example of the unit pixel 200 having Y coordinates of 1, 2, 5, and 6 in FIG. The unit pixel 200 has one photodiode 301. A transfer switch 302E is connected to the photodiode 301. A floating diffusion 303E is connected to the transfer switch 302E. A reset switch 304E and a source follower amplifier 305E are connected to the floating diffusion 303E. A select switch 306E is connected to the source follower amplifier 305E. The drains of the reset switch 304E and the source follower amplifier 305E are connected to a reference potential (VDD) node 308.

  The photodiode 301 is a photoelectric conversion unit that receives light that has passed through the microlens 1020 and generates a charge corresponding to the amount of light received. The transfer switch 302E transfers the charge generated in the photodiode 301 to the floating diffusion 303E in response to the transfer pulse signal PTXE. The floating diffusion 303E is a charge-voltage converter that temporarily holds charges and converts the held charges into a voltage.

The reset switch 304E resets the potential of the floating diffusion 303E to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The source follower amplifier 305E is a source follower circuit including a MOS transistor and a reference potential VDD 308. The source follower amplifier 305E amplifies a voltage based on the charge held in the floating diffusion 303E and outputs it as a pixel signal.
The select switch 306E outputs the pixel signal amplified by the source follower amplifier 305E to the vertical output line 307A according to the select pulse signal PSEL. The vertical output line 307A is shared by a plurality of unit pixels 200 in the same column.

  FIG. 16A is a layout diagram of the unit pixel 200 (FIG. 5) whose Y coordinate is 0 or 4 in FIG. 15 and is the same as FIG. 4A. FIG. 16B is a layout diagram of the unit pixel 200 (FIG. 6) whose Y coordinate is 3 or 7 in FIG. 15, and is the same as FIG. 4B. The unit pixel 200 in FIG. 16A is the first unit pixel 200, and the unit pixel 200 in FIG. 16B is the second unit pixel 200.

  FIG. 16C is a layout diagram of the unit pixel 200 (FIG. 17) whose Y coordinate is 1, 2, 5, 6 in FIG. The photodiode 301 is connected to the transfer switch 302E. The transfer switch 302E is connected to the floating diffusion 303E. The unit pixel 200 in FIG. 16C is the third unit pixel 200.

  FIG. 18 is a diagram illustrating a configuration example of the image sensor 103, the vertical shift register 401, the column circuit 403, the horizontal shift register 404, and the output amplifier 407. Differences of the present embodiment (FIG. 18) from the first embodiment (FIG. 7) will be described. The image sensor 103 has a plurality of unit pixels 200 arranged in a matrix. A signal based on the charges of the floating diffusions 303A to 303E in the unit pixel 200 is input to the column circuit 403 through the vertical output line 307A or 307B. The signal processed by the column circuit 403 is transferred to the output amplifier 407 through the horizontal output lines 405 and 406 by the horizontal shift register 404.

  Next, the readout timing of signal readout from the unit pixel 200 will be described. However, the timing of signal readout from the unit pixel 200 in FIG. 16A and the unit pixel 200 in FIG. 16B is the driving of the first embodiment described with reference to FIGS. Since it is the same as the timing, the description is omitted.

  FIG. 19 is a timing chart illustrating a fourth driving method of the imaging device. The fourth driving method is a driving method for reading a pixel signal from the unit pixel 200 shown in FIGS. First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switch 304E is turned on, and the floating diffusion 303E is reset. At time t1, the signal PTXE becomes high level, the transfer switch 302E is turned on, and the photodiode 301 is reset. At time t2, the signal PTXE becomes low level, the transfer switch 302E is turned off, and the charge accumulation period of the photodiode 301 starts.

  After that, at time t3, the signal PSEL becomes high level, the select switch 306E is turned on, and the source follower amplifier 305E is activated. At time t4, the signal PRES goes low, the reset switch 304E is turned off, and the reset of the floating diffusion 303E is released. The source follower amplifier 305E outputs a voltage based on the charge of the floating diffusion 303E as a reset signal level (noise component) to the column circuit 403 via the vertical output line 307A. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signal PTXE becomes high level, the transfer switch 302E is turned on, and the charge accumulated in the photodiode 301 is transferred to the floating diffusion 303E. At time t10, the signal PTXE becomes low level, the transfer switch 302E is turned off, and the charge transfer from the photodiode 301 to the floating diffusion 303E is completed. That is, the charge accumulation period of the photodiode 301 ends. The source follower amplifier 305E outputs a voltage based on the charge of the floating diffusion 303E as an optical signal level (light component + noise component) to the column circuit 403 via the vertical output line 307A. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. Thereafter, at time t15, the signal PRES goes high, and the floating diffusion 303E is reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, the image processing circuit 106 uses a signal based on the charge of the floating diffusion 303E as a captured image. The image processing circuit 106 may add signals.

  The image processing circuit 106 does not use the output signal of the unit pixel 200 in FIG. 16C, and similarly to the first embodiment, the unit pixel 200 in FIG. 16A or the unit pixel in FIG. The distance to the subject is calculated based on the 200 output signals. The output signal of the unit pixel 200 in FIG. 16C is used to form an image signal.

  Next, the readout timing of the image sensor 103 will be described with reference to FIGS. FIG. 20 is a read timing chart illustrating the driving method in step S804 in FIG. The unit pixel 200 in FIG. 16A is driven by the first driving method in FIG. 8, the unit pixel 200 in FIG. 16B is driven by the second driving method in FIG. 9, and the unit pixel 200 in FIG. The unit pixel 200 is driven by the fourth driving method of FIG.

  In FIG. 20, periods H00 to H15 separated by dotted lines represent the readout periods of the respective rows. In addition, the pixel address of the notation is the same as the address of the pixel arrangement described with reference to FIG. In the present embodiment, reading is performed by a rolling method in which each row is read sequentially. Regarding the order of the rows to be read, first, two rows of the unit pixel 200 in FIG. 16A capable of focus detection by the pupil division phase difference detection method are read in ascending order of the Y coordinate address. Subsequently, four rows of the unit pixel 200 in FIG. 16C used only for normal imaging are read in ascending order of the Y coordinate address, and subsequently, TOF focus detection is possible in FIG. 16B. Two rows of unit pixels 200 are read out in ascending order of the Y coordinate address. At this time, for example, a period in which the unit pixel (0, 0) accumulates charges is a period from H00 to H07, and a period in which the unit pixel (0, 1) accumulates charges is a period from H02 to H09. The period in which the pixel (0, 3) accumulates charges is a period from H06 to H13.

  FIG. 21 is a read timing chart illustrating the driving method in step S805 of FIG. The unit pixel 200 in FIG. 16A is driven by the first driving method in FIG. 8, the unit pixel 200 in FIG. 16B is driven by the third driving method in FIG. 10, and the unit pixel 200 in FIG. The unit pixel 200 is driven by the fourth driving method of FIG.

  Also in FIG. 21, as in FIG. 20, the order of rows to be read is as follows. First, the rows of the unit pixels 200 in FIG. 16A capable of focus detection by the pupil division phase difference detection method are arranged in ascending order of the Y coordinate address. Read two lines. Subsequently, four rows of the unit pixel 200 in FIG. 16C used only for normal imaging are read in ascending order of the Y coordinate address, and subsequently, TOF focus detection is possible in FIG. 16B. Two rows of unit pixels 200 are read out in ascending order of the Y coordinate address. Further, light emission by the light emitting device 112 used for the focus detection of the TOF method is performed in the periods of H05, H06, H13, and H14 in accordance with the signal PLIGHT. At this time, for example, the period in which the unit pixels (0, 0) in the row of the unit pixels 200 in FIG. 16A capable of focus detection by the pupil division phase difference detection method accumulate charges is a period from H00 to H07. .

  On the other hand, the period in which the unit pixels (0, 3) in the row of the unit pixels 200 of FIG. 16B capable of detecting the focus of the TOF method accumulate charges is the period of H06 to H13 in FIG. In FIG. 21, since it is desirable to minimize the influence of external light other than the reflected light from the light emitting object emitted by the light emitting device 112, the charge of the photodiode 301 is reset in the period H12 immediately before the light emission in the period H13. Therefore, the charge accumulation period is only H13.

  Further, the period in which the unit pixel (0, 1) in FIG. 16C used only for normal imaging accumulates electric charge is the period from H02 to H09 in FIG. In FIG. 21, it is desirable to suppress reflection of reflected light from the light-emitting subject on the image by the light-emitting device 112 as much as possible. Therefore, the charge accumulation period is limited to only H07 to H09 by resetting the charge of the photodiode 301 immediately after light emission by the light emitting device 112 performed in the period H06.

  The charge accumulation period of the unit pixel 200 in FIG. 16A capable of focus detection by the pupil division phase difference detection method, the charge accumulation period of the unit pixel 200 in FIG. 16B capable of focus detection by the TOF method, The charge accumulation period of the unit pixel 200 of FIG. 16C used only for normal imaging is different. The charge accumulation period of the unit pixel 200 in FIG. 16B in which the TOF focus detection is possible includes a period during which the light emitting device 112 projects light. The charge accumulation period of the unit pixel 200 in FIG. 16C used only for normal imaging does not include a period during which the light emitting device 112 projects light.

  As described above, the photodiode 301 of the row of the unit pixel 200 in FIG. 16A, the row of the unit pixel 200 in FIG. 16C, and the row of the unit pixel 200 in FIG. Reset the charge. Thereby, the accuracy improvement of the distance measurement of a different system and acquisition of a suitable captured image can be performed simultaneously.

(Third embodiment)
Next, an imaging apparatus according to the third embodiment of the present invention will be described. In the third embodiment, the configuration of all unit pixels 200 is the same regardless of the focus detection method, and a plurality of focus detection methods are realized by changing the driving method of the unit pixels 200.

  22 is a diagram of the image sensor 103 and the microlens 1020 observed from the direction of the optical axis Z in FIG. One unit pixel 200 is provided corresponding to one microlens 1020. The image sensor 103 has 8 × 8 = 64 unit pixels 200. Each unit pixel 200 is expressed as (X, Y) with addresses of X coordinate and Y coordinate. All unit pixels 200 have the same configuration, and have two divided pixels 201A and 201B for one microlens 1020. The unit pixel 200 composed of the two divided pixels 201A and 201B is used for each row for either normal imaging, pupil division phase difference detection focus detection, or TOF focus detection.

  FIG. 24 is a circuit diagram illustrating a configuration example of the unit pixel 200. The unit pixel 200 includes a first photodiode 301F and a second photodiode 301G. Two transfer switches 302F and 302H are connected to the first photodiode 301F. Two transfer switches 302G and 302J are connected to the second photodiode 302G. Floating diffusions 303F, 303G, 303H, and 303J are connected to the transfer switches 302F, 302G, 302H, and 302J, respectively. Reset switches 304F, 304G, 304H, and 304J and source follower amplifiers 305F, 305G, 305H, and 305J are connected to the floating diffusions 303F, 303G, 303H, and 303J, respectively. Select switches 306F, 306G, 306H, and 306J are connected to the source follower amplifiers 305F, 305G, 305H, and 305J, respectively. The drains of the reset switches 304F and 304G and the source follower amplifiers 305F and 305G share a reference potential (VDD) node 308. The drains of the reset switches 304H and 304J and the source follower amplifiers 305H and 305J share the reference potential node 308.

  The photodiodes 301 </ b> F and 301 </ b> G are photoelectric conversion units that receive light that has passed through the same microlens 1020 and generate electric charges according to the amount of light received. The transfer switch 302F transfers the charge generated in the photodiode 301F to the floating diffusion 303F in response to the transfer pulse signal PTXF. The transfer switch 302H transfers the charge generated in the photodiode 301F to the floating diffusion 303H in response to the transfer pulse signal PTXH. The transfer switch 302G transfers the charge generated in the photodiode 301G to the floating diffusion 303G in response to the transfer pulse signal PTXG. The transfer switch 302J transfers charges generated in the photodiode 301G to the floating diffusion 303J in response to the transfer pulse signal PTXJ. The floating diffusions 303F, 303G, 303H, and 303J are charge-voltage conversion units that temporarily hold charges and convert the held charges into voltage signals.

  The reset switches 304F, 304G, 304H, and 304J respectively reset the potentials of the floating diffusions 303F, 303G, 303H, and 303J to the potential of the reference potential node 308 in response to the reset pulse signal PRES. The source follower amplifiers 305F, 305G, 305H, and 305J are source follower circuits each including a MOS transistor and a reference potential VDD308. The source follower amplifiers 305F, 305G, 305H, and 305J amplify voltages based on the charges held in the floating diffusions 303F, 303G, 303H, and 303J, respectively, and output them as pixel signals.

  The select switches 306F, 306G, 306H, 306J output the pixel signals amplified by the source follower amplifiers 305F, 305G, 305H, 305J to the vertical output lines 307F, 307G, 307H, 307J according to the select pulse signal PSEL. As shown in FIG. 25, the vertical output lines 307F, 307G, 307H, and 307J are shared by a plurality of unit pixels 200 in the same column.

  FIG. 23 is a layout diagram of the unit pixel 200. Two transfer switches 302F and 302H are connected to both ends of the photodiode 301F. The transfer switch 302F is connected to the floating diffusion 303F, and the transfer switch 302H is connected to the floating diffusion 303H. Two transfer switches 302G and 302J are connected to both ends of the photodiode 301G. The transfer switch 302G is connected to the floating diffusion 303G, and the transfer switch 302J is connected to the floating diffusion 303J.

  FIG. 25 is a diagram illustrating a configuration example of the image sensor 103, the vertical shift register 401, the column circuit 403, the horizontal shift register 404, and the output amplifier 407. The description of the same components in this embodiment (FIG. 25) as those in the first embodiment (FIG. 7) is omitted. The image sensor 103 has a plurality of unit pixels 200 arranged in a matrix. Signals based on charges in the floating diffusions 303F, 303G, 303H, and 303J in the unit pixel 200 are input to the column circuit 403 through the vertical output lines 307F, 307G, 307H, and 307J, respectively. The signal processed by the column circuit 403 is transferred to the output amplifier 407 through the horizontal output lines 405 and 406 by the horizontal shift register 404.

  FIG. 26 is a timing chart corresponding to FIG. 8 and illustrating a first driving method of the imaging apparatus. The first driving method is a driving method for reading pixel signals in normal imaging and pupil division phase difference detection type focus detection. In the first driving method, charges generated in the two photodiodes 301F and 301G are transferred to the floating diffusions 303F and 303G, respectively.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304F and 304G are turned on, and the floating diffusions 303F and 303G are reset. At time t1, the signals PTXF and PTXG become high level, the transfer switches 302F and 302G are turned on, and the photodiodes 301F and 301G are reset. At time t2, the signals PTXF and PTXG become low level, the transfer switches 302F and 302G are turned off, and charge accumulation of the photodiodes 301F and 301G starts. Here, the transfer switches that are turned on / off for resetting are not limited to those used for charge transfer after charge accumulation. In this example, transfer switches 302H and 302J using signals PTXH and PTXJ may be used.

  After that, at time t3, the signal PSEL becomes high level, the select switches 306F and 306G are turned on, and the source follower amplifiers 305F and 305G are in an operating state. At time t4, the signal PRES goes low, the reset switches 304F and 304G are turned off, and the reset of the floating diffusions 303F and 303G is released. The source follower amplifiers 305F and 305G output the voltage based on the charges of the floating diffusions 303F and 303G as a reset signal level (noise component) to the column circuit 403 via the vertical output lines 307F and 307G, respectively. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signals PTXF and PTXG become high level, the transfer switches 302F and 302G are turned on, and the charges accumulated in the photodiodes 301F and 301G are transferred to the floating diffusions 303F and 303G, respectively. At time t10, the signals PTXF and PTXG become low level, the transfer switches 302F and 302G are turned off, and the charge transfer from the photodiodes 301F and 301G to the floating diffusions 303F and 303G is completed. The source follower amplifiers 305F and 305G output voltages based on the charges of the floating diffusions 303F and 303G as optical signal levels (light component + noise component) to the column circuit 403 via the vertical output lines 307F and 307G, respectively. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. After that, at time t15, the signal PRES goes high, the reset switches 304F and 304G are turned on, and the floating diffusions 303F and 303G are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, when driven for normal shooting, the image processing circuit 106 may add a signal based on the charges of the photodiodes 301F and 301G to form a shot image. On the other hand, in the case of pupil division phase difference detection, the image processing circuit 106 performs correlation calculation on the above-described A image and B image, and acquires distance information. Also in this case, the image processing circuit 106 may add the signals of the A image and the B image after obtaining the distance information.

  In the timing chart of FIG. 26, a combination of F and G is used, but a combination of F and J or G and H can also be used. It is determined as appropriate according to the specifications of horizontal transfer and subsequent processing circuits.

  FIG. 27 corresponds to FIG. 10 and is a timing chart showing a third driving method of the imaging apparatus. The third driving method is a driving method for performing pixel signal reading in TOF focus detection. In the third driving method, in addition to the photodiode 301F, the photodiode 301G is also used and read simultaneously.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304F, 304G, 304H, and 304J are turned on, and the floating diffusions 303F, 303G, 303H, and 303J are reset. At time t1, the signals PTXF, PTXG, PTXH, and PTXJ become high level, the transfer switches 302F, 302G, 302H, and 302J are turned on, and the photodiodes 301F and 301G are reset. At time t2, the signals PTXF, PTXG, PTXH, and PTXJ are at a low level, the transfer switches 302F, 302G, 302H, and 302J are turned off, and charge accumulation in the photodiodes 301F and 301G starts.

  Thereafter, at time t3, the signal PSEL becomes high level, the select switches 306F, 306G, 306H, and 306J are turned on, and the source follower amplifiers 305F, 305G, 305H, and 305J are in an operating state. At time t4, the signal PRES goes low, the reset switches 304F, 304G, 304H, and 304J are turned off, and the reset of the floating diffusions 303F, 303G, 303H, and 303J is released. The source follower amplifiers 305F, 305G, 305H, and 305J output voltages based on the charges of the floating diffusions 303F, 303G, 303H, and 303J to the column circuit 403 as reset signal levels (noise components), respectively. The voltage is output to the column circuit 403 via the vertical output lines 307F, 307G, 307H, and 307J. At time t5, the signal PC0R becomes low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signals PTXF and PTXG become high level, the transfer switches 302F and 302G are turned on, and the charges accumulated in the photodiodes 301F and 301G start to be transferred to the floating diffusions 303F and 303G. Thereafter, at time t9, the signal PLIGHT becomes high level, and the light emitting device 112 starts projecting pulsed light. At time t10, the signals PTXF and PTXG become low level, the signals PTXH and PTXJ become high level, the transfer switches 302F and 302G are turned off, and the transfer switches 302H and 302J are turned on. Then, the charge transfer from the photodiodes 301F and 301G to the floating diffusions 303F and 303G ends, and the charge transfer from the photodiodes 301F and 301G to the floating diffusions 303H and 303J starts. At time t11, the signal PLIGHT becomes a low level, and the light emitting device 112 finishes projecting the pulsed light. At time t12, the signals PTXH and PTXJ become low level, the transfer switches 302H and 302J are turned off, and the charge transfer from the photodiodes 301F and 301G to the floating diffusions 303H and 303J is completed.

  The source follower amplifiers 305F, 305G, 305H, and 305J each output a voltage based on the charges of the floating diffusions 303F, 303G, 303H, and 303J to the column circuit 403 as an optical signal level (light component + noise component). The voltage is output to the column circuit 403 via the vertical output lines 307F, 307G, 307H, and 307J. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. After that, at time t15, the signal PRES goes high, the reset switches 304F, 304G, 304H, and 304J are turned on, and the floating diffusions 303F, 303G, 303H, and 303J are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  FIG. 28 is a timing chart corresponding to FIG. 19 and showing a fourth driving method of the imaging apparatus. The fourth driving method is a driving method for reading a pixel signal from the unit pixel 200 used only for normal imaging.

  First, in the period HBLK, at time t0, the signal PRES is at a high level, the reset switches 304F, 304G, 304H, and 304J are turned on, and the floating diffusions 303F, 303G, 303H, and 303J are reset. At time t1, the signals PTXF, PTXG, PTXH, and PTXJ become high level, the transfer switches 302F, 302G, 302H, and 302J are turned on, and the photodiodes 301F and 301G are reset. At time t2, the signals PTXF, PTXG, PTXH, and PTXJ become low level, the transfer switches 302F, 302G, 302H, and 302J are turned off, and charge accumulation of the photodiodes 301F and 301G starts.

  Thereafter, at time t3, the signal PSEL becomes high level, the select switches 306F, 306G, 306H, and 306J are turned on, and the source follower amplifiers 305F, 305G, 305H, and 305J are in an operating state. At time t4, the signal PRES goes low, the reset switches 304F, 304G, 304H, and 304J are turned off, and the reset of the floating diffusions 303F, 303G, 303H, and 303J is released. The source follower amplifiers 305F, 305G, 305H, and 305J output voltages based on the charges of the floating diffusions 303F, 303G, 303H, and 303J to the column circuit 403 as reset signal levels (noise components), respectively. The voltage is output to the column circuit 403 via the vertical output lines 307F, 307G, 307H, and 307J. At time t5, the signal PC0R is at a low level, the switch 412 is turned off, and the reset state of the operational amplifier 410 is released. At time t6, the signal PTN goes high, the switch 416 is turned on, and the output signal of the operational amplifier 410 is written to the capacitor 414 as the reset signal level. At time t7, the signal PTN becomes low level, the switch 416 is turned off, and writing to the capacitor 414 is completed.

  Next, at time t8, the signals PTXF, PTXG, PTXH, and PTXJ become high level, and the transfer switches 302F, 302G, 302H, and 302J are turned on. Then, charges accumulated in the photodiodes 301F and 301G are transferred to the floating diffusions 303F, 303G, 303H, and 303J. At time t10, the signals PTXF, PTXG, PTXH, and PTXJ become low level, and the transfer switches 302F, 302G, 302H, and 302J are turned off. Then, the charge transfer from the photodiodes 301F and 301G to the floating diffusions 303F, 303G, 303H, and 303J is completed. The source follower amplifiers 305F, 305G, 305H, and 305J each output a voltage based on the charges of the floating diffusions 303F, 303G, 303H, and 303J to the column circuit 403 as an optical signal level (light component + noise component). The voltage is output to the column circuit 403 via the vertical output lines 307F, 307G, 307H, and 307J. At time t13, the signal PTS goes high, the switch 415 is turned on, and the output signal of the operational amplifier 410 is written into the capacitor 413 as the optical signal level. The operational amplifier 410 amplifies and outputs with an inversion gain corresponding to the ratio of the clamp capacitor 408 and the feedback capacitor 409. At time t14, the signal PTS becomes low level, the switch 415 is turned off, and writing to the capacitor 413 is completed. After that, at time t15, the signal PRES goes high, the reset switches 304F, 304G, 304H, and 304J are turned on, and the floating diffusions 303F, 303G, 303H, and 303J are reset.

  Next, in a period HSR, during times t16 to t17, the horizontal shift register 404 sequentially outputs high-level pulse signals PHS and PHN for each column circuit 403 in each column. Then, the switches 417 and 418 of the column circuit 403 of each column are sequentially turned on, and the signals held in the capacitors 413 and 414 of the column circuit 403 of each column are output to the horizontal output lines 405 and 406, respectively. The output amplifier 407 outputs the difference between the signals of the horizontal output lines 405 and 406 as a differential signal level (light component).

  Thereafter, in order to drive for normal photographing, the image processing circuit 106 adds signals based on the charges of the floating diffusions 303F, 303G, 303H, and 303J and uses them as a photographed image.

  As described above, in this embodiment, the signal readout driving method from the unit pixel 200 is changed while all the unit pixels 200 have the same configuration. As a result, the driving method of one of the row of unit pixels that performs focus detection in the pupil division phase difference detection method, the row of unit pixels that are used only for normal imaging, and the row of unit pixels that perform focus detection in the TOF method Selection can be made for each row. A plurality of focus detection methods can be freely realized by using one image sensor 103. Thereby, appropriate focus detection can be performed regardless of the shooting environment and the state of the subject.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

DESCRIPTION OF SYMBOLS 100 Image pick-up device, 103 Image pick-up element, 106 Image processing circuit, 112 Light-emitting device, 200 Unit pixel

Claims (18)

An imaging device having first and second unit pixels for converting light into electric charge;
A light emitting device that projects light onto a subject;
Driving means for driving the image sensor so as to accumulate and read out signals of the first and second unit pixels within one frame period;
A first focus detection means for performing focus detection of an object by the pupil division phase difference detection method using an output signal of the previous SL first unit pixel,
Based on the time from the projecting front Symbol light to the subject by the light emitting device by using an output signal of the second unit pixel until receiving the reflected light by the second unit pixel TOF (Time of Flight) method Second focus detection means for detecting the focus of the subject by
Generating means for generating a captured image using output signals of the first unit pixel and the second unit pixel;
An imaging device comprising:   When the focus detection of the subject is performed by the first focus detection unit, the generation unit generates a captured image using output signals of the first unit pixel and the second unit pixel, and the second unit pixel When the focus detection unit performs focus detection of a subject, a captured image is generated using the output signal of the first unit pixel without using the output signal of the second unit pixel. Item 2. The imaging device according to Item 1.   The first focus detection unit detects the focus of the subject based on the output signal of the first unit pixel when the light emitting device does not project light, and then according to the output signal of the first unit pixel. The imaging apparatus according to claim 1, wherein focus detection of an object is performed by any one of the first focus detection unit and the second focus detection unit. Furthermore, either a first mode for performing focus detection using the first focus detection unit or a second mode for performing focus detection using the second focus detection unit is selected according to the shooting conditions. the imaging apparatus according to claim 1, characterized in that it comprises a control means for. The imaging apparatus according to claim 4 , wherein the control unit selects one of the first mode and the second mode in accordance with luminance of the subject. When the luminance of the subject is higher than a first threshold, the control means selects the first mode to detect the focus of the subject, and the luminance of the subject is lower than the first threshold The imaging apparatus according to claim 5 , wherein the second mode is selected and control is performed to detect the focus of the subject. Wherein, the distance to the subject selects the previous SL first mode when farther than the second threshold perform focus detection of the object, the distance to the subject than the second threshold value the imaging apparatus according to claim 4, wherein the controller controls to perform focus detection of pre-Symbol the subject selects the second mode when close. The first unit pixel has a plurality of photoelectric conversion units sharing a single microlens,
Said first focus detection means, the imaging apparatus according to any one of claims 1 to 7, characterized in that said plurality of focus detection of the object based on the signals of the photoelectric conversion unit. 2. The second focus detection unit performs focus detection of a subject based on a phase difference between light projected by the light emitting device and light received by the second unit pixel. The imaging device according to any one of 8 . The second unit pixel imaging device according to any one of claims 1-9, characterized in that it comprises a single photoelectric conversion unit. The second unit pixel, the image pickup apparatus according to any one of claims 1 to 9, characterized in that it comprises a plurality of photoelectric conversion portions share a single microlens. Wherein the first unit pixel and the second unit pixel, the imaging apparatus according to any one of claims 1 to 11, characterized in that each of the driving method is different. A first row in which a plurality of the first unit pixel are arranged, according to any one of claims 1 to 12, characterized in that a second row in which a plurality of the second unit pixel are arranged Imaging device. 2. The focus accumulation period of the first unit pixel is different from the charge accumulation period of the second unit pixel when the focus detection of the subject is performed by the second focus detection unit. 14. The imaging device according to any one of items 13 . The imaging device has a third unit pixels that converts light into electric charge,
The driving unit drives the imaging element to store and read signals of the first to third unit pixels within one frame period,
The generation unit generates a captured image using output signals of the first to third unit pixels ,
Said first focus detection means and the second focus detection means is any one of claims 1 to 13, characterized in that for performing focus detection of an object without using an output signal of said third unit pixel 1 The imaging device according to item . When the focus detection of the subject is performed by the second focus detection unit, the charge accumulation period of the first unit pixel, the charge accumulation period of the second unit pixel, and the charge accumulation period of the third unit pixel are different. The imaging apparatus according to claim 15 . The imaging apparatus according to claim 16 , wherein the charge accumulation period of the second unit pixel includes a period during which the light emitting device projects light. The imaging apparatus according to claim 16 , wherein the charge accumulation period of the third unit pixel does not include a period during which the light emitting device projects light.

Images