JP2021100253A - Image pickup device and image pickup apparatus - Google Patents

Abstract

To provide an image pickup device and an image pickup apparatus that perform focus detection by a pupil-division phase-difference method using infrared light and that capture a visible light image.SOLUTION: In an image pickup unit of an image pickup apparatus, a first image pickup device 31 is a so-called surface-illuminated image pickup device, which photoelectrically converts light incident on a surface side of a semiconductor substrate 50 by means of a photodiode PD, and outputs a photoelectric conversion signal via a wiring layer 51 formed on the surface side of the semiconductor substrate 50. A second image pickup device 32 using an organic photoelectric film is stacked and arranged on a surface side of the first image pickup device 31. A microlens array in which a plurality of microlenses ML are arranged in a two-dimensional manner is stacked and arranged above the second image pickup device 32 (i.e., on the opposite side of the first image pickup device 31). Each pixel of the first image pickup device 31 and the second image pickup device 32 is provided corresponding to each of the microlenses ML. A color filter CF is arranged between each pixel of the second image pickup device 32 and each microlens ML.SELECTED DRAWING: Figure 5

Description

The present invention relates to an image pickup device and an image pickup apparatus.

A photoelectric conversion film that absorbs infrared light and performs photoelectric conversion is arranged above a semiconductor substrate having a photoelectric conversion unit that photoelectrically converts visible light (RGB), and can take a visible light image and an infrared light image. An image pickup device configured to simultaneously perform the above is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-169676

With the above-mentioned conventional technique, it is not possible to perform focus detection by the pupil division phase difference method.

The imaging device of the first aspect includes an organic photoelectric film that photoelectrically converts light to generate an electric charge, a photoelectric conversion unit that photoelectrically converts light transmitted through the organic photoelectric film to generate an electric charge, and the organic photoelectric film. A wiring layer having a first wiring that outputs a signal based on the electric charge generated in the above and a second wiring that outputs a signal based on the electric charge generated by the photoelectric conversion unit, and an electric charge generated by the organic photoelectric film are output. A semiconductor substrate having a through electrode electrically connected to the first wiring is provided.
The image pickup apparatus of the second aspect is based on at least one of the image pickup device of the first aspect, the charge-based signal generated by the organic photoelectric film, and the charge-based signal generated by the photoelectric conversion unit. It is provided with a detection unit that detects a focusing position at which an image by an optical system is focused on the image pickup device.

According to the present invention, it is possible to perform focal detection by a pupil division retardation method using infrared light, and to capture a visible light image.

It is a block diagram explaining the configuration example of a digital camera. It is a figure explaining the outline of the imaging unit. (A) is a diagram for explaining an arrangement example of a color filter, (b) is a diagram for explaining the output of each pixel of the second image sensor, and (c) is a diagram for explaining the output of each pixel of the first image sensor. It is a figure explaining. In the first embodiment, (a) is a diagram illustrating an example of the semiconductor layout of the imaging unit, (b) is a cross-sectional view when the imaging unit is cut along the cutting lines AB, and (c). ) Is a cross-sectional view when the imaging unit is cut along the cutting line CD. (A) is a cross-sectional view when the imaging unit is cut along the cutting line AB, and (b) is a diagram for explaining the pixel position of the cutting line AB in the second image sensor. (A) is a cross-sectional view when the imaging unit is cut along the cutting line CD, and (b) is a diagram for explaining the pixel position of the cutting line CD in the second image sensor. FIG. 6A is an enlarged view of a part of FIG. 6A. It is a figure which illustrates the circuit structure of one pixel in the image pickup unit. It is a flowchart explaining the flow of a focus detection process. It is a flowchart explaining the flow of a shooting process. In the second embodiment, (a) is a diagram illustrating an example of the semiconductor layout of the imaging unit, (b) is a cross-sectional view when the imaging unit is cut along the cutting lines AB, and (c). ) Is a cross-sectional view when the imaging unit is cut along the cutting line CD. It is sectional drawing when the image pickup part by the modification is cut at the cutting line CD. It is a figure explaining the outline of the imaging unit by a modification.

(First Embodiment)
FIG. 1 is a diagram illustrating a configuration of a digital camera 1 according to the first embodiment of the present invention. The digital camera 1 includes a photographing optical system 10, a control unit 11, an imaging unit 12, an operation unit 13, a liquid crystal monitor 15, and a buffer memory 16. A memory card 17 is mounted on the digital camera 1.

The photographing optical system 10 is composed of a plurality of lenses, and forms a subject image on the imaging surface of the imaging unit 12. The plurality of lenses constituting the photographing optical system 10 include a focus lens driven in the optical axis direction in order to adjust the focus position. The focus lens is driven in the optical axis direction by a lens driving unit (not shown).

The control unit 11 is composed of a microprocessor and its peripheral circuits, and performs various controls of the digital camera 1 by executing a control program stored in a ROM (not shown). Further, the control unit 11 functionally has a focus detection unit 11a and an image generation unit 11b. Each of these functional units is implemented in software by the above control program. It is also possible to configure each of these functional units with an electronic circuit.

The focus detection unit 11a detects the focus adjustment state of the photographing optical system 10 based on the output signal from the image pickup unit 12. The image generation unit 11b generates a color image signal based on the output signal from the image pickup unit 12.

The subject image formed on the imaging unit 12 by the photographing optical system 10 is photoelectrically converted by the imaging unit 12. The output signal from the imaging unit 12 is converted into a digital image signal by an A / D conversion unit (not shown) and stored in the buffer memory 16.

The digital image signal stored in the buffer memory 16 is displayed on the liquid crystal monitor 15 or stored in the memory card 17 after various image processing is performed. The memory card 17 is composed of a non-volatile flash memory or the like, and is removable from the digital camera 1.

The operation unit 13 is composed of various operation buttons such as a release button, a mode switching button, and a power button, and is operated by the photographer. The operation unit 13 outputs an operation signal corresponding to the operation of each of the above operation buttons by the photographer to the control unit 11.

<Explanation of imaging unit>
FIG. 2 is a diagram showing an outline of the imaging unit 12 according to the present embodiment. Note that FIG. 2 shows a state in which the light incident side of the imaging unit 12 is on the upper side. Therefore, in the following description, the direction of the image pickup unit 12 on the light incident side is referred to as "upward" or "up", and the direction opposite to the light incident side is referred to as "downward" or "downward". The image pickup unit 12 has a first image pickup element 31 and a second image pickup element 32 arranged on the same optical path on the light incident side (upper side) of the first image pickup element 31. The second image sensor 32 is composed of an organic photoelectric film that absorbs infrared light (photoelectric conversion). Visible light that has not been absorbed (photoelectrically converted) by the second image sensor 32 passes through the second image sensor 32, enters the first image sensor 31, and is photoelectrically converted by the first image sensor 31. The first image sensor 31 performs photoelectric conversion by a photodiode. The second image sensor 32 and the first image sensor 31 are formed on the same semiconductor substrate, and each pixel position has a one-to-one correspondence. For example, the pixels in the first row and first column of the second image sensor 32 correspond to the pixels in the first row and first column of the first image sensor 31.

FIG. 3A shows the arrangement of the color filter CF provided above the second image sensor 32. FIG. 3B shows the output of each pixel of the second image sensor 32. FIG. 3C shows the output of each pixel of the first image sensor 31. In FIGS. 3A to 3C, the horizontal direction is the x-axis, the vertical direction is the y-axis, and the coordinates of the pixel P are expressed as P (x, y).

As shown in FIG. 3A, a color filter CF that transmits three colors of red (R), green (G), and blue (B) is provided above the second image sensor 32 in a so-called Bayer arrangement. Has been done. This color filter CF is configured to transmit infrared light as well as RGB light. An infrared filter is not provided above the color filter CF. Specifically, the color filter CF (1,1) arranged above the pixel P (1,1) transmits the light of G and the infrared light. Therefore, the light of G and the infrared light are incident on the pixels P (1,1) of the second image sensor. Similarly, the color filter CF (2,1) arranged above the pixel P (2,1) transmits the light of B and the infrared light. Therefore, the light of B and the infrared light are incident on the pixels P (2, 1) of the second image sensor.

In the second image sensor 32 shown in FIG. 3B, each pixel P is an organic photoelectric film that photoelectrically converts infrared light. Therefore, infrared light is absorbed by each pixel P and visible light is transmitted. For example, the pixels P (1,1) to which the infrared light and the G light are incident are photoelectrically converted into the infrared light (IR) and transmitted through the G light. Similarly, the pixels P (2, 1) to which the infrared light and the light of B are incident are photoelectrically converted into the infrared light (IR) and transmitted through the light of B. Pixels P (1, 2) to which infrared light and R light are incident are photoelectrically converted into infrared light (IR) and transmitted through R light. Further, as will be described in detail later, each pixel P of the second image sensor 32 is composed of focus detection pixels corresponding to the pupil division phase difference method.

In the first image sensor 31 shown in FIG. 3C, visible light transmitted through the second image sensor 32 is photoelectrically converted in each pixel P. In the first image sensor 31, for example, the image signal of the G component is obtained at the pixels P (1,1) to which the light of G is incident. Similarly, the pixel P (2,1) on which the light of B is incident obtains an image signal of the B component, and the pixel P (1,2) on which the light of R is incident obtains an image signal of the R component.

As described above, in the image pickup unit 12 according to the present embodiment, the second image pickup device 32 composed of the organic photoelectric film acts as an infrared filter for the first image pickup device 31. Therefore, it is possible to omit the infrared filter installed in front of the image sensor in the conventional digital camera.

Further, in the image pickup unit 12 according to the present embodiment, the image pickup pixels are arranged on the first image pickup element 31, and the focus detection pixels are evenly arranged on the entire surface of the second image pickup element 32. Therefore, it is not necessary to perform complicated pixel interpolation processing as in the conventional technique in which the focus detection pixel is arranged in a part of the image pickup pixels, and the focus detection signal by infrared light from the second image sensor 32, the first A color (RGB) image signal by visible light can be independently obtained from the image sensor 31. Since the image sensor 12 is configured so that the first image sensor 31 and the second image sensor 32 can be driven independently, it is possible to control the storage time with a high degree of freedom.

FIG. 4A is an example of the semiconductor layout of the imaging unit 12. Note that FIG. 4A corresponds to pixels P (1,1) to pixels P (2, 2) of FIG. 3 described above. 4 (b) and 5 (a) are cross-sectional views when the pixels P (1, 1) and the pixels (2, 1) of FIG. 4 (a) are horizontally cut along the cutting lines AB. is there. Further, FIG. 5B is a diagram drawn so that the pixel positions of the cutting lines AB in the second image sensor 32 can be easily understood. The cutting line AB cuts the portions of the photoelectric conversion unit PC_A (1,1) and PC_A (2,1). 4 (c) and 6 (a) are cross-sectional views when the pixels P (2, 1) and the pixels (2, 2) of FIG. 4 (a) are vertically cut along the cutting line CD. is there. Further, FIG. 6B is a diagram drawn so that the pixel positions of the cutting lines CD in the second image sensor 32 can be easily understood. The cutting line CD cuts the portions of the photoelectric conversion units PC_A (2,1), PC_B (2,1), PC_A (2,2), and PC_B (2,2). FIG. 7 is an enlarged view of a part of FIG. 6A.

As shown in FIG. 5, the first image sensor 31 is a so-called surface-illuminated image sensor, and the light incident on the surface side of the semiconductor substrate 50 is photoelectrically converted by the photodiode PD to the surface side of the semiconductor substrate 50. A photoelectric conversion signal is output via the formed wiring layer 51. The second image sensor 32 using the organic photoelectric film is laminated and arranged on the surface side of the first image sensor 31. Further, above the second image sensor 32 (that is, on the side opposite to the first image sensor 31), a microlens array in which a plurality of microlens MLs are arranged two-dimensionally is stacked. Each pixel of the first image sensor 31 and the second image sensor 32 is provided corresponding to each of the microlenses ML. Further, the above-mentioned color filter CF is arranged between each pixel of the second image sensor 32 and each microlens ML.

The photoelectric conversion signal output from the second image sensor 32 is taken out from the signal output end 52 via the wiring layer 51. Separation layers 53 and 54 are arranged on both sides of the signal output terminal 52. In addition, photodiodes PD (1,1) and PDs (2,1) are arranged with the separation layers 53 and 54 separated from each other. Each pixel P of the first image sensor 31 is an image pickup pixel, and one photodiode PD is provided for one image pickup pixel. Although the wiring layer 51 has a three-layer structure in FIGS. 4 to 6, it may have a two-layer structure.

Further, as shown in FIG. 6, each pixel P of the second image sensor 32 is a focus detection pixel, and one pixel P is divided into two in the vertical direction in the drawing, and two photoelectric conversions are performed for each pixel P. Parts PC_A and PC_B are provided. For example, the pixel P (2,1) is provided with a photoelectric conversion unit PC_A (2,1) and a photoelectric conversion unit PC_B (2,1). Specifically, as shown in FIG. 7, in the pixel P (2,1), the photoelectric conversion unit PC_A (2) sandwiches the organic photoelectric film m between the common electrode u and the partial electrode d_A (2,1). , 1) is configured. Further, the photoelectric conversion unit PC_B (2,1) is configured with the organic photoelectric film m sandwiched between the common electrode u and the partial electrode d_B (2,1). The common electrode u is arranged on the microlens ML side, and the partial electrodes d_A and d_B are arranged on the first image sensor 31 side. Further, the common electrode u and the partial electrodes d_A and d_B are transparent electrodes. The partial electrode d_A (2,1) is provided on the upper side in the drawing with respect to the center O (2,1) of the microlens ML (2,1), and the partial electrode d_B (2,1) is a microlens ML (2,1). It is provided on the lower side in the figure with respect to the center O (2,1) of 2,1). The partial electrode d_A (2,1) and the partial electrode d_B (2,1) are separated from the center O (2,1) of the microlens ML (2,1) so as not to come into contact with each other. It is provided. The other pixels P are similarly configured.

Each pixel P of the second imaging element 32 receives a pair of light fluxes that have passed through the pair of pupil regions of the photographing optical system 10 by the photoelectric conversion unit PC_A and the photoelectric conversion unit PC_B, respectively, and photographs by the pupil division phase difference method. A focus detection signal for detecting the focus adjustment state of the optical system 10 is output. Here, the wiring layer 51 of FIG. 5A corresponds to the wiring layer 51A and the wiring layer 51B in FIG. 6A. The wiring layer 51A outputs the signal of the photoelectric conversion unit PC_A (2,1) of the organic photoelectric film to the signal output terminal 52A, and the wiring layer 51B outputs the signal of the photoelectric conversion unit PC_B (2,1) of the organic photoelectric film. Output to end 52B. In the wiring layer 51 of FIG. 5A, only one is visible because the wiring layer 51A and the wiring layer 51B overlap each other.

The pixels P at the same position of the first image sensor 31 and the second image sensor 32 receive the subject light incident from the same microlens ML. For example, as shown in FIG. 5, among the subject lights incident from the microlens ML (1,1) corresponding to the pixel P (1,1), the infrared light and the G light are the color filter CF (1,1). It passes through 1) and the light of R and B is blocked by the color filter CF (1,1). In the second imaging element 32, the photoelectric conversion unit PC_A (1,1) and the photoelectric conversion unit PC_B (1,1) using the organic photoelectric film constituting the photoelectric conversion unit of the pixel P (1,1) are formed by the color filter CF (1,1). The infrared light transmitted through 1, 1) is photoelectrically converted, and G light is transmitted. In the first image sensor 31, the photodiode PD (1,1) constituting the pixel P (1,1) receives the light of G transmitted through the photoelectric conversion unit PC (1,1) and performs photoelectric conversion.

Further, as shown in FIG. 5, between the second image sensor 32 and the first image sensor 31, a boundary portion between adjacent pixels of the first image sensor 31 (signal output end 52 and separation layers 53, 54). ), A flattening layer 55 including a wiring layer 51 inside is formed. Further, between the second image sensor 32 and the first image sensor 31, a material having a higher refractive index than the flattening layer 55 is formed on the photodiode PD of each pixel P of the first image sensor 31. The optical waveguide 56 is formed. The optical waveguide 56 totally reflects the visible light transmitted through each pixel P of the second image sensor 32 at the interface 57 between the optical waveguide 56 and the flattening layer 55, and the photo of each pixel P of the first image sensor 31 is reflected. Guide the light to the diode PD. Further, the dimension of the optical waveguide 56 on the second image sensor 32 side is larger than the dimension of the first image sensor 31 side (photodiode PD side). With the above configuration, crosstalk occurs between the light flux transmitted through each pixel of the second image sensor 32, and a plurality of pixels of the second image sensor 32 pass through one pixel of the first image sensor 31. It is possible to prevent the light flux from being incident. That is, the pixels at the same positions of the second image sensor 32 and the first image sensor 31 can receive the incident light from the same microlens ML.

FIG. 8 is a diagram illustrating a circuit configuration of one pixel P (x, y) in the imaging unit 12. The pixel P (x, y) has a photodiode PD, a transfer transistor Tx, a reset transistor R, an output transistor SF, and a selection transistor SEL as circuits for forming the first image sensor 31. The photodiode PD accumulates an electric charge according to the amount of incident light. The transfer transistor Tx transfers the electric charge accumulated in the photodiode PD to the floating diffusion region (FD unit) on the output transistor SF side. The output transistor SF constitutes a current source PW and a source hollower via the selection transistor SEL, and outputs an electric signal corresponding to the electric charge accumulated in the FD unit to the vertical signal line VLINE as an output signal OUT. The reset transistor R resets the electric charge of the FD portion to the power supply voltage Vcc.

Further, the pixels P (x, y) include photoelectric conversion units PC_A and PC_B using an organic photoelectric film, reset transistors R_A and R_B, and output transistors SF_A and SF_B as circuits for forming the second image sensor 32. It has selection transistors SEL_A and SEL_B. The photoelectric conversion units PC_A and PC_B using an organic photoelectric film convert non-transmitted light into an electric signal according to the amount of light, and output transistors SF_A and SF_B that form a source holower with current sources PW_A and PW_B via selection transistors SEL_A and SEL_B, respectively. The output signals OUT_A and OUT_B are output to the vertical signal lines VLINE_A and VLINE_B, respectively. The reset transistors R_A and R_B reset the output signals of the photoelectric conversion units PC_A and PC_B to the reference voltage Vref. Further, a high voltage Vpc is given for the operation of the organic photoelectric film. Each transistor is composed of MOS_FET.

Here, the operation of the circuit related to the first image sensor 31 will be described. First, when the selection signal φSEL becomes “High”, the selection transistor SEL is turned on. Next, when the reset signal φR becomes “High”, the FD unit resets the power supply voltage Vcc, and the output signal OUT also reaches the reset level. Then, after the reset signal φR becomes “Low”, the transfer signal φTx becomes “High”, the charge accumulated in the photodiode PD is transferred to the FD section, and the output signal OUT changes according to the amount of charge. Start and stabilize. Then, the transfer signal φTx becomes “Low”, and the signal level of the output signal OUT read from the pixels to the vertical signal line VLINE is determined. Then, the output signal OUT of each pixel read out on the vertical signal line VLINE is temporarily held for each line in a horizontal output circuit (not shown), and then output from the imaging unit 12. In this way, signals are read from each pixel of the first image sensor 31 of the image pickup unit 12.

Further, the operation of the circuit related to the second image sensor 32 will be described. First, when the selection signal φSEL_A (or φSEL_B) becomes “High”, the selection transistor SEL_A (or SEL_B) is turned on. Next, the reset signal φR_A (or φR_B) becomes “High”, and the output signal OUT_A (or φOUT_B) also becomes the reset level. Immediately after the reset signal φR_A (or φR_B) becomes “Low”, charge accumulation of the photoelectric conversion unit PC_A (or PC_B) by the organic photoelectric film is started, and the output signal OUT_A (or output signal OUT_B) is started according to the amount of charge. ) Changes. Then, the output signal OUT_A (or output signal OUT_B) is temporarily held for each line in a horizontal output circuit (not shown), and then output from the imaging unit 12. In this way, signals are read from each pixel of the second image sensor 32 of the image pickup unit 12.

<Focus detection process>
Next, the flow of the focus detection process executed by the control unit 11 will be described with reference to the flowchart shown in FIG. The focus detection process shown in FIG. 9 is included in the control program executed by the control unit 11. When a predetermined focus detection operation (for example, an operation of pressing the release button halfway) is performed by the photographer, the control unit 11 starts the focus detection process shown in FIG.

First, in step S1, the imaging unit 12 is made to image the subject image. As a result, photoelectric conversion signals are output from each of the first image sensor 31 and the second image sensor 32.

In step S2, the focus detection unit 11a executes the focus detection process by the pupil division phase difference method based on the focus detection signal by the infrared light output from the second image sensor 32, and adjusts the focus of the photographing optical system 10. Calculate the defocus amount indicating the state. Since the focus detection process by the pupil division phase difference method is a well-known technique, detailed description thereof will be omitted.

In step S3, the control unit 11 determines whether the defocus amount can be calculated by the focus detection process in step S2, that is, whether the focus can be detected. If the focus can be detected, the control unit 11 positively determines step S3 and proceeds to step S4. On the other hand, if the focus cannot be detected, the control unit 11 negatively determines step S3 and proceeds to step S6.

In step S4, the focus detection unit 11a controls a lens drive unit (not shown) based on the defocus amount calculated in step S2 to drive the focus lens of the photographing optical system 10 and move to the in-focus position. Let me.

In step S5, the control unit 11 determines whether or not the end of the focus detection process is instructed. When the end of the focus detection process is instructed, step S5 is determined affirmatively, and the control unit 11 ends the focus detection process. On the other hand, if the end of the focus detection process is not instructed, step S5 is negatively determined, and the control unit 11 returns to step S1.

In step S6, which proceeds when step S3 is negatively determined, the focus detection unit 11a adds the photoelectric conversion signals output from two adjacent photoelectric conversion units PC_A and PC_B in each pixel P of the second imaging element 32. Then, one signal is generated for each pixel P. As a result, the infrared light image captured by the second image sensor 32 can be acquired.

In step S7, the focus detection unit 11a executes the focus detection process by the contrast method based on the infrared light image acquired in step S6. Specifically, the focus detection unit 11a calculates a focus evaluation value (contrast value) based on the acquired infrared light image. To briefly explain the contrast method, the subject image is periodically imaged while driving the focus lens, the contrast amount of the subject image is examined for each position of the focus lens, and the position of the focus lens at which the contrast amount is maximized is determined. This is a method of specifying the focus position.

In step S8, the focus detection unit 11a determines whether the focus lens is in the in-focus position, that is, whether the focus is detected, based on the focus evaluation value calculated in step S7. When it is determined that the focus lens is in the in-focus position, the control unit 11 positively determines step S8 and proceeds to step S5. On the other hand, if it is determined that the focus lens is not in the in-focus position, or if it is determined that the focus cannot be detected by the infrared light signal, the control unit 11 negates step S8 and steps. Proceed to S9.

In step S9, the focus detection unit 11a determines whether or not the focus can be detected by the infrared light signal. If it is determined that the focus cannot be detected by the infrared light signal, the accuracy is insufficient, or the focus detection by visible light is necessary, the control unit 11 negatively determines step S9 and proceeds to step S11. .. On the other hand, when it is determined that the focus can be detected by the infrared light signal, the control unit 11 positively determines step S9 and proceeds to step S10.

In step S10, the focus detection unit 11a controls a lens driving unit (not shown) to drive the focus lens in a predetermined direction by a predetermined amount in order to search for the focusing position by a so-called mountain climbing operation, and in step S6. Return. That is, the focus detection unit 11a repeats steps S6 to S10 to search for the position of the focus lens at which the focus evaluation value becomes maximum, that is, the in-focus position, and drives the focus lens to the in-focus position.

In step S11, which proceeds when step S9 is negatively determined, the focus detection unit 11a executes the focus detection process by the contrast method based on the visible light image captured by the first image sensor 31. Specifically, the focus detection unit 11a calculates a focus evaluation value (contrast value) based on the acquired visible light image.

In step S12, the focus detection unit 11a determines whether the focus lens is in the in-focus position, that is, whether the focus is detected, based on the focus evaluation value calculated in step S11. When it is determined that the focus lens is in the in-focus position, the control unit 11 positively determines step S12 and proceeds to step S5. On the other hand, if it is determined that the focus lens is not in the in-focus position, the control unit 11 negatively determines step S12 and proceeds to step S13.

In step S13, the focus detection unit 11a controls a lens driving unit (not shown) to drive the focus lens in a predetermined direction by a predetermined amount in order to search for the focusing position by a so-called mountain climbing operation, and in step S11. Return. That is, the focus detection unit 11a repeats steps S11 to S13 to search for the position of the focus lens at which the focus evaluation value becomes maximum, that is, the in-focus position, and drives the focus lens to the in-focus position.

As described above, in the digital camera 1 of the present embodiment, the following three types of focus detection processes can be performed based on the output signal from the imaging unit 12, and the following three types of focus detection processes can be appropriately used. it can.
(1) Focus detection process by pupil division phase difference method based on infrared light signal from second image sensor 32 (2) Focus detection process by contrast method based on infrared light signal from second image sensor 32 (3) Focus detection processing by the contrast method based on the visible light signal from the first image sensor 31

<Shooting process>
FIG. 10 is a flowchart illustrating a flow of shooting processing executed by the control unit 11. When the main switch is turned on, the control unit 11 causes the imaging unit 12 to start photoelectric conversion at a predetermined frame rate, sequentially reproduces and displays a through image based on the image signal on the liquid crystal monitor 15, and is illustrated in FIG. Start the program that executes the processed processing. The through image is an image for a monitor acquired before the shooting instruction.

In step S21 of FIG. 10, the control unit 11 determines whether or not a shooting instruction has been given. When the release button is pressed, the control unit 11 positively determines step S21 and proceeds to step S22. If the release button is not pressed, the control unit 11 determines that step S21 is negative and proceeds to step S28.

In step S28, the control unit 11 determines whether or not the time is up. When the predetermined time (for example, 5 seconds) is timed, the control unit 11 positively determines step S28 and ends the process according to FIG. If the time counting time is less than the predetermined time, the control unit 11 negatively determines step S28 and returns to step S21.

In step S22, the control unit 11 performs AE processing and AF processing. In the AE process, the exposure calculation is performed based on the level of the image signal for the through image, and the aperture value AV and the shutter speed TV are determined so that an appropriate exposure can be obtained. In the AF process, the focus is adjusted by performing the focus detection process described above based on the output signal sequence from the pixel sequence included in the set focus detection area. When the control unit 11 performs the above AE and AF processes, the control unit 11 proceeds to step S23.

In step S23, the control unit 11 performs a shooting process and proceeds to step S24. Specifically, an aperture (not shown) is controlled based on the AV, and the imaging unit 12 is made to perform photoelectric conversion for recording at the accumulation time based on the TV.

In step S24, the image generation unit 11b of the control unit 11 generates an image signal using the output signal from the first image sensor 31. As described above, since the image signal of the so-called bayer arrangement can be acquired from the first image sensor 31, the image generation unit 11b uses a known color interpolation process to obtain RGB 3 colors from the output signal of the first image sensor 31. Color image signal can be generated. Further, the image generation unit 11b performs predetermined image processing (gradation conversion processing, contour enhancement processing, white balance adjustment processing, etc.) on the generated color image signal.

In step S25, the control unit 11 displays the captured image after the image processing in step S24 on the liquid crystal monitor 15, and proceeds to step S26.

In step S26, the control unit 11 generates an image file for recording the captured image after the image processing in step S24, and proceeds to step S27.

In step S27, the control unit 11 records the image file generated in step S26 on the memory card 17 and ends the shooting process.

According to the embodiment described above, the following effects can be obtained.
The image sensor 12 of the digital camera 1 photoelectrically converts the light incident on the surface side of the semiconductor substrate 50 to generate a photoelectric conversion signal, and the photoelectric conversion signal is generated via the wiring layer 51 formed on the surface side of the semiconductor substrate 50. A first image pickup device 31 for outputting the light is provided. Further, the image pickup unit 12 is laminated and arranged on the surface side of the semiconductor substrate 50 of the first image pickup element 31, and uses an organic photoelectric film that absorbs infrared light and transmits visible light to photoelectrically convert infrared light into photoelectric light. A second image sensor 32 that generates a conversion signal and outputs a photoelectric conversion signal via the wiring layer 51 is provided. Further, the image pickup unit 12 includes a microlens array in which a plurality of microlens MLs are arranged in a two-dimensional manner and are stacked on the side of the second image pickup element 32 opposite to the first image pickup element 31. The first image pickup device 31 has a plurality of imaging pixels provided corresponding to each of the microlens MLs. The second image pickup device 32 has a plurality of focus detection pixels provided corresponding to each of the microlens MLs. The focus detection pixel is a focus detection pixel that outputs a focus detection signal for detecting the focus adjustment state of the photographing optical system 10 by the pupil division phase difference method. Between the second image sensor 32 and the first image sensor 31, an optical waveguide 56 that guides visible light transmitted through the focus detection pixels of the second image sensor 32 to the image pickup pixels of the first image sensor 31 is provided. It is formed. With such a configuration, it is possible to perform focus detection by a high-speed and highly accurate pupil division phase difference method based on the focus detection signal of infrared light from the second image sensor 32. Since the focus is detected by infrared light, the focus detection accuracy can be improved even in a dark field and low brightness. Further, it is possible to prevent crosstalk from occurring between the light beams transmitted through the focal detection pixels of the second image sensor 32, and the visible light image can be appropriately captured by the first image sensor 31.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the following description, the imaging unit 112 in the second embodiment will be mainly described in that it is different from the first embodiment, and the description of the same configuration as in the first embodiment will be omitted.

The configuration of the imaging unit 112 according to the second embodiment will be described with reference to FIG. FIG. 11A is an example of the semiconductor layout of the imaging unit 112. Note that FIG. 11A corresponds to pixels P (1,1) to pixels P (2, 2) of FIG. 3 described above. 11 (b) is a cross-sectional view of the pixels P (1, 1) and the pixels (2, 1) of FIG. 11 (a) when they are cut in the horizontal direction along the cutting lines AB. 11 (c) is a cross-sectional view of the pixels P (2, 1) and the pixels (2, 2) of FIG. 11 (a) when they are cut in the vertical direction along the cutting line CD.

Unlike the first embodiment, the first image sensor 131 in the second embodiment is a so-called back-illuminated image sensor. Therefore, the first image sensor 131 photoelectrically converts the light incident on the back surface side of the semiconductor substrate 150 by the photodiode PD to generate a photoelectric conversion signal, and transmits the photoelectric conversion signal via the wiring layer 151 formed on the front surface side of the semiconductor substrate 150. The above photoelectric conversion signal is output. The second image pickup device 132 using the same organic photoelectric film as in the first embodiment is laminated and arranged on the back surface side of the first image pickup device 131. Similar to the first embodiment, a microlens array in which a plurality of microlens MLs are arranged two-dimensionally is laminated above the second image sensor 132 (that is, on the side opposite to the first image sensor 131). It is arranged. Further, a color filter CF is arranged between each pixel of the second image sensor 132 and each microlens ML as in the first embodiment.

A through silicon via (TSV: Through Silicon Via) 152 for connecting the second image sensor 132 and the wiring layer 151 is formed on the semiconductor substrate 150 of the first image sensor 131. The photoelectric conversion signal output from the second image sensor 132 is output from the wiring layer 151 via the through via 152. The penetration via 152 is provided for each photoelectric conversion unit PC_A, PC_B (partial electrodes d_A, d_B) of each pixel P of the second image sensor 132.

In the image pickup unit 112 of the second embodiment, since the first image pickup element 131 is configured as a back-illuminated type, unlike the first embodiment, the second image pickup element 132 and the light receiving surface of the first image pickup element 131 No wiring layer is provided between the two. Therefore, in the second embodiment, the distance between the light receiving surface of the second image sensor 132 and the light receiving surface of the first image sensor 131 can be shortened as compared with the first embodiment, so that each pixel of the second image sensor 132 can be shortened. It is possible to prevent the light flux that has passed through the plurality of pixels of the second image sensor 132 from being incident on one pixel of the first image sensor 31 due to cross talk between the light flux transmitted through the light beam. That is, the pixels at the same positions of the second image sensor 132 and the first image sensor 131 can receive the incident light from the same microlens ML.

According to the embodiment described above, the following effects can be obtained.
The image pickup unit 112 photoelectrically converts the light incident on the back surface side of the semiconductor substrate 150 to generate a photoelectric conversion signal, and outputs the photoelectric conversion signal via the wiring layer 151 formed on the front surface side of the semiconductor substrate 150. An irradiation type first image pickup device 131 is provided. Further, the image pickup unit 112 is laminated and arranged on the back surface side of the semiconductor substrate 150 of the first image pickup element 131, and uses an organic photoelectric film that absorbs infrared light and transmits visible light to photoelectrically convert infrared light into photoelectric light. A second image sensor 132 that generates a conversion signal and outputs a photoelectric conversion signal via the wiring layer 151 is provided. Further, the image pickup unit 112 includes a microlens array in which a plurality of microlens MLs are arranged in a two-dimensional manner and are laminated on the side of the second image pickup element 132 opposite to the first image pickup element 131. The first image pickup device 131 has a plurality of imaging pixels provided corresponding to each of the microlens MLs. The second image pickup device 132 has a plurality of focus detection pixels provided corresponding to each of the microlens MLs. The focus detection pixel is a focus detection pixel that outputs a focus detection signal for detecting the focus adjustment state of the photographing optical system 10 by the pupil division phase difference method. With such a configuration, it is possible to perform focus detection by a high-speed and highly accurate pupil division phase difference method based on the focus detection signal of infrared light from the second image sensor 32. Further, it is possible to prevent crosstalk from occurring between the light beams transmitted through the focal detection pixels of the second image sensor 32, and the visible light image can be appropriately captured by the first image sensor 31.

(Modification example 1)
In the above-described embodiment, each of the focus detection pixels of the second image sensor 32 has two photoelectric conversion units PC_A and PC_B, and the photoelectric conversion unit PC_A has passed through the pair of pupil regions of the photographing optical system 10, respectively. An example in which one of the pair of luminous fluxes is received and the photoelectric conversion unit PC_B receives the other of the pair of luminous fluxes has been described.

However, the focus detection pixel is not limited to the above configuration. For example, as shown in FIG. 12, the focus detection pixel having only the photoelectric conversion unit PC_A and the focus detection pixel having only the photoelectric conversion unit PC_B are used. The pixels may be arranged alternately. For example, the pixel P (2,1) has only the photoelectric conversion unit PC_A (2,1) formed by sandwiching the organic photoelectric film m between the common electrode u and the partial electrode d_A (2,1). The partial electrode d_A (2,1) is provided on the upper side in the drawing with respect to the center O (2,1) of the microlens ML (2,1). Further, the pixel P (2, 2) has only the photoelectric conversion unit PC_B (2, 2) formed by sandwiching the organic photoelectric film m between the common electrode u and the partial electrode d_B (2, 2). The partial electrode d_B (2,2) is provided on the lower side in the drawing with respect to the center O (2,2) of the microlens ML (2,2). In the configuration shown in FIG. 12, since the partial electrodes d_A and the partial electrodes d_B can be provided at intervals from the center O of the microlens ML, respectively, the photoelectric is compared with the configuration of the above-described embodiment (FIG. 7). The areas of the conversion unit PC_A and the photoelectric conversion unit PC_B can be increased.

In the second image sensor 32, the pupil division direction in each focus detection pixel is not limited to the above-mentioned direction, and may be any of a vertical direction, a horizontal direction, and an oblique direction, and may be in various directions. The focus detection pixels may be mixed.

(Modification 2)
In the image pickup unit 112 according to the second embodiment described above, the focus detection of the pupil division phase difference method may also be performed on the first image pickup element 131. That is, two photodiodes PD_A and PD_B are provided in each pixel P of the first image sensor 131, and the two photodiodes PD_A and PD_B receive a pair of light fluxes that have passed through the pair of pupil regions of the photographing optical system 10, respectively. It may be configured as follows. Since a light beam in which infrared light is cut is incident on the first image sensor 131, it is possible to perform focal detection by the pupil division phase difference method with visible light. In this case, for example, as shown in FIG. 13, in the second image sensor 132, the pupil division direction of each focus detection pixel is set to the vertical direction, and in the first image sensor 131, the pupil division of each focus detection pixel is set. The pupil division direction may be changed between the second image sensor 132 and the first image sensor 131, such as when the direction is lateral. By doing so, for example, in the case of a subject whose focus detection is easier when the pupil division direction is the vertical direction, focus detection can be performed using the output signal from the second image sensor 132. On the other hand, in the case of a subject whose focus detection is easier when the pupil division direction is the lateral direction, focus detection can be performed using the output signal from the first image sensor 131. In this way, the first image sensor 131 and the second image sensor 132 can be appropriately used, and the focus detection accuracy can be improved. Further, in the first image sensor 131, the focus detection pixels whose pupil division direction is in the vertical direction and the focus detection pixels in the horizontal direction may be mixed, or in the second image sensor 132, the pupil division direction is in the vertical direction. The focus detection pixel and the lateral focus detection pixel may be mixed.

(Modification example 3)
An infrared projector may be provided in the digital camera 1 of the above-described embodiment. By projecting infrared rays with an infrared projector, the focus detection accuracy can be further improved in the above-mentioned focus detection process using infrared light. The infrared projector may irradiate a uniform infrared light, or the infrared light to be irradiated may have a pattern. When the infrared light to be irradiated has a pattern, it may be a projected image that assists the focus detection, or a multipoint image arranged in a grid may be projected. When projecting a multipoint image arranged in a grid shape, distance detection can also be performed by calculating the amount of distortion of the projected grid.

(Modification example 4)
In the above-described embodiment, an infrared light image is acquired by adding photoelectric conversion signals between adjacent photoelectric conversion units in each focus detection pixel of the second image sensor 32, and is used for focus detection by a contrast method. However, the infrared light image may be used for other purposes. For example, by comparing the visible light image captured by the first image sensor 31 with the infrared light image captured by the second image sensor 32, a person hidden by strong light such as a headlight is displayed on the visible light image. It may be detected on an infrared light image. Infrared light has a long wavelength and an image deeper than the outermost surface of an object can be obtained. Therefore, the inside of human skin is analyzed based on an infrared light image of a human face, for example, of the skin. It is also possible to detect stains hidden inside.

(Modification 5)
In the above-described embodiment, the example in which the image signal of the Bayer array can be obtained from the first image sensor 31 with the arrangement of the color filters as the Bayer array has been described, but the arrangement of the color filters may be other arrangements. Further, an example of arranging a color filter that transmits R, G, and B light has been described, but the present invention is not limited to this. For example, a color filter that transmits Cy, Mg, and Ye light may be arranged. It may be.

Although various embodiments and modifications have been described above, the present invention is not limited to these contents. Each embodiment and modification may be combined as appropriate. Other aspects conceivable within the scope of the technical idea of the present invention are also included within the scope of the present invention.

1 ... Digital camera, 10 ... Imaging optical system, 11 ... Control unit, 11a ... Focus detection unit, 11b ... Image generation unit, 12,112 ... Imaging unit, 31,131 ... First image sensor, 32,132 ... Second Image sensor

Claims (8)

An organic photoelectric film that converts light by photoelectric conversion to generate an electric charge,
A photoelectric conversion unit that photoelectrically converts the light transmitted through the organic photoelectric film to generate an electric charge, a first wiring that outputs a signal based on the charge generated by the organic photoelectric film, and an electric charge generated by the photoelectric conversion unit. A semiconductor substrate having a wiring layer having a second wiring that outputs a signal based on the above, and a through electrode that outputs a charge generated by the organic photoelectric film and is electrically connected to the first wiring.
An image sensor comprising. In the image pickup device according to claim 1,
The photoelectric conversion unit is an image pickup device provided between the organic photoelectric film and the wiring layer on the semiconductor substrate. In the image pickup device according to claim 1 or 2.
The semiconductor substrate has a plurality of the photoelectric conversion units, and has a plurality of photoelectric conversion units.
The through electrode is an image sensor that penetrates the semiconductor substrate between the plurality of photoelectric conversion units. The image sensor according to any one of claims 1 to 3.
With a micro lens
A plurality of electrodes for outputting the electric charge generated by the organic photoelectric film, and
With
The organic photoelectric film photoelectrically converts the light transmitted through the microlens to generate an electric charge.
The photoelectric conversion unit is an image pickup device that photoelectrically converts light transmitted through the organic photoelectric film and the plurality of electrodes. The image sensor according to any one of claims 1 to 4.
An image pickup device including a control unit that controls the organic photoelectric film and the photoelectric conversion unit with different charge accumulation times. The image sensor according to any one of claims 1 to 5.
An image pickup device including a waveguide that guides light transmitted through the organic photoelectric film to the photoelectric conversion unit. The image sensor according to any one of claims 1 to 6,
Detection of the in-focus position where the image by the optical system is focused on the image sensor based on at least one of the charge-based signal generated by the organic photoelectric film and the charge-based signal generated by the photoelectric conversion unit. The detector to perform and
An imaging device comprising. In the imaging device according to claim 7,
If the detection unit cannot detect the in-focus position based on the charge-based signal generated by the organic photoelectric film, the detection unit detects the in-focus position based on the charge-based signal generated by the photoelectric conversion unit. An imaging device that detects the in-focus position based on a signal based on the charge generated by the organic photoelectric film when the in-focus position cannot be detected based on the signal based on the charge generated by the photoelectric conversion unit.

Images