JP6399817B2 - IMAGING DEVICE, IMAGING DEVICE CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

  The present invention relates to an imaging apparatus capable of performing phase difference type focus detection using an imaging element.

  2. Description of the Related Art Conventionally, an imaging apparatus capable of executing focus detection by a phase difference method (imaging surface phase difference method) using an image sensor is known. Japanese Patent Application Laid-Open No. 2004-26883 discloses an imaging apparatus that discretely arranges a pair of two focus detection pixels between imaging pixels at a predetermined arrangement pitch and performs phase difference type focus detection using an output signal of the focus detection pixels. Is disclosed. Japanese Patent Application Laid-Open No. 2004-228688 discloses an imaging apparatus in which single focus detection pixels are discretely arranged between imaging pixels at a predetermined arrangement pitch, and phase difference type focus detection is performed based on output signals of the imaging pixels and focus detection pixels. Is disclosed. In Patent Document 3, a photoelectric conversion unit of an imaging pixel is divided into a plurality of regions, and phase difference type focus detection is performed by using output signals of a plurality of photoelectric conversion units of each pixel independently, and this output is added. An imaging apparatus that acquires an imaging signal by using it is disclosed.

Japanese Patent Laying-Open No. 2009-3122 (page 23, FIG. 9) JP 2007-155929 A (page 20, FIG. 4) Japanese Patent Laying-Open No. 2013-7998 (page 34, FIG. 2)

  In the imaging device of Patent Document 1, in order to improve the accuracy of focus detection, it is necessary to increase the ratio of focus detection pixels. However, since the output of the focus detection pixel is incomplete as the imaging information, the number of missing portions of the imaging information increases and the image quality is impaired. Further, in the imaging device of Patent Document 2, the imaging pixel is configured to be able to efficiently receive a light beam from a wide area of the exit pupil of the photographing optical system for high sensitivity. For this reason, the image signal output from the imaging pixel has a large amount of image blur at the time of out-of-focus, and the focus detection performance at the time of large defocus is deteriorated. Moreover, in the imaging apparatus of Patent Document 3, the imaging pixel and the focus detection pixel are combined, and the photoelectric conversion unit of the pixel is divided into a plurality of regions. In this case, as the number of divisions of the photoelectric conversion unit is increased, the degree of freedom regarding optimization of the focus detection performance is increased, but the number of readout signals is increased, and thus the readout speed is lowered.

  Therefore, the present invention provides an imaging apparatus, an imaging apparatus control method, a program, and a storage medium that can perform high-precision focus detection in various situations while satisfying desired imaging performance.

An imaging device according to one aspect of the present invention includes a first pixel group having a first pixel divided by a first pattern, and a second pixel divided by a second pattern different from the first pattern. An image pickup device including a second pixel group having two pixels, and a composition in which a base line length or a pupil division characteristic is different from that of the first pixel group and the second pixel group for detecting a defocus amount And generating means for generating a signal based on the output signal of the first pixel and the second output signal .

According to another aspect of the present invention, there is provided a control method for an image pickup apparatus using a first pixel group having a first pixel divided by a first pattern, and a second pattern different from the first pattern. A control method of an imaging apparatus that performs focus detection based on an output signal from an imaging device including a second pixel group having divided second pixels, the first method for detecting a defocus amount Generating a composite signal having a baseline length or pupil division characteristic different from that of the pixel group and the second pixel group based on the output signal of the first pixel and the output signal of the second pixel; And performing focus detection based on a defocus amount detected from the combined signal .

According to another aspect of the present invention, a program includes a first pixel group having a first pixel divided by a first pattern, and a second pattern divided by a second pattern different from the first pattern. A program for performing focus detection based on an output signal from an imaging device including a second pixel group having two pixels, the first pixel group for detecting a defocus amount and the first pixel group Generating a composite signal having different baseline lengths or pupil division characteristics for two pixel groups based on the output signal of the first pixel and the output signal of the second pixel, and detecting from the composite signal And causing the computer to execute focus detection based on the defocus amount .

  A storage medium according to another aspect of the present invention stores the program.

  Other objects and features of the present invention are illustrated in the following examples.

  According to the present invention, it is possible to provide an imaging apparatus, an imaging apparatus control method, a program, and a storage medium that can perform high-precision focus detection in various situations while satisfying desired imaging performance.

It is a block diagram of the imaging device in each Example. FIG. 3 is a pixel array diagram of an image sensor in Embodiment 1. 1 is a configuration diagram of a readout circuit of an image sensor in Embodiment 1. FIG. In Example 1, it is explanatory drawing of the positional relationship of the exit pupil of an imaging optical system, and the photoelectric conversion part of an image pick-up element. 6 is an explanatory diagram of a shape of a projection pupil on an exit pupil plane of the photographing optical system in Embodiment 1. FIG. 6 is an explanatory diagram of a focus detection signal of a pixel group in Embodiment 1. FIG. 6 is an explanatory diagram of a method for synthesizing output signals of a photoelectric conversion unit in Embodiment 1. FIG. 6 is an explanatory diagram of a shape of a projection pupil of a virtual pixel group in Embodiment 1. FIG. 6 is an explanatory diagram of a focus detection signal of a virtual pixel group in Embodiment 1. FIG. It is a flowchart which shows the imaging | photography process in each Example. 3 is a flowchart illustrating focus detection processing according to the first exemplary embodiment. 6 is a flowchart illustrating focus detection processing as a modification example of Embodiment 1. FIG. 6 is a pixel array diagram of an image sensor in Embodiment 2. 10 is an explanatory diagram of a method for synthesizing output signals of a photoelectric conversion unit in Embodiment 2. FIG. FIG. 6 is a relationship diagram between a subject image and a focus detection area in Embodiment 2. 10 is a flowchart illustrating focus detection processing in the second embodiment. 10 is a flowchart illustrating a focus detection process as a modified example in the second embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  First, with reference to FIG. 1, an imaging apparatus according to Embodiment 1 of the present invention will be described. FIG. 1 is a configuration diagram of an imaging apparatus 100 (imaging system). In the imaging apparatus 100, an imaging apparatus main body (camera main body) including an imaging element 107 and a photographing optical system (photographing lens) are integrally configured. However, the present embodiment is not limited to this, and can also be applied to an imaging system that includes an imaging apparatus main body and a lens device (imaging optical system) that can be attached to and detached from the imaging apparatus main body. The imaging apparatus 100 can record a moving image and a still image.

  In FIG. 1, reference numeral 101 denotes a first lens group. The first lens group 101 is disposed at the tip of the photographing optical system (imaging optical system), and is held so as to be able to advance and retreat in the direction of the optical axis OA (optical axis direction). Reference numeral 102 denotes an aperture / shutter. The aperture / shutter 102 (aperture means) adjusts the light amount at the time of shooting by adjusting the aperture diameter. The aperture / shutter 102 has a function as an exposure time adjustment shutter when taking a still image. Reference numeral 103 denotes a second lens group constituting the photographing optical system. The diaphragm / shutter 102 and the second lens group 103 integrally move forward and backward in the optical axis direction, and perform a zooming function (zoom function) in conjunction with the forward / backward movement of the first lens group 101. Reference numeral 105 denotes a third lens group constituting the photographing optical system. The third lens group 105 performs focus adjustment by moving back and forth in the optical axis direction. Reference numeral 106 denotes an optical low-pass filter. The optical low-pass filter 106 is an optical element for reducing false colors and moire in the captured image.

  Reference numeral 107 denotes an image sensor. The image sensor 107 photoelectrically converts a subject image (optical image) obtained via the photographing optical system and outputs an image signal. The image sensor 107 includes a C-MOS sensor and its peripheral circuit. The image sensor 107 of the present embodiment is a two-dimensional single-plate color sensor in which Bayer array primary color mosaic filters are formed on-chip on light receiving pixels of m pixels in the horizontal direction and n pixels in the vertical direction.

  Reference numeral 111 denotes a zoom actuator. The zoom actuator 111 rotates a cam cylinder (not shown) to drive the first lens group 101, the second lens group 103, and the third lens group 105 forward and backward in the optical axis direction, and performs a zooming operation. . Reference numeral 112 denotes an aperture shutter actuator. The aperture shutter actuator 112 controls the aperture diameter of the aperture / shutter 102 to adjust the amount of photographing light, and also controls the exposure time when taking a still image. Reference numeral 114 denotes a focus actuator. The focus actuator 114 performs focus adjustment by driving the third lens group 105 (focus lens) forward and backward in the optical axis direction.

  Reference numeral 115 denotes a wireless communication means (communication means). The wireless communication unit 115 includes an antenna and a signal processing circuit for communicating with a server (computer) through a network such as the Internet. Reference numeral 116 denotes posture detection means of the imaging apparatus 100 (camera body). The posture detection means 116 is configured to include an electronic level for determining the shooting posture of the imaging apparatus 100, that is, whether it is horizontal position shooting or vertical position shooting.

  Reference numeral 121 denotes a CPU (control device). The CPU 121 is a camera CPU (camera control unit) that controls various controls of the camera body, and includes a calculation unit, a ROM, a RAM, an A / D converter, a D / A converter, a communication interface circuit, and the like. The CPU 121 drives various circuits of the camera body based on a predetermined program stored in the ROM, and executes a series of operations such as AF, shooting, image processing, and recording. In this embodiment, the CPU 121 functions as a focus detection unit that performs focus detection based on an output signal from the image sensor 107.

  122 is a communication control circuit. The communication control circuit 122 transmits a captured image from the imaging apparatus 100 to a server (computer) via the wireless communication unit 115 or receives an image and various information from the server (computer). Reference numeral 123 denotes an attitude detection circuit. The attitude detection circuit 123 determines the attitude of the imaging apparatus 100 based on the output signal of the attitude detection unit 116. Reference numeral 124 denotes an image sensor driving circuit. The image sensor driving circuit 124 controls the image capturing operation of the image sensor 107 and A / D converts the image signal output from the image sensor 107 and transmits the image signal to the CPU 121. Reference numeral 125 denotes an image processing circuit. The image processing circuit 125 performs image processing such as γ conversion, color interpolation, and JPEG compression on the image signal obtained from the image sensor 107.

  Reference numeral 126 denotes a focus driving circuit. The focus drive circuit 126 controls the focus actuator 114 based on the focus detection result, and adjusts the focus by driving the third lens group 105 back and forth in the optical axis direction. Reference numeral 128 denotes an aperture shutter drive circuit. The aperture shutter drive circuit 128 controls the aperture of the aperture / shutter 102 by drivingly controlling the aperture shutter actuator 112. Reference numeral 129 denotes a zoom drive circuit. The zoom drive circuit 129 drives the zoom actuator 111 according to the zoom operation of the photographer.

  Reference numeral 131 denotes a display such as an LCD. The display 131 displays information related to the shooting mode of the camera body, a preview image before shooting and a confirmation image after shooting, a focus state display image at the time of focus detection, and the like. Reference numeral 132 denotes an operation switch group. The operation switch group 132 includes a power switch, a shooting start switch, a zoom operation switch, a shooting mode selection switch, and the like. Reference numeral 133 denotes a detachable flash memory. The flash memory 133 records captured images.

  Next, with reference to FIG. 2, the pixel arrangement of the image sensor 107 in the present embodiment will be described. FIG. 2 is a pixel array diagram of the image sensor 107, and is manufactured using, for example, a technique described in Japanese Patent Application Laid-Open No. 09-046596 by the present applicant.

  FIG. 2 shows a state in which a range of 6 rows and 6 columns in the horizontal (X direction) of the two-dimensional C-MOS area sensor is observed from the photographing optical system side. The color filters are in a Bayer array, and green (G: Green) and red (R: Red) color filters are alternately provided in order from the left on the pixels in the odd rows (row numbers 1, 3, 5). . Further, blue (B: Blue) and green (G: Green) color filters are alternately provided in order from the left in the pixels in even rows (row numbers 2, 4, and 6). Each pixel circle represents an on-chip microlens. A plurality of rectangles arranged inside the on-chip microlens each indicate a photoelectric conversion unit. In the image sensor 107 of this embodiment, the photoelectric conversion units of all the pixels are divided into two regions. However, the division pattern (size and shape of the divided pixels) is not uniform for all pixels, and is composed of a plurality of pixel groups having different division patterns. Hereinafter, characteristics of these pixel groups (pixels) will be described.

  As shown in FIG. 2, the pixel group of row numbers 1 and 2 is composed of a plurality of pixels 211. In a pixel group composed of a plurality of pixels 211, the division positions of the two photoelectric conversion units 211a and 211b in each pixel (boundary positions of the photoelectric conversion units 211a and 211b) are shifted leftward with respect to the pixel center. . The pixel group of row numbers 3 and 4 is composed of a plurality of pixels 212. In a pixel group composed of a plurality of pixels 212, the division positions of the photoelectric conversion units 212a and 212b (boundary positions of the photoelectric conversion units 212a and 212b) in each pixel coincide with the pixel center. The pixel group of row numbers 5 and 6 is composed of a plurality of pixels 213. In a pixel group including a plurality of pixels 213, the division positions of the photoelectric conversion units 213a and 213b in each pixel (boundary positions of the photoelectric conversion units 213a and 213b) are deviated leftward with respect to the pixel center. Note that pixel groups having the same array as the row numbers 1 to 6 are repeatedly arranged outside the region shown in FIG.

  Next, the configuration of the readout circuit of the image sensor 107 will be described with reference to FIG. FIG. 3 is a configuration diagram of a readout circuit of the image sensor 107. In FIG. 3, 151 is a horizontal scanning circuit, and 153 is a vertical scanning circuit. Horizontal scanning lines 152a and 152b and vertical scanning lines 154a and 154b are wired at the boundary between the pixels. Outputs of the photoelectric conversion units 211a, 211b to 213a, and 213b are read out to the outside of the image sensor 107 via each scanning line (horizontal scanning line and vertical scanning line).

  The image sensor 107 of the present embodiment has the following two types of readout modes (first readout mode and second readout mode). The first readout mode is called an all-pixel readout mode and is a mode for capturing a high-definition still image. In this case, signals of all pixels are read out. On the other hand, the second reading mode is called a thinning-out reading mode, and is a mode for performing moving image recording or preview image display (live view display) during shooting preparation. In this case, the number of pixels required is smaller than all the pixels. For this reason, with respect to the pixel group of the image sensor 107, only the signals of some pixels thinned out at a predetermined ratio in the X direction and the Y direction are read out.

  Next, the relationship between the exit pupil of the photographing optical system and the position of the pixel (photoelectric conversion unit) of the image sensor 107 will be described with reference to FIG. FIG. 4 is an explanatory diagram of the positional relationship between the exit pupil of the photographing optical system and the photoelectric conversion unit of the image sensor. FIG. 4A shows two pixels located in the central part (near the optical axis) and the peripheral part (screen edge part) of the image sensor in the pixel group included in the row numbers 1 and 2 shown in FIG. And the optical relationship between the photographic optical system. Similarly, FIG. 4B shows an optical relationship between two pixels located in the central part and the peripheral part of the image sensor in the pixel group included in the row numbers 3 and 4 shown in FIG. 2 and the imaging optical system. It shows the relationship. Similarly, FIG. 4C shows an optical relationship between two pixels located in the central portion and the peripheral portion of the image pickup element in the pixel group included in the row numbers 5 and 6 shown in FIG. Showing the relationship.

  When performing phase difference focus detection (imaging surface phase difference focus detection) using the image sensor 107, the optically conjugate relationship between the exit pupil plane of the imaging optical system and the photoelectric conversion unit of the image sensor 107. It is necessary to arrange so that it becomes. In FIG. 4, the exit pupil of the photographing optical system is considered to coincide with the arrangement surface of the iris diaphragm for adjusting the light amount. On the other hand, the distance between the on-chip microlens of the image sensor 107 and the photoelectric conversion unit is substantially determined by the process of the image sensor 107. Therefore, by appropriately designing the optical power of the on-chip microlens, a conjugate relationship between the exit pupil plane of the photographing optical system and the photoelectric conversion unit is realized.

  In addition, in the peripheral portion of the imaging region, that is, in the region where the image height is large, the principal ray of the imaging light beam is incident with an inclination with respect to the normal line of the imaging element 107 (imaging surface). Therefore, the on-chip microlens is shifted by a minute amount in the direction orthogonal to the optical axis OA so that the light receiving condition of the pixel is substantially the same as the central portion of the imaging region.

  FIG. 4A is a diagram illustrating a conjugate relationship between the pixel 211 and the imaging optical system. In FIG. 4A, the lens group of the photographing optical system is omitted. Reference numeral 102a denotes an aperture plate that defines the aperture diameter when the aperture is open, and 102b is an aperture blade for adjusting the aperture diameter when the aperture is closed. The exit pupil position of the photographing optical system is assumed to be coincident with the position of the diaphragm / shutter 102 (aperture means), and the distance from the image plane is D.

  The pixel 211 is composed of the photoelectric conversion units 211a and 211b, the wiring layers 211e to 211g, the color filter 211h, and the on-chip microlens 211i that are divided into two parts from the lowest layer. Regarding the center pixel, the optical axis OA of the on-chip microlens 211i coincides with the center of the pixel. On the other hand, with respect to the peripheral pixels, the optical axis OA of the on-chip microlens 211i is shifted to the center of the imaging region from the center of the pixels. With such a configuration, the photoelectric conversion units 211a and 211b and the exit pupil of the photographing optical system are conjugated with each other by the on-chip microlens 211i. This conjugate relationship is established without depending on the image height. In FIG. 4A, EP1a and EP1b are projection pupils (projected images) of the photoelectric conversion units 211a and 211b projected onto the exit pupil plane of the photographing optical system. The projection pupils EP1a and EP1b correspond to pupil regions (divided pupil regions) through which a pair of light beams pass in the photographing optical system (phase difference type focus detection system).

  The boundary between the photoelectric conversion units 211a and 211b of the pixel 211 is biased leftward with respect to the pixel center. For this reason, the boundary between the projection pupils EP1a and EP1b is deviated in the right direction with respect to the optical axis OA. The line segment CL1a is a line connecting the center of the photoelectric conversion unit 211a and the center of the projection pupil EP1a, and the line segment CL1b is a line connecting the center of the photoelectric conversion unit 211b and the center of the projection pupil EP1b. Line segments CL1a and CL1b correspond to principal rays of a pair of light beams (focus detection light beams).

  FIG. 4B is a diagram illustrating a conjugate relationship between the pixel 212 and the photographing optical system. The photoelectric conversion units 212a and 212b of the pixel 212 are symmetrical with respect to the pixel center. Therefore, the respective projection pupils EP2a and EP2b corresponding to the photoelectric conversion units 212a and 212b have a bilaterally symmetric shape (symmetric with respect to the optical axis OA) on the exit pupil. FIG. 4C is a diagram illustrating a conjugate relationship between the pixel 213 and the imaging optical system. The boundary between the photoelectric conversion units 213a and 213b of the pixel 213 is biased to the right with respect to the pixel center. For this reason, the boundary portions of the projection pupils EP3a and EP3b corresponding to the photoelectric conversion units 213a and 213b are deviated leftward with respect to the optical axis OA.

  Next, the shape of the projection image (projection pupil) of the photoelectric conversion unit projected on the exit pupil plane of the photographing optical system will be described with reference to FIG. FIG. 5 is an explanatory diagram of the shape of the projection pupil on the exit pupil plane.

  FIG. 5A is a plan view of the projection pupils EP1a and EP1b corresponding to the photoelectric conversion units 211a and 211b of the pixel 211. FIG. TL (F2.8) is an exit pupil in the photographing optical system having an F value (aperture value) of F2.8. TL (F5.6) is an exit pupil when the F value is reduced to F5.6. EP1a and EP1b are projection pupils (projected images) corresponding to the photoelectric conversion units 211a and 211b of the pixel 211, respectively. As described with reference to FIG. 4A, since the boundaries of the photoelectric conversion units 211a and 211b of the pixel 211 are biased to the left with respect to the pixel center, the boundaries of the projection pupils EP1a and EP1b are It is biased to the right with respect to the center of the exit pupil TL.

  FIG. 5B is a plan view of the projection pupils EP2a and EP2b corresponding to the photoelectric conversion units 212a and 212b of the pixel 212. The boundary between the pair of projection pupils EP2a and EP2b coincides with the center of the exit pupil TL of the photographing optical system (the position of a line passing through the center of the exit pupil TL). FIG. 5C is a plan view of the projection pupils EP3a and EP3b corresponding to the photoelectric conversion units 213a and 213b of the pixel 213. The boundary between the pair of projection pupils EP3a and EP3b is deviated leftward with respect to the center of the exit pupil TL of the photographing optical system.

  Next, the relationship between the projection pupil and focus detection characteristics will be described. A pair of projection pupils are cut out within the range of the exit pupil of the photographing optical system, and the center-of-gravity separation amount in the X-axis direction of each cut out region corresponds to the baseline length in the phase difference type focus detection system. Here, the base line length is defined as an angle θ (unit: radians) obtained by dividing the center-of-gravity separation amount (unit: mm) on the pupil plane of the photographing optical system by the pupil distance (unit: mm). The lateral shift amount of the pair of two images at the time of focus detection is u (unit: mm), and the defocus amount at that time is DEF (unit: mm). At this time, these relationships are expressed as the following formula (1).

θ × DEF = u (1)
That is, the larger the baseline length θ, the larger the lateral deviation amount u of the pair of two images with respect to the unit defocus amount. For this reason, the larger the baseline length θ, the higher the focus detection resolution and the higher accuracy focus detection becomes possible.

  Next, the blur amount of the pair of two images for focus detection will be considered. Each projection pupil is cut out within the range of the exit pupil of the photographing optical system, and the width of the cut projection pupil in the X-axis direction defines the blur width of each focus detection image. Here, the width of each projection pupil is defined as an angle α (unit: radians) obtained by dividing the width (unit: mm) of the projection pupil on the pupil plane of the photographing optical system by the pupil distance (unit: mm). The blur width of each image at the time of focus detection is w (unit: mm), and the defocus amount at that time is DEF (unit: mm). At this time, these relationships are expressed as the following formula (2).

α × DEF = w (2)
That is, the greater the projection pupil width α, the greater the blur width of each focus detection image with respect to the unit defocus amount. For this reason, when the defocus amount is large, the contrast of the focus detection image is lowered, and the focus detection is likely to be impossible.

  FIG. 6 is an explanatory diagram of a focus detection signal of each pixel group in the image sensor 107, and shows output waveforms of the pixels 211 to 213 in focus detection for three types of defocus amounts. In FIG. 6, the horizontal axis represents the coordinates (X coordinate) of the pixel in the focus detection area, and it is assumed that the focus detection area includes about 100 pixels 211 to 213 in the X axis direction. The vertical axis indicates the output signal (pixel output) of the pixel located in the focus detection area. The F value of the photographing optical system is assumed to be F2.8, and the subject is assumed to have two white thin lines drawn on a black background. The brightness of the thin line located on the left side is low and the brightness of the thin line located on the right side is high.

  FIG. 6A shows a waveform (pixel output) output from the pixel 211, and shows waveforms of focus detection signals with defocus amounts of 0 mm, 4 mm, and 10 mm in order from the top. First, a waveform with a defocus amount of 0 mm will be described. A thick line output waveform AF1a indicates an output of the photoelectric conversion unit 211a (photoelectric conversion unit group), and a thin line output waveform AF1b indicates an output of the photoelectric conversion unit 211b (photoelectric conversion unit group). Since the defocus amount is 0 mm, the relative lateral deviation amount of both waveforms is zero, that is, the phases match. In addition, since the subject image is in focus, the output waveform of the pair of two images is not blurred, and the edge of the thin line signal is sharply cut. On the other hand, regarding the projection pupil EP1a of the pixel 211 cut out by the exit pupil of the photographing optical system, as shown in FIG. 5A, the projection pupil EP1a corresponding to the photoelectric conversion unit 211a corresponds to the photoelectric conversion unit 211b. The projection pupil EP1b to be larger is larger. That is, since the sensitivity of the photoelectric conversion unit 211b is higher than the sensitivity of the photoelectric conversion unit 211a, the output waveform AF1b has a higher output than the output waveform AF1a.

  When the defocus amount increases to 4 mm, a lateral shift amount u (image shift amount) occurs between the two images, and the blur amount of the two images also increases. In this state, since the contrast of each waveform is still relatively high, it is possible to detect the lateral shift amount of the two images by performing a predetermined phase difference detection calculation. When the defocus amount is 10 mm, the lateral displacement amount u between the two images further increases, the contrast between the two images further decreases, and the probability that the focus detection becomes impossible increases.

  FIG. 6B shows a waveform (pixel output) output from the pixel 212. The thick line output waveform AF2a is the output of the photoelectric conversion unit 212a (photoelectric conversion unit group), and the thin line output waveform AF2b is the photoelectric conversion unit. The output of 212b (photoelectric conversion unit group) is shown. In FIG. 6B, the lateral shift amount of the output waveform of the pair of two images and the decrease in contrast at each defocus amount (DEF = 0, 4, 10 mm) are substantially the same as in FIG. 6A. It is. On the other hand, regarding the projection pupils EP2a and EP2b of the pixel 212 cut out by the exit pupil of the photographing optical system, as shown in FIG. 5B, the projection pupils EP2a and EP2b are bilaterally symmetric and equal in size. That is, since the photoelectric conversion unit 212a (photoelectric conversion unit group) and the photoelectric conversion unit 212b (photoelectric conversion unit group) have the same sensitivity, the output levels of the output waveforms AF2a and AF2b are also equal to each other.

  FIG. 6C shows a waveform (pixel output) output from the pixel 213. A thick line output waveform AF3a is an output of the photoelectric conversion unit 213a (photoelectric conversion unit group), and a thin line output waveform AF2b is a photoelectric conversion unit. Outputs of 213b (photoelectric conversion unit group) are shown. In FIG. 6C, the lateral shift amount of the output waveform of the pair of two images and the decrease in contrast at each defocus amount (DEF = 0, 4, 10 mm) are substantially the same as those in FIGS. It is the same as the case of. On the other hand, regarding the projection pupils EP3a and EP3b of the pixel 213 cut out by the exit pupil of the photographing optical system, as shown in FIG. 5C, the projection pupil EP3a is larger than the projection pupil EP3b. That is, since the sensitivity of the photoelectric conversion unit 213a (photoelectric conversion unit group) is higher than the sensitivity of the photoelectric conversion unit 213b (photoelectric conversion unit group), the output waveform AF3a has a higher output than the output waveform AF3b.

  As described above, when generating a focus detection signal independently without combining (combining) the outputs of the photoelectric conversion units, the width of the projection pupil in the focus detection system by each pixel group (pixels 211 to 213) is wide. In addition, since the base line length is large, the focus detection accuracy in the vicinity of in-focus is excellent. However, focus detection may not be possible during large defocus.

  Next, with reference to FIG. 7 to FIG. 9, a case will be described in which focus detection calculation is performed after the photoelectric conversion unit combining processing is performed between different pixel groups. FIG. 7 is an explanatory diagram of a method for synthesizing the output signals of the photoelectric conversion units. In the pixel array of the image sensor 107 shown in FIG. 2, the outputs of the photoelectric conversion units between different pixel groups are combined to create a virtual The conceptual diagram in the case of producing | generating the pixel group corresponding to a projection pupil (composite projection pupil) is shown. In FIG. 2, pixels in a range of 6 rows and 6 columns are shown. Among them, a specific column, for example, the extracted 6 pixels in the first column at the left end is shown on the left side of FIG. 7. These six pixels are, in order from the top, a green (G) and blue (B) pixel 211, a green (G) and blue (B) pixel 212, and a green (G) and blue (B) pixel 213. It is.

  Here, a method of synthesizing output signals from green (G) pixels of each pixel group, that is, a total of three types of G pixels of row numbers 1, 3, and 5 will be described. The three types of green (G) pixels (pixels 211 to 213) have a total of six photoelectric conversion units 211a to 213b. Therefore, two of the outputs from the six photoelectric conversion units 211a to 213b are taken out and combined (subtracted) by the following equation (3) to obtain four virtual photoelectric conversions as shown in the center of FIG. The output signals of the units 221s to 224s are generated.

S221s = S212a-S211a (3)
S222s = S211b-S212b (4)
S223s = S213a-S212a (5)
S224s = S212b-S213b (6)
In Expressions (3) to (6), S211a to S213b indicate output signals of the photoelectric conversion units 211a to 213b. S221s to S224s indicate output signals of the virtual photoelectric conversion units 221s to 224s, respectively.

  Subsequently, two output signals are extracted from the output signals S221s to S224s of the four virtual photoelectric conversion units 221s to 224s, and the following combination operation is performed. As a result, three types of virtual pixel groups (virtual pixel groups: pixels 214 to 216) as shown on the right side of FIG. 7 are obtained. First, the pixel 231 includes virtual photoelectric conversion units 221s and 222s. The pixel 232 includes virtual photoelectric conversion units 221s and 224s. The pixel 233 includes virtual photoelectric conversion units 223s and 224s. The same operation is performed for the green (G) pixels in the third and fifth rows, and three matrix pixel groups (pixels 214 to 216) as shown on the right side of FIG. 7 are obtained. . The same operation is performed for red (R) and blue (B) pixels other than green (G), but the description thereof is omitted here.

  FIG. 8 is an explanatory diagram of the projection pupil of the virtual pixel group (pixels 214 to 216) projected on the exit pupil plane of the photographing optical system. FIG. 8A is a plan view of the projection pupils EP4a and EP4b corresponding to the photoelectric conversion units 214a and 214b of the virtual pixel 214 shown in FIG.

  TL (F2.8) is an exit pupil in the photographing optical system when the F value (aperture value) is set to F2.8, and TL (F5.6) is an exit pupil when the aperture is limited to F5.6. EP4a and EP4b are projection pupils corresponding to the photoelectric conversion units 214a and 214b of the pixel 214. As shown in FIG. 7, the X-direction dimensions of the photoelectric conversion units 214a and 214b of the pixel 214 are small, the two are close to each other, and the boundary is biased to the left with respect to the pixel center. For this reason, the width of the projection pupils EP4a and EP4b corresponding to the photoelectric conversion units 214a and 214b, that is, the dimension in the X direction is small, the pair of two projection pupils EP4a and EP4b are close to each other, and the boundary portion is It is biased to the right with respect to the center of the exit pupil TL.

  FIG. 8B is a plan view of the projection pupils EP5a and EP5b corresponding to the photoelectric conversion units 215a and 215b of the virtual pixel 215. FIG. The width of the pair of projection pupils EP5a and EP5b is narrow, but the distance between the projection pupils EP5a and EP5b, that is, the base line length is large, and the center of gravity of each projection pupil is located at the same distance from the center of the exit pupil TL. FIG. 8C is a plan view of the projection pupils EP6a and EP6b corresponding to the photoelectric conversion units 216a and 216b of the virtual pixel 216. FIG. The boundary between the pair of projection pupils EP6a and EP6b is deviated leftward with respect to the center of the exit pupil.

  FIG. 9 is an explanatory diagram of a focus detection signal of each virtual pixel group, and shows output waveforms of the pixel group (pixels 214 to 216) at the time of focus detection for three types of defocus amounts. In FIG. 9, the horizontal axis represents pixel coordinates (X coordinate), and the vertical axis represents pixel output signals (pixel output). The F value of the photographing optical system is assumed to be F2.8, and the subject is assumed to have two white thin lines drawn on a black background. The brightness of the thin line located on the left side is low and the brightness of the thin line located on the right side is high.

  FIG. 9A shows waveforms (pixel outputs) output from a plurality of pixels 214, and shows the waveforms of focus detection signals with defocus amounts of 0 mm, 4 mm, and 10 mm in order from the top. First, a waveform with a defocus amount of 0 mm will be described. A thick line output waveform AF4a indicates an output of the virtual photoelectric conversion unit 214a (photoelectric conversion unit group), and a thin line output waveform AF4b indicates an output of the virtual photoelectric conversion unit 214b (photoelectric conversion unit group). Since the defocus amount is 0 mm, the relative lateral shift amount of both waveforms is zero, and the areas of the pair of projection pupils EP4a and EP4b are the same, so the phases and intensities of the output waveforms AF4a and AF4b match each other. ing.

  When the defocus amount increases to 4 mm, a lateral shift amount u is generated in the two images, and the blur amount of the two images also increases. However, as compared with the waveform under the same conditions shown in FIG. 6A, regarding the pixel 214, the projection pupils EP4a and EP4b are narrower and the base line length is shorter. For this reason, the blurring situation of the two images is slight, and the lateral deviation amount u is also small. When the defocus amount is 10 mm, the lateral displacement amount u of the two images further increases and the contrast of the two images also decreases. However, since two peaks corresponding to the two thin lines remain sufficiently, focus detection is not disabled.

  FIG. 9B shows a waveform (pixel output) output from a plurality of pixels 215, and a thick output waveform AF5a indicates an output of a virtual photoelectric conversion unit 215a (photoelectric conversion unit group), and an output waveform of a thin line. AF5b indicates the output of the virtual photoelectric conversion unit 215b (photoelectric conversion unit group). FIG. 9C shows a waveform (pixel output) output from a plurality of pixels 216. A thick line output waveform AF6a indicates a virtual photoelectric conversion unit 216a (photoelectric conversion unit group) output, and a thin line output waveform. AF6b indicates the output of the virtual photoelectric conversion unit 216b (photoelectric conversion unit group).

  Here, with reference to FIGS. 8A to 8C and FIGS. 9A to 9C, the properties of the projection pupil and the focus detection characteristics will be described. As shown in FIGS. 8A to 8C, the widths of the projection pupils of the virtual pixel group (virtual pixels 214 to 216) are substantially equal. On the other hand, the baseline length of the projection pupil of each virtual pixel group is small in the pixels 214 and 216 and large in the pixel 215. When comparing the waveforms in FIGS. 9A to 9C, the contrast reduction state of the output waveforms of a pair of two images at each defocus amount is substantially the same in any of the cases of FIGS. 9A to 9C. Are identical. However, the relative lateral displacement amount u is small in the cases of FIGS. 9A and 9C and large in FIG. 9B. That is, in the focus detection characteristics using the pixels 214 and 216, since the lateral deviation amount u of the pair of two images with respect to a predetermined defocus amount is small, the focus detection resolution is low, but focus detection is possible up to a larger defocus amount. is there. On the other hand, the focus detection characteristic using the pixel 215 has a large lateral shift amount u of the pair of two images with respect to a predetermined defocus amount. There is a possibility of disappearing. Therefore, by using an optimal pixel group according to the magnitude of the defocus amount, focus detection is possible even when the defocus amount is large, and highly accurate focus detection can be achieved in the vicinity of the in-focus point.

  Table 1 is a comparison table of focus detection characteristics in each pixel group. In Table 1, pixel groups 1 to 6 correspond to pixels 211 to 216, respectively. When the defocus amount is large, virtual pixel groups 4 to 6 with less blur and lateral displacement of the focus detection image are preferably used. On the other hand, pixel groups 1 to 3 having a long base line length and high sensitivity are preferably used in the vicinity of the focus.

  Next, imaging processing (imaging device control method) including focus detection processing of the imaging device 100 according to the present embodiment will be described with reference to FIGS. FIG. 10 is a flowchart (main control flow) showing the photographing process in the present embodiment. Each step in FIG. 10 is mainly executed by each unit of the imaging apparatus 100 based on a command from the CPU 121 of the imaging apparatus 100.

  First, in step S101, when the photographer turns on the power switch of the imaging apparatus 100, the process proceeds to step S102. In step S102, the CPU 121 confirms the operations of the actuators and the image sensor 107 of the image capturing apparatus 100, initializes the memory contents and the execution program, and executes the photographing preparation operation (initial state confirmation). Subsequently, in step S103, the CPU 121 accepts shooting condition settings. Specifically, the CPU 121 accepts settings such as an exposure adjustment mode, a focus adjustment mode, an image quality (recording pixel number and compression rate), a white balance mode, and the like by the photographer. In step S104, the CPU 121 controls the aperture value (F value) to a specified value. Here, the designated value is an aperture value selected by the photographer in aperture priority AE, and an aperture value based on a preset exposure control program in shutter priority AE or program AE.

  In step S105, the CPU 121 detects the lens state, that is, the zoom state, the focus lens state, and the aperture state of the photographing optical system, and reads information such as the exit pupil size and exit pupil distance from the storage unit. The storage means is an internal memory (ROM) of the CPU 121, for example. In step S <b> 106, the CPU 121 starts the imaging operation of the imaging device 107 using the imaging device driving circuit 124, and reads the pixel signal output from the imaging device 107. In this embodiment, the photoelectric conversion unit of each pixel of the image sensor 107 is composed of two regions that are independent of each other. Therefore, the CPU 121 reads out the pixel signals output from the respective photoelectric conversion units independently. In step S107, the CPU 121 generates a display preview image from the read pixel signal, and displays this image on the display 131 provided on the back surface (camera back surface) of the imaging device 100. At this time, the CPU 121 adds the output signals of a pair of two photoelectric conversion units of each pixel to convert it into an imaging signal, and further reduces the image according to the number of pixels of the display 131.

  Subsequently, in step S131, the CPU 121 performs focus detection processing (focus control), and calculates the driving amount (lens driving amount) of the third lens group 105 (focus lens). Details of the focus detection process will be described later. In step S151, the CPU 121 determines whether the lens driving amount calculated in step S131 is equal to or less than a predetermined value. If the lens driving amount is less than or equal to the predetermined value, the CPU 121 determines that the in-focus state is in effect, and proceeds to step S153. On the other hand, if the lens driving amount is larger than the predetermined value, the CPU 121 determines that the in-focus state is not in effect, and proceeds to step S152. In step S152, the CPU 121 drives the focus lens (third lens group 105) using the focus drive circuit 126 and the focus actuator 114, and the process proceeds to step S153.

  In step S153, the CPU 121 determines whether or not the photographing switch has been turned on by the photographer. If the photographing switch is not turned on, the process proceeds to step S155. On the other hand, when the photographing switch is turned on, in step S154, the CPU 121 records the photographed image in the flash memory 133 or the like, and proceeds to step S155. In step S155, the CPU 121 determines the state of the main switch. When the main switch is not turned off, that is, when the on state is maintained, the process returns to step S102 and steps S102 to S154 are repeated. On the other hand, if the main switch is turned off in step S155, the process proceeds to step S156, and the shooting is terminated.

  Next, the focus detection process (step S131 in FIG. 10) in the present embodiment will be described in detail with reference to FIG. FIG. 11 is a flowchart showing focus detection processing (focus detection subroutine). Each step of FIG. 11 is mainly executed based on a command from the CPU 121.

  When the process proceeds to step S131 in FIG. 10 (main routine), first, in step S132 in FIG. 11, the CPU 121 recognizes the subject pattern from the preview image, performs face image determination, contrast analysis of the entire photographing screen, and the like. In step S133, the CPU 121 determines a main subject (main subject region) to be focused on based on the result of the subject pattern recognition performed in step S132.

  Subsequently, in step S134, the CPU 121 calculates exit pupil information of the imaging optical system based on the lens information acquired in step S105 of FIG. 10 (exit pupil calculation). Subsequently, in step S135, the CPU 121 extracts signals from the photoelectric conversion units of the pixels 211 to 213 shown in FIG. 2 in the main subject area determined in step S133, and a pair of two images shown in FIG. Are generated for a predetermined number of rows. In step S136, the CPU 121 synthesizes (subtracts) the output signals of the photoelectric conversion units according to the procedure described with reference to FIG. 7, and outputs signals corresponding to the virtual pixels 214 to 216 (virtual pixel group) ( Pixel signal) is calculated. The CPU 121 generates a pair of two image focus detection signals shown in FIG. 9 for a predetermined number of rows.

  Subsequently, in step S137, the CPU 121 uses the exit pupil information calculated in step S134 for each focus detection signal generated in steps S135 and S136 to reduce the light amount imbalance of each signal. Apply shading correction. Thereby, the intensity difference between the pair of two images is reduced, and the focus detection accuracy is improved. Subsequently, in step S138, the CPU 121 performs a correlation operation for calculating the lateral shift amount u with respect to all the focus detection signals subjected to the shading correction. In step S139, the CPU 121 calculates the defocus amount by multiplying the lateral deviation amount u calculated in step S138 by a proportionality constant according to the baseline length of the projection pupil.

  Subsequently, in step S141, the CPU 121 determines the defocus amount Def in the pixel 214 having the short baseline length among the six types of defocus amounts calculated in step S139, and the two types of threshold values DEF1 and DEF2 (DEF1 <DEF2). And compare. For example, 4 mm is set as the threshold DEF1 (first defocus amount), and 10 mm is set as the threshold DEF2 (second defocus amount). When the detected defocus amount Def is larger than the threshold value DEF2, the process proceeds to step S142, and the CPU 121 employs the defocus amount detected based on the focus detection signals of the pixels 214 and 216. On the other hand, when the detected defocus amount Def is equal to or smaller than the threshold value DEF2 and larger than the threshold value DEF1, the process proceeds to step S143, and the CPU 121 determines the defocus amount detected based on the focus detection signal of the pixel 215. adopt. If the detected defocus amount Def is less than or equal to the threshold value DEF1, the process proceeds to step S144, and the CPU 121 adopts the defocus amount detected based on the focus detection signals of the pixels 211 to 213. In steps S142 and S144, since a plurality of focus detection results (a plurality of focus detection signals) are employed, the CPU 121 calculates a final defocus amount by averaging the plurality of focus detection results. . Subsequently, in step S145, the CPU 121 converts the determined defocus amount into a focus lens drive amount (lens drive amount). In step S146, the process returns to the main routine of FIG.

  According to the present embodiment, it is possible to obtain focus detection characteristics suitable for various focus states without increasing the number of divisions of the photoelectric conversion unit of each pixel. Specifically, since the number of divisions of the photoelectric conversion units of all the pixels is set to 2, complication of the structure of the image sensor 107 is avoided. Further, when the defocus amount is small, the output of the photoelectric conversion unit of the existing pixel group is used without being synthesized, so that the base line length for phase difference detection is long and the focusing accuracy is kept high. On the other hand, when the defocus amount is large, the virtual pixel group is generated by synthesizing the outputs of the photoelectric conversion units between different actual pixel groups to perform focus detection. In this case, since a hypothetical projection pupil pair with a narrow projection pupil width and a short base length is generated, the probability that focus detection will be impossible is reduced, and the defocus detection range can be expanded. That is, when the defocus is large, the defocus amount and the defocus direction can be detected without making a mistake, and the focus lens can be driven quickly to the vicinity of the in-focus point. On the other hand, since the focus detection accuracy can be increased in the vicinity of the in-focus state where the defocus amount is small, it is possible to obtain a high-definition image that is focused at high speed and with high accuracy regardless of the focus state. .

  Next, with reference to FIG. 12, the focus detection process (step S131 in FIG. 10) as a modification of the present embodiment will be described in detail. FIG. 12 is a flowchart showing a focus detection process (focus detection subroutine) as a modification. Each step of FIG. 11 is mainly executed based on a command from the CPU 121. Note that steps S132 to S139 in FIG. 12 are the same as the corresponding steps in FIG.

  When the CPU 121 calculates the defocus amount in step S139 in FIG. 12, the process proceeds to step S161. In step S161, the CPU 121 determines correlation reliability regarding the correlation calculation performed in step S138. Here, the correlation reliability is the degree of coincidence between a pair of two image signals (focus detection signals). A high degree of coincidence of signals means that the reliability and accuracy of the focus detection result are high.

  Subsequently, in step S162, the CPU 121 selects a defocus amount (focus detection result) with high correlation reliability from the six types of defocus amounts calculated in step S139. Here, the number of defocus amounts to be selected may be one or plural. When selecting a plurality of defocus amounts, the CPU 121 may use an average value of the selected plurality of defocus amounts. In step S163, the CPU 121 converts the determined defocus amount into a focus lens drive amount (lens drive amount), and returns to the main routine in FIG. 10 in step S164.

  According to the above modification, a highly reliable focus detection result is selected from the calculated plurality of focus detection results. For this reason, an optimum result can be selected regardless of the amount of defocus, and high-speed and high-precision focus adjustment is possible.

  As described above, in this embodiment, the image sensor 107 includes the first pixel group having the first pixels (one of the pixels 211 to 213) divided in the first pattern. The image sensor 107 includes a second pixel group having second pixels (another one of the pixels 211 to 213) divided by a second pattern different from the first pattern. The focus detection unit (CPU 121) performs focus detection based on the output signal of the first pixel and the output signal of the second pixel.

  Preferably, the focus detection unit generates a focus detection signal by combining (by adding, subtracting, or combining) the output signal of the first pixel and the output signal of the second pixel, and based on the focus detection signal Control the focus. More preferably, the focus detection unit performs focus control based on one of the first pixel signal and the second pixel signal according to a predetermined condition. The first pixel signal is a pixel signal (a signal from the photoelectric conversion units 211a to 213b) obtained independently from the output signal of the first pixel or the output signal of the second pixel. The second pixel signal is a pixel signal (a signal from the photoelectric conversion units 214a to 216b) obtained by combining the output signal of the first pixel and the output signal of the second pixel.

  Preferably, the focus detection unit performs focus control based on the first pixel signal when the calculated defocus amount Def is equal to or less than a predetermined defocus amount. The focus detection unit performs focus control based on the second pixel signal when the calculated defocus amount Def is larger than a predetermined defocus amount. That is, the pixel signal (focus detection signal) used for focus control is changed according to the defocus state.

  Further preferably, the focus detection unit outputs either the first pixel signal or the second pixel signal according to a comparison result between the correlation reliability of the first pixel signal and the correlation reliability of the second pixel signal. Based on this, focus control is performed. Here, the first pixel signal and the second pixel signal are each a pair of image signals, and the correlation reliability is the degree of coincidence of the pair of image signals.

  Preferably, the first pixel signal is a signal based on a light beam passing through the first pupil region (at least one of the projection pupils EP1a to EP3b) of the photographing optical system. The second pixel signal is a signal based on a light beam passing through the second pupil region (at least one of the projection pupils EP4a to EP6b) of the photographing optical system. More preferably, the first pupil region is longer than the second pupil region in the pupil division direction (the horizontal direction in FIG. 5 which is the division direction of the pair of projection pupils). Also preferably, the baseline length of the first pupil region is larger than the baseline length of the second pupil region.

  Preferably, the image sensor 107 further includes a third pixel group having a third pixel (the remaining one of the pixels 211 to 213) divided in the third pattern. The first pixel (for example, the pixel 211) includes a first photoelectric conversion unit (photoelectric conversion unit 211a) and a second photoelectric conversion unit (photoelectric conversion unit) that respectively receive light beams that pass through different pupil regions of the photographing optical system. 211b). The second pixel (for example, the pixel 212) includes a third photoelectric conversion unit (photoelectric conversion unit 212a) and a fourth photoelectric conversion unit (photoelectric conversion unit) that respectively receive light beams that pass through different pupil regions of the photographing optical system. 212b). The third pixel (for example, the pixel 213) includes a fifth photoelectric conversion unit (photoelectric conversion unit 213a) and a sixth photoelectric conversion unit (photoelectric conversion unit) that respectively receive light beams that pass through different pupil regions of the photographing optical system. 213b).

  Preferably, the focus detection unit generates a first focus detection signal from output signals of the first photoelectric conversion unit and the second photoelectric conversion unit. A second focus detection signal is generated from the output signals of the third photoelectric conversion unit and the fourth photoelectric conversion unit. Further, a third focus detection signal is generated from the output signals of the fifth photoelectric conversion unit and the sixth photoelectric conversion unit. The first difference signal (S221s), which is the difference between the output signals of the third photoelectric conversion unit and the first photoelectric conversion unit, and the output signals of the second photoelectric conversion unit and the fourth photoelectric conversion unit A fourth focus detection signal is generated by combining the second difference signal (S222s) that is the difference between the first focus detection signal and the second difference signal. Also, a fifth focus detection signal is generated by combining the first difference signal and the third difference signal (S224s) that is the difference between the output signals of the fourth and sixth photoelectric conversion units. To do. Further, a sixth focus detection signal is generated by combining the third difference signal and the fourth difference signal (S223s) which is the difference between the fifth photoelectric conversion unit and the output signal of the third photoelectric conversion unit. To do.

  Preferably, when the calculated defocus amount is equal to or smaller than the first defocus amount (DEF1 ≧ Def), the focus detection unit preferably includes the first focus detection signal, the second focus detection signal, and the third focus detection signal. Focus control is performed based on at least one of the focus detection signals. When the defocus amount is larger than the first defocus amount and less than or equal to the second defocus amount (DEF2 ≧ Def> DEF1), focus control is performed based on the fifth focus detection signal. When the defocus amount is larger than the second defocus amount (Def> DEF2), focus control is performed based on at least one of the fourth focus detection signal and the sixth focus detection signal.

  Preferably, the focus detection means includes a first focus detection signal, a second focus detection signal, a third focus detection signal, a fourth focus detection signal, a fifth focus detection signal, and a sixth focus detection. Focus control is performed based on the focus detection signal selected according to the correlation reliability of each signal.

  Next, a second embodiment of the present invention will be described. The first embodiment generates signals of virtual pixel groups (pixels 214 to 216) having different baseline lengths based on signals of pixel groups (pixels 211 to 213) having a predetermined baseline length, and sets the defocus amount. These signals are selected accordingly. On the other hand, in this embodiment, based on signals of pixel groups having different pupil division characteristics, signals of virtual pixel groups having different pupil division directions are generated and focused on a light and dark pattern in an arbitrary direction of the subject. Perform detection.

  First, the pixel configuration of the image sensor in the present embodiment will be described with reference to FIG. FIG. 13 is a pixel array diagram of the image sensor according to the present embodiment. Similar to the first embodiment, the vertical (Y direction) 6 rows and horizontal (X direction) 6 columns of the two-dimensional C-MOS area sensor are shown. The state observed from the photographing optical system side is shown. A Bayer arrangement is applied to the color filter, and green (G: Green) and red (R: Red) color filters are alternately provided in order from the left in pixels in odd rows. Further, blue (B: Blue) and green (G: Green) color filters are alternately provided in order from the left on the pixels in even rows. Each pixel circle indicates an on-chip microlens, and a plurality of rectangles arranged inside the on-chip microlens indicate a photoelectric conversion unit. Also in this embodiment, the photoelectric conversion units of all the pixels are divided into two regions, but the division form is not uniform for all the pixels, and is composed of a plurality of pixel groups having different division forms. Hereinafter, characteristics of the pixel group of the present embodiment will be described.

  In FIG. 13, row numbers 1 and 2 are configured by a pixel group including a plurality of pixels 311, and one photoelectric conversion unit 311 a in each pixel occupies an upper left quarter region in the square region, and the other The photoelectric conversion unit 311b occupies the remaining three quarters. Row numbers 3 and 4 are formed of a pixel group including a plurality of pixels 312, and photoelectric conversion units 312 a and 312 b in each pixel are equally divided into two in the X direction. Row numbers 5 and 6 are composed of a pixel group including a plurality of pixels 313. One photoelectric conversion unit 313b in each pixel occupies the upper right quarter of the square region, and the other photoelectric conversion unit 313a. Occupies the remaining three quarters. Note that pixels having the same arrangement as row numbers 1 to 6 are repeatedly arranged outside the area shown in FIG.

  Since the correspondence between the division form of the photoelectric conversion unit and the projection pupil in the phase difference detection is as described with reference to FIGS. 2, 4, and 5 of the first embodiment, detailed description is omitted. The characteristics of the projection pupil in FIG. 13 are as follows. Since the pupil division direction of the pixel 311 is a direction (−45 degrees) obtained by rotating the X axis clockwise by 45 degrees (−45 degrees), it is suitable for a subject having a light / dark pattern in this direction. Further, since the area of the pair of photoelectric conversion units is 1: 3, the pair of focus detection signals are subjected to gain correction corresponding to this ratio. Since the pupil division direction of the pixel 312 is a direction along the X axis (0 degree), it is suitable for a subject having a light and dark pattern in this direction. Further, since the areas of the pair of photoelectric conversion units are equal to each other, gain correction is unnecessary. The pupil division direction of the pixel 313 is a direction obtained by rotating the X axis 45 degrees counterclockwise (+45 degrees), and is suitable for a subject having a light and dark pattern in this direction. Further, since the area of the pair of photoelectric conversion units is 3 to 1, gain correction corresponding to this ratio is performed on the pair of focus detection signals.

  Next, with reference to FIG. 14, a method for synthesizing output signals of the photoelectric conversion unit in the present embodiment will be described. FIG. 14 is an explanatory diagram of a method for synthesizing the output signals of the photoelectric conversion units. In the pixel array of the image sensor of FIG. 13, the output signals of the photoelectric conversion units between different pixel groups are synthesized (subtracted), and virtual It is a conceptual diagram in the case of producing | generating the pixel group corresponding to this projection pupil. In FIG. 13, pixels in a range of 6 rows and 6 columns are shown. Among them, a specific column, for example, the extracted 6 pixels in the first column at the left end is shown on the left side of FIG. 14. These 6 pixels are a green (G) and blue (B) pixel 311, a green (G) and blue (B) pixel 312, and a green (G) and blue (B) pixel 313 in order from the top. . FIG. 14 illustrates a method of synthesizing output signals based on green (G) pixels in each pixel group, that is, a total of three types of pixels of row numbers 1, 3, and 5.

  The three types of green (G) pixels (pixels 311 to 313) have a total of six photoelectric conversion units 311a, 311b, 312a, 312b, 313a, and 313b. In the present embodiment, two output signals are taken out from the output signals of these six photoelectric conversion units 311a to 313b and synthesized (subtracted) by the following equations (7) to (10). Then, output signals S321s to S324s of four virtual photoelectric conversion units 321s to 324s as shown on the right side are obtained.

S321s = S311a (7)
S322s = S311b−S312b (8)
S323s = S313a-S312a (9)
S324s = S313b (10)
Here, S311a, S311b, S312a, S312b, S313a, and S313b are output signals of the photoelectric conversion units 311a, 311b, 312a, 312b, 313a, and 313b, respectively. S321s, S322s, S323s, and S324s are output signals of the virtual photoelectric conversion units 321s, 322s, 323s, and 324s, respectively. That is, the four virtual photoelectric conversion units 321 s to 324 s correspond to the output signals of a total of four photoelectric conversion units obtained by dividing the photoelectric conversion unit of a pixel having no pupil division function into two in the X direction and the Y direction. .

  In the present embodiment, two output signals are extracted from the output signals of the four photoelectric conversion units 321 s to 324 s and the following combination operation is performed, whereby a virtual pixel 314 as illustrated on the right side of FIG. 315 (virtual pixel group) is obtained. First, the added value of the output signals of the virtual photoelectric conversion units 321s and 322s is set to 314a, and the added value of the output signals of the virtual photoelectric conversion units 323s and 324s is set to 314b, whereby the virtual pixel 314 (virtual pixel Group). The projection pupil of the pixel 314 is divided in the X direction (lateral direction). Similarly, the addition value of the output signals of the virtual photoelectric conversion units 321s and 323s is 315a, the addition value of the output signals of the virtual photoelectric conversion units 322s and 324s is 315b, and the virtual pixel 315 (virtual pixel group) Get. The projection pupil of the pixel 314 is divided in the Y direction (vertical direction). The same operation is performed on the other green (G) pixels, red (R), and blue (B) pixels, and virtual pixels 314 and 315 arranged in a two-dimensional matrix can be obtained. it can. In addition, since all the pixels virtually generated are described, the diagram becomes complicated. Therefore, on the right side of FIG. 14, only one row for pixels 314 and only one column for pixels 315 are illustrated.

  Next, with reference to FIG. 15, the relationship between the subject image to be focused and the selected pixel group will be described. FIG. 15 is a relationship diagram between a subject image to be detected by a focus detection and a selected pixel group (focus detection region). Four types of subject images as focus detection targets and a pixel group (focus detection selected by each subject image). It is a figure explaining the relationship with an area | region. In FIG. 15, 401 is an imaging region, 411 to 414 are four types of subject images, and 421 to 424 are four types of focus detection regions. The angle is positive in the counterclockwise direction and negative in the clockwise direction with respect to the X axis.

  Reference numeral 411 denotes a subject image in which vertical lines of white and black are alternately arranged, that is, a subject image having a contrast difference in the X-axis direction. When performing phase difference focus detection on such a subject image 411, it is necessary to use pixels that are pupil-divided in the X direction. Therefore, focus detection is performed using the pixel 312 in FIG. 13 and the pixel 314 in FIG. Specifically, a focus detection region 421 extending in the X direction is set at the position of the subject image 411, and focus detection is performed using the pixel 312 and the virtual pixel 314 included in the focus detection region 421. Note that the pixel 312 and the pixel 314 have different focus detection characteristics such as the baseline length of the projection pupil. For this reason, focus detection calculation is performed independently for each pixel group, and processing such as simple averaging or weighted averaging based on the reliability of the results is performed on the plurality of calculated results. It is preferable to output as one defocus amount.

  Reference numeral 412 denotes a subject image in which vertical lines of white and black extending in the +45 degree direction are alternately arranged, that is, a subject image having a contrast difference in the -45 degree direction. In this case, since it is necessary to use pixels that are pupil-divided in the -45 degree direction, focus detection is performed using the pixels 311 in FIG. Specifically, a focus detection area 422 extended in the −45 degree direction is set at the position of the subject image 412, and focus detection is performed using the pixels 311 included in the focus detection area 422.

  Reference numeral 413 denotes a subject image in which vertical lines of white and black extending in the -45 degree direction are alternately arranged, that is, a subject image having a contrast difference in the +45 degree direction. In this case, since it is necessary to use pixels that are pupil-divided in the +45 degree direction, focus detection is performed using the pixels 313 in FIG. Specifically, a focus detection area 423 extending in the +45 degree direction is set at the position of the subject image 413, and focus detection is performed using the pixels 313 included in the focus detection area 423.

  Reference numeral 414 denotes a subject image in which white and black horizontal lines are alternately arranged, that is, a subject image having a contrast difference in the Y-axis direction. When performing phase difference focus detection on such a subject image 414, it is necessary to use pixels that are pupil-divided in the Y direction. Therefore, focus detection is performed using the virtual pixel 315 of FIG. Specifically, a focus detection region 424 extended in the Y direction is set at the position of the subject image 414, and focus detection is performed using a virtual pixel 315 included in the focus detection region 424.

  Next, with reference to FIG. 16, the focus detection process (step S131 in FIG. 10) in the present embodiment will be described in detail. FIG. 16 is a flowchart showing focus detection processing (focus detection subroutine). Each step of FIG. 16 is mainly executed based on a command from the CPU 121. Since the main flow is the same as that of the first embodiment shown in FIG.

  When the process proceeds to step S131 in FIG. 10 (main routine), first in step S232 in FIG. 16, the CPU 121 recognizes the subject pattern from the preview image, and performs face image determination, contrast analysis of the entire shooting screen, and the like. In step S233, the CPU 121 determines a main subject (main subject region) to be focused on based on the result of subject pattern recognition in step S232.

  Subsequently, in step S234, the CPU 121 determines the direction dependency of the light / dark information of the main subject area (the directionality of the subject pattern). That is, the CPU 121 determines which of the subject images 411 to 414 described with reference to FIG. In step S235, the CPU 121 calculates exit pupil information of the imaging optical system based on the lens information acquired in step S105 in FIG. 10 (exit pupil calculation).

  Subsequently, in step S236, the CPU 121 selects a suitable pixel group based on the main subject area determined in step S233 and the determination result in step S234, and generates a focus detection signal. Specifically, the CPU 121 selects a pixel group suitable for focus detection (appropriate for focus detection) from the pixels 311 to 315. Then, the CPU 121 generates a pair of two focus detection signals for a predetermined number of rows from the output signal of the selected pixel group.

  Subsequently, in step S237, the CPU 121 performs so-called shading correction that reduces the light amount imbalance of each focus detection signal generated in step S236, using the exit pupil information calculated in step S235. Thereby, the intensity difference between the pair of two images is reduced, and the focus detection accuracy is improved. Subsequently, in step S238, the CPU 121 performs a correlation operation for calculating the lateral shift amount u with respect to the focus detection signal subjected to the shading correction. In step S239, the CPU 121 calculates the defocus amount by multiplying the lateral deviation amount u calculated in step S238 by a proportional constant according to the baseline length of the projection pupil. In step S240, the defocus amount calculated in step S239 is converted into a focus lens drive amount, and the process returns to the main routine in step S241.

  According to the present embodiment, it is possible to obtain focus detection characteristics suitable for various subject patterns without increasing the number of divisions of the photoelectric conversion unit of each pixel. Specifically, since the number of divisions of the photoelectric conversion units of all the pixels is set to 2, it is possible to avoid complication of the structure of the image sensor. When the direction of the light and dark pattern of the subject image matches the pupil division direction of the actual pixel group, focus detection is performed without combining the outputs of the pixel group. In addition, when the light and dark pattern direction of the subject image does not match the pupil division direction of the actual pixel group, the output of the photoelectric conversion unit is synthesized between a plurality of pixel groups having different pupil division forms, and the pupil division direction of the existing pixel group A virtual projection pupil is generated in which the pupil is divided in a different direction. This makes it possible to detect the focus with respect to any light / dark pattern.

  Next, with reference to FIG. 17, the focus detection process (step S131 in FIG. 10) as a modification of the present embodiment will be described in detail. FIG. 17 is a flowchart showing a focus detection process (focus detection subroutine) as a modification. Each step in FIG. 17 is mainly executed based on a command from the CPU 121. Note that steps S232, S233, and S235 in FIG. 17 are the same as the corresponding steps in FIG. 16, and thus description thereof is omitted.

  When the main subject (main subject region) is determined in step S233, in step S235, the CPU 121 calculates exit pupil information of the photographing optical system based on the lens information acquired in step S105 of FIG. Exit pupil calculation).

  Subsequently, in step S251, the CPU 121 generates a pair of two focus detection signals in the pixels 311 to 313 in the main subject area determined in step S233. In step S252, the CPU 121 generates a pair of two image focus detection signals in the virtual pixels 314 and 315 (virtual pixel group) in the main subject region.

  Subsequently, in step S253, the CPU 121 performs so-called shading correction that reduces the light amount imbalance of each of the generated focus detection signals using the exit pupil information calculated in step S235. Thereby, the intensity difference between the pair of two images is reduced, and the focus detection accuracy is improved. In step S254, the CPU 121 performs a correlation calculation for calculating the lateral shift amount u for all the focus detection signals subjected to the shading correction. In step S <b> 255, the CPU 121 calculates the defocus amount by multiplying the calculated lateral deviation amount u by a proportionality constant according to the baseline length of the projection pupil.

  Subsequently, in step S256, the CPU 121 determines the correlation reliability with respect to the correlation calculation executed in step S254. Here, the correlation reliability is the degree of coincidence between a pair of two image signals, and a high degree of coincidence means that the reliability and accuracy of the focus detection result are high. Subsequently, in step S257, the CPU 121 selects a result (defocus amount) with high correlation reliability from the calculation results of the five types of defocus amounts calculated in step S255. Here, the number to be selected may be one or plural. In the case of a plurality, a value obtained by averaging them may be used. In step S258, the CPU 121 converts the determined defocus amount into a focus lens drive amount, and returns to the main routine in step S259.

  According to the above modification, by selecting a highly reliable result from the calculated plurality of focus detection results, it is possible to select an appropriate result according to the direction characteristics of the light and dark pattern of the subject image. For this reason, high-speed and high-precision focus adjustment is possible.

  As described above, in this embodiment, the image sensor 107 includes the first pixel group having the first pixels (one of the pixels 311 to 313) divided in the first pattern. The image sensor 107 includes a second pixel group having second pixels (another one of the pixels 311 to 313) divided by a second pattern different from the first pattern. The focus detection unit (CPU 121) performs focus detection based on the output signal of the first pixel and the output signal of the second pixel. Preferably, the focus detection unit performs focus control based on one of the first pixel signal and the second pixel signal in accordance with the light / dark distribution information (light / dark pattern) of the subject image. Preferably, the pupil division directions of the first pupil region and the second pupil region are different from each other.

[Other Embodiments]
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. To be executed. In this case, a computer-executable program describing the procedure of the imaging apparatus control method and a storage medium storing the program constitute the present invention.

  According to each embodiment, focus detection is possible even in a large defocus state or a situation where a light and dark pattern exists only in a specific direction without sacrificing imaging performance such as high pixel count, high image quality, high sensitivity, and high-speed readout. It is possible to provide an imaging apparatus that has high focus detection accuracy near the in-focus state. Therefore, according to each embodiment, it is possible to provide an imaging apparatus, an imaging apparatus control method, a program, and a storage medium that can perform high-precision focus detection in various situations while satisfying desired imaging performance. .

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

100 imaging device 107 imaging device 121 CPU (focus detection means)

Claims (18)

A first pixel group having a first pixel divided by a first pattern, and a second pixel group having a second pixel divided by a second pattern different from the first pattern; An image sensor comprising:
A combined signal having a base length or pupil division characteristic different from that of the first pixel group and the second pixel group for detecting the defocus amount is obtained as an output signal of the first pixel and the second pixel group. An imaging device comprising: generating means for generating based on an output signal . The imaging apparatus according to claim 1, further comprising a focus detection means for performing focus detection based on the defocus amount detected from the composite signal.   The focus detection unit may be a first pixel signal obtained independently from an output signal of the first pixel or an output signal of the second pixel, or the first pixel, according to a predetermined condition. The imaging apparatus according to claim 1, wherein focus control is performed based on one of a second pixel signal obtained by combining the output signal of the second pixel and the output signal of the second pixel. The focus detection means includes
When the calculated defocus amount is equal to or less than a predetermined defocus amount, the focus control is performed based on the first pixel signal,
The imaging apparatus according to claim 3, wherein the focus control is performed based on the second pixel signal when the calculated defocus amount is larger than the predetermined defocus amount.   The focus detection unit may select one of the first pixel signal and the second pixel signal according to a comparison result between the correlation reliability of the first pixel signal and the correlation reliability of the second pixel signal. The image pickup apparatus according to claim 3, wherein the focus control is performed based on the image pickup device. Each of the first pixel signal and the second pixel signal is a pair of image signals,
The imaging apparatus according to claim 5, wherein the correlation reliability is a degree of coincidence between the pair of image signals.   4. The focus detection unit according to claim 3, wherein the focus detection unit performs the focus control based on one of the first pixel signal and the second pixel signal in accordance with brightness distribution information of a subject image. Imaging device. The first pixel signal is a signal based on a light beam passing through a first pupil region of the photographing optical system,
The imaging device according to any one of claims 3 to 7, wherein the second pixel signal is a signal based on a light beam passing through a second pupil region of the photographing optical system.   The imaging apparatus according to claim 8, wherein the first pupil region is longer than the second pupil region in a pupil division direction.   10. The imaging apparatus according to claim 8, wherein a baseline length of the first pupil region is larger than a baseline length of the second pupil region.   The imaging apparatus according to claim 8, wherein pupil division directions of the first pupil region and the second pupil region are different from each other. The imaging device further includes a third pixel group having third pixels divided by a third pattern,
The first pixel includes a first photoelectric conversion unit and a second photoelectric conversion unit that respectively receive light beams that pass through different pupil regions of the imaging optical system,
The second pixel includes a third photoelectric conversion unit and a fourth photoelectric conversion unit that respectively receive light beams that pass through different pupil regions of the photographing optical system,
12. The third pixel includes a fifth photoelectric conversion unit and a sixth photoelectric conversion unit that respectively receive light beams that pass through different pupil regions of the photographing optical system. The imaging device according to any one of the above. The focus detection means includes
Generating a first focus detection signal from output signals of the first photoelectric conversion unit and the second photoelectric conversion unit;
Generating a second focus detection signal from output signals of the third photoelectric conversion unit and the fourth photoelectric conversion unit;
Generating a third focus detection signal from the output signals of the fifth photoelectric conversion unit and the sixth photoelectric conversion unit;
A first difference signal that is a difference between an output signal of the third photoelectric conversion unit and an output signal of the first photoelectric conversion unit, and an output signal of the second photoelectric conversion unit and the fourth photoelectric conversion A fourth focus detection signal is generated by combining the second difference signal, which is a difference from the output signal of the unit,
The fifth focus detection signal is obtained by combining the first difference signal and the third difference signal that is the difference between the output signal of the fourth photoelectric conversion unit and the output signal of the sixth photoelectric conversion unit. Generate
The sixth focus detection signal is obtained by combining the third difference signal and the fourth difference signal that is the difference between the output signal of the fifth photoelectric conversion unit and the output signal of the third photoelectric conversion unit. The imaging device according to claim 12, wherein the imaging device is generated. The focus detection means includes
When the calculated defocus amount is equal to or less than the first defocus amount, based on at least one of the first focus detection signal, the second focus detection signal, and the third focus detection signal. Focus control,
When the defocus amount is larger than the first defocus amount and less than or equal to the second defocus amount, the focus control is performed based on the fifth focus detection signal,
The focus control is performed based on at least one of the fourth focus detection signal and the sixth focus detection signal when the defocus amount is larger than the second defocus amount. Item 14. The imaging device according to Item 13.   The focus detection means includes the first focus detection signal, the second focus detection signal, the third focus detection signal, the fourth focus detection signal, the fifth focus detection signal, and the first focus detection signal. The imaging apparatus according to claim 13, wherein focus control is performed based on a focus detection signal selected according to a correlation reliability of each of the six focus detection signals. A first pixel group having a first pixel divided by a first pattern, and a second pixel group having a second pixel divided by a second pattern different from the first pattern; A method for controlling an imaging apparatus that performs focus detection based on an output signal from an imaging element including:
A combined signal having a base length or pupil division characteristic different from that of the first pixel group and the second pixel group for detecting the defocus amount is obtained as an output signal of the first pixel and the second pixel group. Generating based on the output signal of the pixel;
And a step of performing focus detection based on a defocus amount detected from the combined signal . A first pixel group having a first pixel divided by a first pattern, and a second pixel group having a second pixel divided by a second pattern different from the first pattern; A program for performing focus detection based on an output signal from an image pickup device including:
A combined signal having a base length or pupil division characteristic different from that of the first pixel group and the second pixel group for detecting the defocus amount is obtained as an output signal of the first pixel and the second pixel group. Generating based on the output signal of the pixel;
A program for causing a computer to execute focus detection based on a defocus amount detected from the combined signal .   A storage medium storing the program according to claim 17.