JP2013246375A - Image capturing device and control method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image capturing device which offers a high-speed AF function with high focusing accuracy.SOLUTION: An image capturing device includes; an image sensor; a pupil division unit which confines a light flux that enters each pixel on the image sensor to each specific pupil region of an imaging lens; an image generation unit which, using an electrical signal obtained from the image sensor, alters combination of pixels on the image sensor depending on a chosen specific imaging area such that an image signal for the specific imaging area is obtained, and adds the result to generate an image signal for the specific imaging area; an image addition control unit which controls the pupil regions to be added by the image generation unit; a contrast evaluation unit which computes contrast of the image signal obtained from the image generation unit; and a focus evaluation unit which derives a focus evaluation value from an output from the contrast evaluation unit obtained when the specific imaging area is changed.

Description

  The present invention relates to an autofocus (hereinafter referred to as AF) technique in an imaging apparatus such as a digital camera.

  Conventionally, as an autofocus device for focus adjustment, many digital single-lens reflex cameras have so-called phase difference AF, and compact cameras have contrast AF. As a feature of these AFs, the phase difference AF is capable of high-speed focus adjustment, and the contrast AF is capable of strict focus adjustment.

  For example, Patent Document 1 discloses a method of simultaneously operating a diaphragm when acquiring an image for performing contrast evaluation. Japanese Patent Application Laid-Open No. 2004-228561 discloses an image pickup apparatus that can individually receive light beams that have passed through different pupil regions, and that generates an image that has been subjected to focus adjustment after image pickup.

JP 2003-279838 A JP 2007-4471 A

  However, in the conventional techniques disclosed in the above-mentioned patent documents, it may be difficult to achieve both the focusing speed and the focusing accuracy. That is, in Patent Document 1, since it is necessary to acquire a large number of images in order to perform contrast evaluation, the focusing speed may be slow. In Patent Document 2, an image in which the focus position is changed can be obtained later, but a method of utilizing a signal for so-called focusing is not sufficiently described.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an imaging apparatus capable of performing AF with high speed and high focus accuracy.

  An image pickup apparatus according to the present invention includes an image pickup device, pupil dividing means for limiting a light beam incident on a pixel on the image pickup device to only a specific pupil region of a photographing lens, and an electric signal obtained from the image pickup device. The image shift means for giving a different shift for each pupil region that has passed through, the image generation means for multiplying the electrical signal obtained from the image shift means by a predetermined positive value, and the addition in the image generation means Image addition control means for controlling the pupil region to be performed, contrast evaluation means for calculating the contrast of the image obtained from the image generation means, and output of the contrast evaluation means when the shift amount of the image shift means is changed And a focus evaluation means for obtaining a focus evaluation value from.

  According to the present invention, it is possible to provide an imaging apparatus capable of performing AF with high speed and high focus accuracy.

1 is a block diagram showing an electrical configuration of a digital camera and a lens that are an embodiment of an imaging apparatus of the present invention. FIG. 3 is a diagram illustrating a main part of a photographing optical system according to an embodiment of the present invention. The flowchart which shows the operation | movement for obtaining the focus evaluation value of one Embodiment. The flowchart which shows the operation | movement for obtaining the focus evaluation value of one Embodiment. The schematic diagram explaining operation | movement of contrast AF. The schematic diagram which shows the reconstruction operation | movement of an image. FIG. 6 is a schematic diagram showing another example of an optical system applicable to the present invention.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing an electrical configuration of a digital camera and a lens which are an embodiment of an imaging apparatus of the present invention. A camera system including a camera 1 and a lens (photographing lens) 2 has an imaging system, an image processing system, a recording / reproducing system, and a control system. The imaging system includes a photographing optical system 3 and an imaging element 6, and the image processing system includes an image processing unit 7. The recording / reproducing system includes a memory 8 and a display unit 9, and the control system includes a camera system control circuit 5, an operation detection unit 10, a lens system control circuit 12, and a lens driving unit 13. The lens driving unit 13 can drive a focus lens, a shake correction lens, a diaphragm, and the like.

  The imaging system is an optical processing system that forms an image of light from an object on the imaging surface of the imaging device 6 via the imaging optical system 3. The image sensor 6 converts the formed optical image into a predetermined electric signal. Microlenses are arranged in a lattice pattern on the surface of the imaging element 6 to form a so-called microlens array (hereinafter referred to as MLA). In this embodiment, the MLA constitutes pupil dividing means. Details of the functions and arrangement of the MLA will be described later with reference to FIG. As will be described later, since the focus evaluation amount / appropriate exposure amount can be obtained from the imaging element 6, the imaging optical system 3 is appropriately adjusted based on this signal, so that an appropriate amount of object light can be obtained from the imaging element 6. And an object image is formed in the vicinity of the image sensor 6.

  The image processing unit 7 includes an A / D converter, an interpolation calculation circuit, a white balance circuit, a gamma correction circuit, and the like, and can generate an image for recording. In addition, an image shift unit, an image generation unit, a contrast evaluation unit, a correlation calculation unit, and the like, which are main parts of the present embodiment, can be included. However, in the present embodiment, description will be given assuming that these elements are arranged in the camera system control circuit 5.

  The memory 8 includes a processing circuit necessary for recording in addition to an actual storage unit. The memory 8 outputs to the storage unit and generates and stores an image to be output to the display unit 9. Further, the memory 8 compresses images, moving images, sounds, and the like using a predetermined method.

  The camera system control circuit 5 generates and outputs a timing signal at the time of imaging. In response to an external operation, the imaging system, the image processing system, and the recording / reproducing system are controlled. For example, the operation detection circuit 10 detects that a shutter release button (not shown) is pressed, and controls driving of the image sensor 6, operation of the image processing unit 7, compression processing of the memory 8, and the like. Further, the display unit 9 controls the state of each segment of the information display device that displays information on a liquid crystal monitor or the like.

  Next, the adjustment operation of the optical system by the control system will be described. An image processing unit 7 is connected to the camera system control circuit 5 and an appropriate focal position and aperture position are obtained based on a signal from the image sensor 6. The camera system control circuit 5 issues a command to the lens system control circuit 12 via the electrical contact 11, and the lens system control circuit 12 appropriately controls the lens driving unit 13. Further, a camera shake detection sensor (not shown) is connected to the lens system control circuit 12, and in the camera shake correction mode, the camera shake correction sensor is appropriately controlled based on the signal of the camera shake detection sensor. To do.

  FIG. 2 is a diagram for explaining a main part of the photographing optical system in the present embodiment. In the present embodiment, it is necessary to acquire angle information in addition to the position of the light beam, which is so-called light space information. In the present embodiment, an MLA is arranged in the vicinity of the imaging plane of the photographing optical system 3 for obtaining angle information, and a plurality of pixels are associated with one lens constituting the MLA.

  FIG. 2A is a diagram schematically illustrating the relationship between the image sensor 6 and the MLA 20. FIG. 2B is a schematic diagram showing the correspondence between the pixels of the image sensor and the MLA. FIG. 2C is a diagram showing that pixels provided under the MLA by the MLA are associated with a specific pupil region.

  As shown in FIG. 2A, the MLA 20 is provided on the image sensor 6, and the front principal point of the MLA 20 is arranged in the vicinity of the imaging plane of the photographing optical system 3. FIG. 2A shows a state where the MLA is viewed from the side and from the front of the imaging apparatus, and the lens of the MLA is disposed so as to cover the pixels on the image sensor 6 when viewed from the front of the imaging apparatus. In FIG. 2A, the microlenses constituting the MLA are illustrated in large size so that the microlenses are easy to see, but each microlens is actually only about several times as large as a pixel. The actual size will be described with reference to FIG.

  FIG. 2B is a partially enlarged view from the front of the apparatus of FIG. A grid-like frame shown in FIG. 2B indicates each pixel of the image sensor 6. On the other hand, each microlens constituting the MLA is indicated by thick circles 20a, 20b, 20c, and 20d. As apparent from FIG. 2B, a plurality of pixels are assigned to one microlens. In the example of FIG. 2B, 5 rows × 5 columns = 25 pixels are one microlens. Is provided against. That is, the size of each microlens is 5 × 5 times the size of the pixel.

  FIG. 2C is a diagram in which the image pickup element 6 is cut so that the longitudinal direction of the sensor includes the optical axis of the microlens and the horizontal direction of the drawing. 2, 21, 22, 23, 24, and 25 in FIG. 2C indicate pixels (one photoelectric conversion unit) of the image sensor 6. On the other hand, the figure shown above FIG. 2C shows the exit pupil plane of the photographing optical system 3. Actually, when the direction is aligned with the sensor diagram shown in the lower part of FIG. 2 (c), the exit pupil plane is in the direction perpendicular to the paper plane of FIG. 2 (c). It is changing. In FIG. 2C, one-dimensional projection / signal processing will be described to simplify the description. In an actual device, this can be easily extended to two dimensions.

  The pixels 21, 22, 23, 24, and 25 in FIG. 2C are in a positional relationship corresponding to 21a, 22a, 23a, 24a, and 25a in FIG. As shown in FIG. 2C, each pixel is designed to be conjugate with a specific area on the exit pupil plane of the photographing optical system 3 by the microlens 20. In the example of FIG. 2C, the pixel 21 and the region 31 correspond to the pixel 22 and the region 32, the pixel 23 and the region 33, the pixel 24 and the region 34, and the pixel 25 and the region 35, respectively. That is, only the light beam that has passed through the region 31 on the exit pupil plane of the photographing optical system 3 enters the pixel 21. The same applies to the other pixels. As a result, it is possible to obtain angle information from the passing area on the pupil plane and the positional relationship on the image sensor 6.

  Processing for obtaining a focus evaluation value (focus evaluation value) from a signal from the image sensor 6 using the photographing optical system shown in the present embodiment will be described with reference to FIGS.

  3 and 4 are flowcharts showing the operation for obtaining the focus evaluation value of this embodiment. 3A shows the overall operation for obtaining the focus evaluation value, FIG. 3B shows the operation of the image shift means, FIG. 4A shows the operation of the image generation means, and FIG. 4B shows the contrast evaluation. The operation of each means is shown.

  Description will be given in order of each step from FIG. Step S1 indicates the start of the focus evaluation value acquisition operation. For example, this is the case when the operation detection unit 10 shown in FIG. 1 detects a specific operation from the photographer (for example, pressing the release button).

  Step S2 corresponds to acquiring data by exposing the image sensor 6 for an appropriate time and reading (A / D conversion). An appropriate exposure amount in photographing can be calculated from the exposure time and the exposure amount at this time. However, since it is not the main part of this embodiment, description is omitted.

  Steps S3 to S11 form a loop. In step S3, a loop for performing focus evaluation using contrast in a plurality of times is formed. That is, it is rough at first, and then the focus is adjusted precisely. The operation for coarse focus adjustment is called coarse adjustment, and the operation for fine focus adjustment is called fine adjustment.

  In step S4, a pupil position to be added is determined in accordance with the current adjustment. In coarse adjustment, evaluation is performed while roughening the evaluation position in the next step S5 and thereafter. For this reason, in order to avoid falling into the local minimum, the pupil position to be added so that the contrast evaluation for the focus position becomes slow (= physically corresponds to obtaining an image with a deep depth of field). To decide. As an example, the area to be added may be set to be small and adjacent. For example, in FIG. 2C, only the region 33 and the region 32 may be added. In order to make it more general, the number of pupil positions added corresponding to coarse tone may be larger than 2 and smaller than the number of pupil divisions.

  On the other hand, since there is little possibility of falling into a local minimum at the time of fine adjustment, in order to improve the focus accuracy, the contrast evaluation for the focus position becomes steep (= physically obtaining an image with a shallow depth of field) Corresponding), the pupil position to be added is determined. As an example, a large number of areas to be added may be set. For example, in FIG. 2C, all of the areas 31 to 35 may be added. With this setting, the focus accuracy can be improved.

  As another method, in order to improve the sensitivity to focus, it may be set so as not to include the center of the exit pupil (= a so-called ring-shaped pupil). In FIG. 2C, the region 31 and the region 35 may be added. With this setting, the focus accuracy can be improved.

  As another method, the pupil position to be added is determined so as to coincide with the F number of the recorded (= development) image so that the contrast at the time of recording becomes optimum. For example, in FIG. 2C, when developing a slightly narrowed image, the region 32 to the region 34 may be added. With this setting, it is possible to improve the focus accuracy by eliminating the influence of aberration (when there is an aberration, the best focus position changes depending on the position of the passing pupil).

  Furthermore, it is possible to increase the speed by performing the coarse / fine scan in the loop from step S3 to step S11. As described above, in order to avoid the local minimum, in the rough adjustment, the pupil position to be added is adjusted so that the contrast evaluation with respect to the focus position becomes slow. If an attempt is made to obtain a global optimal solution without performing such an operation, a very large number of sampling points are required. That is, it is necessary to obtain a focus evaluation value at many points with a very fine pitch by using an evaluation value having high sensitivity to focus. On the other hand, when the method of coarse / fine scan is used, a rough rough peak is searched for during coarse adjustment, and fine sampling is performed in the vicinity of the peak obtained by coarse adjustment during fine adjustment. By performing this operation, the global optimum solution can be reached with fewer sampling points. In this way, as is well known in general block matching and the like, it is possible to increase the speed by performing the coarse / fine scan even in the focus evaluation by contrast.

  As described above, step S4 controls the pupils to be added, and operates as image addition control means in the present embodiment.

  Steps S5 to S9 form a loop, and an evaluation value (corresponding to an image reconstruction position, which will be described later with reference to FIG. 6) is set in advance by shifting the evaluation position by an appropriate step from a predetermined initial value. Calculate until. The above-mentioned initial values can be set differently depending on the coarse / fine adjustment. In step S5, the coarse adjustment is set to be coarse and wide, and the fine adjustment is set to fine and narrow.

  In step S6, the image shift means is operated to obtain a result. Details of the operation of the image shift means will be described later with reference to FIG. In step S7, the image generating means is operated to obtain a result. Details of the operation of the image generating means will be described later with reference to FIG.

  In step S8, the contrast evaluation means is operated to obtain a result. Details of the operation of the contrast evaluation means will be described later with reference to FIG.

  In step S10, the focus position is determined based on the last result obtained in step S8 (since step S8 is in the loop from step S5 to step S9, a plurality of evaluation values can be obtained while changing the focus position). That is, the position where the contrast is highest is determined and set as the best focus position. In rough adjustment, an approximate focus position is determined, and initial values, end values, and the like set in S5 of the next loop are set. In fine adjustment, the result obtained here becomes the focus position of the final image.

  A series of operation | movement is complete | finished by step S12. As a result, reading of the image sensor 6 is performed only once in step S2, and a focus evaluation value including a contrast evaluation value can be obtained.

  Details of the operation of the image shift means (image shift processing) will be described with reference to FIG. Step S21 indicates the start of the operation of the image shift means.

  Steps S22 to S26 form a loop. In step S22, loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 2, since it is divided into 25, calculation according to each pupil position of 25 is performed. As will be described later with reference to FIG. 6, considering the reconstruction of the image, if the incident angle is different even on the same reconstruction surface (if the exit pupil is sufficiently far away, it is almost synonymous with the different pupil areas passing through. ) The amount of image shift is different. This is a loop for appropriately reflecting this.

  In step S23, the image shift amount in each pupil region corresponding to the evaluation position is calculated based on the data from step S24. In step S24, the correspondence between each pixel stored in the memory 8 and the MLA, that is, information indicating which pupil region of each pixel is receiving light is read.

  In step S25, pixels that have obtained light rays with the same incident angle (obtain light rays from the same pupil region) are shifted based on the information in step S23. As an actual operation, when selecting an addition target pixel for image generation by image generation means (image generation processing) from output data from the image sensor 6 stored in the memory 8, the selection address is shifted. Let Alternatively, the pixels corresponding to the evaluation positions are rearranged on the memory 8. For example, pixels 25a and 25b in FIG. 2 correspond to pixels that obtain light rays having the same incident angle. There are as many such pixels as the number of microlenses constituting the MLA. The image shift process will be illustrated later using FIG.

  In step S27, the process returns to the caller step S6.

  Details of the operation of the image generating means will be described with reference to FIG. Step S31 indicates the start of the operation of the image generating means.

In step S32, the area data for addition in step S35 is initialized (filled with 0). The size of the data area at this time may be as much as the number of MLA, and it is convenient if the data gradation can store the product of the original data gradation and the number of pupil divisions. For example, if the original data is 8 bits and divided into 25, if 13 bits (> 8 bits + log ₂ 25), it is not necessary to consider overflow of data.

  Steps S33 to S37 form a loop. In step S33, loop calculation is executed according to the number of microlenses constituting the MLA. For example, in the example illustrated in FIG. 4, the number of microlens is the number of pixels of the original image sensor ÷ 25 (number of pupil divisions).

  Steps S34 to S37 form a loop. In step S34, the loop calculation is executed by the number corresponding to the number of pupil divisions. For example, in the example shown in FIG. 2, since the light is divided into 25, there are light fluxes from 25 pupil positions.

  In step S35, it is determined whether or not the region is to be added. As described in FIG. 3A, the pupil to be added is instructed by the image addition control means in step S4. When it corresponds to the pupil to be added, the process proceeds to step S36 and is added. In other cases, the process proceeds to step S37. If the shift amount is not an integer multiple of the pixel, the addition is performed while being appropriately divided in addition step S36. What is necessary is just to add appropriately according to the overlapping area. The image generating means will be illustrated later using FIG.

  In step S39, the process returns to the caller step S7.

  The details of the operation of the contrast evaluation means will be described with reference to FIG. Step S41 indicates the start of the operation of the contrast evaluation means.

  In step S42, the number of evaluation points for contrast evaluation and the size of the evaluation frame are set. When the number of evaluation points increases, it becomes possible to cover the entire screen, but there is a problem that it takes time to evaluate. Set appropriately according to user settings. On the other hand, if the size of the evaluation frame is increased, it is possible to focus even on textures that do not have a pattern locally, but if it is too large, images of subjects with different distances are evaluated simultaneously. A so-called perspective conflict will occur. The size of the evaluation frame is set appropriately so that these problems can be solved.

  In step S43, appropriate filtering is performed. Appropriate filtering may be performed by focusing on a lower frequency during coarse adjustment, and by focusing on a higher frequency than during coarse adjustment during fine adjustment. By doing so, it is possible to improve the focus accuracy while avoiding the local minimum.

  Steps S44 to S52 form a loop. In step S44, the calculation is repeated so as to obtain an evaluation value corresponding to the evaluation number determined in step S42.

  Steps S45 to S47 form a loop. In step S45, the primary contrast calculation is performed in the range of the number of pixels corresponding to the size of the evaluation frame determined in step S42. The primary contrast calculation may be calculated by Σ | Si-Si-1 | as in step S46. Here, Si represents the luminance output from the i-th image generating means. In this way, the luminance difference between adjacent pixels can be integrated. However, in the description of this step, an expression in which images are arranged one-dimensionally is shown for the sake of clarity. In a two-dimensional image, the luminance difference in both the vertical and horizontal directions may be integrated, or only one luminance may be integrated.

Steps S48 to S50 form a loop. In step S49, the secondary contrast calculation is performed in the range of the number of pixels corresponding to the size of the evaluation frame determined in step S42. The secondary contrast calculation may be calculated by Σ (Si−Si−1) ² as in step S49. In the description of this step, an expression in which images are arranged one-dimensionally is shown for the sake of clarity.

  FIG. 5 schematically shows changes in Si when the focus position is changed. In FIG. 5, the horizontal axis is an arrangement of pixels of an image generated on each image plane (that is, pixels after addition by image generation processing) arranged in one direction (any of vertical, horizontal, diagonal, etc.) The vertical axis represents the luminance value of each pixel. In FIG. 5, the middle figure is in the best focus state, and the upper figure and the lower figure show that the image is out of focus. Further, when the focus position is changed as shown in FIG. 5, a location where the brightness change value becomes large (= a location where the contrast is high, the middle diagram in FIG. 5) appears, and the contrast decreases before and after that. However, even when the contrast is reduced, the integral (= primary contrast) of the luminance difference between adjacent pixels does not change much. In FIG. 5, the primary contrast of Si-3 to Si + 3 is almost the difference between Si-3 and Si + 3 and does not change greatly with respect to the focus position. On the other hand, the secondary contrast changes greatly depending on the focus position.

  In step S51, the contrast evaluation value is obtained by dividing the secondary contrast by the square of the primary contrast. In this way, it is possible to obtain the contrast evaluation value of the subject according to the focus position while being normalized with respect to the luminance.

  In this embodiment, the contrast evaluation value is obtained by the method as described above. However, other calculation methods can be used as long as the contrast evaluation value according to the focus variation is obtained. For example, a method of preparing band pass filters having different pass bands and using the output ratio can be considered.

  In step S53, the process returns to the caller step S8.

  Next, image shift and image generation are schematically shown in FIG. 6, and the usefulness of contrast calculation by image reconstruction will be described. FIG. 6 is lined up from the top (a), (b), and (c). FIG. 6 (b) shows the surface where the image sensor 6 actually exists and the image is acquired, and FIG. 6 (b) shows a reconstruction surface on the object side (referred to as reconstruction surface 1), and FIG. 6 (c) shows a reconstruction surface on the side farther from the object side than FIG. 6 (b) (reconstruction surface 2 and Each).

  In FIG. 6B, X1, i, X2, i, X3, i, X4, i, and X5, i pass through pupil regions 1, 2, 3, 4, and 5, respectively, and enter the microlens Xi. The obtained data is shown. That is, of the subscripts, the first half indicates the passing pupil region, and the second half indicates the pixel number. Also, in FIG. 6, data is described as having only a one-dimensional spread for the sake of clarity. In relation to the physical position, X1, i is the data obtained from the 21 region in FIG. 2 (c), X2, i is the data obtained from the 22 region in FIG. 3, 4, and 5 indicate that they correspond to the regions 23, 24, and 25, respectively.

  In order to generate an image on the acquisition surface, data incident on the microlens Xi may be added as shown in FIG. Specifically, the integrated value in the angular direction of the light incident on Xi can be obtained with Si = X1, i + X2, i + X3, i + X4, i + X5, i. As a result, an image similar to that of a normal camera is generated.

  What is necessary is just to restrict | limit the pupil to add at the time of rough adjustment. FIG. 6 shows an example limited to three regions closer to the center. Specifically, it is assumed that Si = X2, i + X3, i + X4, i and a light beam incident on Xi. By doing so, an image having a deep depth of field can be obtained in the same manner as when the aperture diameter is physically reduced. For fine adjustment, a method of adding all of them (Si = X1, i + X2, i + X3, i + X4, i + X5, i), a method of using a ring-shaped pupil (Si = X1, i + X5, i), a method of setting the same area as the recording, etc. It is done. FIG. 6 shows an example of adding all during fine adjustment.

  Next, a method for generating an image on the reconstruction plane 1 will be considered. As described with reference to FIG. 2, the imaging optical system of the present embodiment limits the luminous flux incident on each pixel to a specific pupil region, and thus the incident angle is known. The position of each pixel on the reconstruction plane is reconstructed along this angle. Specifically, it is assumed that a pupil region with a subscript of 1 such as X1, i is incident at an angle as shown at 41 on the right side of FIG. Hereinafter, it is assumed that the subscripts 2, 3, 4, and 5 of the pupil region correspond to 42, 43, 44, and 45, respectively. At this time, the light beam incident on the microlens Xi on the reconstruction surface 1 is scattered and incident on the acquisition surface from Xi-2 to Xi + 2. More specifically, it is distributed in X1, i-2, X2, i-1, X3, i, X4, i + 1, X5, i + 2. In order to restore the image on the reconstruction plane 1 as well as Xi, it is understood that the image may be shifted and added according to the incident angle. In order to generate an image on the reconstruction plane 1, a pupil region with a subscript of 1 shifts 2 pixels to the right, a pupil region with a subscript of 2 shifts to the right with a 1 pixel shift, and a pupil region has a subscript. Shifting according to the incident angle by shifting 3 characters without shift, shifting pupil pixels with 4 subscripts by 1 pixel to the left, and shifting pupil regions with 5 subscripts by 2 pixels to the left Can be given. Thereafter, the data in the reconstruction plane 1 can be obtained by adding in the vertical direction of FIG. Specifically, the integrated value in the angular direction of the light incident on Xi can be obtained on the reconstruction plane 1 with Si = X1, i-2 + X2, i-1 + X3, i + X4, i + 1 + X5, i + 2. As a result, an image on the reconstruction surface was obtained.

  In coarse adjustment, the pupils to be added may be limited in the same manner as in FIG. Specifically, it is assumed that Si = X2, i-1 + X3, i + X4, i + 1 and the light beam incident on Xi on the reconstruction surface 1. A method of adding all during fine adjustment (Si = X1, i-2 + X2, i-1 + X3, i + X4, i + 1 + X5, i + 2), a method of using a ring-shaped pupil excluding the center (Si = X1, i-2 + X5) , i + 2), a method of making the same area as recording (at the time of generating a recorded image), and the like.

  Here, if Xi has a bright spot on the reconstruction plane 1, X1, i-2, X2, i-1, X3, i, X4, i + 1, X5, i + 2 on the acquisition plane. It is dispersed and in a so-called blurred state. However, when the image on the reconstruction plane 1 described above is generated, a bright spot is generated again in Xi, and an image with high contrast is obtained. That is, so-called contrast AF can be performed by reconstructing the image and calculating the contrast.

  Further, as can be seen from FIG. 6C, an image can be generated on the reconstruction plane 2 in the same manner as the reconstruction plane 1. If the direction in which the reconstruction plane is arranged is different (meaning opposite to the object), it is only necessary to reverse the direction of shifting.

  As described above, in order to obtain an image signal of a predetermined image plane regardless of whether it is the acquisition plane or the reconstruction planes 1 and 2, each pixel on the image sensor is combined according to the predetermined image plane. What is necessary is just to produce | generate an image signal by making it differ.

  An example of another optical system applicable to the present embodiment will be described with reference to FIG. FIG. 7 is a diagram schematically showing a state in which light rays from an object (subject) form an image on the image sensor 6. FIG. 7A corresponds to the optical system described in FIG. 2 and is an example in which the MLA 20 is disposed in the vicinity of the imaging surface of the photographing optical system 3. FIG. 7B shows an example in which the MLA 20 is arranged closer to the object than the imaging plane of the photographing optical system 3. FIG. 7C shows an example in which the MLA 20 is arranged on the side farther from the object than the imaging plane of the photographing optical system 3.

  In FIG. 7, 6 is an image sensor, 20 is an MLA, 31 to 35 are pupil regions used in FIG. 2, 51 is an object plane, 51a and 51b are appropriate points on the object, and 52 is imaging optics. In the pupil plane of the system, 61, 62, 71, 72, 73, 81, 82, 83, and 84 indicate specific microlenses on the MLA, respectively. In FIG. 7B and FIG. 7C, 6a represents a virtual image sensor, and 20a represents a virtual MLA. These are shown for reference in order to clarify the correspondence with FIG. Also, a light beam that passes from the point 51a on the object and passes through the regions 31 and 33 on the pupil plane is indicated by a solid line, and a light beam that passes from the point 51b on the object and passes through the regions 31 and 33 on the pupil plane is indicated by a broken line. did.

  In the example of FIG. 7A, as described with reference to FIG. 2, the MLA 20 is disposed in the vicinity of the imaging plane of the photographing optical system 3, so that the imaging device 6 and the pupil plane 52 of the photographing optical system have a conjugate relationship. is there. Furthermore, the object plane 51 and the MLA 20 are in a conjugate relationship. Therefore, the light beam emitted from the point 51a on the object reaches the microlens 61, the light beam emitted from the point 51b reaches the microlens 62, and the light beams that have passed through the regions 31 to 35 respectively correspond to those provided below the microlens. Reach the pixel.

  In the example of FIG. 7B, the light beam from the photographing optical system 3 is imaged by the microlens 20, and the imaging element 6 is provided on the imaging surface. By arranging in this way, the object plane 51 and the image sensor 6 are in a conjugate relationship. The light beam that has exited from the point 51a on the object and passed through the region 31 on the pupil plane reaches the microlens 71, and the light beam that has exited from the point 51a on the object and passed through the region 33 on the pupil plane reaches the microlens 72. To do. The light beam that has exited from the point 51b on the object and passed through the region 31 on the pupil plane reaches the microlens 72, and the light beam that has exited from the point 51b on the object and passed through the region 33 on the pupil plane reaches the microlens 73. To do. The light beam that has passed through each microlens reaches a corresponding pixel provided under the microlens. In this way, images are formed at different positions depending on the point on the object and the passing area on the pupil plane. If these are rearranged at positions on the virtual imaging surface 6a, the same information as in FIG. 7A can be obtained. That is, information on the pupil region (incident angle) that has passed through and the position on the imaging device can be obtained.

  In the example of FIG. 7C, the microlens 20 re-images the light beam from the photographic optical system 3 (this is called re-imaging because the light beam once imaged is diffused). ), An image pickup device 6 is provided on the image plane. By arranging in this way, the object plane 51 and the image sensor 6 are in a conjugate relationship. The light beam that has exited from the point 51a on the object and passed through the region 31 on the pupil plane reaches the micro lens 82, and the light beam that has exited from the point 51a on the object and passed through the region 33 on the pupil plane reaches the micro lens 81. To do. The light beam that has exited from the point 51b on the object and passed through the region 31 on the pupil plane reaches the microlens 84, and the light beam that has exited from the point 51b on the object and passed through the region 33 on the pupil plane reaches the microlens 83. To do. The light beam that has passed through each microlens reaches a corresponding pixel provided under the microlens. Similar to FIG. 7B, information similar to that in FIG. 7A can be obtained by rearranging the positions on the virtual imaging surface 6a. That is, information on the pupil region (incident angle) that has passed through and the position on the imaging device can be obtained.

  FIG. 7 shows an example in which the position information and the angle information can be acquired by using the MLA (phase modulation element) as the pupil dividing means. However, the position information and the angle information (equivalent to restricting the passing area of the pupil) are shown. Other optical configurations can be used as long as they can be obtained. For example, a method of inserting a mask (gain modulation element) with an appropriate pattern into the optical path of the photographing optical system can be used.

  As described above, according to the present embodiment, high-speed AF with high focus accuracy can be achieved by simultaneously realizing phase difference AF and contrast AF based on information on light beams that have passed through different pupil regions in the image sensor. An imaging device that can be performed can be provided.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

Claims (6)

An image sensor;
Pupil dividing means for restricting a light beam incident on a pixel on the image sensor to a specific pupil region of the photographing lens;
The electrical signals obtained from the image sensor are added with different combinations of the pixels on the image sensor according to the predetermined image plane so that an image signal of a predetermined image plane is obtained, Image generating means for generating an image signal of a predetermined image plane;
Image addition control means for controlling a pupil region to be added in the image generation means;
Contrast evaluation means for calculating the contrast of the image signal obtained from the image generation means;
A focus evaluation unit that obtains a focus evaluation value from the output of the contrast evaluation unit when the predetermined image plane is changed;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the focus evaluation unit performs a plurality of focus evaluations while performing a focus adjustment while changing a range designated by a pupil region by the image addition control unit.   The imaging apparatus according to claim 1, wherein the image addition control unit instructs all pupil regions in the final focus evaluation.   The imaging apparatus according to claim 1, wherein the image addition control unit excludes at least a pupil region closest to the center in the final focus evaluation.   The image pickup apparatus according to claim 1, wherein the image addition control unit indicates the same pupil region as that of the recorded image generation in the final focus evaluation. An imaging device control method comprising: an imaging device; and pupil dividing means for limiting a light beam incident on a pixel on the imaging device to a specific pupil region of a photographing lens,
The electrical signals obtained from the image sensor are added with different combinations of the pixels on the image sensor according to the predetermined image plane so that an image signal of a predetermined image plane is obtained, An image generating step for generating an image signal of a predetermined image plane;
An image addition control step for controlling a pupil region to be added in the image generation step;
A contrast evaluation step of calculating a contrast of the image signal obtained in the image generation step;
A focus evaluation step for obtaining a focus evaluation value from the output of the contrast evaluation means when the predetermined image plane is changed;
An image pickup apparatus control method comprising:

Images