CN102713513B - Camera head, image capture method, program and integrated circuit - Google Patents

Abstract

Do not make the light quantity of exposure reduce, and solve the ambiguity of the differentiation of the point spread function before and after the picture point that the subject distance with subject is corresponding, and, the estimation of subject distance is carried out according to the photographed images of few number.Camera head (10) possesses: imaging apparatus (15), shooting image; Optical system (11), images in imaging apparatus (15) for making shot object image; Disdiaclast (12), has birefringent effect; And distance measuring portion (18), according to the image be taken and the point spread function that there occurs change in the front and back of the picture point corresponding with the subject of subject distance because of optical element, measure the distance from imaging apparatus to subject.

Description

Imaging device, imaging method, program, and integrated circuit

technical field

The present invention relates to an imaging device for measuring the depth of a scene using a plurality of images captured from a single point of view.

Background technique

Conventionally, various methods have been proposed for non-contact measurement of the depth of a three-dimensional scene, that is, the distance from an imaging device to each subject (hereinafter, "subject distance"). It can be roughly distinguished as an active method that calculates the distance to the subject based on the time until the reflected wave returns, the angle of the reflected wave, etc., by irradiating infrared rays, ultrasonic waves, lasers, etc., and calculating the subject distance from the image of the subject. Passive method for distance. In particular, for imaging devices such as cameras, passive methods that do not require devices for irradiating infrared rays or the like are widely used.

As for the passive method, several methods have been proposed, and one of the passive methods is a method called Depth from Defocus (hereinafter, "DFD"). With DFD, the subject distance is measured in terms of blur that changes in size and shape depending on the subject distance. The DFD has features that do not require a plurality of cameras, can measure a distance from a small number of images, and the like.

Hereinafter, the principle of DFD will be briefly described.

It can be said that a captured image including blur (hereinafter "blurred image") is a point spread function (PSF: Point Spread Function) that will be a function of the subject distance for an omni-focus image representing a state without blur by the lens. ) Convolved image. Since the point spread function is a function of the subject distance, according to DFD, the subject distance can be obtained by detecting blur from a blurred image. However, at this time, the distance between the all-focus image and the subject becomes unknown. For a blurred image, a formula related to the blurred image, the all-in-focus image, and the subject distance is established. Therefore, a new formula is obtained by retaking blurred images with different focus positions. That is, a plurality of the above-described calculation expressions related to a plurality of blurred images having different focus positions are obtained. The subject distance is calculated by solving a plurality of formulas thus obtained. There are various proposals for DFD, including patent document 1 and non-patent document 1, regarding the method of obtaining the formula, the method of solving the formula, and the like.

DFD is a method of obtaining a subject distance by using a point spread function for blur contained in a blurred image. However, there is a problem in DFD that since the shape shown by the point spread function before and after the image point corresponding to the object distance is similar, it is not clear due to the influence of noise contained in the image, and after passing the image point The point spread function, or the point spread function before the image point, its discrimination becomes difficult.

To solve this problem, for example, by further increasing the number of images with different focus positions, the accuracy of object distance estimation can be improved. In addition, as in Non-Patent Document 1, by using an opening whose overall shape is not point-symmetrical, it is possible to solve the problem of unclear discrimination of the shape shown by the point spread function before and after the image point corresponding to the object distance of the object. question.

(Prior art literature)

(patent documents)

Patent Document 1: Japanese Patent No. 2963990

(non-patent literature)

Non-Patent Document 1: C.Zhou, S.Lin and S.Nayar, "Coded Aperture Pairs for Depth from Defocus" In International Conference on Computer Vision, 2009

Non-Patent Document 2: "Flexible Depth of Field Photography", H.Nagahara, S.Kuthirummal, C.Zhou, S.K.Nayer, Euro pean Conference on Computer Vision, 2008

Summary of the invention

The problem to be solved by the invention

As a solution to the problem of the ambiguity of the shape indicated by the point spread function before and after the image point corresponding to the subject distance of the subject, a method of increasing the number of images with different focus positions is proposed. , and a method of using an opening of a shape that is not point-symmetrical as a whole. However, the former has a problem in that imaging time increases due to an increase in the number of images. In addition, the latter has a problem in that since a part of the opening is blocked, the amount of light decreases, and the estimation accuracy of the subject distance decreases.

Contents of the invention

Therefore, in view of the above situation, an object of the present invention is to provide an imaging device that solves the problem shown by the point spread function before and after the image point corresponding to the subject distance of the subject without reducing the amount of light exposed. The ambiguity of the discrimination of the shape estimates the subject distance from the captured images of a small number of sheets.

means of solving problems

In order to achieve the above object, an imaging device according to one embodiment of the present invention includes: an imaging element for capturing an image; an optical system for forming an image of an object on the imaging element; an optical element having a birefringent effect; and a distance measurement section that measures the distance from the image based on the captured image and the point spread function that changes due to the optical element before and after the image point corresponding to the subject distance of the subject. The distance from the camera element to the subject.

According to this configuration, since the optical element having an effect of birefringence acts, the shape indicated by the point spread function before and after the image point corresponding to the subject distance of the subject can be made into a different shape. According to this, ambiguity in distinguishing point spread functions before and after an image point corresponding to the subject distance of the subject can be resolved, and the subject distance can be estimated from a small number of captured images. In addition, since the birefringent substance does not need to shield light compared to the method using openings that are not point-symmetrical, it is possible to suppress a reduction in the amount of light.

In addition, the optical element having the effect of birefringence (in particular, if the birefringent substance is a parallel plate and the optical system is a telecentric optical system), unlike other optical elements, mainly affects only astigmatism, so even if the The shape indicated by the point spread function is different before and after the image point corresponding to the object distance of the object, and the influence on other aberrations is also small. Therefore, there is no need to redesign the optical system. That is, it can be realized only by inserting into an existing device and adding a device for performing point spread function processing.

Here, preferably, the direction of the optical axis of the optical element is not parallel to the optical axis of the optical system.

Here, preferably, the optical element is disposed between the imaging element and the optical system on the optical axis of the optical system, and the optical element intersecting the optical axis of the optical system plane, perpendicular to the optical axis of the optical system.

Here, it is preferable to further include an optical element moving part for making the effect of the birefringence on the optical axis of the optical system by inserting or withdrawing the optical element from the optical axis of the optical system. Enabled or disabled, the distance measuring unit utilizes an image captured by the imaging element in a state where there is no effect of the birefringence due to the optical element, and a position where the optical element is located in the optical system. An image is captured in a state on the optical axis, and the distance from the imaging element to the subject is measured.

Here, it is preferable that the optical element is capable of enabling or disabling the effect of birefringence electrically or magnetically, and that the distance measuring unit utilizes a An image captured by the imaging element in a state and an image captured in a state in which the optical element is positioned on the optical axis of the optical system are used to measure the distance to the subject.

Here, it is preferable to further include a reference image generation unit for generating a reference image based on an image captured by the imaging element in a state where the effect of birefringence due to the optical element is absent, and the distance measurement unit , using the image captured by the optical element and the reference image to estimate the point spread function and measure the distance to the object.

Here, it is preferable that the reference image generating unit generates an all-in-focus image as the image captured by the imaging element without the effect of birefringence due to the optical element. Reference image.

Here, it is preferable that the optical system has an optical characteristic of image space telecentricity.

Here, it is also possible to further include a light beam splitting unit for splitting the light beam into a plurality of optical paths, the plurality of imaging elements corresponding to the plurality of optical paths separated by the light beam splitting unit, The subject is photographed, and the optical element is disposed on at least one optical path among a plurality of optical paths separated by the light beam splitter.

Furthermore, the present invention can be realized not only as such an imaging device, but also as an imaging method including operations of characteristic components included in the imaging device as steps. Furthermore, it can also be realized as a program for causing a computer to execute the imaging method. Such a program can also be distributed via a storage medium such as a CD-ROM or a transmission medium such as the Internet. Furthermore, the present invention can also be realized as an integrated circuit that performs processing by each processing unit.

Invention effect

According to the imaging device of the present invention, by calculating the shape represented by the point spread function included in the image based on at least two images, it is possible to obtain the subject distance stably and with high accuracy.

Description of drawings

FIG. 1 is a block diagram showing the configuration of an imaging device according to Embodiment 1 of the present invention.

FIG. 2 is a diagram showing the state of light rays passing through a birefringent substance.

FIG. 3 is a diagram showing the arrangement of components of an imaging device according to Embodiment 1 of the present invention.

FIG. 4 is a diagram showing how a birefringent material changes the shape indicated by the point spread function before and after an image point corresponding to the subject distance of the subject.

In FIG. 5, (a-1) is a diagram showing the shape shown by the point spread function of the extraordinary ray at the position (a) in FIG. 4 when a birefringent substance is used, and (b-1) is a diagram showing When a birefringent material is used, the figure of the point spread function of the extraordinary ray at the position (b) in Fig. 4 is shown, and (a-2) shows the position ( a) is a graph of the point spread function, and (b-2) is a graph showing the point spread function at the position (b) in FIG. 4 when a birefringent substance is not used.

FIG. 6 is a graph showing point spread functions corresponding to different object positions when a birefringent substance is used.

FIG. 7 is a graph showing point spread functions corresponding to different object positions when no birefringent substance is used.

FIG. 8 is a diagram showing the shape of a cubic phase plate.

FIG. 9 is a diagram showing the flow of operations of the imaging device according to Embodiment 1 of the present invention.

FIG. 10 is a block diagram showing the configuration of an imaging device according to Embodiment 2 of the present invention.

FIG. 11 is a diagram showing the arrangement of components of an imaging device according to Embodiment 2 of the present invention.

Detailed ways

Hereinafter, this embodiment will be described with reference to the drawings. Furthermore, the embodiments described below all show a preferred specific example of the present invention. The constituent elements, arrangement positions and connection forms of the constituent elements, steps, the order of the steps, etc. shown in the following embodiments are examples and do not limit the gist of the present invention. The present invention is limited only by the claims. Therefore, among the constituent elements of the following embodiments, the constituent elements not described in the independent claims showing the most general concept of the present invention are not necessarily required in order to achieve the problems of the present invention, but are described as constitutional elements. Elements of a more preferable form.

(Example 1)

FIG. 1 is a block diagram showing the configuration of an imaging device according to Embodiment 1 of the present invention.

The imaging device 10 includes an optical system 11, a birefringent substance 12, an actuator 13, a focal range control unit 14, an imaging element 15, an image obtaining unit 16, a reference image generating unit 17, and a distance measuring unit 18.

In FIG. 1 , an optical system 11 forms a subject image on an imaging element 15 . On the optical path between the imaging element 15 and the optical system 11, a birefringent substance 12 is provided as an optical element having a birefringent effect. In particular, the shape shown by the point spread function of the extraordinary ray among the rays passing through the birefringent material 12 is changed, and the shape shown by the point spread function before and after the image point corresponding to the object distance of the object is changed to be different. shape. The actuator 13 inserts and withdraws the birefringent substance 12 from the optical path. Since the actuator 13 inserts and exits the birefringent material 12 into and out of the optical path, the imaging device 10 can obtain an image of an object transmitted through the birefringent material 12 and an image of an object not transmitted through the birefringent material 12 . like image. The in-focus range control unit 14 moves at least one of the optical system 11 and the imaging device 15 to control the in-focus position and depth of field. Specifically, control is performed by operating the optical system 11 in a specific mode, switching a specific optical element, or the like. The imaging element 15 is composed of CCD, CMOS, etc., and converts light received on the imaging surface into an electrical signal for each pixel, and outputs it. The image obtaining unit 16 obtains a plurality of images from the imaging element 15 and holds each image. The reference image generating unit 17 generates a reference image (omnifocus image) in which no blur caused by the optical system is estimated based on a plurality of images obtained by the effect of the focus range control unit 14 and having different focus positions and depths of field. . The distance measurement unit 18 performs distance measurement according to the DFD method using a blurred image focused at an arbitrary distance and a reference image obtained by the reference image generation unit 17 .

Next, a method of using the birefringent substance 12 to make the shape shown by the point spread function before and after the image point corresponding to the object distance of the object into a different shape will be described.

The birefringent substance 12 is a substance having optical anisotropy, and has a property of separating light rays entering the substance into ordinary rays and extraordinary rays according to the polarization directions of the rays. The ordinary ray and the extraordinary ray depend on the direction of the inherent optical axis of the birefringent substance 12 . An ordinary ray is a ray having an electric field vibrating vertically with respect to a plane formed by the optical axis and the incident ray, and an extraordinary ray is a ray having an electric field vibrating in this plane. Furthermore, the direction of the optical axis and the number of axes differ depending on the type of substance, and when there is one optical axis, it appears as a single axis, and when there are two optical axes, it appears as a double axis. In Example 1, as the birefringent substance 12, calcite (Calcite) which is a uniaxial crystal is used.

The difference between the ordinary ray and the extraordinary ray means that when passing through the birefringent material 12, the ordinary ray, the speed of light has nothing to do with the direction of light propagation and is constant, and the extraordinary ray, the speed of light is different due to the direction of propagation. Furthermore, the refractive index no for ordinary rays is different from the refractive index ne for extraordinary rays. According to the difference between the refractive index no of the ordinary ray and the refractive index ne of the extraordinary ray, and the property that the speed of the light of the extraordinary ray is different due to the direction of propagation, as shown in Figure 2, when the light enters the birefringent material 12 , there is a difference in the direction of travel between ordinary and extraordinary rays. Therefore, a phenomenon occurs in which the light rays incident on the birefringent substance 12 are split into ordinary rays and extraordinary rays in the birefringent substance 12 . In FIG. 2 , light is incident perpendicularly to the plane of the birefringent substance 12 from the left of the birefringent substance 12 .

In the present invention, in order to make the shape shown by the point spread function before and after the image point corresponding to the subject distance of the subject to be different, extraordinary rays are used in particular.

The positional relationship among optical system 11 , birefringent substance 12 , and imaging element 15 is that birefringent substance 12 is disposed between optical system 11 (lens) and imaging element 15 as shown in FIG. 3 . That is, the three are arranged and arranged on the optical axis in the order of the optical system 11 , the birefringent material 12 , and the imaging element 15 . The birefringent material 12 refers to a parallel plate having a shape and arrangement in which all planes of the birefringent material intersecting the optical axis are perpendicular to the optical axis. Moreover, the "vertical" at this time can also be used, not strictly vertical. Furthermore, the birefringent substance 12 is uniaxial, and in FIG. 3 , the direction of the optical axis is the y direction. Furthermore, the relation of the refractive indices for ordinary rays and extraordinary rays is no>ne. Here, the positional relationship between the optical system 11 , the birefringent substance 12 , and the imaging element 15 has been described. However, in order to obtain effects from the positional relationship described above, the birefringent substance 12 is preferably a parallel plate. In the case of a parallel plate, it may have the property of giving only the influence of principal astigmatism. The parallel plate referred to here refers to a substance in which the first surface on the light-incident side and the second surface on the light-emitting side are parallel to each other. That is, the angles and shapes of surfaces other than the first surface and the second surface are not limited.

In the case of using the structure of FIG. 3 , there is a difference in the shape shown by the point spread function of the extraordinary rays before and after the image point corresponding to the subject distance of the subject, and at the subject distance corresponding to the subject The front of the image point of is long in the y direction, and the rear of the image point corresponding to the object distance of the object is long in the x direction. FIG. 4 is a diagram showing the behavior of ordinary rays and extraordinary rays on the yz plane and the xz plane in the structure of FIG. 3 . According to the property that the speed of light is different due to the direction of the optical axis and the relaying direction of the extraordinary ray, for the extraordinary ray, in the y-z plane, compared with the ordinary ray, the refraction is stronger, and compared with the ordinary ray, the refraction is stronger than that of the photographed The subject is farther away from the corresponding image point. On the other hand, in the xz plane, for extraordinary rays, the refraction angle is smaller than that of ordinary rays, and the position of the image point corresponding to the subject distance of the subject is closer than that of ordinary rays. If only the extraordinary light is considered, the position of the image point corresponding to the distance between the light of the xz plane and the light of the yz plane is different from that of the subject. Therefore, for the shape shown by the point spread function of the extraordinary ray, at the position (a) in front of the image point corresponding to the subject distance of the subject in Fig. 4, the degree of blur in the y direction is larger than that in the x direction Since the degree of blur is large, it becomes a shape elongated in the y direction as shown in FIG. 5(a-1). On the other hand, for the shape shown by the point spread function of the extraordinary ray, at the position (b) behind the image point corresponding to the subject distance of the subject in Fig. 4, since the degree of blur in the x direction is larger than the Since the degree of ambiguity in the direction is large, it becomes a shape elongated in the x direction as shown in FIG. 5(b-1). 5(a-2) and (b-2) are point spread functions of ordinary rays at the positions of FIG. 4(a) and (b), respectively. It can also be said that they are point spread functions of the light rays without the birefringent substance 12 . That is, it was confirmed that in the absence of the birefringent substance 12 , the front and rear of the image point corresponding to the subject distance of the subject have similar shapes (for example, a circle in this case).

Since the shape shown by the point spread function is different before and after the image point corresponding to the subject distance of the subject, the ambiguity of distance discrimination is resolved, and the subject distance can be uniquely estimated. Hereinafter, it will be described based on FIGS. 6 and 7 that when a birefringent substance is used, it is effective for estimating the subject distance. Furthermore, FIG. 6 is a graph showing point spread functions corresponding to different object positions when the birefringent substance 12 is used. FIG. 7 is a graph showing point spread functions corresponding to different object positions when no birefringent substance is used. And, for the so-called "subject distance" here, in the description, it is defined as the distance from the imaging device to the subject, but it may be the distance from the optical system 11 to the subject, or it may be from The distance from the imaging element 15 to the subject.

In the case where there is a birefringent substance 12 as shown in FIG. 6 , when considering the point spread function corresponding to the subject position (a) and the subject position (b), at the subject distance corresponding to the subject The shape shown by the point spread function before and after the image point is different, therefore, the shape shown by the point spread function corresponding to the position (a) and the shape shown by the point spread function corresponding to the position (b) of the imaging element 15 are imaged. The shapes are different from each other. That is, the subject distance can be uniquely estimated from the point spread function obtained by the imaging element 15 . On the other hand, in FIG. 7, since there is no birefringent substance 12, the point spread functions corresponding to the positions (a) and (b) have similar shapes, but are unclear due to noise, and the obtained point spread functions are the same as the positions ( a) or corresponding to position (b), it is difficult to uniquely estimate the subject distance. That is to say, in the case where there is a birefringent substance 12 as shown in FIG. 6, compared with the case where there is no birefringent substance 12 as shown in FIG. It is effective that the subject distance can be easily and uniquely estimated based on whichever of the image point corresponds to the distance. Furthermore, for the calculation of the point spread function of FIGS. 5 to 7 , optical simulation software "ZEMAX (product name)" manufactured by ZEMAX Development Corporation was used.

Actually, according to the configuration of FIG. 3 , ordinary rays and extraordinary rays are simultaneously detected. However, since the extraordinary ray is included, the shape indicated by the point spread function before and after the image point corresponding to the subject distance of the subject still differs.

Next, a method of obtaining a reference image will be described.

For the imaging device 10 of Embodiment 1, a reference image (omnifocal image) free from blurring by the optical system 11 is used. An image without blurring by the optical system 11 can be said to be an image with a depth of field. By narrowing the aperture of the optical system, it is possible to easily achieve a deeper depth of field. However, according to this method, the amount of light received by the imaging element 15 decreases, and therefore, a number of methods have been proposed to increase the depth of field without narrowing the aperture. One of these methods is a method called Extended Depth of Field (hereinafter referred to as EDoF). Hereinafter, a specific method of EDoF will be described.

The simplest EDoF method is a method of taking a plurality of images while gradually shifting the in-focus position slightly, and extracting in-focus parts from such images to synthesize them.

In contrast, Non-Patent Document 2 discloses a method of changing the focus position during exposure to generate a blur-free image.

Specifically, if the imaging element or the lens is moved in the direction of the optical axis during exposure, the point spread function becomes substantially constant regardless of the subject distance, and a uniformly blurred image can be obtained. For the obtained blurred image, if deconvolution is performed using an invariant point spread function that is not affected by the subject distance, an image that is not blurred in the entire image can be obtained.

On the other hand, a method of EDoF using a special optical element has also been proposed. For example, a method using an optical element called a cubic phase mask (Cubic Phase Mask). The shape is shown in FIG. 8 for an example of a three-dimensional phase plate. When an optical element having such a shape is mounted near the aperture of the optical system, an image with substantially constant blur regardless of the subject distance can be obtained. Similar to Non-Patent Document 1, if deconvolution is performed using an invariant point spread function that is not affected by the subject distance, an image without blurring in the entire image can be obtained. In addition, a method using a multifocal lens, etc. are mentioned.

In addition, in the following description, it is assumed that a method of changing the focus position during the exposure time is used as a method of expanding the depth of field to obtain a reference image.

Next, the flow of processing for calculating the subject distance will be described. FIG. 9 is a flowchart showing an example of a flow of processing for calculating a subject distance. This processing is a process of calculating, among predetermined n stages of subject distances d1 , d2 , .

First, the image I of the object and the reference image I' obtained through the birefringent material are photographed and obtained (steps S101, S102). Moreover, the order of steps S101 and S102 can also be reversed. However, the reference image obtained here is an image of a subject that is not transmitted through the birefringent substance 12 .

Here, between the image I and the reference image I′, the relationship shown in the following formula 1 is established.

【Equation 1】

I(x, y)=I'(x, y)*h(x, y, d(x, y))···(Formula 1) 

Here, h represents the point spread function at the position (x, y) in the image, and d represents the subject distance at the position (x, y) at (x, y). Also, * in the formula represents a convolution operation. The point spread function differs depending on the subject distance, therefore, in the case where a plurality of subjects exist at different subject distances, the point spread function with different subject distances for each position of the image is obtained with no The blurred image is convolved with the image as image I.

Next, an initial value of 1 is substituted into the counter i (step 103), and an error function C(x, y, di) for the subject distance of the i-th stage is calculated for each pixel of the image (step S104). The error function is represented by Equation 2 below.

【Equation 2】

C(x, y, d _i )=|I(x, y)-I′(x, y)*h(x, y, d _i )|(i=1, 2,...,n)... ·(Formula 2)

Here, h(x, y, di) represents a point spread function corresponding to the subject distance di. The point spread function corresponding to the subject distance di (i=1 to n: n is a natural number greater than or equal to 2) is stored in advance in, for example, a memory of the imaging device 10 . Equation 2 is equivalent to taking the convoluted image of the unblurred reference image I′ and the point spread function h(x, y, di) corresponding to the i-th stage object distance di, and the actual captured image I difference between. The error function C(x, y, di) which is the difference becomes the minimum when the photographed subject actually exists at the subject distance of the i-th stage.

Moreover, in Equation 2, the error function C(x, y, di) is the point spread function h(x, y, di) corresponding to the object distance di of the i-th stage between each pixel and the unblurred The absolute value of the difference between the image convolved with the image of , and the actual captured image I, but the error function may be determined in any form representing the L2 norm equidistant.

After calculating the error function, determine whether the value of counter i reaches n (step S105), and if not, increase the value of counter i by 1 (step S106), and repeat until the value of counter i reaches n.

After calculating all the error functions of the first stage to the nth stage, the subject distance is calculated (step S107). The subject distance d(x, y) at the position (x, y) is represented by Equation 3 below.

【Equation 3】

$d(x,\mathrm{the\; y})=\underset{\mathrm{di}}{\arg \min }C(x,\mathrm{the\; y},d_i)$ ····(Formula 3)

Actually, in order to reduce the influence of noise included in the captured image I, the image is divided into a plurality of blocks, the sum of the error functions in the blocks is obtained, and the subject distance at which the error function becomes the smallest is set as the distance of the entire block. The processing of the subject distance and the like of the photographed subject enables more stable distance measurement.

According to such a configuration, since the shape indicated by the point spread function is different before and after the image point corresponding to the subject distance of the subject, it is possible to uniquely estimate the subject distance.

In this embodiment, in FIG. 3 , the direction of the optical axis of the birefringent material 12 is set upward, but the direction of the optical axis is not limited to upward, but can be any direction, even if it is any direction , the shape shown by the point spread function before and after the image point corresponding to the object distance of the object can also be made into a different shape. When the direction of the optical axis is changed, the shape represented by the obtained point spread function also changes. Regardless of which direction the optical axis faces, the positions of the image point corresponding to the subject distance of the subject of ordinary rays and the position of the image point corresponding to the subject distance of extraordinary rays of light are different, but only in When the optical axis is parallel to the optical axis, the difference in shape indicated by the point spread function before and after the image point corresponding to the subject distance of the subject becomes smaller. Therefore, it is preferable that the birefringent substance 12 is arranged so that the direction of the optical axis of the birefringent substance 12 is not parallel to the optical axis.

In addition, in the description of this embodiment, calcite (Calcite), which is a uniaxial crystal, is used as the birefringent substance 12 , but other substances having an effect of birefringence may also be used. Regarding the optical axis, in addition to the direction, the number of axes can also be used as an element controlling the shape represented by the point spread function, and the effect can be obtained from a biaxial material as well as a uniaxial birefringent material. By arranging a plurality of uniaxial, biaxial, or both birefringent substances, the range of change can also be enlarged. Furthermore, depending on the thickness and type of the birefringent substance, the shape represented by the obtained point spread function can also be changed before and after the image point corresponding to the subject distance of the subject.

Moreover, in this embodiment, the acquisition of the image of the subject transmitted through the birefringent material 12 and the image of the subject not transmitted through is realized by the movement of the birefringent material performed by the actuator, however, there are still Other methods, in short, can be achieved by physically driven movement of the birefringent substance itself in and out of the optical path, and by using optical elements that can control the effect of birefringence.

For the former, there is a method in which a birefringent substance exists and a birefringent substance does not exist on the optical path by linearly moving the actuator and rotating the plate of the birefringent substance in a state perpendicular to the optical axis. wait. As for the latter, for example, an element capable of electrical control such as the electro-optic effect, an element capable of magnetic control, and the like are mentioned. In such a case, the presence or absence of the effect of birefringence can be controlled by switching the presence or absence of the application of the voltage and the magnetic field. In addition, instead of using a birefringent substance, liquid crystals or the like that can electrically and magnetically control the effect of the birefringent substance may be used.

Regarding the position of the birefringent substance, other than the position in FIG. 3 , a shape different from the shape shown by the point spread function before and after the image point corresponding to the subject distance of the subject can be obtained from any position. However, it is preferable to arrange them immediately before the imaging element as shown in FIG. 3 because a large effect can be obtained.

Furthermore, the optical system 11 is preferably an optical system in which the shapes indicated by point spread functions of all image heights are the same, and is particularly preferably an optical system in which the image is telecentric. The image-space telecentric optical system refers to an optical system in which the chief ray and the optical axis of all angles of view become parallel on the image side. In the configuration of FIG. 3 , when the optical system 11 is an image-space telecentric optical system, even if the birefringent material 12 is arranged on the optical path, the shape indicated by the point spread function of all image heights becomes the same. That is, when the optical system has the property that the shapes shown by the point spread functions of all image heights become the same, even if the birefringent material 12 is arranged on the optical path, this property can be preserved. Therefore, in this case, no redesign of the optical system including the birefringent substance is required. In the case where the shapes represented by the point spread functions of all image heights are the same, only one point spread function is required for the calculation of the distance measurement, and the cost required for the calculation can be suppressed.

(Example 2)

The imaging device 19 according to the second embodiment of the present invention has a configuration that separates the ordinary ray and the extraordinary ray and acquires only images of the respective rays. FIG. 10 is a block diagram showing the configuration of an imaging device 19 according to Embodiment 2 of the present invention. In FIG. 10 , the same reference numerals are used for the same constituent elements as those of the imaging device 10 in FIG. 1 , and a part of description is omitted. The imaging device 19 includes an optical system 11, a light beam separating unit 20, a birefringent material 12, a focus range control unit 14, an imaging element A21, an imaging element B22, an image obtaining unit A23, an image obtaining unit B24, a reference image generating unit 17, and a distance measuring unit 18 .

In FIG. 10 , the optical system 11 forms a subject image on an imaging element A21 and an imaging element B22 . The light beam splitter 20 spatially separates the light beams with an arbitrary light quantity ratio. The imaging element A21 and the imaging element B22 are composed of CCD, CMOS, etc., and convert the light received on the imaging surface into an electrical signal for each pixel, and output it. Then, one of the light beams separated by the light beam splitter 20 undergoes a change in shape represented by a point spread function by the birefringent substance 12 and is received by the imaging element A21. The imaging element B22 receives the other light beam that has not been transmitted through the birefringent material 12 and that has not received the effect of the birefringent material 12 and that has been separated by the light beam splitter 20 . The image obtaining unit A23 and the image obtaining unit B24 obtain images from the imaging device A21 and the imaging device B22 respectively, and store the obtained images.

Specifically, with the structure shown in FIG. 11 , the birefringent material 12 produces a blurred image with a different shape shown by the point spread function before and after the image point corresponding to the subject distance of the subject, and the imaging element A21 get. On the other hand, for light rays that do not pass through the birefringent material 12 , the in-focus position and depth of field are controlled by the in-focus range control unit 14 and captured by the imaging device B22 as in the first embodiment. Furthermore, the reference image generation unit 17 generates a reference image based on the image obtained by the imaging device B22. The blurred image obtained by the imaging device A21 and the reference image generated from the image captured by the imaging device B22 are processed in the same manner as in FIG. 9 and used for calculation of the subject distance. In Example 2, the blurred image obtained by the imaging element A21 and the reference image obtained by the reference image generating unit 17 correspond to the image I and the reference image I' in FIG. 9 , respectively. Furthermore, the subject distance can be calculated by the same calculation as in Expressions 1 to 3.

Examples of the light beam splitter 20 for splitting light include a non-polarized beam splitter, a polarized beam splitter, and the like. In the case of no polarizing beam splitter, the obtained image I is an image including both extraordinary rays and ordinary rays, as in the first embodiment. Also in the case of using a polarizing beam splitter, it is possible to control the direction of the polarized light separated from the optical axis of the birefringent material so that the light included in the image I becomes only the extraordinary light. Furthermore, by making the rays included in the image I only extraordinary rays, it is possible to capture an image that does not contain noise due to ordinary rays, and thus it is possible to obtain an image with higher accuracy for deriving the subject distance. Furthermore, when using a polarizing beam splitter, a birefringent substance may be arranged between the polarizing beam splitter and the optical system. In this case, it is necessary to select the polarization direction so that only ordinary rays reach the imaging element B22.

Furthermore, although the amount of light decreases for an image containing only extraordinary rays, it can be obtained by using an optical element such as a polarizer that transmits only specific polarized light to transmit only extraordinary rays.

According to such a configuration, the image I and the reference image I′ can be obtained at the same time, and therefore, there is no difference between the two images except blur, and the subject distance can be obtained more accurately. In Embodiment 1, since the image I and the reference image I' are not obtained at the same time, according to the movement of the subject and the camera itself, the relative position of the subject corresponding to the camera changes, and the two images If a difference other than blur occurs, the accuracy of distance measurement tends to decrease. However, when the imaging time for one image is the same, in Example 1 where light is not separated, the amount of light incident on one imaging element is larger, so the signal-to-noise ratio (S/N ratio) is higher.

In the first embodiment, the reference image I' and the image I are obtained by time division, whereas in the second embodiment, it can be said that the image I and the reference image I' are obtained by space division. In Example 2, due to the division of light rays, the light quantities corresponding to the image I and the reference image I′ respectively decrease. However, if the light quantities of the two images are combined, the light quantity does not decrease and there is no loss. In addition, when the time required to obtain both images is the same, the total light intensity is the same in Example 1 and Example 2.

Furthermore, in the first and second embodiments, the omni-focus image is used as the reference image in order to obtain the subject distance of the subject, but it is not limited to this, and a blurred image may also be used as the reference image. to use to derive the subject distance of the subject.

Moreover, the control unit of the actuator 13 as the birefringence effect imparting unit and the image acquisition unit 16 as the imaging unit in each functional block of the block diagrams of the first and second embodiments (FIG. 1, FIG. 10, etc.) , and the distance measuring unit 18 are realized by an LSI which is a typical integrated circuit. These may be single-chip or a part or all of them may be single-chip. For example, functional blocks other than the memory may be formed into a single chip.

Here, it is referred to as an LSI, but it may also be called an IC, a system LSI, a super LSI, or a super LSI depending on the degree of integration.

In addition, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array: Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection or setting of circuit cells inside the LSI can also be used.

Furthermore, as a matter of course, if an integrated circuit technology that replaces LSI appears due to progress in semiconductor technology or other derived technologies, the functional blocks can be integrated using that technology. There is a possibility of application of biotechnology and the like.

In addition, in each functional block, only a unit for storing data to be processed may be separately configured instead of being integrated into a single chip.

Industrial Applicability

The imaging device according to the present invention can perform distance measurement based on an image captured from a single point of view, and therefore can be widely used in imaging equipment.

Symbol Description

10 camera device

11 optical system

12 birefringent substances

13 actuators

14 Focus range control unit

15 camera elements

16 Image Acquisition Department

17Reference image generation unit

18 Distance Measurement Department

19 camera device

20 light separation part

21 Imaging element A

22 Imaging element B

23 Image Acquisition Part A

24 Image Acquisition Part B

Claims (12)

1. A camera device comprising: camera element, to capture images; an optical system, configured to image the subject image on the imaging element; optical elements, having the effect of birefringence; and The distance measurement unit measures the distance from the imaging element to the distance from the imaging element based on the captured image and a point spread function that changes with the optical element before and after an image point corresponding to an object distance of the object. The distance of the subject. 2. The imaging device according to claim 1, The direction of the optical axis of the optical element is not parallel to the optical axis of the optical system. 3. The imaging device according to claim 2, On the optical axis of the optical system, the optical element is arranged between the imaging element and the optical system, and the plane of the optical element intersecting the optical axis of the optical system is opposite to the The optical axis of the optical system is vertical. 4. The imaging device according to claim 3, further comprising an optical element moving unit for enabling or disabling the effect of birefringence on the optical axis of the optical system by inserting or withdrawing the optical element relative to the optical axis of the optical system, The distance measuring unit utilizes an image captured by the imaging element in a state where there is no effect due to the birefringence of the optical element, and a state in which the optical element is located on an optical axis of the optical system. Under the captured image, the distance from the imaging element to the subject is measured. 5. The imaging device according to claim 3, said optical element capable of enabling or disabling the effect of birefringence electrically or magnetically, The distance measuring unit utilizes an image captured by the imaging element in a state where there is no effect due to the birefringence of the optical element, and a state in which the optical element is located on an optical axis of the optical system. Under the captured image, measure the distance to the subject. 6. The imaging device according to any one of claims 1 to 5, further comprising a reference image generation unit for generating a reference image based on an image captured by the imaging element in a state in which the effect of birefringence due to the optical element is absent, The distance measurement unit estimates the point spread function using the image captured by the optical element and the reference image, and measures the distance to the subject. 7. The imaging device according to claim 6, The reference image generation unit generates an all-in-focus image as the reference image based on an image captured by the imaging element without the effect of the birefringence due to the optical element. 8. The imaging device according to any one of claims 1 to 5, The optical system has an optical characteristic of image telecentricity. 9. The imaging device according to any one of claims 1 to 5, It also has a light separation unit that separates the light into multiple light paths, There are a plurality of imaging elements, and the plurality of imaging elements respectively correspond to the plurality of optical paths separated by the light beam separating part, and photograph the subject, The optical element is arranged on at least one optical path among the plurality of optical paths separated by the light beam splitter. 10. The imaging device according to any one of claims 1 to 5, There are multiple optical elements. 11. An imaging method, which is an imaging method of an imaging device, the imaging device having an imaging element for capturing an image, and an optical system for imaging a subject image on the imaging element, the imaging method comprising: The step of imparting a birefringence effect is to impart a birefringence effect on the optical axis of the optical system. The birefringence effect refers to making a shape represented by a point spread function determined by the optical system appear on the optical axis of the optical system. The effect of different shapes at the front and rear positions of the image point corresponding to the subject distance of the subject; an imaging step of capturing an image by the imaging element in a state where the effect of birefringence is imparted on the optical axis of the optical system; and In the distance measuring step, the distance to the subject is measured based on the image captured by the imaging device and the point spread function. 12. An integrated circuit, which is an integrated circuit of an imaging device, the imaging device having an imaging element for capturing an image, and an optical system for imaging a subject image on the imaging element, the integrated circuit comprising: The birefringence effect imparting unit imparts a birefringence effect on the optical axis of the optical system. The birefringence effect is to make a shape represented by a point spread function determined by the optical system appear on the optical axis of the optical system. The effect of different shapes at the front and rear positions of the image point corresponding to the subject distance of the subject; an imaging unit that captures an image with the imaging element in a state where the effect of birefringence is imparted to the optical axis of the optical system; and The distance measurement unit measures the distance to the subject based on the image captured by the imaging element and the point spread function.