JP2001304821A - Image capturing apparatus and distance measuring method - Google Patents

Abstract

(57) [Problem] To provide an image pickup apparatus and a distance measuring method capable of photographing outgoing light obtained from a subject irradiated with light and easily measuring a depth distance of the subject. SOLUTION: A first irradiation light having a first wavelength and a second irradiation light having second and third wavelengths different from the first wavelength are applied to a subject from optically different radiation positions. Irradiation unit 100 for irradiation, first emission light having a first wavelength, and second emission light having a second wavelength are obtained from emission light obtained from a subject irradiated with the first and second irradiation lights by the irradiation unit. 2
And an imaging unit that images the first, second, and third outgoing lights separated by the dispersing unit 30. The light separating unit 30 optically separates the outgoing light from the third outgoing light having the third wavelength. Part 2
0, 40, a light intensity detection unit 64 that detects the intensity of the first, second, and third emitted light, and the intensity of the first, second, and third emitted light. And a depth calculation unit 66 for calculating the depth distance of the image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image pickup apparatus and a distance measuring method for acquiring information on a depth distance of a subject. In particular, the present invention relates to an image capturing apparatus and a distance measuring method for capturing information on the depth of a subject by shooting outgoing light obtained from the subject irradiated with light.

[0002]

2. Description of the Related Art In order to obtain information on the distance to an object and information on the position of the object, three-dimensional image measurement in which a pattern light such as a slit or a striped pattern is projected on the object, and the pattern projected on the object is photographed and analyzed. Are known. Typical measurement methods include slit light projection (also known as light sectioning) and coded pattern light projection, which are detailed in Seiji Iguchi and Kosuke Sato, "Three-dimensional image measurement" (Shokodo).

[0003] JP-A-61-155909 (publication date: July 15, 1986) and JP-A-63-233312
Japanese Patent Laid-Open Publication (Publication Date: September 29, 1988) discloses a distance measuring device that irradiates a subject with light from different light source positions and measures a distance to the subject based on an intensity ratio of reflected light from the subject. A distance measuring method is disclosed.

Japanese Patent Application Laid-Open No. 62-46207 (published on Feb. 28, 1987) discloses that a subject is irradiated with two lights having different phases, and the light is reflected on the subject based on the phase difference of the reflected light from the subject. A distance detecting device for measuring a distance to a vehicle is disclosed.

[0005] Also, Kahoku et al., "Axi-Vision three-dimensional imaging device"
Camera Development ”(3D Image Conference 99, 199
9 years), ultra-high-speed intensity modulation is applied to the projection light, and a subject illuminated with the intensity-modulated light is photographed with a camera equipped with a high-speed shutter function. Is disclosed.

[0006]

The conventional measuring method based on the light projection method measures a distance to a region of a subject on which a projection pattern is projected based on the principle of triangulation. Therefore, in order to obtain a high resolution of the distance measurement, it is necessary to arrange the projection optical system and the photographing optical system sufficiently in principle, and there has been a problem that the size of the measuring apparatus cannot be avoided. Also, since the optical axes of the projection optical system and the imaging optical system are far apart, when viewed from the imaging optical system, there is a place where the projected pattern is hidden behind the subject and cannot be observed, and distance information can be obtained. There was also the problem that there was no "blind spot".

In the distance measuring device and the distance measuring method disclosed in JP-A-61-155909 and JP-A-63-233312, light is sequentially irradiated at different radiation positions, and each reflected light is reflected. Because it needs to be measured,
There is a time difference in measurement. Therefore, in the case of a moving subject, there is a problem that the distance cannot be measured. Also, during irradiation with the position of the light source changed, there is a possibility that a measurement error may occur due to blurring of the imaging device.

Further, in the case of lights having different wavelength characteristics, the reflected light can be irradiated at the same time, the reflected light can be separated using a filter adapted to the wavelength characteristics of the irradiated light, and the reflected light intensity can be measured. However, if the spectral reflectance of the object is different, the reflected light intensity will differ due to the difference in the wavelength of the illuminating light, and this will be an error factor when calculating the depth distance from the ratio of the reflected light intensity, and the accurate depth distance will be calculated. There was a problem of not being able to do so.

The distance detecting device disclosed in Japanese Patent Application Laid-Open No. 62-46207 requires a highly accurate phase detector for detecting a phase difference, which makes the device expensive and lacks simplicity. Further, since the phase of the reflected light from the point of the subject is measured, the depth distribution of the entire subject cannot be measured.

Also, Kahoku et al., "Axi-Vision three-dimensional imaging device"
Camera Development ”(3D Image Conference 99, 199
9), the distance measurement method using intensity modulation,
It is necessary to perform light modulation and light shutter operation at a very high speed, and there is a problem that the measuring device is large and expensive, and cannot be measured easily.

Accordingly, an object of the present invention is to provide an image pickup device and a distance measuring device which can solve the above-mentioned problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous embodiments of the present invention.

[0012]

According to a first aspect of the present invention, there is provided an image pickup apparatus for acquiring information relating to the depth of a subject, wherein a first wavelength is set to a main wavelength. A first irradiation light as a component and a second irradiation light having second and third wavelengths different from the first wavelength as main wavelength components are radiated to a subject from optically different radiation positions. An irradiation unit, and a depth calculation unit that calculates a depth distance to the subject based on light emitted from the subject to which the first and second irradiation lights have been irradiated by the irradiation unit.

[0013] The irradiating section may simultaneously irradiate the first and second irradiation lights. An optical imaging unit that forms the outgoing light obtained from the subject irradiated with the first and second irradiation light by the irradiation unit, and a first outgoing light having a first wavelength from the outgoing light obtained from the subject A light separating unit that optically separates the second outgoing light having the second wavelength from the third outgoing light having the third wavelength; and an optical image forming unit that is separated by the light separating unit. Further comprising a light receiving unit for receiving the first, second and third outgoing lights, and a light intensity detecting unit for detecting the intensities of the first, second and third outgoing lights received by the light receiving unit. The calculating unit uses the first, second, and third outgoing light intensities,
The depth distance to the subject may be calculated.

The light receiving section has three solid-state imaging devices, and the spectroscopic portion separates the first, second, and third outgoing lights by using an optical path dividing means, and separates each of the light beams into three solid-state imaging devices. May be received by any one of them. The light receiving unit includes a solid-state imaging device, and the light splitting unit includes a first optical filter that transmits light of a first wavelength, a second optical filter that transmits light of a second wavelength, and a third optical filter. A third optical filter that transmits light of a wavelength; and the first, second, and third optical filters may be alternately arranged on the light receiving surface of the solid-state imaging device.

The irradiating section includes a first optical filter transmitting light in a wavelength region shorter than a predetermined first boundary wavelength, and a second optical filter transmitting light in a wavelength region longer than a predetermined second boundary wavelength.
Irradiating a subject with first irradiation light passing through the first optical filter and second irradiation light passing through the second optical filter from optically different radiation positions; The light splitting unit includes a first optical filter that transmits light having a first wavelength shorter than a shorter one of the first and second boundary wavelengths, and a second optical filter that is longer than a longer one of the first and second boundary wavelengths. And third and third light, respectively, that transmit light of the third and third wavelengths, respectively.
The first optical filter transmits the outgoing light obtained from the subject to the first optical filter.
By separating the first outgoing light having the second wavelength and transmitting the outgoing light through the second and third optical filters, respectively, the second outgoing light having the second wavelength and the third outgoing light having the third wavelength are provided. 3 may be separated.

The depth calculator may calculate the depth distance to the subject using a value based on the intensity of the second and third emitted light and the intensity of the first emitted light. The depth calculation unit is configured to, based on the intensities of the second and third outgoing lights, provide a temporary outgoing light from the subject assuming that the light having the first wavelength is emitted from the radiation position of the second irradiation light. Find the strength of
The depth distance to the subject may be calculated using the intensity of the first emission light and the intensity of the temporary emission light. The depth calculation unit may calculate the depth distance to the subject using the average intensity of the second emission light and the third emission light and the intensity of the first emission light.

According to a second aspect of the present invention, there is provided an image pickup apparatus for acquiring information relating to the depth of a subject,
The first irradiation light having the first wavelength as the main wavelength component and the second irradiation light having the second and third wavelengths different from the first wavelength as the main wavelength components are optically combined. An irradiation unit that irradiates the subject from different radiation positions, and a depth calculation unit that calculates a depth distance to the subject based on light emitted from the subject that has been irradiated with the first and second irradiation lights by the irradiation unit It is characterized by having.

The irradiating section may irradiate the first and second irradiation lights simultaneously. The irradiating unit includes a first irradiation light having the first wavelength as a main wavelength component, a second wavelength shorter than the first wavelength, and a third wavelength longer than the first wavelength as a main wavelength component. An optical imaging unit that irradiates the subject with the second irradiation light to be emitted from an optically different radiation position, and forms the light emitted from the subject irradiated with the first and second irradiation light by the irradiation unit. A spectroscopic unit that optically separates first outgoing light having a first wavelength and second outgoing light having second and third wavelengths from outgoing light obtained from a subject; The first outgoing light and the second
And a light intensity detector for detecting the intensities of the first and second outgoing light beams received by the light receiving unit, and the depth calculator includes a first and a second outgoing light beam. May be used to calculate the depth distance to the subject.

The light receiving section has two solid-state image pickup devices, and the light splitting section optically splits the optical paths of the first and second emitted lights by using an optical path splitting means. Any one of the solid-state imaging devices of the plate may receive light. The light receiving unit includes a solid-state imaging device, and the light splitting unit includes a first optical filter that transmits light of a first wavelength, and a second optical filter that transmits light of second and third wavelengths. The first optical filter and the second optical filter may be alternately arranged on the light receiving surface of the solid-state imaging device.

The depth calculator may calculate the depth distance to the subject based on a ratio of the intensity of the first emitted light to half the intensity of the second emitted light.

In the above first and second embodiments, the optical axis when the irradiating section irradiates the subject with the first and second irradiating lights and the optical axis when the image forming section captures the outgoing light from the photographic subject. The optical axis may be substantially the same. The light intensity detection unit detects the intensities of the first and second emitted lights at each pixel of the image of the subject captured by the light receiving unit, and the depth calculation unit determines the depth up to the region of the subject corresponding to each of the pixels. , The depth distribution of the subject may be calculated.

In the first and second embodiments, the first
And the second irradiation light is light in the infrared region,
The image forming apparatus further includes means for optically separating visible light from emitted light obtained from the subject, and the light receiving unit is optically separated by the spectroscopic unit, and the optical imaging unit receives visible light to form an image. The image processing apparatus may further include a solid-state imaging device, and may further include a recording unit that records an image of the subject captured by the solid-state imaging device for visible light together with the depth distribution of the subject calculated by the depth calculation unit.

In the first and second embodiments, the irradiation is performed based on at least one of the intensity of the outgoing light from the subject detected by the light intensity detection unit and the depth distance to the subject calculated by the depth calculation unit. The image processing apparatus may further include a control unit that controls at least one of a light emission time, an intensity, a radiation position, and an exposure time of the light receiving unit of the first and second irradiation lights emitted by the unit.

According to a third aspect of the present invention, there is provided a distance measuring method for acquiring information on the depth of a subject,
The first irradiation light having the first wavelength as the main wavelength component and the second irradiation light having the second and third wavelengths different from the first wavelength as the main wavelength components are optically combined. An irradiation step of simultaneously irradiating a subject from different radiation positions; and a first emission light having a first wavelength and a second wavelength from an emission light obtained from the subject irradiated with the first and second irradiation lights. And an imaging step of imaging the separated first, second, and third emission lights, and a second emission light having a third wavelength and a third emission light having a third wavelength. And a light intensity detecting step of detecting the intensity of the imaged first, second, and third emitted light, and the intensity of the first, second, and third emitted light,
Calculating a depth distance to a subject.

In the depth calculation step, a value based on the intensities of the second and third outgoing lights and the intensity of the first outgoing light are used.
The depth distance to the subject may be calculated. In the depth calculation step, the first and second output light intensities are determined based on the first and third output light intensities.
Is obtained from the radiation position of the second irradiation light, the intensity of the provisional emission light from the subject is obtained, and the ratio of the intensity of the first emission light to the intensity of the provisional emission light is obtained. May be used to calculate the depth distance to the subject. In the depth calculation step, the depth distance to the subject may be calculated based on a ratio between the average intensity of the second emission light and the third emission light and the intensity of the first emission light.

According to a fourth aspect of the present invention, there is provided a distance measuring method for acquiring information on the depth of a subject,
A first irradiation light having the first wavelength as a main wavelength component, a second irradiation light having a second wavelength shorter than the first wavelength, and a second irradiation light having a third wavelength longer than the first wavelength as a main wavelength component. Irradiation light,
An irradiation step of simultaneously irradiating the subject from optically different radiation positions, and a first emission light having a first wavelength from an emission light obtained from the subject irradiated with the first and second irradiation lights; A spectroscopic step of optically separating the second outgoing light having the second and third wavelengths, an imaging step of imaging the separated first outgoing light and the second outgoing light, and a first received light
A light intensity detecting step of detecting the intensities of the outgoing light and the second outgoing light, respectively, and a depth calculation for calculating a depth distance to the subject using the intensities of the first outgoing light and the second outgoing light. And a step.

In the depth calculation step, the depth distance to the subject may be calculated based on a ratio between the intensity of the first emitted light and half the intensity of the second emitted light.

Note that the above summary of the present invention does not list all of the necessary features of the present invention, and sub-combinations of these features can also constitute the present invention.

[0029]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described through embodiments of the present invention. However, the following embodiments do not limit the invention according to the claims and are described in the embodiments. Not all combinations of the features described above are essential to the solution of the invention.

(Embodiment 1) A first embodiment of the present invention will be described. First, the basic principle of measuring the depth distance to the subject from the intensity of the reflected light from the subject will be described. FIG. 1 is a diagram illustrating the principle of the present invention. The light sources 6 and 8 are point light sources having the same wavelength characteristics and having radiated light intensities I ₁ and I ₂ , respectively. Light sources 6 and 8 each have a distance R from object 2
₁ and R ₂ , and the interval between the radiation positions of the light sources 6 and 8 is L. The light source 6 is caused to emit light, and the reflected light from the irradiated object 2 is photographed by the camera 5. Next, the light source 8 is caused to emit light, and the reflected light from the irradiated object 2 is photographed by the camera 5.

The light emitted from the light source 6 is emitted in all directions. Considering a sphere with a radius r around the light source 6, the light density per unit area on a sphere with a radius r of light is

It is given by I ₁ / (4πr ² ). Therefore, the reflected light intensity W from the region 4 of the object 2 existing at a position separated from the light source 6 by the distance R _1
1 is assuming that the surface reflectance of the object 2 is Rf.

[0033] is given by _{W 1 = Rf · I 1 /} (4πR 1 2). Similarly, the reflected light intensity W from the region 4 of the object 2 present at a position distant from the light source 8 by a distance R _2
2 is

[0034] is given by _{W 2 = Rf · I 2 /} (4πR 2 2).

[0035] The ratio _{W R} of the reflected light intensity _{W 2} by the reflected light intensity _{W 1} and the light source 8 by the light source _{6, W R = W 1 / W} 2 = (I 1 · R 2 2) / (I 2 ·
Obtained as R ₁ ^2). _This results from the relationship of R 2 _-R 1 = L, if the radiation position distance L of the light source 6 and the light source 8 is known, by measuring the reflected light intensity ratio _{W R,} the distance _{R 1} wherein

[0036] R ₁ = L / can be determined by _{{(W R · I 2 /} I 1) 1/2 -1}.

[0037] Thus, by using the same color of the light source, in the process of obtaining the reflected light intensity ratio W _R, since the influence of the surface reflectance Rf is canceled, it is possible to obtain information about the depth distance of the object. In this method, since the light source 6 and the light source 8 emit light in order and take an image, there is a time difference in photographing. Therefore, it cannot be applied to a moving subject. Therefore, the wavelength characteristics of the light source 6 and the light source 8 are made different,
A method is considered in which the light sources 6 and 8 emit light at the same time, the reflected light from the light source 6 and the reflected light from the light source 8 are wavelength-separated from the reflected light from the subject, and the intensity of each reflected light is measured. The surface reflectivity of the object 2 generally differs depending on the wavelength. The surface reflectance when irradiating light of wavelength λ is Rf
(Λ). The wavelength of the light source 6 is λ ₁ and the wavelength of the light source 8 is λ
When _2, the reflected light intensity _{W 1} by the light source 6,

[0038] is given by _{W 1 = Rf (λ 1)} · I 1 / (4πR 1 2). On the other hand, the reflected light intensity _{W 2} by the light source 8,

[0039] is given by _{W 2 = Rf (λ 2)} · I 2 / (4πR 2 2).

Since there is a difference in the surface reflectance depending on the wavelength,
Reflected light intensity ratio W_RIs obtained, the term of the surface reflectance Rf is key.
Information about the subject's depth
Information cannot be obtained. Wavelength λ₁And λ₂The difference between
The surface reflectance Rf (λ₁) And Rf (λ₂) Difference
Ignoring the reflected light intensity ratio W_RFind the depth of the subject
Distance can be calculated, but errors may occur in the calculation.
You. Surface reflectance Rf (λ ₁) And Rf (λ₂Depending on the difference
In order to reduce the calculation error, the wavelength λ₁And λ ₂Difference
Must be small enough, but the wavelength λ₁And λ₂
If the difference is small, the accuracy of wavelength separation will deteriorate, and the wavelength
An error will be included in each intensity measurement. Accordingly
To improve the accuracy of intensity measurement by increasing the resolution of wavelength separation
For the wavelength λ₁And λ₂The difference between
Also, the surface reflectance Rf (λ₁) And Rf (λ₂)The difference of
In order to reduce the distance and improve the accuracy of the distance measurement,
λ₁And λ₂That the difference between the two must be reduced
To limit the accuracy of distance measurement.
A world arises.

Therefore, in the present embodiment, the subject is irradiated with light having the first wavelength characteristic and light having the second wavelength characteristic different from the first wavelength characteristic simultaneously from optically different radiation positions. The light reflected by the light having the first wavelength characteristic is optically separated from the light reflected by the light having the second wavelength characteristic.
When the light having the wavelength characteristic is irradiated from the radiation position of the light having the second wavelength characteristic, a temporary reflected light intensity that would be obtained from the subject is obtained, and the reflected light intensity of the first wavelength characteristic and the temporary reflected light intensity are calculated. The depth distance of the subject is calculated based on the ratio with the reflected light intensity of the subject. By determining the temporary reflected light intensity, it is possible to cancel the difference in the surface reflectance depending on the wavelength, so that the depth distance can be accurately determined.

FIG. 2 shows an image pickup apparatus 200 according to this embodiment.
FIG. As the image capturing device 200, a digital still camera, a digital video camera capable of capturing a still image, and the like can be considered. The image capturing device 200 includes the irradiation unit 1
00, an imaging unit 120, a processing unit 60, and a control unit 80.

The irradiating section 100 irradiates the subject with light, and the imaging section 120 takes an image of the subject illuminated by the irradiating section 100.
The processing unit 60 processes the image of the subject captured by the imaging unit 120, obtains the depth distance of the captured subject, and records it as depth distribution information of the subject. The processing unit 60 also
The image of the subject captured by the imaging unit 120 can also be recorded. The control unit 80 performs feedback control based on the depth distance of the subject obtained by the processing unit 60, controls the intensity of light emitted by the irradiation unit 100, emission timing, emission time, emission position, and the like. And the exposure time.

The irradiation unit 100 includes light sources 10A and 10B,
It has optical filters 12A and 12B. Light source 10
A, 10B are installed at different positions, and the light sources 10A, 10B
The light from B passes through the optical filters 12A and 12B, each transmitting a specific wavelength component, and is simultaneously irradiated on the subject. The irradiation unit 100 inserts an optical lens such as a condenser lens into the optical path of the irradiation light when efficiently using the light amount or when optically increasing the difference between the radiation positions of the light sources 10A and 10B. The light may be condensed, or the optical emission position of the irradiation light may be changed by a lens effect.

The image pickup section 120 has an optical lens 20 as an example of an optical image forming section, a light splitting section 30, and a light receiving section 40. The optical lens 20 forms an image of light reflected from the subject. The spectroscopy unit 30 transmits the reflected light from the subject to the irradiation unit 10
0 separates the wavelength according to the wavelength characteristic of the irradiation. The light receiving unit 40 receives the reflected light on which the optical lens 20 forms an image and the wavelength of which is separated by the spectroscopic unit 30.

The light receiving section 40 is, for example, a solid-state image sensor. The subject image is formed on the light receiving surface of the solid-state imaging device. Charges are accumulated in each sensor element of the solid-state imaging device according to the light quantity of the formed subject image, and the accumulated charges are scanned in a certain order and read out as electric signals.

The solid-state image sensor has an S / N ratio so that the intensity of the reflected light from the subject can be detected with high accuracy in pixel units.
A charge-coupled device (CCD) image sensor with a good ratio and a large number of pixels is desirable. As a solid-state imaging device, other than a CCD, any of a MOS image sensor, a CdS-Se contact image sensor, an a-Si (amorphous silicon) contact image sensor, or a bipolar contact image sensor may be used.

The processing section 60 includes an image memory 62, a light intensity detecting section 64, a depth calculating section 66, and an image correcting section 67.
And a recording unit 68. The image memory 62 stores the image of the subject captured by the imaging unit 120 in accordance with the wavelength characteristics of the irradiation light emitted by the irradiation unit 100. The light intensity detection unit 64 detects the intensity of the reflected light from the image of the subject stored in the image memory 62 in pixel units or pixel area units. The depth calculation unit 66 calculates the depth distance to the subject area captured in each pixel area based on the reflected light intensity detected by the light intensity detection unit 64. The recording unit 68 records the distribution of the depth distance of the subject calculated by the depth calculation unit 66. The image correction unit 67 performs correction such as gradation correction and white balance on the image of the subject stored in the image memory 62. The recording unit 68 records the image of the subject processed by the image correction unit 67. In addition, the light intensity detection unit 64 and the depth calculation unit 66 output information on the detection level of the reflected light from the subject and the depth distribution of the subject to the control unit 80, respectively. The recording unit 68 records image data and depth distribution information in a semiconductor memory such as a flash memory or a memory card.

The control unit 80 performs feedback control based on the depth of the subject obtained by the processing unit 60, controls the intensity of light emitted by the irradiating unit 100, the timing of light emission, the light radiating position, and the like. The light receiving sensitivity and the exposure time of the light receiving unit 40 of the unit 120 are controlled. The control unit 80 may control the irradiation unit 100 and the imaging unit 120 using photometric data of a photometric sensor (not shown) or distance data of a distance sensor. Further, the control unit 80 may adjust the focus, aperture, exposure time, and the like of the imaging unit 120 when capturing an image of the subject based on the depth distance of the subject obtained by the processing unit 60.

FIG. 3 is a configuration diagram of the irradiation unit 100 and the imaging unit 120 of the present embodiment. The light sources 10A and 10B are located at distances R1 and R2 from the object 2, respectively.
The radiation interval between A and 10B is L. The optical filter 12A is mainly transmit light of wavelength lambda _A, the optical filter 1
2B mainly transmits light having wavelengths λ _B and λ _C. Irradiation unit 100 irradiates light having a wavelength lambda _A from the position of the light source 10A, the light having a wavelength lambda _B and lambda _C from the position of the light source 10B at the same time on the object 2.

The optical lens 20 of the imaging section 120 forms an image of the reflected light from the object 2 irradiated with the light from the light sources 10A and 10B. The light splitting unit 30 is a prism that splits an optical path by wavelength separation into three lights of wavelengths λ _A , λ _B , and λ _C. The light receiving units 40A, 40B, and 40C are three solid-state imaging devices. The wavelengths λ _A , λ split by the spectroscopy unit 30
_B and λ _C are respectively received by light receiving units 40A, 40B,
The light is received at 40C. Each light receiving section 40A, 40B, 40C
Is read out as electric charges by the photoelectric effect, converted into a digital electric signal by an A / D converter (not shown), and input to the processing unit 60.

FIG. 4 is a configuration diagram of the processing section 60 of the present embodiment. The subject images output from the light receiving units 40A, 40B, and 40C are stored in image memories 62A, 62B, and 62C, respectively.
Is stored in The light intensity detector 64 is provided in each image memory 62.
A, 62B, using the image data stored in 62C,
The intensity of the reflected light at the wavelengths λ _A , λ _B , and λ _C is detected. The depth calculation unit 66 uses the reflected light intensities of the wavelengths λ _A , λ _B , and λ _C detected by the light intensity detection unit 64 to calculate the light source 10A.
From the object to the region 4 of the object 2 is obtained. The depth calculation unit 66 calculates a depth distance to a pixel or an area of a subject in a pixel area of a captured image in units of pixels or pixels, obtains and outputs a depth distribution of the object. The recording unit 68 records depth distribution information of the subject.

The light intensity detector 64 detects the detected wavelength λ _A ,
The intensity of the reflected light of λ _B and λ _C is output to the control unit 80. The depth calculation unit 66 outputs the depth distribution information of the subject to the control unit 8.
Output to 0. When the intensity level is not appropriate or when the depth distance measurement accuracy is not good, the control unit 80
The intensity of the emitted light of A or 10B is adjusted, or the interval between the emission positions of the light sources 10A and 10B is adjusted. Control unit 80
Alternatively, some of the radiated light intensity ratios of the light sources 10A and 10B may be made easy in advance, and the radiated light intensity ratio may be selected according to the depth distance of the subject. For example, when the subject is at a short distance, the radiation light intensity ratio is set to a value close to 1, and when the subject is at a long distance, the radiation light intensity of the light source 10B at a position far from the subject is increased. In addition, a radiation light intensity ratio may be set.

FIG. 5 is an explanatory diagram of a depth distance calculating method by the light intensity detecting section 64 and the depth calculating section 66. Light intensity detector 64, the wavelength lambda _A strength _{W A} of the reflected light of the wavelength lambda _B
Detecting the _{W B} of the reflected light, the _{W C} of the reflected light of wavelength lambda _C respectively. The intensities of the light sources 10A and 10B are I ₁ and I
₂ and the surface reflectance at wavelength λ of the object 2 is Rf
When (lambda), the intensity _{W A} of the reflected light of wavelength lambda _A is

[0055] _{W A = Rf (λ A)} · I 1 / represented (4πR ₁ ²⁾ and the wavelength lambda intensity _{W C} of the reflected light intensity _{W B,} the wavelength lambda _C of reflected light _B is, _W B = _{Rf (λ B) · I 2} / (4πR 2 2) W C = Rf (λ C) · I expressed as ^₂ / (4πR 2 2).

[0056] Depth calculating unit 66, using the strength _{W B} of the reflected light of the wavelength lambda _B, and the intensity _{W C} of the reflected light of wavelength lambda _C,
If it is assumed that the light of the emitted light intensity I ₂ having a wavelength lambda _A was irradiated from the radiation position of the light source 10B, obtains the intensity W _D of the reflected light of the provisional obtained from the subject. The value of the obtained W _D is, ideally

[0057] is a _{W D = Rf (λ A)} · I 2 / (4πR 2 2). Therefore, the intensity W _A of the reflected light by the light from the light source 10A having a wavelength lambda _A, when determining the ratio between the intensity W _D of the reflected light of the temporary by the light from the light source 10B having the same wavelength lambda _A, the surface reflection The term of the rate Rf (λ _A ) is canceled,

[0058] _{W A / W D = (I} 1 · R 2 2) / (I 2 · R 1 2) is obtained, from which the _R 2 _-R 1 = L, to calculate the depth distance _{R 1} of the object be able to.

[0059] Wavelength λ intensity _{W B} of the reflected light _B, and the wavelength λ
Using intensity W _C of the reflected light and _C, the intensity of the reflected light of the provisional W _D
There are many possible variations in the method of finding. FIG.
FIG. 4 is an explanatory diagram of a method for obtaining a temporary reflected light intensity by interpolation. Intensity of the reflected light of the wavelength lambda _{B W B,} and the wavelength lambda _C
By interpolating the intensity W _C of the reflected light, obtaining a temporary intensity W _D in the case of wavelength lambda _A. Temporary intensity W by linear interpolation
It may also be determined _D, the strength W _B of simply wavelength lambda _B reflected _light, and an intermediate value of intensity W _C of the reflected light of wavelength lambda _C may be intensity W _D tentative.

FIG. 7 shows the provisional reflected light intensity obtained by extrapolation.
FIG. 4 is an explanatory diagram of a method for obtaining the value. Wavelength λ _BOf reflected light W
_B, And wavelength λ_COf reflected light W_CIs extrapolated to the wavelength λ
_BShorter wavelength λ_ATemporary strength W in case of_DAsk for.

Further, there is the following modified example. FIG. 8 shows the light source 1
It is a figure explaining the method of calculating | requiring temporary reflected light intensity from each reflected light by 0A, 10B. The irradiation unit 100
Irradiation light having wavelengths λ1 and λ2 is simultaneously emitted from the light source 10A, and irradiation light having wavelengths λ3 and λ4 is simultaneously emitted from the light source 10B, and the imaging unit 120 converts reflected light obtained from the subject into wavelengths λ1, λ2 from the light source 10A. And the respective reflected lights having wavelengths λ3 and λ4 from the light source 10B. The light intensity detection unit 64
Wavelength λ1 from the light source 10A, obtains the intensity of the reflected light .lambda.2, depth calculation unit 66, calculates the position of the light source 10A, the reflected light intensity W _A assuming that was irradiated with light of wavelength λ5 same intensity I do. Further, the light intensity detection unit 64 obtains the intensity of the reflected light of the wavelengths λ3 and λ4 from the light source 10B, and it is assumed that the depth calculation unit 66 irradiates the light of the wavelength λ5 with the same intensity from the position of the light source 10B. Reflected light intensity W
Calculate _D. Reflected light intensity of the depth calculation unit 66 tentatively W _A
The ratio of the W _D determined, it is possible to calculate the depth distance of the object.

Further, there is the following modified example. FIG. 9 (a)-
(D) is a diagram for explaining a method of obtaining a temporary reflected light intensity when the light is irradiated using a band-pass filter that transmits only light of a long wavelength or a short wavelength. The optical filter 12A of the light source 10A is a band-pass filter which transmits only light of the first wavelength longer than the boundary wavelength lambda _1, the light radiated from the light source 10A passes through the optical filter 12A, FIG. 9 (a) The object is irradiated as light having the wavelength characteristics shown in FIG. Optical filter 12 of light source 10B
B is a band-pass filter which transmits only light of the second wavelength shorter than the boundary wavelength lambda _2, the light radiated from the light source 10B is transmitted through the optical filter 12B, FIG. 9 (b)
The object is irradiated as light having the wavelength characteristics shown in FIG. The first boundary wavelength λ ₁ may be shorter than the second boundary wavelength λ ₂ . That is, the optical filter 12A,
The wavelength characteristic of the light transmitted through the first boundary wavelength 12B is the first boundary wavelength λ.
It may have an overlap in the wavelength region of from ₁ to ₂ second boundary wavelength lambda. Therefore, when the first boundary wavelength λ ₁ is shorter than the second boundary wavelength λ ₂ , the wavelength characteristics of the irradiation light from the light sources 10A and 10B are from the first boundary wavelength λ ₁ to the second boundary wavelength λ _2. Have an overlap in the wavelength region.

The spectroscopic section 30 has a wavelength λ_A, Λ_B, Λ_CLight of
Has an optical filter that transmits
The obtained reflected light is wavelength-separated, and the wavelength λ_A, Λ_B, Λ_CTo
The received light is received by the light receiving unit 40. For light source 10A
Before passing through the spectroscopy unit 30, the reflected light from FIG.
FIG. 9 shows the wavelength characteristic of FIG.
As shown in FIG._AOnly the components of are extracted.
The same applies to the light reflected by the light source 10B.
Before passing through 30, it has the wavelength characteristic of FIG.
However, as shown in FIG.
λ_B, Λ_CAre extracted respectively. here
And the wavelength λ_AIs the first boundary wavelength λ₁And the second boundary wavelength
λ₂Which is longer than the longer one,
λ_B, Λ_CIs the first boundary wavelength λ₁And the second boundary wavelength λ
₂Wavelength must be shorter than the shorter one of
is there. Because the separated wavelength λ_AWith reflected light
Must not include the interference due to the irradiation light of the light source 10B.
Wavelength separated by λ_B, Λ_CHave each anti
The emitted light may include interference due to the light emitted from the light source 10A.
Because it does not become. Wavelength λ_AOf reflected light having
_AAnd the wavelength λ_B, Wavelength λ _CEach having a reflection
Extrapolating the light intensity, the wavelength λ_AThat the ingredients contained
Temporary reflected light intensity W assuming_DAsk for. Reflected light intensity
Degree ratio W_A/ W_DThe process of finding the depth distance of the subject from
As described above.

In any of the above methods, the wavelengths λ _B and λ _C are linearly interpolated or linearly extrapolated so that the temporary reflected light intensity can be accurately obtained by processing such as interpolation, extrapolation, and averaging. It is more preferable to set the values as close to each other as possible within the range in which insertion is possible. FIG. 10 is a diagram showing the surface reflectance of three types of objects. The horizontal axis of the graph is the wavelength, and the vertical axis is the reflectance. Graphs 302, 304, and 306 are the results of measuring the surface reflectance of three types of objects, human skin, roads, and leaves, with a spectrometer. 630 nm, 650 nm, 6
Points on each graph for a wavelength of 70 nm were marked.
In the wavelength region of 630 nm, 650 nm, and 670 nm,
Linear interpolation is possible for any object. Light sources in the same wavelength range are easily available. As the wavelengths λ _A , λ _B , and λ _C , such linear interpolation can be performed, and values in a wavelength region that can be easily obtained as a light source can be selected. Further, if image correction processing such as gradation correction performed by an ordinary digital camera or the like is performed on the output signal of the solid-state imaging device of the light receiving unit 40, the linearity of the signal is lost. Therefore, it is preferable to detect the intensity at the stage of the signal intensity having linearity with respect to the intensity of the incident light to the solid-state imaging device, and to perform processing such as interpolation, extrapolation, and averaging. Alternatively, a table representing an inverse function of a signal conversion function by an image correction process such as a gradation correction is prepared, and the signal output after the image correction is temporarily referred to the table of the inverse function, and the incident light to the solid-state imaging device is determined. After converting to a signal strength that has linearity with respect to the strength, the strength is detected, interpolated,
Processing such as extrapolation and averaging may be performed.

Further, in the above description, an optical splitting element for splitting an optical path by wavelength separation, for example, a prism or a beam splitter is used as the light splitting unit 30.
As 0, an optical filter arranged on the light receiving surface of the light receiving unit 40 may be used. FIG. 11 is a diagram illustrating an optical filter provided in the light receiving unit 40 and transmitting a specific wavelength component. A single-plate solid-state imaging device is used as the light-receiving unit 40, and an optical filter 32 is provided on the light-receiving surface of the solid-state imaging device.
In the optical filter 32, filters that respectively transmit only the wavelengths λ _A , λ _B , and λ _C are alternately arranged. Accordingly, the wavelengths λ _A , λ _B ,
lambda _C understand any one is obtained by receiving light, the wavelength lambda _A, lambda _B, the light having a lambda _C can be received by wavelength separation. Compared to using a prism or beam splitter, the single-chip solid-state image sensor receives light,
The device can be downsized.

In the description of the above embodiment, the irradiation
Subject with a large difference in surface reflectance depending on the wavelength of light
If you are targeting, wavelength λ_A, Λ_B, Λ_CIs the temporary reflected light intensity
As close as possible to avoid errors in the calculation of degrees
Is desirable. On the other hand, the intensity of the reflected light of each wavelength component
In order to improve the accuracy of degree detection, the wavelength λ_A, Λ_B, Λ _C
Try to minimize the wavelength components other than
Or the wavelength λ_A, Λ_B, Λ _CValues that are far from each other
And the wavelength λ_A, Λ_B, Λ_CResolution and wavelength interference
Is desirably as small as possible. Therefore,
Depending on the surface reflectance characteristics of the subject and the required measurement accuracy
The wavelength characteristic of the light source 10 of the irradiation unit 100, the optical filter
-12, wavelength transmission characteristics of the spectroscopic unit 30 of the imaging unit 120
It is preferable to design long transmission characteristics.

FIG. 12 is a flowchart of the distance measuring method according to the present embodiment. Irradiation unit 100, a first irradiation light to the wavelength lambda _A as a main wavelength component and a second irradiation light whose main wavelength component different wavelengths lambda _B and the wavelength lambda _C the wavelength lambda _A, The subject is simultaneously irradiated from different optically radiating positions (S100).

The optical lens 20 of the imaging unit 120 is
The reflected light from the subject irradiated with the second and the second irradiation light
(S102). The spectroscopy unit 30 reflects light reflected from the subject.
From the wavelength λ_AA first reflected light having a wavelength λ_BWith
And the wavelength λ _CA third reflected light having
Are optically separated (S104).

The light receiving section 40 includes the first, second, and
3 is received (S106). Light intensity of processing unit 60
The degree detection unit 64 determines the intensity of the first, second, and third reflected lights.
W_A, W _B, W_CIs detected (S108).

[0070] Depth calculating unit 66, first, second, the intensity of the third reflected light _{W A,} W B, by using the _{W C,} and calculates the depth distance to the object (S110).

FIG. 13 is a flowchart of the depth distance calculation processing S110. Second and third reflected light intensities W
_B , based on W _C , the light of wavelength λ _A and the intensity of I 2
Determining the intensity W _D of the reflected light tentative from the object when it is assumed from the radiation position of the irradiation light that is irradiated (S11
2). Intensity W _D of the reflected light The temporary strength W _B and W _C of the second and third reflected light determined by interpolation or extrapolation. First reflected light intensity W _A and intensity W _D of the reflected light of temporary
Determining the ratio _W A / _{W D} of (S114). The distance to the object is calculated based on the first and second irradiation light intensities I ₁ and I ₂ , the first and second irradiation light emission position intervals L, and the reflected light intensity ratio W _A / W _D. (S116).

FIG. 14 is a flow chart of the depth distance calculation processing S110.
It is a flowchart of a modification. Second and third reflected light
Strength W_B, W_CAverage value of W_D= (W_B+ W_C) / 2
It is determined (S118). Intensity W of the first reflected light_AAnd the second
And the average intensity W of the third reflected light_DRatio W_A/ W_DAsk for
(S120). First and second irradiation light intensities I₁,
I ₂, The distance L between the radiation positions of the first and second irradiation light, and the intensity of the reflected light
Degree ratio W_A/ W_DCalculate the distance to the subject based on
(S122).

As described above, according to the image pickup apparatus of the present embodiment, light having different wavelength characteristics is simultaneously radiated on the subject from different optically radiating positions, and the wavelength characteristic is changed from the reflected light obtained from the subject. In addition, the wavelength separation is performed, and the depth distance to the subject can be easily obtained by using the intensity of the wavelength-separated reflected light.

Further, since an image formed by reflected light from a subject is captured by a solid-state image sensor and stored as image data, the depth distance can be calculated by detecting the reflected light intensity for each pixel or pixel area. It is possible to obtain the depth distribution of the region of the subject that has been set. Therefore, 2
By obtaining the depth distribution of the subject from the three-dimensional image, it is possible to create a three-dimensional stereoscopic image of the subject.

(Embodiment 2) The second embodiment of the present invention
explain. The image pickup apparatus according to the present embodiment is a first embodiment.
Irradiation unit 100 and the imaging unit 1
20 has the same configuration because only a part of the configuration is different.
Description of the elements is omitted, and different components are
I will explain only. FIG. 15 illustrates the irradiation unit 100 of the present embodiment.
FIG. 2 is a configuration diagram of an imaging unit 120. In the present embodiment, the irradiation unit
The 100 optical filters 12A mainly have a wavelength λ._AThrough the light of
The optical filter 12B mainly has a wavelength λ_AShorter
Wavelength λ _BAnd the wavelength λ_ALonger wavelength λ_CWith light
Spend. The irradiating unit 100 has a wavelength λ from the position of the light source 10A.
_AAnd the wavelength λ from the position of the light source 10B._BAnd λ_CWith
The object 2 at the same time.

[0076] spectroscopic unit 30 of the imaging unit 120 is a prism for splitting the light having the wavelength lambda _A, the optical path to the wavelength separation and light having a wavelength lambda _B and lambda _C. The light receiving units 40A and 40B are two solid-state imaging devices. By the spectroscopic unit 30, light having a spectral wavelength lambda _A is the light receiving portion 40A
The light having the wavelengths λ _B and λ _C is received by the light receiving unit 40B. The light received by the light receiving units 40A and 40B is converted into an electric signal and input to the processing unit 60.

FIG. 16 is a configuration diagram of the processing section 60 of the present embodiment. The subject images output from the light receiving units 40A and 40B are stored in the image memories 62A and 62B, respectively.
Light intensity detecting unit 64 using the image data stored the image memory 62A, the 62B, the reflected light having a wavelength lambda _A,
The intensity of the reflected light having the wavelengths λ _B and λ _C is detected. The depth calculator 66 calculates the wavelength λ _A detected by the light intensity detector 64.
The distance R1 from the light source 10A to the area 4 of the object 2 is obtained using the intensity of the reflected light at the wavelengths λ _B and λ _C. The depth calculation unit 66 calculates a depth distance to a pixel or an area of a subject in a pixel area of a captured image in units of pixels or pixels, obtains and outputs a depth distribution of the object. The recording unit 68 records depth distribution information of the subject.

FIG. 17 is an explanatory diagram of a depth distance calculation method by the light intensity detection unit 64 and the depth calculation unit 66. Light intensity detector 64, the intensity W _A of the reflected light of wavelength lambda _A, the wavelength lambda
Respectively detect the intensity W _E of the reflected light having a _B and lambda _C. Light source 10A, the intensity of 10B as _I 1, _{I 2} respectively, when the surface reflectance at a wavelength of the object 2 lambda and Rf (lambda), the intensity _{W A} of the reflected light of wavelength lambda _A is

[0079] _{W A = Rf (λ A)} · I is expressed as ₁ / (4πR 1 ^2), the intensity of the reflected light having a wavelength lambda _B and lambda _C W
_{E is, W E = Rf (λ B} ) · I 2 / (4πR 2 2) + Rf (λ
_{C) ·} I expressed as ^₂ / (4πR 2 2).

The depth calculator 66 calculates the wavelength λ _B and the wavelength λ _C
The half value of the intensity W _E of the reflected light having a W _D. Since the wavelength λ _A is an intermediate value between the wavelength λ _B and the wavelength λ _C ,
The value of W _D, when it is assumed that irradiation with light of the radiation intensity I ₂ having a wavelength lambda _A from the radiation position of the light source 10B, approximately equal to the intensity of the reflected light of the provisional obtained from the subject. The obtained value of W _D is ideally

[0081] is a _{W D = Rf (λ A)} · I 2 / (4πR 2 2). Therefore, the intensity W _A of the reflected light by the light from the light source 10A having a wavelength lambda _A, when determining the ratio between the intensity W _D of the reflected light of the temporary by the light from the light source 10B having the same wavelength lambda _A, the surface reflection The term of the rate Rf (λ _A ) is canceled,

[0082] _{W A / W D = (I} 1 · R 2 2) / (I 2 · R 1 2) is obtained, from which the _R 2 _-R 1 = L, to calculate the depth distance _{R 1} of the object be able to.

Temporary reflected light intensity W_DTo get exactly
And the wavelength λ_AIs the wavelength λ_B, Λ_CWavelength between
Is more preferable. In the above description, the spectroscopy unit 30
Long λ _AAnd a wavelength λ_BAnd λ_CWith light having
Although the wavelength is separated, the wavelength λ is used as a filtering method.
_B, Λ_CIt is not necessary to selectively transmit
λ_AThe same applies when using a band cut filter that cuts
Has the same effect. FIGS. 18A to 18D show band brackets.
Explains how to separate reflected light using a multi-filter
FIG. As shown in FIG. 18A, illumination from the light source 10A is performed.
The emission wavelength is λ _AHas wavelength characteristics with the main wavelength component
I do. As shown in FIG. 18B, irradiation light from the light source 10B
Is the wavelength λ_AThe wavelength λ sandwiching_B, Λ_CThe main wavelength components
Wavelength characteristics. The spectroscopy unit 30 is configured as shown in FIG.
As shown in FIG._AA van that only penetrates
And a pass filter, as shown in FIG.
Wavelength λ _ABand cut filter that cuts wavelength components
And a band-pass filter for reflected light from the subject.
Wavelength λ_AReflected light with
Band cut filter that separates and reflects light from the subject
The wavelength λ_BAnd wavelength λ_CHaving
Separates reflected light. Wavelength λ_BAnd wavelength λ_CReflected light with
Half the intensity of the_AHaving the intensity of the reflected light
The method of obtaining the depth value of the subject based on the ratio is described above.
It is on the street.

FIG. 19 is a flowchart of the distance measuring method according to the present embodiment.
It is a low chart. The irradiation unit 100 has a wavelength λ_AThe major
Irradiation light having a wavelength component_AShorter waves
Long λ _BAnd the wavelength λ_ALonger wavelength λ_CThe main wavelength components
From the optically different radiation position
At the same time, the object is irradiated (S200).

The optical lens 20 of the image pickup section 120 forms an image of the reflected light from the subject irradiated with the first and second irradiation lights (S202). Spectroscopic unit 30, the reflected light from the object, a first reflected light having a wavelength lambda _A, and a second reflected light optically separated with a wavelength lambda _B and wavelength λ _C (S204).

The light receiving section 40 receives the separated first and second reflected lights (S206). Light intensity detecting unit 64 of the processor 60, first, the intensity of the second reflected light _W A, detects the _{W E} (S208).

[0087] Depth calculating unit 66, first reflected light intensity _{W A,} the ratio _{W A} half value _{W D} intensity _{W E} of the second reflected light
/ _W obtains the _D (S212), based on the first, the intensity of the second irradiation light _I 1, _{I 2,} first, the radiation position interval of the second illumination light L, the reflected light intensity ratio _W A / _{W D} Then, the distance to the subject is calculated (S214).

In the above description, an optical splitting element for splitting an optical path by wavelength separation, such as a prism or a beam splitter, is used as the light splitting unit 30, but the light receiving unit is used as the light splitting unit 30 as in the first embodiment. A single-plate solid-state imaging device may be used as 40, and an optical filter 32 may be provided on the light receiving surface of the solid-state imaging device. The optical filter 32 is
Filters that transmit only the wavelength λ _A and filters that transmit the wavelengths λ _B and λ _C are alternately arranged. Compared to the case where a prism or a beam splitter is used, a single solid-state imaging device receives light, so that the device can be downsized.

As described above, according to the image pickup apparatus of the present embodiment, the first wavelength having the first wavelength as the main wavelength component is used.
And simultaneously irradiating the subject from the optically different radiation positions with the second irradiation light having the second and third wavelengths as main wavelength components sandwiching the first wavelength in the middle. The obtained reflected light is separated into a first reflected light having a first wavelength and a second reflected light having second and third wavelengths, and a first reflected light intensity and a second reflected light are obtained. The depth distance of the subject can be calculated based on the ratio to half the light intensity. Only by halving the intensity of the second reflected light, it is possible to obtain a temporary reflected light intensity when the light having the first wavelength is irradiated from the radiation position of the second irradiation light. The depth distance of the subject can be calculated.
Also, a solid-state imaging device that receives reflected light from a subject is provided with two
A plate can be used, and the size of the device can be reduced.

(Embodiment 3) A third embodiment of the present invention will be described. The image capturing apparatus according to the present embodiment is different from the image capturing apparatus according to the first embodiment in that the irradiation unit 100 and the image capturing unit 1
Since only a part of 20 differs, the description of the same components will be omitted, and only different components will be described. FIG. 20 is a configuration diagram of the irradiation unit 100 and the imaging unit 120 of the present embodiment. In the present embodiment, the light sources 10A and 10B of the irradiation unit 100 are infrared light sources. The optical filter 12A transmits light having a wavelength λ _A in the infrared region, and the optical filter 12B transmits light having a wavelength λ _A in the infrared region.
_B and light having a wavelength λ _C are transmitted. Irradiation unit 100
Is light at a wavelength λ _{A in} the infrared region from the position of the light source 10A,
The object 2 is simultaneously irradiated with light having wavelengths λ _B and λ _C in the infrared region from the position of the light source 10B. The object 2 is further irradiated with light in the visible light region, for example, natural light or illumination light.

The spectroscopic unit 30 of the image pickup unit 120 separates the wavelength into light having a wavelength λ _A in the infrared region, light having the wavelengths λ _B and λ _C in the infrared region, and light in the visible light region. A prism that divides the optical path. Light receiving units 40A, 40B, and 4
0C is a three-plate solid-state imaging device. By the spectroscopic unit 30, light having a spectral wavelength lambda _A is the light receiving portion 40A
The light having the wavelengths λ _B and λ _C is received by the light receiving unit 40B, and the visible light is received by the light receiving unit 40C. The light receiving unit 40 is provided so that the captured image of the reflected light in the infrared region is not out of focus.
A and 40B are set in advance at the positions where focus is achieved. The light received by the light receiving units 40A, 40B, and 40C is
The signal is converted into an electric signal and input to the processing unit 60.

FIG. 21 is a configuration diagram of the processing section 60 of the present embodiment. The subject images output from the light receiving units 40A and 40B are stored in the image memories 62A and 62B, respectively.
The light intensity detector 64 detects the intensity of the reflected light using the image data stored in each of the image memories 62A and 62B, and the depth calculator 66 uses the intensity of the reflected light detected by the light intensity detector 64. Thus, the distance R1 from the light source 10A to the area 4 of the object 2 is obtained. Light intensity detector 64 and depth calculator 6
The operation of No. 6 is the same as that of the second embodiment, and the description is omitted. The depth calculation unit 66 calculates a depth distance to a pixel or a region of a subject in a pixel region of a captured image, and obtains and outputs a depth distribution of the subject. The recording unit 68 records depth distribution information of the subject. Further, the subject image output by the light receiving unit 40C is stored in the image memory 62C. The image correction unit 67 performs image correction such as gradation correction on the image data stored in the image memory 62 and outputs the image data as subject image data. The recording unit 68 converts the subject image data into the depth distribution of the subject. Record with information.

In the above description, the reflected light having the wavelengths λ _B and λ _C was received by the light receiving section 40B without being separated.
Thus, the reflected light having the wavelength λ _B and the reflected light having the wavelength λ _C may be separated, and different solid-state imaging devices may receive the respective reflected lights. In that case, in the same manner as in the first embodiment, the depth distance of the subject can be obtained by using the intensities of the reflected lights of the wavelengths λ _A , λ _B , and λ _C.

As described above, according to the image pickup apparatus of the present embodiment, since infrared light is used for measuring the depth distance of a subject, the subject is exposed to natural light or natural light when the subject is irradiated with illumination light. However, the depth distance of the subject can be measured. Therefore, it is not necessary to make the room a dark room for measuring the depth distance of the subject. Further, since the reflected light in the visible light region can be imaged while being separated, the image of the subject can be photographed at the same time as measuring the depth distribution of the subject. It is possible to perform image processing of a subject using the depth distribution of the subject, such as extracting a main subject from a captured image of the subject based on the depth distribution, and separating a background and a human image.

(Embodiment 4) A fourth embodiment of the present invention will be described. The image capturing apparatus according to the present embodiment includes first, second,
The irradiation unit 10 is different from the image pickup apparatus according to the third embodiment.
The only difference is that the half mirrors 17 and 18 are used in order to make the optical axis of the imaging unit 120 the same as that of 0. FIG. 22 is a configuration diagram of the irradiation unit 100 and the imaging unit 120 of the present embodiment. In the first embodiment, the half mirrors 17 and 18
Although an example using is described, a similar configuration can be adopted in the second and third embodiments. Light source 1 of irradiation unit 100
The radiation positions of 0A and 10B are separated by a distance L, and the irradiation light from the light source 10B is reflected on the half mirror 17, further reflected on the half mirror 18, and irradiated on the object 2.
The irradiation light from the light source 10A passes through the half mirror 17,
The light is reflected on the half mirror 18 and is irradiated on the object 2. The reflected light from the subject passes through the half mirror 18 and is imaged by the optical lens 20 of the imaging unit 120.

In this embodiment, the irradiation unit 100 and the imaging unit 1
Since the optical axis of 20 is optically coaxial, when the imaging unit 120 captures a subject illuminated by the illumination unit 100,
There is no shadowed area that cannot be captured. Therefore, the depth distribution of the entire illuminated subject can be calculated, and there is no occurrence of a blind spot where the depth distance cannot be calculated. In addition, by making the optical axis of the irradiation unit 100 and that of the imaging unit 120 coaxial, the size of the entire imaging device 200 can be reduced.

(Fifth Embodiment) FIG. 23 is a view for explaining the principle according to a fifth embodiment of the present invention. Lights of the intensities P1 and P2 are emitted to the object 120 from the irradiation positions 112 and 114, respectively, and reflected light of each light by the object 120 is imaged by the camera 110. The camera 110 is, for example, a charge-coupled device (CCD) and has a plurality of pixels.
The reflected light from the measured portion 122 of the object 120 and the vicinity 124 of the arsenic separated portion of the object 120 are photographed for each pixel, and the intensity is detected for each pixel. Based on the reflected light captured by the camera 110, the distance L of the object 120 to the measured portion 122, the inclination θ ₀ of the surface of the measured portion 122, and the reflectance R _Obj of the surface of the measured portion 122 are calculated. Irradiation position (112,
114) may be located at any position. Either of the irradiation angles of the light irradiated from the irradiation positions (112, 114) to the object 120 is calculated.
The camera 110 is arranged at an optically same position as any one of the irradiation positions (112, 114), and calculates the irradiation angle of the light irradiated on the object 120 based on the incident angle of the reflected light captured by the camera 110. I do. In this example, the irradiation position 1
The case where the camera 12 and the camera 110 are optically arranged at the same position will be described.

As shown in FIG. 23, the irradiation position 114 is
L from irradiation position 112₁₂It is placed at a remote location. Distance
Release L₁₂Is a known value specific to the measurement system. Also irradiation
The position to be measured 12 of the object 120 from the position (112, 114)
2 is the angle at which light is irradiated₁, Θ₂And irradiation
From the position (112, 114) near the measured part of the object 120
The angle at which the side 124 is irradiated with light is θ₁’, Θ₂’
And In addition, the camera 110 is optically
Is located at the same position,
The angle at which the reflected light is received is θ_C, Near the part to be measured 12
Θ is the angle at which the reflected light from_C’, Θ
_C= Θ₁, Θ_C’= Θ₁’. The irradiation position 11
The distance from 2 to the measured part 122 is L₁, Measured
The distance to the neighborhood 124 of the fixed part is L₁’And irradiation position 1
The distance from 14 to the part to be measured is L ₂, Near the part to be measured 1
L to the distance to 24₂’.

Assuming that the intensities of the light emitted from the irradiation positions (112, 114) from the portion to be measured 122 are D ₁ and D ₂ , D ₁ and D ₂ are given as follows.

[0100]

(Equation 1) That is,

(Equation 2) Becomes

Also,

(Equation 3) So that

(Equation 4) Becomes Also,

[0102]

(Equation 5) Since L ₁₂ and θ ₁ are known values, θ ₂ is L
Given by That is,

[0103]

(Equation 6) Becomes Therefore,

[0104]

(Equation 7) Becomes _{Here, D1, D 2, P 1} , P 2, θ 1 , so is the measurement value or known values, unknowns above formula are two of theta ₀ and L. Therefore, a combination of (θ ₀ , L) satisfying the above condition can be obtained. That is, when the distance L to the measured portion 122 is assumed, the surface inclination θ ₀ of the measured portion corresponding to the assumed distance L can be calculated. Similarly, for the vicinity 124 of the measured portion, a combination of the distance L 'to the measured portion vicinity 124 and the surface inclination θ ₀ ′ of the measured portion vicinity 124 can be obtained.

The present invention relates to a combination of the distance L of the object 120 to the measured portion 122 and the surface inclination θ ₀ of the measured portion 122, the distance L ′ of the object 120 to the vicinity 124 of the measured portion, and the measured value The distance L to the part to be measured 122 and the plane inclination θ ₀ are calculated based on the combination with the plane inclination θ ₀ ′ of the vicinity 124 of the section. Hereinafter, the calculation method will be described in detail.

FIG. 24 shows the distance L to the measured portion 122,
Is an illustration of an example of a method of calculating the surface inclination theta ₀ of the measured portion 122. First, the distance L to the measured part 122 is La
Assume that Based on the equations described in connection with FIG. 23, it calculates the surface inclination theta ₀ of the measured portion 122 corresponding to the assumed distance La. Next, the surface of the measured part 122 determined by the calculated plane inclination θ ₀ is extended,
Assuming that the vicinity 124 of the measured part exists at the point where the optical path of the reflected light from the light source 24 intersects, the vicinity 12 of the measured part
The temporary distance Lb up to 4 is calculated. At this time, the tentative distance Lb to the vicinity 124 of the measured part is the tentative distance La between the camera 110 and the measured part 122, the incident angle θ ₁ of the reflected light from the measured part 122, and the vicinity 124 of the measured part. Can be geometrically calculated based on the incident angle θ ₁ ′ of the reflected light from the object and the surface inclination θ ₀ of the measured portion 122.

Next, corresponding to the calculated temporary distance Lb,
The plane inclination θ ₀ ′ in the vicinity 124 of the measured part is calculated. The plane inclination θ ₀ ′ can be calculated by the mathematical expression described with reference to FIG. Since the distance between the measured portion 122 of the object 120 and the vicinity 124 of the measured portion is very small, the surface inclination of the measured portion 122 and the vicinity 124 of the measured portion are substantially the same. Therefore, by comparing the calculated plane inclination θ ₀ and θ ₀ ′, it is possible to determine whether the initially assumed distance La to the measured portion 122 is a correct value. That is, if the difference between the plane inclinations θ ₀ and θ ₀ ′ is within a predetermined range, the assumed distance La can be set as the distance L to the measured portion 122.

If the difference between the plane inclinations θ ₀ and θ ₀ ′ is not within the predetermined range, it is determined that there is an error in setting the initially assumed distance La, and the distance L to the measured part 122 is determined. Similar calculations may be performed by setting other values. It is clear that the distance L to the measured portion 122, which satisfies the mathematical expression described with reference to FIG. 23, has an upper limit and the lower limit is zero. Just check it. For example, the true distance L to the measured part 122 may be extracted by a binary search method for the distance La in a finite range. Further, for the temporary distance La, a difference between θ ₀ and θ ₀ ′ is calculated at predetermined distance intervals in a finite range,
The true distance L may be extracted. In addition, for the temporary distance La, a difference between θ ₀ and θ ₀ ′ may be calculated for a plurality of values, and the temporary distance La that minimizes the difference may be set as the true distance L.
The surface inclination information of the measured portion can be calculated based on the distance information to the measured portion, using the mathematical formula described with reference to FIG. In addition, the surface reflectance information of the measured portion can be calculated by the mathematical expression described with reference to FIG.

FIG. 25 is a graph showing a predetermined distance La for the provisional distance La.
Surface inclination θ at a predetermined distance interval in the range ₀And surface inclination θ₀’
An example of the result of calculating is shown. In the graph of Figure 25
The horizontal axis indicates the temporary distance La to the measured part 122.
The vertical axis represents the measured part 122 and the vicinity 124 of the measured part.
Indicates the surface inclination. In the present example, the measured part 1 of the object 120
The distance L up to 22 is 200 mm,
Is 0 degree. The irradiation position 112 and the irradiation position 11
10 mm, θ₁20 degrees, the provisional distance La
The calculation was performed with an interval of 10 mm.

Data 192 indicates the surface inclination of the measured portion 122 corresponding to the provisional distance La shown on the horizontal axis, and data 194 indicates the surface inclination of the vicinity 124 of the measured portion. At the provisional distance of 200 mm, the surface inclination of the measured portion 122 and the vicinity 124 of the measured portion coincide at 0 degree.

(Embodiment 6) FIG. 26 is a view for explaining the principle of an information acquiring method for acquiring information on a distance to an object and the like. From the irradiation position 212, the irradiation position 214, and the irradiation position 216,
The object 220 is irradiated with light having the intensities P1, P2, and P3, and the reflected light of each light by the object 220 is reflected on the camera 2
An image is taken at 10. The camera 210 is, for example, a charge-coupled device (CCD), has a plurality of pixels, captures reflected light from the irradiated portion 224 of the object 220 for each pixel, and detects the intensity of each pixel. . Based on the reflected light captured by the camera 210, the distance L to the irradiated part 224 of the object 220, the inclination θ ₀ of the surface of the irradiated part 224, and the reflectance R _Obj of the surface of the irradiated part 224 are calculated. The irradiation positions (212, 214, 216) and the camera 210
Although it may be arranged at an arbitrary position, in this example, a case where the irradiation position 212, the irradiation position 216, and the camera 210 are arranged on a straight line will be described.

As shown in FIG. 26, the irradiation position 216 is
L from irradiation position 212₁₂The irradiation position 21
4 is L from the irradiation position 212₂₃Camera at a distance
210 is located at a position X away from the irradiation position 212
You. Distance L₁₂, L₂₃, X are known values specific to the measurement system
It is. The irradiation position (212, 214, 216)
Is the angle at which the irradiated part 224 of the object 220 is irradiated with light.
To θ₁, Θ₂, Θ ₃Camera 210
The angle at which the reflected light at the irradiation unit 224 is received is θ_CToss
You. Of these, θ_CIs taken for each pixel of the camera 210
It is calculated based on the reflected light obtained. In addition, the irradiation position (2
12, 214, 216) to the irradiated part 224
Is L₁, L₂, L₃And

[0113] The light emitted from the irradiation position (212, 214, 216), when _{D 1,} D 2, _{D 3} respectively the intensity of the reflected light at the irradiated portion _{224, D 1, D 2,} D
₃ is given as follows.

[0114]

(Equation 8) That is,

(Equation 9) Here, f () is a function. Also,

[0115]

(Equation 10) So that

[Equation 11] Becomes Also,

[0116]

(Equation 12) Since X, L ₁₂ and θ _C are known values,
θ ₁ , θ ₂ , and θ ₃ are each given by a function of L. That is,

[0117]

(Equation 13) Becomes Therefore,

[0118]

[Equation 14] Becomes D ₁ , D ₂ , D _3, θ _C , P ₁ , P ₂ , P
_{Since 3} is a measured value or a known value, from the above two equations,
The unknowns θ ₀ and L can be obtained. Also, if θ ₀ is obtained, R _obj can be calculated from any of D ₁ , D ₂ , and D _3.
Can be requested. As described above, by imaging the reflected light from the object of light emitted from three or more optically different irradiation positions, distance information to the object, inclination information of the surface of the object, and reflectance of the surface of the object are obtained. Information can be calculated.

In this example, the irradiation position 212, the irradiation position 216, and the camera 210 are arranged in a straight line. However, if the distance between the elements is known, the distance information to the object and the surface of the object can be obtained by reflected light. It is clear that the inclination information and the reflectance information of the surface of the object can be calculated.

According to the information acquisition method described above, errors due to the inclination of the surface of the object 220 are eliminated,
You can obtain distance information up to. Also, object 2
It is possible to acquire the inclination information of the surface of the object 20 and the reflectance information of the surface of the object 220.

However, the irradiation positions (212, 214, 21
When 6) is arranged on a straight line, distance information to the object cannot be calculated by the method described with reference to FIG. Hereinafter, the irradiation positions (212, 214,
The method of acquiring distance information to the object, information on the inclination of the surface of the object, and information on the reflectance of the object surface when (216) is arranged on a straight line will be described.

FIG. 27 is an explanatory diagram of an information acquisition method for acquiring distance information to an object and the like when the irradiation positions are arranged on a straight line. Irradiation position 212, irradiation position 214,
The object 220 is irradiated with light having the intensities P1, P2, and P3 from the irradiation position 216, and the camera 210 captures the reflected light of the light from the object 220.

As shown in FIG. 27, the irradiation position 216 is
The irradiation position 21 is located at a position L ₁₃ away from the irradiation position 212.
Reference numeral 4 denotes a position L ₁₂ away from the irradiation position 212, and the camera 210 is disposed substantially at the same position as the irradiation position 212. The distances L ₁₂ and L ₁₃ are known values unique to the measurement system. Further, the irradiation positions (212, 214, 216) are such that the angles of irradiating the irradiated portion 224 of the object 220 with light are θ ₁ , θ ₂ , and θ ₃ , respectively, the angle at which the light is referred to as θ _C. Of these, θ _C is calculated based on the reflected light captured for each pixel of the camera 210. In this example, the camera 2
Since 10 is located at the same position as the irradiation position 212 is θ _{C =} θ _1. The irradiation position (212, 21)
4, 216) to the irradiated portion 224 are denoted by L ₁ , L ₂ , and L ₃ , respectively.

Assuming that the intensities of the light emitted from the irradiation positions (212, 214, 216) at the irradiated portion 224 are D ₁ , D ₂ , and D ₃ , respectively, D ₁ , D ₂ , D
₃ is given as follows.

[0125]

(Equation 15) That is,

(Equation 16) It is. Also,

[0126]

[Equation 17] So that

(Equation 18) Becomes Also,

[0127]

[Equation 19]And θ₁Is θ_CEqual to L₁₂, L₁₃, Θ_CIs
Since it is known, θ₂, Θ ₃Is given by a function of L. You
That is,

[0128]

(Equation 20) Becomes Therefore,

[0129]

(Equation 21) Becomes D ₁ , D ₂ , D _3, θ _C , P ₁ , P ₂ , P
_{Since 3} is a measured value or a known value, from the above two equations,
The unknowns θ ₀ and L can be obtained. Also, if θ ₀ is obtained, R _obj can be calculated from any of D ₁ , D ₂ , and D _3.
Can be requested. From the above, the reflected light from the object 220 of the light irradiated from the three irradiation positions that are optically on a straight line is imaged at the same optical position as any one of the three irradiation positions. The distance information to the object, the inclination information of the surface of the object, and the reflectance information of the surface of the object can be calculated. In the present example, it has been substantially the same the position of the irradiation position 212 and the camera 210, theta _1, theta _2, since theta ₃ either with a theta _C may be the same, the irradiation position 214 or the irradiation position 216 Obviously, the position of the camera 210 may be substantially the same as any one of the above.

In the information acquisition method described with reference to FIG. 27, the camera 210 and the irradiation position (212, 214, 21)
Since the optical axis with any of 6) is optically coaxial, when the camera 210 captures an image of the object 220 irradiated from the irradiation position (212, 214, 216), the area that becomes a shadow and cannot be captured is reduced. be able to. Also, camera 2
By making the position of 10 and one of the irradiation positions (212, 214, 216) substantially the same, there is no need to check the distance between the camera 210 and the irradiation position in advance, and the distance between the camera 210 and the irradiation position is not required. An error in distance measurement can be eliminated.

In the information acquisition method described with reference to FIG. 26 or FIG. 27, the object 220 is irradiated with light from three irradiation positions. In other examples, the object 220 is irradiated from three or more irradiation positions. May be irradiated with light. Further, based on the intensity of the reflected light, distance information to the object 220, the object 2
It is preferable to calculate the inclination information of the surface of the object 20 and the reflectance information of the surface of the object 220. Further, the distance information, the inclination information, and the reflectance information may be calculated for each pixel of the camera 210, and the distribution of the distance information, the inclination information, and the reflectance information of the object 220 may be calculated.

In the information acquisition method described with reference to FIG. 26 or FIG. 27, the light emitted from the three irradiation positions has different wavelengths as main wavelength components, and the light is substantially different at each irradiation position. Object 220 simultaneously
May be irradiated with light. At this time, it is preferable that the camera 210 has a wavelength selection unit that selectively transmits light of different wavelengths, and captures the reflected light from the object 220 with the light irradiated from each irradiation position. By simultaneously irradiating the object 220 with light having different wavelengths as main wavelength components, it is possible to accurately acquire distance information, tilt information, and reflectance information even for a moving object. However, since the reflectivity of the surface of the object 220 varies depending on the wavelength of light, an error occurs in the measurement result. In order to reduce the error, it is necessary to correct the difference in reflectance depending on the wavelength.

FIG. 28 is an explanatory diagram of an information acquisition method in which the difference in the reflectance of the surface of the object 220 due to the wavelength is corrected.
As an example, as shown in FIG.
Λ _a , the wavelengths of the light radiated from the irradiation position 214 are λ _b and λ _C , and the wavelengths of the light radiated from the irradiation position 216 are λ _d and λ _e . Further, the intensity of light irradiated from the irradiation position 212 is P ₁ , and the irradiation position 21 is
The intensity of light emitted from the 4 _{P 2,} the intensity of light emitted from the irradiation position 216 and _{P 3,} the intensity of the reflected light of each respective wavelength _{W a, W b, W c} , W d, W _e . As shown in FIG. 28B, the intensity P ₂ from the irradiation position 214
Based on the intensities W _b and W _c of the reflected light of the wavelengths λ _b and λ _c illuminated with the light, the intensity of the reflected light when the light of the wavelength λ _a is illuminated from the irradiation position 214 at the intensity P ₂ to calculate the W _f. In this example, the wavelength lambda _b, but calculates the intensity W _f based on the lambda _c, in another example, it may calculate the intensity W _f based further on the reflected light of a number of wavelength components . Similarly, as shown in FIG.
16 irradiated at an intensity _{P 3} from, based on the wavelength lambda _d, the intensity of the reflected light of the light λ _{e W} d, _{W e,} if the light of wavelength lambda _a in intensity _{P 3} from the irradiation position 214 is irradiated calculating the intensity W _g of the reflected light. Calculated magnitude W _b, W _c, and the distance information to the object 220 based on the detected intensity W _a, the inclination information of the surface of the object 220 and calculates the reflectivity information of the surface of the object 220. According to the above-described information acquisition method, since light is irradiated simultaneously at three irradiation positions, it is possible to accurately acquire distance information, inclination information, and reflectance information even for a moving object. .

FIG. 29 is a flowchart showing an example of the information acquisition method according to the present invention. The information acquisition method in the present embodiment includes an irradiation step (S200) and an imaging step (S20).
2) and a calculation step (S204).

In the irradiation step (S200), as described with reference to FIG. 26, the object 220 is irradiated with light from three or more optically different irradiation positions. Imaging stage (S20
2) imaging the reflected light from the object by the light irradiated from three or more irradiation positions. Calculation step (S20
4) Calculate distance information to the object 220, inclination information of the surface of the object 220, and reflectance information of the surface of the object 220, based on the reflected light captured in the imaging step.

In the irradiating step (S200), the object 220 is irradiated with light from three optically irradiating positions, similarly to the information acquiring means described with reference to FIG. In (S202), the reflected light of light from the three or more irradiation positions may be imaged at the same optical position as any one of the three or more irradiation positions.

In the calculation step (S204), as described with reference to FIG. 26 or FIG. 2, the distance information to the object 220, the inclination information of the surface of the object 220, the information May be calculated. In the irradiation step (S200), the object 220 may be irradiated with light from three irradiation positions that optically form a triangle.

In the irradiation step (S200), similarly to the information acquisition method described with reference to FIG. 28, light of a different wavelength is irradiated on the object 220 at each irradiation position. The object 220 may be irradiated with light substantially simultaneously. In this case, the imaging stage (S
202) preferably has wavelength selecting means for selectively transmitting each of light of different wavelengths, and preferably captures reflected light from the object 220 due to light at each irradiation position.

As described above, according to the image pickup apparatus and the distance measuring method of the present invention, light having different wavelength characteristics is simultaneously radiated to the subject from different radiation positions, and reflected light from the subject is adjusted to the wavelength characteristics. By optically separating the subject and measuring the intensity, the depth distance of the subject can be easily calculated.

As described above, the present invention has been described using the embodiments. However, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be added to the above embodiment. It should be noted that such modified or improved embodiments may be included in the technical scope of the present invention.
It is clear from the description of the claims.

As an example of such a modification, in the above description, reflected light is imaged as an example of emitted light obtained from a subject irradiated with light, and the distance to the subject is obtained from the difference in the intensity of the reflected light. When the subject is a transparent or translucent object that transmits light, the depth distance of the subject can be obtained from the difference in the intensity of the transmitted light that has passed through the subject.

[0142]

As is apparent from the above description, according to the present invention, the depth of a subject can be easily measured by photographing the outgoing light obtained from the subject irradiated with the light.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a configuration diagram of an image capturing apparatus 200 according to the first embodiment.

FIG. 3 shows an irradiation unit 100 and an imaging unit 12 according to the first embodiment.
FIG.

FIG. 4 is a configuration diagram of a processing unit 60 according to the first embodiment.

FIG. 5 is an explanatory diagram of a depth distance calculation method by a light intensity detection unit 64 and a depth calculation unit 66.

FIG. 6 is an explanatory diagram of a method of obtaining a temporary reflected light intensity by interpolation.

FIG. 7 is an explanatory diagram of a method of obtaining temporary reflected light intensity by extrapolation.

FIG. 8 is a diagram for explaining a method of obtaining a provisional reflected light intensity from respective reflected lights from the light sources 10A and 10B.

FIG. 9 is a diagram for explaining a method of obtaining a provisional reflected light intensity when irradiation is performed using a band-pass filter that transmits only light of a long wavelength or a short wavelength.

FIG. 10 is a diagram showing the surface reflectance of three types of objects.

FIG. 11 is a diagram illustrating an optical filter provided in the light receiving unit 40 and transmitting a specific wavelength component.

FIG. 12 is a flowchart of a distance measuring method according to the first embodiment.

FIG. 13 is a flowchart of a depth distance calculation process S110.

FIG. 14 is a flowchart of a modified example of the depth distance calculation processing S110.

FIG. 15 shows an irradiation unit 100 and an imaging unit 1 according to the second embodiment.
FIG.

FIG. 16 is a configuration diagram of a processing unit 60 according to the second embodiment.

FIG. 17 is an explanatory diagram of a depth distance calculation method by the light intensity detection unit 64 and the depth calculation unit 66.

FIG. 18 is a diagram illustrating a method of separating reflected light using a band cut filter.

FIG. 19 is a flowchart of a distance measuring method according to the second embodiment.

FIG. 20 shows an irradiation unit 100 and an imaging unit 1 according to the third embodiment.
FIG.

FIG. 21 is a configuration diagram of a processing unit 60 according to the third embodiment.

FIG. 22 shows an irradiation unit 100 and an imaging unit 1 according to the fourth embodiment.
FIG.

FIG. 23 is an explanatory view of a principle according to the fifth embodiment of the present invention.

FIG. 24 shows the distance L to the measured section 122, the measured section 1;
FIG. 14 is an explanatory diagram of an example of a method for calculating a plane inclination θ0 of FIG.

FIG. 25 shows an example of a result of calculating the plane inclination θ0 and the plane inclination θ0 ′ at a predetermined distance interval within a predetermined range with respect to a temporary distance La.

FIG. 26 is a diagram illustrating the principle of an information acquisition method according to Embodiment 5 of the present invention.

FIG. 27 shows a case where irradiation positions are arranged on a straight line.
It is an explanatory view of an information acquisition method.

FIG. 28 is an explanatory diagram of an information acquisition method in which a difference in reflectance of the surface of the object 20 due to a wavelength is corrected.

FIG. 29 is a flowchart illustrating an example of an information acquisition method.

[Explanation of symbols]

 Reference Signs List 10 light source 12 optical filter 20 optical lens 30 spectral unit 40 light receiving unit 60 processing unit 62 image memory 64 light intensity detecting unit 66 depth calculating unit 67 image correcting unit 68 recording unit 80 control unit 100 irradiating unit 120 image capturing unit 200 image capturing device

Claims (29)

[Claims] 1. An image capturing apparatus for acquiring information relating to the depth of a subject, comprising: a first irradiation light having a first wavelength as a main wavelength component; and a second and a third irradiation light different from the first wavelength. An irradiating unit that irradiates the subject with second irradiation light having a wavelength of 3 as a main wavelength component from an optically different radiation position; and the first and second irradiation lights are irradiated by the irradiation unit. And a depth calculation unit that calculates a depth distance to the subject based on light emitted from the subject. 2. The apparatus according to claim 1, wherein the irradiating unit simultaneously irradiates the first and second irradiation lights. 3. An optical imaging section for imaging the emitted light obtained from the subject irradiated with the first and second irradiation lights by the irradiation section; and A first emission light having a first wavelength, a second emission light having the second wavelength, and a light splitting unit for optically separating the third emission light having the third wavelength; A light receiving unit that receives the first, second, and third outgoing light beams that are separated by the light splitting unit and formed by the optical image forming unit; and the first, second, and third light receiving units that receive the light receiving unit. And a light intensity detection unit that detects the intensity of the outgoing light. The depth calculating unit calculates a depth distance to the subject using the intensities of the first, second, and third outgoing lights. The image pickup apparatus according to claim 2, wherein: 4. The light-receiving unit has three solid-state imaging devices, and the spectroscopy unit separates the first, second, and third outgoing lights using an optical path splitting unit, and separates each of the first, second, and third outgoing lights. The image pickup apparatus according to claim 3, wherein any one of the three solid-state image pickup devices receives light. 5. The light receiving section has a solid-state image sensor, and the light splitting section has a first optical filter transmitting the first wavelength light and a second optical filter transmitting the second wavelength light. A second optical filter and a third optical filter that transmits light of the third wavelength, wherein the first, second, and third optical filters are alternately arranged on a light receiving surface of the solid-state imaging device. The image pickup apparatus according to claim 3, wherein: 6. The irradiating unit includes a first optical filter that transmits light in a wavelength region shorter than a predetermined first boundary wavelength, and a second optical filter that transmits light in a wavelength region longer than a predetermined second boundary wavelength. And irradiating the subject with first irradiation light transmitted through the first optical filter and second irradiation light transmitted through the second optical filter from optically different radiation positions. The spectroscopic unit includes a first optical filter that transmits light having a first wavelength shorter than a shorter one of the first and second boundary wavelengths, and a first optical filter that transmits light having a longer first and second boundary wavelength. Having second and third optical filters that respectively transmit light of second and third wavelengths longer than that of the first optical filter, by transmitting the emitted light obtained from the subject to the first optical filter, The first emission having the first wavelength And the transmitted light is transmitted through the second and third optical filters, respectively, so that the second emitted light having the second wavelength and the third output light having the third wavelength are separated. The image pickup device according to claim 3, wherein the light is separated. 7. The method according to claim 7, wherein the depth calculation unit is configured to control the second and third depths.
The image capturing apparatus according to claim 3, wherein the depth distance to the subject is calculated using a value based on the intensity of the outgoing light and the intensity of the first outgoing light. 8. The method according to claim 8, wherein the depth calculation unit is configured to determine the second and third depths.
Based on the intensity of the outgoing light, the intensity of the temporary outgoing light from the subject is obtained when assuming that the light having the first wavelength is emitted from the emission position of the second irradiation light, The image capturing apparatus according to claim 7, wherein the depth distance to the subject is calculated using the intensity of the outgoing light and the intensity of the temporary outgoing light. 9. The depth calculation unit calculates the depth distance to the subject using an average intensity of the second emission light and the third emission light and an intensity of the first emission light. The image capturing apparatus according to claim 7, wherein the calculation is performed. 10. The irradiation unit includes a first irradiation light having a first wavelength as a main wavelength component, a second wavelength shorter than the first wavelength, and a third light longer than the first wavelength. And the second irradiation light having the main wavelength component is irradiated to the subject from an optically different radiation position. From the subject irradiated with the first and second irradiation lights by the irradiation unit An optical imaging section for imaging the outgoing light of the first, the first outgoing light having the first wavelength from the outgoing light obtained from the subject, and the second outgoing light having the second and third wavelengths A light separating unit that optically separates outgoing light, a light receiving unit that receives the first outgoing light and the second outgoing light that is separated by the light separating unit and is imaged by the optical imaging unit, A light intensity detector for detecting the intensity of the first and second emitted light received by the light receiver. Comprising the al, the depth calculation unit, the first and with the intensity of the second outgoing light, the image capturing apparatus according to claim 2, characterized in that to calculate the depth distance to the subject. 11. The light-receiving unit has two solid-state imaging devices, and the light-splitting unit uses a light-path dividing unit to perform the first light-receiving operation.
The image pickup apparatus according to claim 10, wherein an optical path of the outgoing light and the second outgoing light is optically branched, and each light is received by any one of the two solid-state imaging devices. 12. The light receiving unit includes a solid-state image sensor.
The spectroscopy unit includes a first optical filter that transmits the light of the first wavelength, and a second optical filter that transmits light of the second and third wavelengths. The image pickup apparatus according to claim 10, wherein filters and the second optical filters are alternately arranged on a light receiving surface of the solid-state image pickup device. 13. The depth calculation unit calculates the depth distance to the subject based on a ratio between the intensity of the first emission light and a half value of the intensity of the second emission light. The image pickup device according to claim 10, wherein: 14. An optical axis when the irradiating unit irradiates the first and second irradiation light to the subject, and an optical axis when the imaging unit captures the outgoing light from the subject. Are substantially the same.
13. The image pickup device according to 1 or 12. 15. The light intensity detection unit detects the intensity of the first and second outgoing light in each pixel of the image of the subject captured by the light receiving unit, and the depth calculation unit includes:
The apparatus according to claim 14, wherein a depth distribution of the subject is calculated by obtaining the depths up to the area of the subject corresponding to each of the pixels. 16. The method according to claim 16, wherein the first and second irradiation lights are light in an infrared region, and the spectroscopic unit further includes a unit for optically separating visible light from the outgoing light obtained from the subject. The light receiving unit further includes a solid-state imaging device for visible light that receives the visible light that is optically separated by the light splitting unit and is imaged by the optical imaging unit, and that the depth calculation unit calculates the subject. 16. The imaging apparatus according to claim 15, further comprising a recording unit that records an image of the subject captured by the visible light solid-state imaging device together with the depth distribution. 17. At least one of an intensity of the outgoing light from the subject detected by the light intensity detection unit and the depth distance to the subject calculated by the depth calculation unit.
The first and second irradiations performed by the irradiation unit based on
13. The control device according to claim 4, further comprising a control unit configured to control at least one of a light emission time, an intensity, a radiation position, and an exposure time of the light receiving unit. The image pickup apparatus according to any one of the preceding claims. 18. A distance measuring method for acquiring information on a depth of a subject, comprising: a first irradiation light having a first wavelength as a main wavelength component; and a second and a third irradiation light different from the first wavelength. An irradiation step of simultaneously irradiating the object with second irradiation light having a wavelength of 3 as a main wavelength component from an optically different radiation position; and the object irradiated with the first and second irradiation light. From the output light obtained from the first light, the first light having the first wavelength, the second light having the second wavelength, and the third light having the third wavelength. Spectrally separating the light, imaging the separated first, second and third emitted light, and detecting the intensity of the imaged first, second and third emitted light. Light intensity detecting step; and using the first, second and third outgoing light intensities, Distance measuring method characterized by comprising a depth calculation step of calculating a depth distance to the object. 19. The depth calculating step calculates the depth distance to the subject using a value based on the intensity of the second and third outgoing light and the intensity of the first outgoing light. 19. The distance measuring method according to claim 18, wherein: 20. The depth calculating step, based on the intensities of the second and third outgoing lights, assumes that light having the first wavelength is irradiated from a radiation position of the second irradiation light. In the case, the intensity of the provisional emission light from the subject is obtained,
20. The distance measuring method according to claim 19, wherein the depth distance to the subject is calculated based on a ratio between the intensity of the first emission light and the intensity of the temporary emission light. 21. The depth calculation step, wherein the depth to the subject is determined based on a ratio of an average intensity of the second emission light and the third emission light to an intensity of the first emission light. 20. The image pickup device according to claim 19, wherein the distance is calculated. 22. A distance measuring method for acquiring information on a depth of a subject, comprising: a first irradiation light having a first wavelength as a main wavelength component; a second wavelength shorter than the first wavelength; And irradiating the subject simultaneously with second irradiation light having a third wavelength longer than the first wavelength as a main wavelength component from optically different radiation positions; and the first and second irradiation steps. Optically converting the first outgoing light having the first wavelength and the second outgoing light having the second and third wavelengths from the outgoing light obtained from the object irradiated with the A separating step; an imaging step of imaging the separated first and second output lights; and detecting the intensities of the received first and second output lights, respectively. Detecting the light intensity, and the intensity of the first emitted light and the second Distance measuring method of using the intensity of the emitted light, characterized in that a depth calculation step of calculating a depth distance to the subject. 23. The depth calculating step includes calculating the depth distance to the subject based on a ratio of an intensity of the first emission light to a half of an intensity of the second emission light. The distance measuring method according to claim 22, characterized in that: 24. An information acquiring method for acquiring distance information to an object, comprising: irradiating the object with light from two optically different irradiation positions; and irradiating from the two irradiation positions. An imaging step of respectively imaging reflected light from the object by the light, and an angle calculation step of calculating any one of angles at which the object is irradiated with light from two optically different irradiation positions, A calculating step of calculating distance information to the measured portion based on reflected light from the measured portion of the object captured at the image capturing position and reflected light from the vicinity of the measured portion. An information acquisition method characterized in that: 25. The calculating step, wherein based on the intensity of the reflected light from the measured part and the intensity of the reflected light from the vicinity of the measured part imaged in the imaging step, The information acquisition method according to claim 24, wherein distance information and surface inclination information of the measured part are calculated. 26. The method according to claim 26, further comprising: calculating temporary distance information to the measured section based on an intensity of reflected light from the measured section and an incident angle of the reflected light. First to calculate the surface inclination information of the part
A second calculating step of calculating temporary distance information near the measured part based on the temporary distance information to the measured part and the surface inclination information of the measured part, Based on the provisional distance information in the vicinity of the measured portion, the intensity of reflected light from the vicinity of the measured portion, and the incident angle of the reflected light, surface inclination information in the vicinity of the measured portion is calculated. And a fourth calculating step of calculating an error between the surface tilt information of the measured part and the surface tilt information near the measured part, wherein the error is within a predetermined range. 26. The information acquisition method according to claim 25, wherein when the value is within, the provisional distance information of the measured part is output as true distance information of the measured part. 27. An information acquisition method for acquiring distance information to an object, comprising: irradiating the object with light from three or more optically different irradiation positions; and irradiating the three or more points. And an imaging step of imaging reflected light from the object by the light emitted from the light source, and a calculating step of calculating the distance information based on the reflected light imaged in the imaging step. Information acquisition method to do. 28. The imaging step includes imaging the reflected light of the light from the three or more irradiation positions at the same optical position as any one of the three or more irradiation positions. The method for acquiring information according to claim 27, characterized in that: 29. The information acquisition method according to claim 28, wherein in the irradiating step, the object is irradiated with the light from three irradiation positions that are optically on a straight line.