JP7565780B2 - Holographic imaging device and image reconstruction device - Google Patents

Description

The present invention relates to a holographic imaging device and an image reconstruction device, and more particularly to a holographic imaging device and an image reconstruction device associated with an imaging technique based on incoherent holography.

Holography is a technology that acquires and reconstructs three-dimensional information about an object. In particular, incoherent holography technology has the advantage that it can capture holograms of objects using light sources with low spatial coherence, such as sunlight, general indoor lighting, and fluorescent light, making it possible to achieve three-dimensional imaging in natural light environments. For this reason, incoherent holography does not require consideration of eye-safety issues that must be considered when introducing three-dimensional imaging methods that use laser light sources, making it a promising candidate for realizing a passive three-dimensional imaging method.

In general, to obtain three-dimensional information using holography, it is necessary to detect phase information or complex amplitude information (a combination of phase information and amplitude information). The phase shift method is often used to detect this information (for example, Patent Document 1). The phase shift method is a technology that detects phase information or complex amplitude information by capturing multiple holograms with different amounts of phase shift of light (i.e., the positions of light and dark stripes) and analyzing them.

When capturing the multiple holograms required for the phase shifting method, a time-division phase shifting method (e.g., Patent Document 1, Non-Patent Document 1) is often used, in which a spatial light modulator such as a piezoelectric element or liquid crystal is used to sequentially shift the phase of light and capture a hologram each time. However, this method requires multiple captures to detect phase information or complex amplitude information, making it difficult to apply to video capture.

Therefore, in order to apply this to video imaging, a spatial division phase shift method has been proposed that simultaneously captures multiple holograms required for the phase shift method (Patent Document 2). In Patent Document 2, light reflected, transmitted, or scattered from an object to be imaged is modulated by a light modulation element having a checkered structure. This light modulation element imparts a checkered phase distribution to the light. The light that has been imparted with the checkered phase distribution is divided into four, each of which is imparted with a different phase shift amount, and further propagates in four directions corresponding to the period of the checkered phase distribution. This allows the four holograms required for the phase shift method to be formed simultaneously on the image sensor surface, and the four holograms can be imaged simultaneously in a single image capture.

Patent No. 6416270 JP 2019-144520 A

Myung K. Kim, “Full color natural light holographic camera,” Optics Express, vol. 21, (2013), pp. 9636-9642 Joseph W. Goodman, “Introduction to Fourier Optics,” -3rd edition, p. 214

However, when using a checkered light modulation element, the incident light is divided into four, so even if the light can be ideally modulated, the light utilization efficiency of the incident light in each hologram is 100 x 1/4 = 25%. Strictly speaking, when modulating light with a checkered light modulation element, high-frequency components of light are also generated. In the case of the checkered light modulation element of Patent Document 2, the light utilization efficiency of the incident light in each hologram is theoretically about 16%. This (25% - 16%) energy corresponds to high-frequency components and does not contribute to the four holograms. In addition, if there is a defect in the checkered light modulation element, unnecessary zero-order light components are generated. In addition, the light utilization efficiency of each hologram is further reduced due to the characteristics of the light modulation element and processing errors.

A decrease in light utilization efficiency causes two main problems. The first is that when capturing images with an image sensor, the signal-to-noise ratio decreases as the amount of light decreases, causing the effects of noise in the image sensor to become more pronounced and degrading the image quality of 3D images. The second is that in order to compensate for the amount of light when capturing holograms, a long exposure time must be set, and when capturing images of subjects or dynamic phenomena that move quickly relative to that exposure time, the 3D images become blurred.

Therefore, in consideration of the above problems, the object of the present invention is to provide an optical modulation element that realizes a spatial division phase shifting method with a higher light utilization efficiency than conventional techniques, and a hologram imaging device and an image reconstruction device that use the spatial division phase shifting method to capture holograms with good image quality and accurately reconstruct three-dimensional information.

In order to solve the above problems, the hologram imaging device of the present invention splits an incoherent light wave into first and second divided lights, imparts different spherical phases to the complex amplitude distributions of these two light waves, and causes the first and second divided lights to interfere with each other to form and image a hologram.The hologram imaging device is characterized in that a two-level phase distribution that changes periodically in a first direction and is constant in a second direction perpendicular to the first direction is imparted to each of the first and second divided lights, and diffracted light generated by this periodic phase distribution is utilized to split the first and second divided lights in three directions each, forming three holograms with different phase shift amounts for each direction on the imaging surface of an imaging element, and acquiring the three holograms simultaneously.

The hologram imaging device further includes two light modulation elements that change periodically in a first direction and impart a constant two-level phase distribution to light in a second direction perpendicular to the first direction, and modulates the first divided light with one light modulation element and modulates the second divided light with the other light modulation element. When modulating, it is preferable to use two light modulation elements in which the phase distribution of the other light modulation element is shifted in the first direction relative to the phase distribution of the one light modulation element, or to modulate the light by arranging the positional relationship between one light modulation element and the first divided light and the positional relationship between the other light modulation element and the second divided light with a shift in the first direction.

The hologram imaging device also preferably includes a means for converting the first split light and the second split light into circularly polarized light with mutually different rotation directions, an optical modulation element that imparts a phase distribution to the light that changes periodically in the first direction and has a constant two-level phase distribution in the second direction perpendicular to the first direction, the optical modulation element splitting each circularly polarized light into three directions, and a three-area split polarizer in which the three areas are composed of polarizers with different transmission axis directions, and transmits the circularly polarized light split into the three directions in correspondence with each area of the three-area split polarizer.

In addition, it is preferable that the means for converting the first split light and the second split light into circularly polarized light having mutually different rotation directions in the hologram imaging device comprises a linear polarizer having a transmission axis in a 45-degree oblique direction that transmits the propagating light, a birefringent lens that modulates the linearly polarized light transmitted through the linear polarizer, and a quarter-wave plate that converts the light transmitted through the birefringent lens into circularly polarized light.

The hologram imaging device preferably includes a linear polarizer having a transmission axis in a 45-degree diagonal direction that transmits propagating light, a birefringent lens that modulates the linearly polarized light transmitted through the linear polarizer, and an optical modulation element that applies a phase distribution of two levels to an electric field vector in one direction of light that changes periodically in a first direction and is orthogonal to the first direction, and applies a phase distribution of two levels to an electric field vector in a direction orthogonal to the electric field vector in the one direction that changes periodically in the first direction and is orthogonal to the first direction, with the phase distribution shifted along the first direction, and the phase distribution of two levels to an electric field vector in a direction orthogonal to the first direction, and the optical modulation element that splits the vertical linearly polarized light and horizontal linearly polarized light transmitted through the birefringent lens into three directions, and a linear polarizer that transmits the vertical linearly polarized light and horizontal linearly polarized light split into three directions by the optical modulation element.

Furthermore, it is preferable that the light modulation element of the hologram imaging device is a reflective or transmissive light modulation element that superimposes a phase distribution of a lens.

To solve the above problems, the image reconstruction device according to the present invention is characterized in that it takes as input image data of a hologram captured by the hologram capture device, extracts three individual holograms from the image data, obtains a complex amplitude distribution using a phase shift method, and reconstructs an image using a propagation calculation.

It is also desirable that the image reconstruction device obtains the complex amplitude distribution by a phase shift method in which the intensity of the hologram is corrected using the ratio of the light intensity or diffraction efficiency of the zeroth order light and the ±1st order light of the light modulation element, which has been obtained in advance.

According to the hologram imaging device and image reconstruction device of the present invention, light can be utilized more efficiently than in the prior art, a hologram with good image quality can be captured, and three-dimensional information can be reconstructed with high accuracy.

1 is a conceptual diagram of a hologram imaging device and an image reconstructing device according to the present invention. 3 is an example of a phase distribution imparted to light by a light modulation element. 1A and 1B are a plan view and a cross-sectional view showing an example of the structure of a light modulation element. 1 is an example of a hologram imaging device according to a first embodiment. 13 is an example of a hologram imaging device according to a second embodiment. 13 is an example of a hologram imaging device according to a third embodiment. 1A and 1B are diagrams illustrating examples of transmission axes of a three-area split polarizer. 13 is an example of a hologram imaging device according to a fourth embodiment. 1A and 1B are diagrams illustrating the optical characteristics of a light modulation element (polarization diffraction grating) according to the present invention FIG. 2 is a diagram showing an object to be imaged; This is a hologram (intensity image) captured by an imaging element. This is the result of reconstructing the target object. 13 is an example of a hologram (intensity image) captured when a phase shift occurs. 13 shows the reconstruction results of the target object with and without correction of the intensity image.

The present invention will now be described with reference to the drawings. FIG. 1 is a conceptual diagram of a holographic imaging device and an image reconstructing device according to the present invention. The present invention constructs an interferometer using incoherent light, which serves as a holographic imaging device.

The holographic imaging device of the present invention comprises an incoherent light splitting means 2, spherical phase modulation elements 3 and 4, light modulation elements 5 and 6, a light combining means 7, and an imaging element 8, and further comprises a calculation device 10 for constituting an image reconstruction device.

The process of imaging a hologram will be outlined below. An object (image target object) 1 is illuminated with light, and the transmitted light, reflected light, or fluorescent light or spontaneous emission is split into two by splitting means 2. As a means for splitting light, for example, a beam splitter or orthogonal polarization can be used. The two lights after splitting are called the first split light and the second split light, respectively.

The first and second divided lights are modulated using spherical phase modulation elements 3 and 4, which have different radii of curvature of the phase distribution. Examples of the spherical phase modulation elements 3 and 4 include general lenses, birefringent lenses, Fresnel diffractive lenses realized by spatial light modulators, polarized diffractive lenses, and polarized direct flat lenses.

Then, the light modulation elements 5 and 6, which have the function of imparting a phase distribution to light, impart a predetermined phase distribution to the first and second divided lights, respectively. As will be described in detail later, the light modulation elements 5 and 6 have the function of imparting to the incident light a phase distribution that changes periodically in a first direction and a constant two-level phase distribution in a second direction perpendicular to the first direction. Here, the phase distribution of the light modulation element 5 is referred to as "phase distribution 1," and the phase distribution of the light modulation element 6 is referred to as "phase distribution 2." Note that the phase distributions imparted to each divided light may be reversed.

After being modulated by the light modulation elements 5 and 6, the first and second divided lights are each split into three. The three split lights originate from the diffraction phenomenon caused by the periodic phase change imparted by the light modulation elements 5 and 6, and correspond to the diffracted lights of the zeroth order light component and the ±1st order light components of the phase distribution that changes in two tones. These lights are then combined by the combining means 7. For example, a beam splitter or the like can be used as a means for combining the lights.

The first and second divided lights are then made to interfere with each other on the imaging surface of the imaging element 8. Three holograms are formed on the imaging surface. These are then captured all at once (simultaneously) by the imaging element 8 to obtain a captured image 9 (i.e., three holograms). As a hologram imaging device, for example, this captured image 9 can be recorded and stored on a recording medium. Furthermore, this captured image 9 can be transferred to a calculation device 10, which is an image reconstruction device, and a three-dimensional image of the object to be imaged can be reconstructed by performing signal processing.

In FIG. 1, three holograms are captured with one imaging element, but three imaging elements may be arranged side by side and each imaging element may capture one hologram. Even when multiple imaging elements are used, it is desirable to simultaneously obtain three holograms. The details of the signal processing for image reconstruction will be described later.

In the present invention, the light used to form a hologram is split into three directions, and therefore the light utilization efficiency is higher than that of the method of Patent Document 2, which splits the light into four directions using a checkerboard grid. Theoretically, the light utilization efficiency of the incident light in each hologram can be improved by approximately 29%. Furthermore, the optical system is relatively small and easy to construct.

The light modulation elements 5 and 6 used in the optical system of the present invention will now be described. Figure 2 shows an example of the phase distribution imparted to light by the light modulation elements of the hologram imaging device, with Figure 2(a) showing the light modulation element 5 and Figure 2(b) showing the light modulation element 6. The light modulation elements 5 and 6 impart a phase distribution that changes periodically in one axial direction (first direction, here the x direction) to the first and second divided light that form a hologram, and impart a constant phase distribution in a direction perpendicular to the one axial direction (second direction, here the y direction).

The light modulation elements 5 and 6 impart a phase +Δφ to light in the regions 51 and 61 with a width of d ₁ , and impart a phase -Δφ to light in the regions 52 and 62 with a width of d ₂ , and the entire light modulation elements 5 and 6 have a two-level phase distribution that changes in one axis direction (x direction) with a period of d ₁ +d _2. For example, the phase distribution of FIG. 2(a) is imparted to the first divided light, and the phase distribution of FIG. 2(b) is imparted to the second divided light. Note that the phase distributions of FIG. 2(a) and FIG. 2(b) are essentially the same, but FIG. 2(b) is shifted by s from FIG. 2(a) along the direction in which the phase distribution changes (x direction). The effect and necessity of shifting by s will be described later.

In this way, the light modulation elements 5 and 6 used in the optical system of the present invention have the function of periodically changing the incident light in one axial direction (first direction) and imparting a constant two-level phase distribution in the direction perpendicular to this (second direction). The first and second divided lights modulated by the light modulation elements 5 and 6 are split into three lights of the same quality as the respective divided lights. By causing these lights to interfere together and imaging them, it is possible to obtain the three holograms required for analyzing the complex amplitude distribution with a relatively high light utilization efficiency.

Figure 3 shows a plan view and a cross-sectional view of an example of the structure of the light modulation element 5. The light modulation element 5 having the function of imparting the phase distribution of Figure 2 (a) can be, for example, a reflective or transmissive diffraction grating having a periodic uneven structure change in one axial direction (first direction, here the x-direction). It can also be realized by fabricating an element having a periodic optical structure in the x-axis direction and a uniform optical structure in the y-axis direction.

In Figure 3(a), a periodic structure of concaves and convexes is formed in the x-axis direction, and there is no change in the structure in the y-direction, which is perpendicular to the x-axis direction. This corresponds to a diffraction grating made of a material with an isotropic refractive index. Light in the z-direction that passes through the light modulation element is given different phases due to differences in the distance it passes through a material with a given refractive index. For example, a phase of +Δφ (or -Δφ) is given to the concave portion 53, and a phase of -Δφ (or +Δφ) is given to the convex portion 54.

Figure 3 (b) shows a metasurface formed with nanorods of a size equal to or smaller than the wavelength, where nanorods of different diameters are periodically arranged in the x-axis direction, and nanorods of the same diameter are arranged in the y-direction. For example, a phase of +Δφ (or -Δφ) is imparted to region 55 where nanorods of a small diameter are arranged, and a phase of -Δφ (or +Δφ) is imparted to region 56 where nanorods of a large diameter are arranged. Note that the structure of the metasurface can be various structures other than nanorods, but is not limited to a specific shape, and any structure can be used as long as it can impart the phase distribution of Figure 2. For example, an element composed of nanopillars, nanoblocks, etc. of a size equal to or smaller than the wavelength of the light to be imaged may be used.

Comparing the light modulation elements 5 of FIG. 3(a) and FIG. 3(b), FIG. 3(a) is relatively easier to process than FIG. 3(b), and it is easier to fabricate a large element. However, in FIG. 3(a), when light is incident at a large angle of incidence, the large step of the uneven structure causes the light to be totally reflected or refracted on the side of the convex structure, reducing the light utilization efficiency. In other words, the light is highly dependent on the incident angle. On the other hand, FIG. 3(b) is more difficult to process than FIG. 3(a), but since the unevenness is arranged to be about the wavelength or smaller, total reflection and unnecessary refraction are suppressed, the incidence angle dependence is low, and the light utilization efficiency can be increased.

Next, we will describe in detail the optical system that realizes the holographic imaging device shown in Figure 1.

First Embodiment
Fig. 4 shows an example of a holographic imaging device according to the first embodiment. In the first embodiment, a holographic imaging device is configured based on a Mach-Zehnder interferometer using the transmission type light modulation elements 5 and 6 that impart the phase distribution shown in Fig. 2.

The holographic imaging device in FIG. 4 includes a magnification adjustment lens 11, a wavelength filter 12, a beam splitter 2, a spherical phase modulation element 3, mirrors 13 and 14, light modulation elements 5 and 6, a beam splitter 7, and an imaging element 8.

The magnification adjustment lens 11 refracts the transmitted light, reflected light, or spontaneous light such as fluorescence from the object 1, and adjusts the magnification by enlarging or reducing it so that an image of the desired size is obtained. Note that the use of the magnification adjustment lens 11 is optional.

The wavelength filter 12 is an optical bandpass filter that transmits only light of a specific band of incoherent light waves, improving temporal coherence. The wavelength width of the transmitted light is, for example, 1 nm to 100 nm. The use of the wavelength filter 12 is optional. The wavelength filter 12 is used when it is desired to improve the spatial resolution of a stereoscopic image.

The beam splitter 2 functions as a light splitting means, and splits the light transmitted through the wavelength filter 12 into a first split light and a second split light. The first split light travels toward the spherical phase modulation element 3, and the second split light travels toward the mirror 14.

The spherical phase modulation element 3 is, for example, a general lens, and imparts a predetermined spherical phase to the first divided light. The spherical phase modulation element 3 may be a birefringent lens, a Fresnel diffraction lens, a polarized diffraction lens, a polarized direct flat lens, or the like. In FIG. 4, a lens is used as the spherical phase modulation element 3 only in one optical path (first divided light) after splitting by the beam splitter 2, but the optical path without a lens is equivalent to performing modulation with an infinitesimal curvature, and the first divided light and the second divided light are given phase distributions with different radii of curvature, so there is no problem in forming a hologram. A spherical phase modulation element may be provided only in the other optical path (second divided light), or different spherical phase modulation elements 3 and 4 may be provided in both optical paths.

Mirror 13 reflects the first split light that has passed through spherical phase modulation element 3 at a specified angle, directing it toward light modulation element 5. Mirror 14 also reflects the second split light from beam splitter 2 at a specified angle, directing it toward light modulation element 6.

The light modulation element 5 is a transmissive element, and gives the first divided light that passes through it a phase distribution as shown in FIG. 2(a). At this time, the first divided light is divided by diffraction into three lights corresponding to the 0th-order light component and ±1st-order light components of the phase distribution, and these lights head toward the beam splitter 7. Similarly, the light modulation element 6 is also a transmissive element, and gives the second divided light that passes through it a phase distribution as shown in FIG. 2(b). At this time, the second divided light is divided by diffraction into three lights corresponding to the 0th-order light component and ±1st-order light components of the phase distribution, and these lights head toward the beam splitter 7. The light modulation elements 5 and 6 have the function of dividing light into three by diffraction, and the function of giving a phase shift to the first divided light and the second divided light, as described below.

The beam splitter 7 functions as a light combining means. That is, it transmits the first divided light in the same direction as it was incident, and reflects the second divided light, changing its direction by 90 degrees. As a result, the first divided light and the second divided light are combined, and both travel toward the image sensor 8.

Three holograms are formed on the imaging surface of the imaging element 8 (see the image of the captured image 9 in Figure 1), and these are captured by the imaging element 8.

Three holograms are extracted from this captured image 9. When the phase shift amount imparted to the central hologram is taken as a reference, the phase shift amounts Δφ _L and Δφ _R imparted to the left and right holograms are expressed by the following equations (1) and (2).

If Δφ _L and Δφ _R ≠ 0, a complex amplitude distribution reflecting three-dimensional information of the object 1 to be imaged can be obtained by applying the algorithm of the three-step phase shift method. As can be seen from the above formula, Δφ _L and Δφ _R can be adjusted by the phase distribution shift s of the light modulation elements 5 and 6. In order to apply the phase shift method with high accuracy, it is desirable to set s = ± (d ₁ + d ₂ ) / 2 or s = ± 2 (d ₁ + d ₂ ) / 3. That is, it is desirable for s to be half or 2/3 the length of the period in one axis direction. Also, when d ₁ = d ₂ and d ₁ and d ₂ are sufficiently large with respect to the wavelength of light, according to the theoretical formula of diffraction efficiency described in Non-Patent Document 2, if Δφ is set to 0.3196πrad, three holograms can theoretically be formed with the same intensity.

In this embodiment, light modulation elements 5 and 6 (light modulation elements that periodically change in a first direction and impart a constant two-level phase distribution to light in a second direction perpendicular to the first direction) are used, and modulation is performed using two light modulation elements in which the phase distribution of the other light modulation element 6 is shifted in the first direction from the phase distribution of one light modulation element 5. However, instead of this, two identical light modulation elements may be used and the positional relationship between one light modulation element and the first divided light and the positional relationship between the other light modulation element and the second divided light may be shifted in the first direction (the direction in which the phase changes periodically) to perform modulation.

Second Embodiment
Fig. 5 shows an example of a holographic imaging device according to the second embodiment. In the second embodiment, a holographic imaging device is configured based on a Michelson interferometer using reflection-type light modulation elements 5 and 6 that impart the phase distribution shown in Fig. 2.

The holographic imaging device in FIG. 5 includes a magnification adjustment lens 11, a wavelength filter 12, a beam splitter 2 (7), a spherical phase modulation element 4, light modulation elements 5 and 6, and an imaging element 8. The description of the components common to the holographic imaging device in FIG. 4 will be simplified.

The magnification adjustment lens 11 adjusts the magnification so that the image of the object 1 has the desired size. The wavelength filter 12 transmits only light of a specific band of incoherent light waves. The use of the magnification adjustment lens 11 and the wavelength filter 12 is optional.

The beam splitter 2 (7) functions as a light splitting means 2 and a light combining means 7. That is, the beam splitter 2 (7) splits the light transmitted through the wavelength filter 12 into a first divided light and a second divided light. The first divided light then travels toward the light modulation element 5, and the second divided light travels toward the spherical phase modulation element 4 and the light modulation element 6. The beam splitter 2 (7) also combines the first divided light and the second divided light reflected by the light modulation elements 5 and 6. The combined first divided light and second divided light then travel toward the image sensor 8.

The spherical phase modulation element 4 is, for example, a lens, and imparts a predetermined spherical phase to the second divided light. The spherical phase modulation element may be provided on the first divided light side, or different spherical phase modulation elements 3 and 4 may be provided on both optical paths.

The light modulation elements 5 and 6 are reflective elements, with the light modulation element 5 providing the phase distribution shown in FIG. 2(a) to the reflected first divided light, and the light modulation element 6 providing the phase distribution shown in FIG. 2(b) to the reflected second divided light. The first and second divided lights are each split into three lights by diffraction, and travel through the beam splitter 7 to the image sensor 8. The light modulation elements 5 and 6 have the function of splitting the light into three by diffraction, and the function of providing a phase shift to the first and second divided lights.

Even when using this optical system in which the split light is modulated by the reflective light modulation elements 5 and 6, three holograms can be formed on the imaging surface of the imaging element 8, as in the first embodiment. The concept of the phase shift amount imparted to the three holograms is the same as in the first embodiment.

In this embodiment, reflective light modulation elements 5 and 6 (light modulation elements that periodically change in a first direction and impart a constant two-level phase distribution to reflected light in a second direction perpendicular to the first direction) are used, and modulation is performed using two light modulation elements in which the phase distribution of the other light modulation element 6 is shifted in the first direction from the phase distribution of one light modulation element 5. However, instead of this, two identical light modulation elements may be used and the positional relationship between one light modulation element and the first divided light and the positional relationship between the other light modulation element and the second divided light may be shifted in the first direction (the direction in which the phase changes periodically) to perform modulation.

Third Embodiment
FIG. 6 is an example of a hologram imaging device according to the third embodiment. In the third embodiment, a hologram imaging device is configured using an interferometer with a single optical path. Although the interferometers in FIGS. 4 and 5 are based on a two-path interferometer, the optical path is divided into two, and therefore the device is susceptible to vibrations and air fluctuations. By using the interferometer in FIG. 6, it is possible to image three holograms similar to those in the first and second embodiments using only one optical modulation element that imparts the phase distribution of FIG. 2(a) or FIG. 2(b) in a single common optical path.

The holographic imaging device of FIG. 6 includes a magnification adjustment lens 11, a linear polarizer 15, a birefringent lens 16, a quarter-wave plate 17, a light modulation element 5, a three-area split polarizer 18, and an imaging element 8, and further includes a calculation device 10 for configuring an image reconstruction device. The description of the configuration common to the holographic imaging devices of FIGS. 4 and 5 will be simplified.

The magnification adjustment lens 11 adjusts the magnification so that the image of the object 1 has a desired size. Use of the magnification adjustment lens 11 is optional.

Linear polarizer 15 is a linear polarizer with a transmission axis in a diagonal direction of 45 degrees. When light propagating from object 1 is transmitted through linear polarizer 15, it becomes linearly polarized light at a diagonal angle of 45 degrees. Linearly polarized light at a diagonal angle of 45 degrees can be thought of as a superposition of vertically linearly polarized light and horizontally linearly polarized light, with one of the vertically linearly polarized light and the horizontally linearly polarized light being the first split light, and the other being the second split light.

The birefringent lens 16 is a lens with optical anisotropy, and the refractive index changes depending on the polarization direction. By modulating linearly polarized light at an angle of 45 degrees using the birefringent lens 16, it is possible to impart spherical phases with different radii of curvature to the vertically linearly polarized light and the horizontally linearly polarized light, i.e., the first and second divided lights.

The quarter-wave plate 17 converts linearly polarized light into circularly polarized light or elliptically polarized light. That is, by making the first and second split lights incident on the quarter-wave plate 17 whose fast or slow axis is tilted at 45 degrees, two orthogonal linearly polarized lights (vertical linearly polarized light and horizontal linearly polarized light) are converted into two orthogonal circularly polarized lights (left-handed circularly polarized light and right-handed circularly polarized light).

The light modulation element 5 is, for example, a light modulation element that imparts the phase distribution 1 in FIG. 2(a). The light modulation element 5 modulates two circularly polarized lights. As a result, the left-handed and right-handed circularly polarized lights, which are the first and second divided lights, are each split into three. These lights are made incident on the three-area split polarizer 18. Note that in this embodiment, the light modulation element 5 is used as an element that splits the incident light into three, and does not impart a phase shift to the two divided lights.

Figure 7 is a diagram showing an example of the transmission axis of the three-area divided polarizer 18. As shown in Figure 7, the three-area divided polarizer 18 is composed of three linear polarizers 181, 182, and 183 with different transmission axes. The three divided circularly polarized light is transmitted through the three areas of the three-area divided polarizer 18 in a corresponding manner. Depending on the angle of the transmission axis of the polarizer, the first divided light (left-handed circularly polarized light) and the second divided light (right-handed circularly polarized light) undergo different phase shifts, and are output from the linear polarizers 181 to 183 as two linearly polarized lights with the same polarization state but different phases.

After passing through this three-area split polarizer 18, the first and second split lights, which have become linearly polarized light, interfere with each other, and three holograms with different phase shift amounts are simultaneously formed on the imaging surface of the imaging element 8. Therefore, in this embodiment as well, the three holograms required for the phase shift method can be obtained simultaneously with a single image capture by the imaging element 8. Image data of the captured hologram (captured image 9) is transmitted to the computing device 10.

Unlike the optical system using two light modulation elements shown in FIG. 4 and FIG. 5, in the optical system of this embodiment, the amount of phase shift given to the hologram is determined as follows. When the angle of the transmission axis of the central polarizer 182 of the three-area split polarizer 18 is used as a reference and the angles of the left and right adjacent polarizers 181 and 183 are rotated by −θ and +θ, respectively, the amount of phase shift Δφ _L =−2θ and Δφ _R =+2θ are given to the holograms formed on the left and right sides, respectively, based on the amount of phase shift given to the central hologram. In order to apply the phase shift method with high accuracy, it is desirable to set θ=±π/4 or θ=±π/3. Also, instead of the birefringent lens 16 and the quarter-wave plate 17, a polarized diffractive lens or a polarized direct flat lens may be used. When the light modulation element 5 is a reflective element, a beam splitter may be placed in front of the light modulation element 5.

In this embodiment, a means is used to make the first and second split lights circularly polarized with different rotation directions, and a single light modulation element that imparts the phase distribution of FIG. 2 is used to modulate the circularly polarized light waves and split them into three directions, and the circularly polarized light split into three directions is transmitted through each region of a three-region split polarizer composed of three polarizers with different transmission axis directions, thereby simultaneously obtaining three holograms. Compared to the two-path interferometer configuration of FIGS. 4 and 5, in this embodiment, an optical system can be constructed with a single optical path, and is highly robust against disturbances and noise such as vibrations and air fluctuations.

Fourth Embodiment
8 shows an example of a holographic imaging device according to the fourth embodiment. The fourth embodiment is another example in which the holographic imaging device is configured by an interferometer with a single optical path.

The holographic imaging device in FIG. 8 includes a magnification adjustment lens 11, a linear polarizer 15, a birefringent lens 16, a light modulation element (polarization diffraction grating) 19, a linear polarizer 20, and an imaging element 8, and further includes a calculation device 10 for configuring an image reconstruction device. The description of the configuration common to the holographic imaging devices in FIGS. 4 to 6 will be simplified.

While the optical system of the third embodiment (Figure 6) required a quarter-wave plate 17, the optical system of this embodiment does not require one, and the number of optical elements can be reduced, suppressing light reflection and absorption and improving light utilization efficiency. In addition, since a normal linear polarizer 20 is placed in front of the image sensor 8, rather than a three-area split polarizer 18, the optical system can be constructed relatively easily.

The magnification adjustment lens 11 adjusts the magnification so that the image of the object 1 has the desired size. The linear polarizer 15 converts the light propagating from the object 1 into linearly polarized light at an angle of 45 degrees. This generates a first split light (vertical linearly polarized light) and a second split light (horizontal linearly polarized light). The birefringent lens 16 then imparts spherical phases with different radii of curvature to the vertical linearly polarized light and the horizontal linearly polarized light, i.e., the first split light and the second split light.

The light modulation element (polarized diffraction grating) 19 imparts both the phase distributions of the light modulation element 5 (FIG. 2(a)) and the light modulation element 6 (FIG. 2(b)) to two orthogonal linearly polarized lights. In other words, it is a light modulation element that imparts a constant two-level phase distribution to the electric field vector of the light in one direction, which changes periodically in the first direction and is orthogonal to the first direction, and imparts a constant two-level phase distribution to the electric field vector in a direction orthogonal to the one direction, which changes periodically in the first direction and is shifted along the first direction, in the second direction orthogonal to the first direction. Furthermore, it splits the first and second split lights (vertical linearly polarized light and horizontal linearly polarized light) that have passed through the birefringent lens 16 into three directions each.

Then, the first and second split lights split in three directions by the linear polarizer 20 become linearly polarized lights with the same polarization direction, and interfere with each other to simultaneously form three holograms with different phase shift amounts on the imaging surface of the imaging element 8. Therefore, the three holograms required for the phase shift method can be obtained simultaneously with a single image capture by the imaging element 8. Image data of the captured hologram (captured image 9) is transmitted to the computing device 10.

9 is a diagram for explaining the optical characteristics of the light modulation element (polarization diffraction grating) 19 of the present invention. As shown in FIG. 9(a), the light modulation element 19 is composed of four types of retarders with different characteristics. Each of the regions A, B, C, and D has a function of a retarder with a different direction of the fast axis or slow axis, and the four types of retarders are repeatedly arranged in a predetermined order in the first direction. When the shift of the phase distribution along the first direction is s and the widths of the two-tone phase distribution are d ₁ and d ₂ , the widths of the regions A, B, C, and D are s, d ₁ -s, s, and d ₂ -s, respectively. In addition, the characteristics of each retarder are expressed by the following formulas (3) to (6) using the Jones matrix.

Here, K is a constant that gives the change in the complex amplitude value due to the refractive index and absorption coefficient of the light modulation element itself, i is an imaginary number, ψ is an arbitrary rotation angle, and Δ is the phase difference between the fast axis and the slow axis, that is, the amount of phase shift between horizontally linearly polarized light and vertically linearly polarized light. It is desirable to set d ₁ , d ₂ , and s in FIG. 9 in the same manner as in FIG. 2 , and in particular, when d ₁ =d ₂ and d ₁ and d ₂ are sufficiently large relative to the wavelength of light, if Δ is set to 0.6391πrad, three holograms can theoretically be formed with the same intensity. In order to apply the phase shift method with high accuracy, it is desirable to set s=±(d ₁ +d ₂ )/2 or s=±2(d ₁ +d ₂ )/3.

Figure 9(b) is a conceptual diagram of an element that realizes the function of Figure 9(a), and the direction of the shaded lines represents the direction of the fast axis or slow axis of the retarder. Figure 9(b) shows the case where ψ = 0, and it can be seen that the fast axis and the slow axis differ by 45 degrees. The light modulation element 19 may be either a reflective type or a transmissive type. This element can be manufactured using liquid crystal polymers, polymer orientation films, optical crystals, structural birefringence, metasurfaces, etc., and there are countless approaches regarding the material and structural shape of the element. Based on the above explanation, as long as the phase distribution corresponding to Figure 2 can be imparted, there are no limitations on the manufacturing technology, method, material, or shape of this element.

The light modulation element (polarization diffraction grating) 19 of this embodiment is a single light modulation element that imparts a phase distribution that changes in one axial direction as shown in Figures 2(a) and 2(b) to each of the first and second divided lights. This single light modulation element 19 is used to modulate light, and the two orthogonal electric field vectors of the modulated light wave are treated as the first and second divided lights, respectively, and these light waves are passed through a linear polarizer and then interfered with each other, thereby forming a hologram imaging device constructed with an interferometer of a common optical path.

Compared to the single-path interferometer configuration of the third embodiment, the configuration of this embodiment does not require the polarization state of the incident light to be circularly polarized, so the number of optical elements can be reduced, light utilization efficiency is high, and holograms can be imaged with higher accuracy.

In the optical system of each embodiment, a lens is arranged to impart a spherical phase to each optical path, but a phase distribution having the same function as a lens may be imparted by a light modulation element by superimposing the phase distribution in FIG. 2 on the phase distribution. Although no specific examples are given for the light modulation element in this case, the structure can be changed according to the phase amount of light to be realized, and it can be manufactured based on the disclosure of this specification and known techniques. In addition, since there are countless structural options, they are not limited to these, and it is sufficient that the two-level phase that changes periodically in one axis direction and the phase of the lens are superimposed. This makes it possible to simplify the configuration of the optical system and improve the light utilization efficiency in all imaging devices.

In addition, in the present invention, there is a risk that the intensity value of the hologram may vary depending on the processing precision of the element, but by applying the phase shift method algorithm described below, a highly accurate image can be obtained. As a result, the present invention can provide a more practical hologram imaging device.

Fifth Embodiment
Next, a process for reconstructing three-dimensional information using the image reconstruction device of the present invention will be described. The image reconstruction device receives as input image data of a hologram captured by the hologram capturing device of the present invention.

Whether any of the optical systems of the first to fourth embodiments is used, the three holograms formed on the imaging surface of the imaging element 8 are essentially the same, and the signal processing performed by the image reconstruction device (arithmetic device) 10 is common to all optical systems. First, three holograms are independently extracted from the captured image 9. At this time, the three holograms need to be extracted while aligning their positions, but by using correlation calculations, the holograms can be extracted with high precision. By applying a three-step phase shift method to these, the complex amplitude distribution o(x,y) of the light propagating from the object is obtained.

At this time, if there is a manufacturing error in the light modulation element, the phase distribution given to the incident light will deviate from the design value, causing a variation in light intensity for each light wave when the incident light is divided into three. If this variation in light intensity occurs, the intensity of the hologram formed on the imaging element 8 will also deviate. In such a situation, if a normal three-step phase shift method algorithm is applied, the measurement error will increase and the image quality of the imaged object will be significantly reduced. In this way, if the manufacturing error of the light modulation element is large and its effect cannot be ignored, it is possible to improve the image quality by measuring and acquiring information on the ratio of the light intensity or diffraction efficiency of the zeroth order light and ±1st order light from the light modulation element before imaging, and then performing signal processing of the phase shift method algorithm that corrects the intensity difference.

Specifically, we use the three-step phase shift method using the following equation (7).

Here, I _C (x,y), I _L (x,y), and I _R (x,y) are the intensity distributions of the holograms formed on the imaging surface of the imaging element 8 at the center, left, and right, respectively, and φ _C , φ _L , and φ _R represent the phase shift amounts given to the respective holograms. α and β are the ratios of the diffraction efficiencies of the −1st order light and the 0th order light of the light modulation element, and the ratios of the diffraction efficiencies of the +1st order light and the 0th order light, respectively. The values of α and β can be measured in advance by measuring the diffraction efficiency. If the manufacturing error is sufficiently small or its influence can be ignored, then α=β=1 is equivalent to the normal three-step phase shift method. o(x,y) is the complex amplitude distribution of the light propagated from the object on the imaging element surface. By using the algorithm based on formula (7), the complex amplitude distribution of the light wave from the object can be measured robustly against the manufacturing error of the light modulation element.

The process of reconstructing the three-dimensional information of an object from o(x,y) is similar to the method described in Patent Document 2, and an image of the object is obtained by applying propagation calculations within a computing device.

(Example and verification of effects)
As an example, a hologram was captured and reconstructed based on the optical system of FIG. 6, and the results are shown below.

The wavelength of the light source was 633 nm, and the optical modulation element had d ₁ =d ₂ =19.5 μm, s=13 μm, and Δφ=0.3196πrad.

In this embodiment, the objects "1" and "2" shown in FIG. 10 were the objects to be imaged. The sizes of the objects "1" and "2" are 117 μm×188.5 μm and 123.5 μm×175.5 μm, respectively. Taking the object "1" as the reference, the center position in the in-plane direction of the object "2" is (x=364 μm, y=-318.5 μm). The distance in the depth (z) direction between the objects "1" and "2" is 20 mm. Note that each screen in FIG. 10 is conceptual, and it is sufficient that the objects "1" and "2" reflect or emit light at the above positions, and the other areas (shown in black) are translucent. These were taken as three-dimensional objects by the holographic imaging device of the present invention. The images in the following figures were created by simulation.

Figure 11 shows a hologram (intensity image) captured by the imaging element 8 of the hologram imaging device. From Figure 11, it can be seen that three holograms with different positions of light and dark stripes are formed. The calculation device 10 extracts three holograms individually from the intensity image of Figure 11. At this time, it is necessary to extract the three holograms accurately so that there is no in-plane shift in the three holograms. To achieve this, a correlation calculation is used. Below, an explanation is given of the case where a matched filter is used, but other correlation calculations such as phase-limited correlation may also be used. Of the three holograms, any one hologram is cut out so that no vignetting occurs, and this is set as the template image t(x,y). Note that t(x,y) is made into an image with the same number of pixels as i(x,y), which will be described later, by zero padding signal processing, if necessary. If the intensity image of Figure 11 is i(x,y), the correlation calculation is given by the following equation (8).

Here, I(u,v) and T(u,v) are the Fourier spectra of i(x,y) and t(x,y), respectively, and * indicates complex conjugate. FT[...] is the Fourier transform operator. Note that in equation (8), even if an inverse Fourier transform is performed, it is substantially equivalent. As a result of the calculation, a clear peak appears in p(x,y) at the center position of each hologram. From the peak position of p(x,y), relative position information of the three holograms is obtained. By referring to this position information, in-plane deviation can be sufficiently suppressed, and the three holograms can be accurately extracted. Furthermore, the hologram extraction position is fine-tuned in pixel units to find the position where the three holograms overlap accurately. The above process only needs to be performed once as a calibration process after constructing the hologram imaging device, and is not necessarily required for each imaging process.

The complex amplitude distribution o(x,y) is obtained by applying the algorithm in equation (7) to the three extracted holograms. Note that the simulation was performed assuming that the light modulation element used in this verification was ideally manufactured, so there is no difference in intensity between the three holograms, and α and β in equation (7) are 1. The light distribution o'(x,y) in any z plane can be reconstructed by applying a propagation calculation to the complex amplitude distribution o(x,y). The propagation calculation is given by the following equation (9).

Here, FT ^-1 [...] is the inverse Fourier transform operator. λ is the wavelength of the light source. When the position z _s of the object to be imaged in the depth direction is known, in order to obtain an image in focus on the object to be imaged, z _r in equation (9) should be set to comply with the following equation (10).

Here, f _d1 and f _d2 are the focal lengths of the lenses which are the spherical phases imparted to the first and second divided beams, respectively, z _h is the distance between the birefringent lens and the image sensor, and z _d is given by the following equation (11).

f ₀ is the focal length of the magnification adjustment lens, and d is the distance between the magnification adjustment lens 11 and the birefringent lens 16. In this embodiment, f ₀ = 250 mm, d = 100 mm, f _d1 = 500 mm, f _d2 → ∞ (corresponding to not adding a spherical phase), and z _h = 300 mm. FIG. 12 shows the result of reconstructing the target object by referring to the information of the arrangement positions z _s = 280, 300 mm of “1” and “2”. It can be seen that the focus positions were successfully adjusted to each of the objects “1” and “2”. When the focus position is adjusted to the object “1”, the image of the object “2” becomes blurred. Similarly, when the focus position is adjusted to the object “2”, the image of the object arranged on a surface different from the focus position becomes blurred.

The above examples are based on the case where the light modulation element is ideally manufactured, but below we will show the effect when a manufacturing error occurs in the light modulation element and the phase Δφ imparted to the light deviates from the desired value, and the improvement results using formula (7).

When the desired Δφ value is 0.3196πrad and a phase shift of +15% occurs from that value, in other words, when the phase Δφ actually imparted to the light is 0.36754πrad, the hologram (intensity image) captured by the imaging element 8 (formed on the imaging surface) is as shown in Figure 13. Comparing it to the result in Figure 11, it can be seen that the brightness of the hologram in the center is lower than the holograms on the left and right.

These holograms were cut out using the procedure described above, and the commonly known three-step phase shift (α=β=1 in equation (7)) was applied to reconstruct the images, as shown in Figure 14(a). Comparing this to the reconstruction result in Figure 12, it can be seen that there is strong background noise.

To reduce this noise, a phase shift method is used to correct the intensity distribution of the hologram using the ratio of the light intensity or diffraction efficiency of the zeroth-order light and ±1st-order light of the light modulation element of the present invention. The intensity of the hologram in the center and the intensity ratio of the left and right holograms can be calculated as α and β by measuring the diffraction efficiency of the zeroth-order light component and ±1st-order light component of the fabricated light modulation element in advance. Formula (7) can be applied using α and β, which are known information for this optical system. As a result, the image in Figure 14(b) is obtained.

Compared with the result in Figure 14(a), it can be seen that background noise has been sufficiently reduced in Figure 14(b). From the above, it has been confirmed that by using the algorithm in equation (7), a high-quality image can be reconstructed even if there is a manufacturing error in the light modulation element. As described above, the present invention makes it possible to realize an optical system with a single optical path, and to capture and reconstruct three-dimensional information in a single capture.

In the above embodiment, the configuration and operation of the hologram capturing device and image reconstruction device are described, but the present invention is not limited to this, and may be configured as a hologram capturing method and an image reconstruction method.

A computer can be suitably used to function as the image reconstruction device (arithmetic device) 10 described above, and such a computer can be realized by storing a program describing the processing contents for realizing each arithmetic procedure of the image reconstruction device in the memory unit of the computer, and having the CPU of the computer read and execute this program. Note that this program can be recorded on a computer-readable recording medium.

The above-mentioned embodiments have been described as representative examples, but it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiments, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple components described in the embodiments into one, or to divide one component.

The present invention can be used as a stereoscopic camera and can be applied to interference measurement and analysis devices such as fluorescent 3D microscopes.

1 Object to be imaged 2 Beam splitter (splitting means)
3 Spherical phase modulation element 4 Spherical phase modulation element 5 Light modulation element 6 Light modulation element 7 Beam splitter (combining means)
Reference Signs List 8: Image sensor 9: Photographed image 10: Calculation device 11: Magnification adjustment lens 12: Wavelength filter 13: Mirror 14: Mirror 15: Linear polarizer 16: Birefringent lens 17: Quarter-wave plate 18: Three-area split polarizer 19: Light modulation element (polarization diffraction grating)
20 Linear polarizer

Claims (8)

A hologram imaging device that divides an incoherent light wave into a first divided light and a second divided light, imparts different spherical phases to the complex amplitude distributions of these two light waves, and causes the first divided light and the second divided light to interfere with each other to form a hologram and image the hologram,
a phase distribution of two gradations that changes periodically in a first direction and is constant in a second direction perpendicular to the first direction is imparted to each of the first and second divided lights, and diffracted light caused by this periodic phase distribution is utilized to split the first and second divided lights in three directions, respectively, to form three holograms with different phase shift amounts for each direction on an imaging surface of an imaging element, and the three holograms are simultaneously acquired. 2. The holographic imaging device according to claim 1,
a first light modulating element that periodically changes in a first direction and that imparts a constant two-level phase distribution to light in a second direction perpendicular to the first direction, the first light modulating element modulating the first divided light and the other light modulating element modulating the second divided light;
A hologram imaging device characterized in that, when modulating, the modulation is performed using two light modulation elements in which the phase distribution of the other light modulation element is shifted in a first direction relative to the phase distribution of one light modulation element, or the modulation is performed by arranging the positional relationship between one light modulation element and the first divided light and the positional relationship between the other light modulation element and the second divided light so as to be shifted in the first direction. 2. The holographic imaging device according to claim 1,
a means for converting the first divided light and the second divided light into circularly polarized lights having mutually different rotation directions;
a light modulation element that applies a phase distribution to light that changes periodically in a first direction and has a constant two-level phase distribution in a second direction perpendicular to the first direction, the light modulation element splitting each circularly polarized light into three directions;
A holographic imaging device comprising: a three-area split polarizer in which three areas are constituted by polarizers whose transmission axes are different from each other; and wherein circularly polarized light split into three directions is transmitted through each area of the three-area split polarizer in a corresponding manner. 4. The holographic imaging device according to claim 3,
The means for converting the first divided light and the second divided light into circularly polarized lights having different rotation directions from each other includes:
A linear polarizer having a transmission axis in a 45 degree direction that transmits propagating light;
a birefringent lens that modulates the linearly polarized light transmitted through the linear polarizer; and
and a quarter-wave plate for converting the light transmitted through the birefringent lens into circularly polarized light. 2. The holographic imaging device according to claim 1,
A linear polarizer having a transmission axis in a 45 degree direction that transmits propagating light;
a birefringent lens that modulates the linearly polarized light transmitted through the linear polarizer; and
an optical modulation element that applies a phase distribution of two constant levels to an electric field vector of light in one direction, the phase distribution of which changes periodically in a first direction and is perpendicular to the first direction, and that applies a phase distribution of two constant levels to an electric field vector in a direction perpendicular to the electric field vector of the one direction, the phase distribution of which changes periodically in the first direction and is perpendicular to the first direction, the phase distribution of which is shifted along the first direction, the optical modulation element splitting vertically linearly polarized light and horizontally linearly polarized light transmitted through the birefringent lens into three directions;
A holographic imaging device comprising a linear polarizer that transmits vertically linearly polarized light and horizontally linearly polarized light split in three directions by the light modulation element. 6. The holographic imaging device according to claim 2,
11. A holographic imaging device, wherein the light modulation element is a reflective or transmissive light modulation element in which a phase distribution of a lens is superimposed. An image reconstruction device that receives image data of a hologram captured by the hologram capture device described in any one of claims 1 to 6, extracts three individual holograms from the image data, obtains a complex amplitude distribution by a phase shift method, and reconstructs an image by propagation calculation. 8. The image reconstruction apparatus according to claim 7,
An image reconstruction device characterized in that a complex amplitude distribution is obtained by a phase shift method in which the intensity of a hologram is corrected using a ratio of the light intensity or diffraction efficiency of the 0th order light and ±1st order light of a light modulation element that has been obtained in advance.

Images