CN101975994B - Three-dimensional imaging system with multi-stage lens - Google Patents

Abstract

The present invention is a three-dimensional imaging system of a multi-stage lens, including a multi-stage lens and a sensor. Level lens, wherein: multi-level lenses and sensors are placed in sequence in the direction of light propagation, the light passes through the first-level lens and is imaged to the N-1th level lens to generate multi-level scene reduction imaging information, and the multi-level scene reduction imaging information continues Propagate to the N-level lens, the N-level lens obtains the multi-view scene two-dimensional information of the object light, the sensor extracts the multi-view scene two-dimensional information, and then obtains the depth information of the object in the scene through stereo matching. Design the pixel size of the sensor, the focal length of each lens of each level of lens, and the distance between adjacent lenses of each level of lens, so as to obtain good depth resolution.

Description

Three-dimensional imaging system with multi-stage lens

technical field

The present invention relates to the field of three-dimensional (3D) imaging. More specifically, the present invention relates to three-dimensional imaging systems for multi-stage lenses.

Background technique

Three-dimensional technology has been developed for more than a century, but its application is usually limited by operator requirements and cost. Usually we say that an objective world is three-dimensional, and the three-dimensional image of the objective world is recorded by a certain technology, then processed, compressed, and then transmitted, displayed, and finally the image of the objective world is reproduced in the human brain. Now, 3D systems are gradually being used in entertainment, business and scientific applications. Along with this, many hardware and software companies also joined the 3D system.

In recent years, the Sharp Mova SH251iS mobile phone released by NTT DoCoMo has used a color screen capable of displaying 3D images for the first time. A single digital camera takes two two-dimensional (2D) images from different angles, and then processes them to obtain a 3D image. When 3D images are sent out, devices with similar capabilities are required to view the 3D images. And because a single digital camera shoots 2D images, it must be converted into 3D images through deliberate processing. In order to obtain a better 3D image, the requirements for the position of the object to be photographed are relatively strict when shooting. So the quality of this image cannot be guaranteed.

Fuji Real 3D W1 uses the newly developed "FinePix REAL 3D lens system" and the natural image processing engine 3D. In fact, it uses binocular vision, that is, uses two lenses to simulate the two eyes of a person. After processing, it forms a sense of hierarchy. 3D images. Firstly, the synchronous 2D information collected by two CCD sensors is fused, and then processed into high-quality images in real time. However, two lenses are required for simulation, which makes the size of the device too large and inconvenient to carry, thus limiting the promotion.

The SwissRanger SR-2 camera adopts TOF principle technology for the first time. Send a beam of light as a reference into the scene to be tested, and calculate the distance of the object in the scene by calculating the time difference or phase difference of the beam back to the light, so as to obtain the depth information. In addition, combined with camera shooting to obtain two-dimensional image information, the entire scene can be acquired at the same time. Due to the use of a reference beam, the device is limited in capturing the entire scene, resulting in low depth resolution and a small viewing angle.

The current research results of 3D acquisition equipment still have defects such as the size of the equipment is too large; the depth resolution is low, the viewing angle is small; the recomputability is weak; the non-rigid complete object cannot be easily obtained.

Contents of the invention

In order to solve the problems of the prior art, the object of the present invention is to provide a multi-level lens that reduces system size, improves depth resolution, has a large field of view, has strong recalculation, and can easily obtain rigid and non-rigid complete objects. 3D imaging system.

In order to achieve the above object, the present invention provides a three-dimensional imaging system of a multi-level lens, and its technical solution includes: a multi-level lens and a sensor, and the multi-level lens is a first-level lens, a second-level lens, ... the N-2th level lens , the N-1st lens, the Nth-level lens, wherein: the multi-level lens and the sensor are placed in sequence in the direction of light propagation, and the light is imaged by the first-level lens to the N-1-th level lens to generate a multi-level scene reduction Imaging information, multi-level scene reduction imaging information continues to spread to the N-level lens, the N-level lens obtains the multi-view scene two-dimensional information of the object light, the sensor extracts the multi-view scene two-dimensional information, and then through stereo matching, to obtain Depth information of objects in the scene.

Wherein, the number of each stage lens in the multistage lens is one lens, or a plurality of lenses arranged symmetrically; each lens is a single-sided convex lens or a double-sided convex lens, and the shape of the lens is circular , or a regular polygon; a plurality of lenses are arranged on a plane or on a curved surface, and the curved surface is a curved surface with various curvature radii; the curved surface is concave to incident light, or convex to incident light.

Wherein, the plurality of lenses in the symmetrical layout are grid-shaped layout lenses, ring-shaped layout lenses and random regular polygon layout lenses.

Wherein, the annular layout lens is at least one annular layout lens or more than one annular layout lens.

Wherein, the light emitted from any point on the real image formed by the first-stage lens can reach each lens in the next-stage lens.

Wherein, the pixel size of the sensor is smaller than the difference of the disparity is the depth resolution; when the multi-stage lens is selected as the two-stage lens, the difference of the disparity (disparity) is expressed as follows:

$\frac{\mathrm{<span\; class="notranslate">\; LG</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; wxya</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}$

和分别为场景中深度不同的两个物体针对第一级透镜的像距，且

Wherein, suppose that the second-order lens is made up of a plurality of lenses, L is the distance between the centers of adjacent lenses in the plurality of lenses in the second-order lens, and t is each lens in the second-order lens to the first-order The distance of the lens imaging datum plane, g is the same focal length of each lens in the second-stage lens,

and are the image distances of two objects with different depths in the scene with respect to the first-stage lens, and

Wherein, the size of the sensor is larger than the imaging range of each lens in the N-th stage lens.

Wherein, the images formed by the layout of the lenses of the Nth stage lens on the sensor do not overlap.

Wherein, in the images of two adjacent lenses on the sensor passing through the lenses of various levels, the ratio of the number of pixels corresponding to the same object part is greater than 1/2.

Among them, the focus adjustment of the three-dimensional imaging system is to obtain clear imaging by adjusting the position of one or several devices among the first-stage lens to the N-stage lens and the sensor.

Among them, the focus adjustment of the three-dimensional imaging system is to obtain clear imaging by adjusting the positions of the first-stage lens to the N-th stage lens and the position of each device in the sensor at the same time.

Beneficial effects of the present invention: the present invention adopts the three-dimensional imaging technology of multi-level lenses and the symmetrical layout of the lenses at all levels, which reduces the size of the system and makes it possible to carry it easily. The flexible lens shape and the selection of various lens layout surfaces expand the field of view, greatly increase the imaging angle of view, and improve the depth resolution. At the same time, the present invention proposes a flexible focusing method to meet the requirements of using the system in different occasions. The invention can conveniently obtain the three-dimensional imaging system of the multi-stage lens of the complete object of the rigid body and the non-rigid body.

Description of drawings

FIG. 1 shows a block diagram of one embodiment of a three-dimensional imaging system with multiple lenses.

Figure 2 shows a block diagram of one embodiment of a three-dimensional imaging system with multi-stage lenses.

3a to 3h show the layout types of the lenses in the lenses of each level in the three-dimensional imaging system of the multi-level lenses.

4a to 4b show basic optical path diagrams of a three-dimensional imaging system with multi-level lenses.

5a to 5b show the optical path diagrams of two objects with different depths in the scene in the three-dimensional imaging system with multi-level lenses.

FIG. 6 shows a schematic diagram 1 of ensuring that images of each lens in the last-stage lens do not overlap and correspond to the same proportion of pixels in a three-dimensional imaging system of multi-stage lenses.

FIG. 7 shows a schematic diagram of lenses in a 2*3 grid layout of the second-level lenses in the three-dimensional imaging system of the second-level lenses.

8a to 8b show that in the three-dimensional imaging system of multi-level lenses, the lenses of each level are arranged on curved surfaces and planes.

9a to 9l show the shape of each lens in the three-dimensional imaging system of the multi-level lens and the arrangement of the corresponding lens composition.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

A three-dimensional imaging system with multistage lenses is described here. The 3D imaging system with multi-level lenses allows users to take advantage of the design of multi-level lenses to obtain 3D scenes and 3D objects conveniently and quickly. The system includes two parts: a multi-level lens and a sensor system. The multi-level lens includes a first-level lens, a second-level lens, ... N-2 level lens, N-1 level lens until N-level lens.

FIG. 1 shows a block diagram of an embodiment of a two-stage lens 3D imaging system, including a first-stage lens and a second-stage lens, and a sensor for obtaining two-dimensional information of multi-dimensional viewing angles and other components to ensure correct functioning of the system. After the light is imaged by the first-stage lens 101, it continues to propagate through each lens in the second-stage lens 102 to generate multi-level scene reduction imaging information, and the multi-level scene reduction imaging information is finally acquired by the sensor 103. The light of the object in the scene passes through the first-stage lens 101 and the information imaged by the second-stage lens 102. The basic optical path diagram of this process is shown in Fig. 4a to Fig. 4b. The distance between the first-stage lens 101 and the second-stage lens 102 is L ₁ , and the distance between the second-stage lens 102 and the sensor 103 is L ₂ . The first-stage lens 101 in the multi-stage lens three-dimensional imaging system selects one lens, and the layout of the lenses in the second-stage lens 102 is a 3*3 grid layout. The layout of the lenses in the first-level lens 101 and the second-level lens 102 is not limited to this layout, and the symmetrical layout of the lenses in several levels of lenses shown in Figure 3a to Figure 3h can be selected, and other symmetrical layouts can also be selected . The direction of light emitted by any point on the real image formed by the first-stage lens 101 has a certain range, so it is necessary to ensure that the light emitted by each point can reach each lens in the second-stage lens 102 . The pixel size of the sensor 103 should be guaranteed to be smaller than the depth resolution, and the size of the sensor 103 is larger than the imaging range of the second-level lens 102, so that the imaging of the second-level lens array is all on the sensor 103, thereby reflecting the depth of objects in the scene information. By adjusting the mutual positions of the lenses in the various levels of lenses, it is ensured that the images on the sensor 103 do not overlap. However, the ratio of the number of pixels corresponding to the same object part in the image formed by two adjacent lenses of the second-stage lens 102 in the reduced scene imaging information is greater than 1/2. The distance between the first-stage lens 101 , the second-stage lens 102 and the sensor 103 is not fixed, and whether the distance between the three is fixed or not is determined by the focusing method. The specific method of focusing can be achieved by adjusting the positions of the first-stage lens 101, the second-stage lens 102 and the sensor 103 at the same time, that is, changing the distance _L1 and the distance _L2 at the same time to make the imaging on the sensor 103 clear; It is also possible to fix the distance L ₂ between the second-level lens 102 and the sensor 103, and adjust the position of the first-level lens 101 to change L ₁ to adjust the focus; it is also possible to fix the distance between the first-level lens 101 and the second-level lens 102 distance L ₁ , adjust the position of the sensor 103 to change the distance L ₂ to obtain a clear image; it is also possible to fix the distance L between the first-stage lens 101 and the sensor 103, and adjust the position of the second-stage lens 102 to change the distance L ₁ and distance L ₂ to get a clear image.

Fig. 2 shows the block diagram of an embodiment of the three-dimensional imaging system of the three-level lens, including the three-level lens system of the first-level lens, the second-level lens, the third-level lens and the two-dimensional information used to obtain the multi-dimensional viewing angle after imaging Sensors and other components that ensure the correct functioning of the system. The light is imaged by the first-stage lens 101 and continues to propagate through the second-stage lens 104 for imaging, then passes through the third-stage lens 102 and finally passes through the sensor 103 to obtain the imaged information of the object passing through the three-stage lenses 101 , 104 and 102 . The optical path diagram of this process is shown in Fig. 4a to Fig. 4b. The distance between the first-stage lens 101 and the second-stage lens 104 is L ₁ , the distance between the second-stage lens 104 and the third-stage lens 102 is L ₂ , and the distance between the third-stage lens 102 and the sensor 103 for L ₃ . The first stage lens 101 in the three-dimensional imaging system of the multistage lens shown in Fig. The lens layout in is a 3*3 grid layout lens. The layout of the lenses in the various levels of lenses is not limited to this layout, and the symmetrical layout of the lenses in the several levels of lenses shown in Figure 3a to Figure 3h can be selected, or other symmetrical layouts can be selected. The direction of light emitted by any point on the real image formed by the first-stage lens 101 has a certain range, so it is necessary to ensure that the light emitted by each point can reach each lens in the next-stage lens. The pixel size of the sensor 103 is guaranteed to be smaller than the depth resolution, and the size of the sensor 103 is larger than the imaging range of the third-level lens 102, so that the imaging of the lens array of the third-level lens 102 is all on the sensor 103, so that the sensor 103 can reflect Information about the imaging of objects in the scene. By adjusting the mutual positions of the lenses in the various levels of lenses, it is ensured that the imaging on the sensor 103 does not overlap, but the light of the object passes through the various levels of lenses and finally in the images of the two adjacent lenses on the sensor 103, corresponding to The ratio of the number of pixels of the same object part is greater than 1/2. The distance between the first-stage lens 101, the second-stage lens 104, the third-stage lens 102, and the sensor 103 is not fixed. Whether the distance between the three is fixed is determined by the focusing method. The specific method can be determined by simultaneously Adjust the positions of the first-stage lens 101, the second-stage lens 104, the third-stage lens 102, and the sensor 103, that is, change the distance L ₁ , distance L ₂ , and distance L ₃ at the same time to achieve the purpose of focusing and make the image on the sensor 204 clear; also can fix the distance L ₁ between the first stage lens 101 to the second stage lens 104 and the distance L ₂ between the second stage lens 104 to the third stage lens 102; adjust the position of the sensor 103 to change the distance L ₃ to obtain clear imaging; it is also possible to fix the positions of the first-level lens 101, the second-level lens 104 and the sensor 103, adjust the position of the third-level lens 102, and change the distance _L2 and distance _L3 to obtain clear imaging It is also possible to fix the positions of the first stage lens 101, the third stage lens 102 and the sensor 103, adjust the position of the second stage lens 104, and change the distance _L1 and the distance _L2 to obtain clear imaging; the second stage can also be fixed Level lens 104, the position of third level lens 102 and sensor 103, adjust the position of first level lens 101, change distance L ₁ to obtain clear imaging; also can fix the position of first level lens 101 and second level lens 104 , adjust the positions of the third-level lens 102 and the sensor 103 at the same time, and change the distance _L2 and the distance _L3 to obtain clear imaging; it is also possible to fix the positions of the first-level lens 101 and the third-level lens 102 while adjusting the second-level The position of lens 104 and sensor 103, change distance L ₁ , distance L ₂ and distance L ₃ to obtain clear imaging; It is also possible to fix the position of first-level lens 101 and sensor 103, and adjust second-level lens 104 and third-level lens 104 simultaneously. The position of the first-stage lens 102 can be changed by changing the distance _L1 , distance _L2 and distance _L3 , thereby obtaining clear imaging; the position between the second-stage lens 104 and the third-stage lens 102 can also be fixed, and the first-stage lens can be adjusted at the same time 101 and the position of the sensor 103, change the distance L ₁ and the distance L ₃ , so as to obtain clear imaging; also can fix the position of the second-level lens 104 and the sensor 103, and adjust the position of the first-level lens 101 and the third-level lens 102 at the same time position, change the distance L ₁ , distance L ₂ and distance L ₃ to obtain clear imaging; the positions of the third-level lens 102 and the sensor 103 can also be fixed, and the positions of the first-level lens 101 and the second-level lens 104 can be adjusted at the same time , change the distance L ₁ , the distance L ₂ and the distance L ₃ , so as to obtain clear imaging.

Fig. 1 and Fig. 2 only illustrate the 3D imaging system of the secondary lens and the 3D imaging system of the tertiary lens. For the three-dimensional imaging system of multi-level lenses, the design of more-level lenses can be extended to perform three-dimensional imaging, and the principle is consistent with the imaging principles in Figure 1 and Figure 2 . For the focusing method of the three-dimensional imaging system of the multi-level lens, Fig. 1 and Fig. 2 are only for the three-dimensional imaging system of the second-level lens and the three-dimensional imaging system of the third-level lens. For a three-dimensional imaging system, clear imaging can be obtained by adjusting the positions of one or more of the lenses and sensors at all levels.

Fig. 3 a-Fig. 3 h show the lens layout type in each stage lens in the three-dimensional imaging system of multistage lens, for example the first stage lens 101 in Fig. 1, the second stage lens 102 etc., the first stage lens in Fig. 2 101, the second-stage lens 104, the third-stage lens 102, etc., and even the Nth-stage lens can be a symmetrical layout of any lens in Fig. 3a-Fig. 3h. Figures 3a to 3h only list some symmetrically arranged lens arrays: a single lens (as shown in Figure 3a), a grid-shaped layout lens (as shown in Figure 3b, Figure 3d, Figure 3e and Figure 3h), a ring-shaped layout lens (such as Figure 3f) and regular polygonal layout lenses (as shown in Figure 3c and Figure 3g) and so on. The lens arrays of all levels of lenses are arranged symmetrically or regularly, and can be in many forms. Figures 3a to 3h show only a part of them, even 100*100 grid-shaped layout lenses and 200*200 grid-shaped layout lenses are also available, and other symmetrical layouts or regular arrangements are also available, so I won’t go into details here.

Each lens in the three-dimensional imaging system of the multi-stage lens can be a single-sided convex lens (as shown in Figure 9a) or a double-sided convex lens (as shown in Figure 9b), and the shape of the lens is not limited to the circle shown in Figure 9c, but can also be Other regular polygons, such as regular triangles (as shown in Figure 9d), squares (as shown in Figure 9e), regular pentagons (as shown in Figure 9f), regular hexagons (as shown in Figure 9g) and regular octagons (as shown in Figure 9h) regular polygon. Then the arrangement of lenses at all levels can also be a regular layout made of multiple regular polygons, such as a regular layout made of regular triangles (as shown in Figure 9i), a regular layout made of squares (as shown in Figure 9j), A regular layout composed of regular hexagons (as shown in FIG. 9k ) and a regular layout composed of regular octagons (as shown in FIG. 91 ) and other regular layouts composed of multiple other regular polygons.

In the three-dimensional imaging system of multi-level lenses, each lens in each level of lens can be arranged on a plane as shown in Figure 1 and Figure 2, and can also be arranged on a curved surface, and the curved surface can be a curved surface with any curvature, and the curved surface is Concave for incident light, or convex for incident light. Fig. 8a shows a cross-sectional view of lenses of various levels arranged on a convex surface, and Fig. 8b shows a cross-sectional view of lenses of various levels arranged on a concave surface.

4a to 4b show a specific optical path diagram of a three-dimensional imaging system with multi-stage lenses. The optical path diagram of the three-dimensional imaging system with multi-stage lenses is illustrated by taking the three-dimensional imaging system with two-stage lenses in which the first-stage lens is a lens and the second-stage lens is a 1*3 grid-shaped lens array as an example. The number of lenses in the first-stage lens 101 and the second-stage lens (the second-stage lens consists of 10201, 10202 and 10203 to form a second-stage lens array) can be one or more symmetrically arranged lens arrays. And the number of stages of the lens can also be extended to multiple stages. The object whose distance from the main optical axis is P is imaged through the first-stage lens 101 with focal length f, the distance between the real image formed by the first-stage lens 101 and the main optical axis is Q, and the optical path of the first-stage lens 101 imaging In , the object distance is u ₁ , and the image distance is v ₁ . The light continues to propagate through the lenses of the second-stage lenses whose focal lengths are all g, where the real image formed by the first-stage lens 101 is regarded as an object, then in the optical path of the second-stage lens imaging, the object distance is u ₂ , the image distance is v ₂ . The size of the sensor 103 is larger than the imaging range of the second-stage lens. In FIG. 4 b , for the image formed by the first-stage lens 101 to pass through the intermediate lens 10202 , the distance from the image on the sensor 103 to the optical axis is D _m . For a lens 10201 adjacent to the middle lens 10202, the distance from its center to the main optical axis (middle lens 10202) is L, and the image formed by the first-stage lens 101 passes through the edge 10201 and the image formed on the sensor 103 reaches this point. The distance of the optical axis of the lens 10201 is D _s . Then, the difference between the distances between the two images of the object formed by the intermediate lens 10201 and its adjacent lens 10202 and the optical axes of the respective lenses is defined as disparity, represented by D, that is, D=D _s −D _m .

和

其中

像距分别为

和

其中

第一级透镜101的焦距为f，第二级透镜中各透镜的焦距均为g，第二级透镜102到第一级透镜101成像基准面的距离为t，在这两个物体中，物体1成像在镜头的成像基准面上，则对于第二级透镜的物距为

物体2成像在镜头成像基准面之前(向第一级透镜方向偏移)，则对于第二级透镜的物距为

物体1和物体2到光轴的距离分别为P₁和P₂，经过第一级透镜成像后，物体1的像和物体2的像到光轴的距离分别为Q₁和Q₂。物体1和物体2发出的光线分别按照图4a至图4b中的光路图，最终在传感器103上分别得到物体1和物体2的视差D⁽¹⁾和D⁽²⁾。那么两个物体在两幅图像中的视差的差别δD(也称为深度分辨率)为：5a to 5b show the optical path diagrams of two objects with different depths in the scene in the three-dimensional imaging system with multi-level lenses. In the scene, objects 1 and 2 with different depths pass through the first-stage lens to form real images with different image distances. For the next-stage lens, the object distance is different, resulting in different parallax. Three-dimensional imaging system with multi-stage lens Two optical paths with different depths The object distances of object 1 and object 2 in Figure 5a to Figure 5b are respectively

and

in

The image distance is

and

in

The focal length of the first-stage lens 101 is f, the focal length of each lens in the second-stage lens is g, and the distance from the second-stage lens 102 to the imaging reference plane of the first-stage lens 101 is t. In these two objects, the object 1 is imaged on the imaging reference plane of the lens, then the object distance of the second-stage lens is

Object 2 is imaged in front of the lens imaging reference plane (shifted toward the first-stage lens), then the object distance for the second-stage lens is

The distances from object 1 and object 2 to the optical axis are respectively P ₁ and P ₂ , and the distances from the optical axis to the image of object 1 and object 2 are Q ₁ and Q ₂ respectively after being imaged by the first-stage lens. The light rays emitted by object 1 and object 2 respectively follow the light path diagrams in FIG. 4a to FIG. 4b , and finally obtain the parallax D ⁽¹⁾ and D ⁽²⁾ of object 1 and object 2 on sensor 103 respectively. Then the difference δD (also called depth resolution) of the parallax of two objects in the two images is:

$\mathrm{<span\; class="notranslate">\; \&delta;D</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; LG</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; wxya</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>$

和

表示为：where image distance

and

Expressed as:

${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}\mathrm{<span\; class="notranslate">\; f</span>}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; ,</span>$ ${\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}\mathrm{<span\; class="notranslate">\; f</span>}}{{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; ,</span>$

In order to obtain the difference δD of the parallax of two objects in the two images on the sensor 103, the pixel size of the required sensor 103 should be smaller than the difference δD of the parallax of the two objects in the two images:

$\mathrm{<span\; class="notranslate">\; \&delta;D</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; LG</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; wxya</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>$

Only then can it reflect the depth information of the object in the scene. At the same time, in order to make the disparity (disparity) of objects at different depths in the two pictures have a difference greater than 1 pixel, it is necessary to use a first-stage lens with a longer focal length.

Fig. 6 shows a three-dimensional imaging system with multi-level lenses. Among them, the following four conditions must be ensured to be satisfied: First, the distance between two adjacent lenses in the last-stage lens must be greater than the diameter of the lens, so that the lenses will not overlap. The second is to take the largest inscribed square in the imaging circle of each lens in the last-stage lens as the final image to be acquired on the sensor, and these squares cannot have images formed by other lenses in the last-stage lens. The third is that the common area imaged between the above-mentioned inscribed squares should be large enough, greater than 1/2. Fourth, each square must fall within the range of the sensor. The position of each lens in the last stage lens is determined by the above four conditions. In the system of Figure 6, the first-stage lens is a lens with a focal length of f and an effective aperture of D, and each lens in the second-stage lens has a focal length of g and a diameter of d, and the array is as shown in Figure 7, wherein, in Figure 7 L _x and _Ly are the distances between the centers of two adjacent lenses in the x-axis and y-axis directions of the second-level lens respectively; a, b, f, and h in the second-level lens are called edge lenses, and c and e It is called the intermediate lens; the distance from the image formed by the light through the first-level lens to the plane where the second-level lens is located is taken as the object distance, and its value is u, and the distance from the second-level lens to the sensor is the image distance, and its value is v, The diameter of the two circles on the first-stage lens imaging plane obtained by the edge lens 10201 and the middle lens 10202 is L1, and the distance between the centers of the two circles is E. The image formed by the light of the object passing through the first-stage lens, Then, the second image is formed on the sensor 103 through the edge lens 10201 and the middle lens 10202, the radius is r, and the distance between the centers is S.

Consider the horizontal orientation:

$\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="L"\; filenames="HSA00000246440800071.png,HSA00000246440800072.png"\; state="\{\{state\}\}">L</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; df</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; ,</span>$

The radius of the imaging circle of the lens in the second stage lens

$\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="1"\; label="wxya"\; filenames="HSA00000246440800071.png,HSA00000246440800072.png"\; state="\{\{state\}\}">wxya</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; ,</span>$

The center distance S of the lens imaging circle in the second stage lens is

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; ]</span>}{<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ,</span>$

Therefore, the four conditions to be met by the arrangement position of the second-level lenses are:

1. Adjacent lenses do not overlap, that is, L _x >= d

即对L_x的要求为，2. The image acquired on the sensor is the inscribed square of the imaging circle of each lens of the second-level lens, so the images obtained on the sensor must not overlap with each other, then

That is, the requirement for L _x is,

${\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; \&Greater\; Equal;</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; +</span>\frac{\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ]</span>}{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>$

The given L _x must satisfy the coincidence ratio R of two adjacent pictures greater than 50%. As the two circles of the images of the two lenses in the second-stage lens, the distance between the centers of the circles can be expressed as:

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\mathrm{<span\; class="notranslate">\; f</span>}}{<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; ,</span>$

Therefore, L _x is the largest, so that the public area R just satisfies the 1/2 condition.

At this point, two circles that are images of the edge lens 10201 and the middle lens 10202 can be calculated, and the distance E between the circle centers should be

此时有

进而 therefore require

At this time there is

and then

${\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; \&le;</span>\frac{\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="df"\; filenames="HSA00000246440800071.png"\; state="\{\{state\}\}">df</figure-callout></span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; .</span>$

The above three also have the same conclusion in the vertical direction.

其中w是传感器的长，即The selected inscribed squares should all be arranged within the range of the sensor and cannot exceed the range of the sensor.

where w is the length of the sensor, i.e.

$\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; \&le;</span>\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; -</span>\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ]</span>}{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; [</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; u</span>}}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; ]</span>}<span\; class="notranslate">\; ,</span>$

3. If you consider the vertical direction, you need to replace w in the formula with the width h of the sensor.

The above is only a specific implementation mode in the present invention, but the scope of protection of the present invention is not limited thereto. Anyone familiar with the technology can understand the conceivable transformation or replacement within the technical scope disclosed in the present invention. All should be covered within the scope of protection of the claims of the present invention.

Claims (11)

1. A three-dimensional imaging system of a multistage lens, characterized in that the system comprises: a multistage lens and a sensor, and the multistage lens is a first stage lens, a second stage lens, ... the N-2 stage Lenses, class N-1 lenses, class N lenses, in which: Place multi-level lenses and sensors in sequence in the direction of light propagation. The light passes through the first-level lens and is imaged to the N-1th level lens to generate multi-level scene reduction imaging information. The multi-level scene reduction imaging information continues to propagate to the Nth level. lens, the Nth stage lens obtains the multi-view scene two-dimensional information of the object light, the sensor extracts the multi-view scene two-dimensional information, and then through stereo matching, thereby obtaining the depth information of the object in the scene; the last stage of the multi-stage lens The distance between two adjacent lenses in the first-level lens must be greater than the diameter of the lens, so that the lenses do not overlap; in the last-level lens, the largest inscribed square in the imaging circle of each lens is the final image on the sensor For the image to be acquired, there must be no image formed by other lenses in the last-level lens inside these squares; the common imaging area between the above-mentioned inscribed squares must be large enough, greater than 1/2; each square must fall on the sensor within range. 2. the three-dimensional imaging system of multistage lens as claimed in claim 1, is characterized in that, the number of each stage lens in described multistage lens is a lens, or a plurality of lenses of symmetrical arrangement; The lens is a single-sided convex lens or a double-sided convex lens, and the shape of the lens is a circle or a regular polygon; multiple lenses are arranged on a plane or on a curved surface, and the curved surface is a curved surface with various radii of curvature; The curved surface is concave to incident light, or convex to incident light. 3. The three-dimensional imaging system of multi-level lenses according to claim 2, characterized in that, the plurality of lenses in the symmetrical layout are grid-shaped layout lenses, ring-shaped layout lenses and arbitrary regular polygon layout lenses. 4 . The three-dimensional imaging system of multi-level lenses according to claim 3 , wherein the ring-shaped lens is at least one ring-shaped lens or more than one ring-shaped lens. 5. The three-dimensional imaging system with multi-stage lenses according to claim 1, wherein the light emitted from any point on the real image formed by the first-stage lens can reach each lens in the next-stage lens. 6. The three-dimensional imaging system of multistage lens as claimed in claim 1, is characterized in that, the difference of the pixel size of described sensor is less than parallax is depth resolution; When selecting multistage lens as two-stage lens, the difference of parallax Expressed as follows: $\frac{\mathrm{<span\; class="notranslate">\; LG</span>}}{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; wxya</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}$ Wherein, suppose that the second-order lens is made up of a plurality of lenses, L is the distance between the centers of adjacent lenses in the plurality of lenses in the second-order lens, and t is each lens in the second-order lens to the first-order The distance of the lens imaging datum plane, g is the same focal length of each lens in the second-stage lens,

and

are the image distances of two objects with different depths in the scene with respect to the first-stage lens, and

7. The three-dimensional imaging system with multi-stage lenses according to claim 1, wherein the size of the sensor is larger than the imaging range of each lens in the Nth-stage lens. 8 . The three-dimensional imaging system with multi-stage lenses according to claim 1 , wherein the images formed by the layout of the lenses of the N-th stage lens on the sensor do not overlap. 9. The three-dimensional imaging system of multistage lens as claimed in claim 1, is characterized in that, in the image of two adjacent lenses on the sensor through various stages of lenses, the ratio of the number of pixels corresponding to the same object part It should be greater than 1/2. 10. The three-dimensional imaging system of multi-level lenses as claimed in claim 1, wherein the focusing of the three-dimensional imaging system is achieved by adjusting the position of one or several devices from the first level lens to the Nth level lens and the sensor Get clear images. 11. The three-dimensional imaging system of multi-level lenses as claimed in claim 1, wherein the focusing of the three-dimensional imaging system is achieved by adjusting the position of each device in the first-level lens to the Nth-level lens and the sensor at the same time. of imaging.

Images