JP2024105178A - Imaging device and ranging system - Google Patents

Abstract

[Problem] In a ToF imaging device in which the optical axes of an illumination means and a light receiving means are not the same, an area in which the dot pattern light received by the light receiving means is excessively close in the angle of view to be measured is eliminated, and accurate distance information is obtained. [Solution] An imaging device comprising a housing, a first light projecting unit that projects structured light, a first light receiving unit into which incident light including the structured light from the first light projecting unit is incident, and a support unit that supports the imaging device, the first light receiving unit, the first light projecting unit, and the support unit being arranged at different positions in a first direction of the housing in the order of the first light receiving unit, the first light projecting unit, and the support unit. [Selected Figure] Figure 9-1

Description

The present invention relates to an imaging device and a distance measurement system.

Conventionally, one method for measuring the distance from an imaging device to an object is known as the ToF (Time of Flight) method, which calculates the distance to the object based on the time between emitting light and receiving the reflected light. In order to obtain distance information using this ToF method, a ToF imaging device has been developed as a distance measuring device that uses a light receiving means including a ToF sensor and an illumination means including a laser light source. ToF imaging devices are used, for example, to detect obstacles in vehicles or to obtain spatial information inside structures.

Patent Document 1 discloses a ToF distance measuring device equipped with an illumination means (structured illumination) that emits a spatial illumination pattern having strong light emission areas (dot-like) and weak light emission areas (uniform light) for the purpose of accurately acquiring distance information in a scene in which various objects with significantly different light reflectances and distances are mixed together.

The distance measuring device disclosed in Patent Document 1 has a spatial irradiation pattern with weak and strong light emission regions, so that even if the return light from the weak light emission region cannot ensure a sufficient S/N ratio to obtain distance information, the return light from the strong light emission region can ensure a sufficient S/N ratio to obtain accurate distance information.

In addition, according to the distance measuring device disclosed in Patent Document 1, even if the return light from the strong light emission region is too strong when the light reflectance is high or the target object is located nearby, accurate distance information can be obtained by receiving the return light to an extent that the light receiving element is not saturated by the light from the weak light emission region.

However, with conventional ToF imaging devices, the optical axes of the illumination means and the light receiving means are not the same, resulting in an area where the dot pattern light received by the light receiving means is excessively close to each other at the angle of view that is the subject of distance measurement. Thus, with conventional ToF imaging devices, if the dot lights received on the light receiving element are too close to each other, the dots will connect with each other and will no longer function as a dot pattern, making it impossible to obtain accurate distance information.

The present invention has been made in consideration of the above, and aims to obtain accurate distance information in a ToF imaging device in which the optical axes of the illumination means and the light receiving means are not the same, by eliminating areas in which the dot pattern light received by the light receiving means is excessively close in the angle of view to be measured.

In order to solve the above-mentioned problems and achieve the object, the present invention provides an imaging device comprising a housing, a first light-projecting unit that projects structured light, a first light-receiving unit to which incident light including the structured light from the first light-projecting unit is incident, and a support unit that supports the imaging device, characterized in that the first light-receiving unit, the first light-projecting unit, and the support unit are arranged at different positions in a first direction of the housing in the order of the first light-receiving unit, the first light-projecting unit, and the support unit.

The present invention has the effect of eliminating areas where the dot pattern light received by the light receiving means is excessively close in the angle of view to be measured in a ToF imaging device in which the optical axis of the illumination means that irradiates the dot pattern light and the optical axis of the light receiving means are not the same, thereby making it possible to obtain accurate distance information.

FIG. 1 illustrates an example of a configuration of a distance measuring system according to a first embodiment. FIG. 2 is a diagram illustrating an example of a configuration of the imaging apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of a schematic configuration of an imaging apparatus. FIG. 4 is a diagram showing an example of the arrangement of the optical system. FIG. 5 illustrates an example of functional blocks of a processing unit according to the first embodiment. FIG. 6 is a diagram showing an example of a supporting manner of the imaging device. FIG. 7 is a diagram illustrating an example of a blind spot of an imaging device. FIG. 8A is a diagram showing an example of a desirable dot pattern image. FIG. 8B is a diagram showing an example of an actually obtained dot pattern image (with sparseness and density due to parallax). FIG. 9A is a diagram showing an example of irradiation of a nearby planar target for distance measurement. FIG. 9B is a diagram showing an example of the pitch between adjacent dots. FIG. 10 is a diagram illustrating an example of a blind spot of a ToF camera. FIG. 11 is a diagram showing the relationship between the ToF viewing angle and D/d. FIG. 12 is a diagram showing another example of a support portion of the imaging device. FIG. 13 is a diagram showing another example of the arrangement of the optical elements of the imaging device. FIG. 14 is a diagram illustrating an example of a configuration of a distance measuring system according to the second embodiment. FIG. 15 is a diagram illustrating an example of a configuration of a distance measuring system according to the third embodiment. FIG. 16 is a diagram showing a modification of the distance measuring system according to the third embodiment. FIG. 17 is a diagram showing another modified example of the distance measuring system according to the third embodiment. FIG. 18 illustrates an example of functional blocks of a processing unit according to the third embodiment. FIG. 19 is a diagram illustrating an example of a configuration of a distance measuring system according to the fourth embodiment. FIG. 20 is a diagram showing an example of the influence of camera shake. FIG. 21 is a diagram illustrating an example of a configuration of a distance measuring system according to the fifth embodiment. In FIG. FIG. 22 is a diagram showing a modification of the distance measuring system according to the fifth embodiment. FIG. 23 is a diagram showing another modified example of the distance measuring system according to the fifth embodiment. FIG. 24 is a diagram showing an example of overlapping of patterned light. FIG. 25 is a diagram illustrating an example of a cutoff unit according to the sixth embodiment. FIG. 26 is a diagram showing another example of the interrupter according to the sixth embodiment. FIG. 27 is a diagram showing another example of the interrupter according to the sixth embodiment. FIG. 28 is a diagram showing another example of the interrupter according to the sixth embodiment. FIG. 29 is a diagram illustrating an example of a configuration of a distance measuring system according to the seventh embodiment. In FIG.

Embodiments of the imaging device and distance measurement system are described in detail below with reference to the attached drawings.

(First embodiment)
FIG. 1 is a diagram showing a system configuration of a distance measuring system 3001 including an imaging device 1001 and an information processing device 4001 according to the first embodiment. As shown in FIG. 1, the distance measuring system 3001 is a system configuration in which the imaging device 1001 is connected to an information processing device 4001 such as a PC or a cloud. In the distance measuring system 3001, the imaging device 1001 performs ToF photography or the like, and the information processing device 4001 performs distance measuring processing based on data acquired by the imaging device 1001. The imaging device 1001 and the information processing device 4001 can communicate with each other by wired communication or wireless communication using a transmitting/receiving unit that each of them has. Data may be transmitted (output) from the imaging device 1001 to the information processing device 4001 via a network, and the transmitting/receiving unit may be configured by an interface circuit with a portable storage medium such as an SD card or a personal computer.

Fig. 2 is an external perspective view showing the configuration of the imaging device 1001 according to the first embodiment, Fig. 3 is a diagram showing an example of the schematic configuration of the imaging device 1001, and Fig. 4 is a diagram showing an example of the arrangement of the optical system. In this embodiment, the imaging device 1001 has a function as a distance measuring device (ToF camera) using the ToF (Time of Flight) method and a function as a luminance camera (RGB camera), and captures images of the entire celestial sphere using the ToF camera and the luminance camera.

As shown in Figures 2 to 4, the imaging device 1001 includes a light projecting unit 21, a ToF light receiving unit 61, a luminance light receiving unit 30, and a control unit 120. The light projecting unit 21 and the ToF light receiving unit 61 function as a ToF distance measuring device, that is, a ToF camera, and the luminance light receiving unit 30 functions as a luminance camera.

The light projection unit 21 irradiates distance measurement light (infrared light, etc.) toward the measurement target area. The light projection unit 21 includes a light source 210 that emits infrared light and a ToF light projection system 111 (light projection optical system) consisting of optical elements that widen the divergence angle, and emits the light of the light source 210 at a wide angle. The optical elements of the ToF light projection system 111 include, for example, a lens, a DOE (diffractive optical element), a diffuser, etc. The light source 210 is, for example, a two-dimensional array of vertical cavity surface emitting lasers (VCSELs). The imaging device 1001 of this embodiment has two light projection units 21 arranged facing in opposite directions.

The two light-projecting units 21 are structured illuminations that project patterned light (a dot pattern in this embodiment) into space, which is structured light. In other words, the two light-projecting units 21 project point-like light (spot light). Ideally, the light is divided into areas that are irradiated and areas that are not irradiated, but in reality, the spot light has a certain degree of light spread and a small amount of multipath interference, so the contrast difference between the points of reflected light of the spot light received by the ToF sensor 110 described below and other areas is not necessarily "100:0".

The distance measurement light (structured light) emitted from the light projection unit 21 is reflected by an object present in the measurement target area. The ToF light receiving unit 61 receives the reflected light from the object in the measurement target area. The ToF light receiving unit 61 includes a ToF sensor 110 having sensitivity to the distance measurement light, and a ToF light receiving optical system 112 (first light receiving optical system) consisting of optical elements that guide the incident light to the ToF sensor 110. The optical elements of the ToF light receiving optical system 112 include, for example, a lens. The ToF sensor 110 is a light receiving element in which light receiving pixels are arranged two-dimensionally, and each pixel corresponds to each position in the measurement target area, so that the ToF light receiving unit 61 can individually receive light from each position in the measurement target area. The imaging device 1001 of this embodiment has four ToF light receiving units 61 arranged facing in different directions. The four ToF light receiving units 61 can capture images 360 degrees around the first direction (Z axis). The multiple light projecting units 21 and the multiple ToF light receiving units 61 are provided around the housing 11 in the first direction (Z axis).

The luminance receiving unit 30 acquires a two-dimensional image using a CMOS sensor 33. The luminance receiving unit 30 includes a CMOS sensor 33 for capturing a luminance image (RGB image) and a luminance receiving optical system 113 (second light receiving optical system) consisting of optical elements that guide incident light to the CMOS sensor 33. The optical elements of the luminance receiving optical system 113 include, for example, a lens.

The imaging device 1001 of this embodiment can obtain a distance image from the light reception data of the ToF light receiving unit 61, and can obtain a luminance image (RGB image) from the light reception data of the luminance light receiving unit 30. The luminance image (RGB image) is mapped to the coordinate point group obtained from the distance image by a processing unit described below. This enables the distance measurement system 3001 to convert distance and shape information of the surrounding space into digital data with color information.

The control unit 120 drives or controls the light projecting unit 21, the ToF light receiving unit 61, and the luminance light receiving unit 30. The control unit 120 is connected to each of the light source 210, the ToF sensor 110, and the CMOS sensor 33 via a cable, FPC, FFC, etc.

Here, the light projecting unit 21 is an example of a first light projecting unit that projects structured light. The ToF light receiving unit 61 is an example of a first light receiving unit to which incident light including structured light is incident. The luminance light receiving unit 30 is an example of a second light receiving unit, and outputs information including at least luminance.

In this embodiment, as shown in FIG. 2 to FIG. 4, the imaging device 1001 has a long, elongated shape in the Z-axis direction. In the first stage of the imaging device 1001, which is the furthest +Z direction, four ToF light receiving optical systems 112, each with an angle of view of 120 degrees or more, are arranged so as to face three directions in the XY plane and one direction, which is the +Z direction. In the second stage, which is the -Z direction side of the first stage of the imaging device 1001, two ToF light projecting systems 111, each with an angle of view of 180 degrees or more, and two luminance light receiving optical systems 113, each with an angle of view of 180 degrees, are arranged. The two ToF light projecting systems 111 face in opposite directions (+X direction and -X direction), and the two luminance light receiving optical systems 113 also face in opposite directions (+Y direction and -Y direction). In the lower stage on the -Z direction side of the imaging device 1001, the control unit 120 and the battery 130 are arranged. This allows for a compact arrangement of optical systems that cover the entire celestial sphere, making it possible to miniaturize the imaging device.

The control unit 120 controls the timing at which the light projection unit 21 projects light, and detects the light received by the ToF light receiving unit 61. First, the control unit 120 controls the timing at which the light source 210 is driven, causing it to emit light toward the area to be measured. Furthermore, the control unit 120 acquires received light data obtained by photoelectric conversion of the light received by the ToF sensor 110. At the same time, the control unit 120 acquires a luminance image obtained by imaging using the CMOS sensor 33.

When a direct ToF sensor is used as the ToF sensor 110, the control unit 120 acquires data including information on the timing of light reception at each pixel, for example. On the other hand, when an indirect ToF sensor is used as the ToF sensor 110, the control unit 120 acquires a phase image based on the amount of light received at each pixel at four different phases, for example. A distance image can be generated from the four phase images by a processing unit described below.

The data acquired by the imaging device 1001 is input to a processing unit 5001 provided in the information processing device 4001. The processing unit 5001 is composed of, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive) of the information processing device 400.

Figure 5 shows an example of a processing unit 5001 that calculates the distance to an object based on light reception data from the ToF light receiving unit 61 in the first embodiment. As shown in Figure 5, the processing unit 5001 realizes the functions of an image storage unit 5010, a distance calculation unit 5011, a correction value calculation unit 5012, a distance correction unit 5013, and an output unit 5014.

The image storage unit 5010 stores the light reception data (multiple phase images) from the ToF light receiving unit 61 in a storage unit such as a RAM. The distance calculation unit 5011 calculates distance information indicating the distance to the target object based on the multiple phase images stored by the image storage unit 5010. The correction value calculation unit 5012 calculates a correction value for correcting the distance information using information obtained, for example, from the area of the directly reflected light of the spot light in the distance image obtained by the distance calculation unit 5011 after the light projection unit 21 is emitted and from the other area (area where the directly reflected light of the spot light does not enter). The distance correction unit 5013 corrects the distance information obtained by activating the light projection unit 21 using the correction value calculated by the correction value calculation unit 5012. Note that the correction value calculation unit 5012 and the distance correction unit 5013 are not essential, but the above-mentioned correction can reduce multipath.

The output unit 5014 outputs distance information indicating the distance to the object corrected by the distance correction unit 5013 to an external device.

Here, we will explain some examples of how the imaging device 1001 can be supported.

Figure 6 is a diagram showing an example of a support form of the imaging device 1001. As shown in Figure 6 (a), the imaging device 1001 has a fixing means 19 (see Figure 2) such as a screw hole for attaching a device 200 such as a tripod to the lower part of the housing 11. As shown in Figure 6 (a), the imaging device 1001 is supported by attaching a device 200 such as a tripod that functions as a support part to the fixing means 19 provided on the housing 11. That is, the imaging device 1001 is arranged in different positions in the first direction (Z axis) of the housing 11 with respect to the ToF light receiving unit 61, the light projecting unit 21, and the support part in this order.

As shown in FIG. 6(b), the imaging device 1001 includes a gripping unit 14 that can be held by a user's hand on the housing 11, below the light projecting unit 21 and the luminance receiving unit 30. As shown in FIG. 6(b), the imaging device 1001 is supported by the user's hand gripping the gripping unit 14 that functions as a support unit.

Here, FIG. 7 is a diagram showing an example of the blind spot of the imaging device 1001. As described above, the imaging device 1001 of this embodiment has multiple ToF light receiving units 61 and light projecting units 21, each facing in different directions. In this way, a distance image of the surroundings centered on the imaging device 1001 can be obtained over a wide range. However, as shown in FIG. 7, the imaging device 1001 has a blind spot that is a non-imaging area in the direction of the light projecting unit 21 and the support unit to prevent the light from the light projecting unit 21 and the support unit from being captured in the ToF light receiving unit 61.

Here, Figure 8-1 shows an example of a desirable dot pattern image, and Figure 8-2 shows an example of an actually obtained dot pattern image (with sparseness and density due to parallax).

When the reflected light of the dot pattern light irradiated from the light projecting unit 21 onto a nearby planar target for distance measurement is received by the ToF sensor 110 in the ToF light receiving unit 61, the image obtained is preferably as shown in FIG. 8-1. However, in reality, particularly when measuring the distance of a nearby target, the parallax between the ToF light receiving unit 61 and the light projecting unit 21 results in a sparse and dense distribution as shown in FIG. 8-2.

Incidentally, as shown in FIG. 6, when the imaging device 1001 is in use, if the ToF light receiving unit 61 is located above the light projecting unit 21, i.e., if the ToF light receiving unit 61 and the light projecting unit 21 cannot be placed in the same location, the dot pattern will be dense at the pixels of the ToF sensor 110 that correspond to the distance measurement points below when viewed from the ToF camera, as in FIG. 8-2.

In areas where the dot pattern is dense like this, the reflected light from multiple dots enters one pixel, making it impossible to resolve each dot, and the weakly illuminated areas disappear as they are filled with areas of medium to strong illumination. In such a state, for example in the device described in Patent Document 1, some of the benefits of the pattern illumination are lost, such as areas without weakly illuminated areas occurring, making it difficult to measure the distance to close-up objects.

Of the dot pattern lights received by the ToF sensor 110 in the ToF light receiving unit 61, if the dot lights received on the ToF sensor 110 are too close to each other, the dots will connect with each other, making the dot density high and the dot pattern will no longer function. This type of area with high dot density is located below the ToF light receiving unit 61 and in the blind spot of the ToF camera, so there is no dot pattern light that is too close within the viewing angle of the ToF camera, and accurate distance information cannot be obtained.

Here we explain the above issues in more detail.

Figure 9-1 is a diagram showing an example of illumination of a nearby planar distance measurement target. As shown in Figure 9-1, for example, consider obtaining a distance image of a planar distance measurement target that is a distance D away in the Y direction from the light receiving surface of the ToF sensor 110 in the ToF light receiving unit 61. In the imaging device 1001, the optical axes of the ToF light receiving unit 61 and the light projecting unit 21 are spaced a distance d apart in the Z direction, and the light projecting unit 21 is disposed on the negative Z side (lower side in the figure) relative to the ToF light receiving unit 61. For example, consider that the dot pattern light emitted from the light projecting unit 21 is emitted at 5 degree intervals. The imaging device 1001 shown in Figure 9-1 is equipped with a device 200 such as a tripod on the negative Z side relative to the light projecting unit 21.

Figure 9-2 is an example diagram showing the pitch between adjacent dots.

Figures 9-2(a) to 9-2(c) are diagrams showing the pitch between adjacent dots when the distance D is changed in the Y direction from the light receiving surface of the ToF sensor 110 in the ToF light receiving unit 61 in the imaging device 1001 shown in Figure 9-1. More specifically, Figures 9-2(a) to 9-2(c) show at what viewing angle the reflected light of each dot of the dot pattern light is received when D/d is set to 0.5, 1, or 2, and how many pixels the pitch between that dot image and the adjacent dot image corresponds to on the ToF sensor 110 in the ToF light receiving unit 61.

As shown in Figures 9-2(a) to 9-2(c), the smaller D/d is, the greater the pitch deviation becomes, and when D/d = 0.5, there are areas where the pitch between adjacent dot images is 1 pixel or less at viewing angles of -60 degrees or less. In such areas, multiple dot images partially overlap on one pixel, and the non-illuminated areas between the dots are lost on the ToF sensor 110 in the ToF light receiving unit 61, and the benefits of pattern illumination are lost as described above.

Here, FIG. 10 is a diagram showing an example of a blind spot of a ToF camera. As described above, in the imaging device 1001 of this embodiment, a blind spot is provided below in FIG. 10 so that the light of the light projecting unit 21 arranged below the ToF light receiving unit 61 does not directly enter the ToF light receiving unit 61, and also for the purpose of preventing the reflection of the equipment 200 such as a tripod. Specifically, as shown in FIG. 10, the imaging device 1001 provides a step 11a in the housing 11. The imaging device 1001 forms a step in the optical system between the ToF light receiving unit 61 and the light projecting unit 21 according to the step 11a provided in the housing 11. The imaging device 1001 blocks incident light with an angle of view of the ToF light receiving unit 61 of -60 degrees or less by the step 11a provided in the housing 11, so that it becomes a blind spot. As a result, the ToF light receiving unit 61 is positioned to avoid the patterned light with the maximum angle of view emitted from the light projecting unit 21.

As shown in FIG. 9-2(c), the imaging device 1001 uses a blind spot of the ToF light receiving unit 61 at or below -60 degrees in the field of view, and in the case of close-range distance measurement with D/d=0.5 or more, it is possible to hide areas where the pitch between adjacent dot images that exist at or below the field of view of the ToF light receiving unit 61 at or below -60 degrees is 1 pixel or less. This removes unresolved dot images from the image received by the ToF light receiving unit 61, eliminating the need to identify areas on the ToF sensor 110 in the ToF light receiving unit 61 where no non-irradiated areas exist and remove them from the data.

Incidentally, the housing 11 of the imaging device 1001 is preferably made of a material that easily absorbs light of wavelengths (for example, visible light of 850 nm, 940 nm, etc.) irradiated from the light-projecting unit 21. The housing 11 of the imaging device 1001 is, for example, an aluminum alloy that has been subjected to a matte black anodizing treatment. By constructing the housing 11 of the imaging device 1001 from a material that easily absorbs light of wavelengths irradiated from the light-projecting unit 21, it is expected that the effect of suppressing the temperature rise of internal heat-generating components by improving heat dissipation to the outside air can be achieved.

The housing 11 of the imaging device 1001 is not limited to the above example, and may be provided with a surface treatment that easily absorbs the wavelength of light emitted from the light projecting unit 21, such as the application of black body tape or the application of black body paint.

In the imaging device 1001 of this embodiment, a blind spot for the ToF light receiving unit 61 is provided by a step 11a provided in the housing 11, but incident light from an angle that is to be a blind spot may be blocked by other methods. For example, the imaging device 1001 shown in FIG. 9-1 may shift the center of the ToF sensor 110 associated with the ToF light receiving unit 61 in the Z direction to prevent incident light from an angle of view in a range that should be a blind spot from entering the light receiving range of the ToF sensor 110.

For example, if you want to obtain a desired distance image at D = 50 mm or more from the imaging device 1001, whose blind spot is the viewing angle of the ToF sensor 110 of -60 degrees or less, the device size of the imaging device 1001 only needs to be D/d ≧ 0.5, so the size should be such that the distance D between the ToF light receiving unit 61 and the light projecting unit 21 is 100 mm or less.

In addition, when the imaging device 1001 is held by the gripping portion 14 as shown in FIG. 6(b), if the gripping portion 14 is a rectangular column with a cross-section of about 40 mm x 30 mm, a cylinder with a diameter of about 40 mm, or an oval shape similar to these, it can be easily and stably held in one hand.

In the imaging device 1001 of this embodiment, the threshold value for the viewing angle that is the blind spot of the ToF light receiving unit 61 is -60 degrees, but this varies depending on the optical design, the target D/d, the pixel size of the ToF sensor 110, etc.

Here, FIG. 11 is a diagram showing the relationship between the ToF viewing angle and D/d. More specifically, FIG. 11 shows D/d when it is assumed that the optical system of the ToF light receiving unit 61 is an fθ lens or equivalent to an fθ lens, and the viewing angle at which the dot pitch is narrowest on the ToF sensor 110 of the ToF light receiving unit 61.

Figures 9-2(a) to 9-2(c) show the change in pitch between adjacent dot images received by the ToF sensor 110 when D/d is 0.5, 1, and 2. Figures 9-2(a) to 9-2(c) show examples where the minimum value is approximately -67 degrees when D/d = 0.5, approximately -57 degrees when D/d = 1, and approximately -50 degrees when D/d = 1. Figure 11 corresponds to the change in pitch that results in the minimum value shown in Figures 9-2(a) to 9-2(c).

Incidentally, in the ToF light receiving unit 61, the viewing angle at which multiple dot lights overlap in a single pixel changes depending on the size of the pixel on the ToF sensor 110. Basically, the dot images are most likely to overlap at the viewing angle at which the pitch between adjacent dot images is smallest, so viewing angles below that angle are blocked.

As shown in Figure 11, the viewing angle at which the pitch between adjacent dot images is smallest basically does not exceed -40 degrees. Although omitted in Figure 11, the viewing angle at which the pitch between adjacent dot images is smallest is approximately -42 degrees even if the distance D is infinite (D/d = ∞). Therefore, basically, the threshold is a maximum of approximately -42 degrees, and blocking angles greater than this tends to unnecessarily reduce the information in the distance image.

In practice, the imaging device 1001 rarely performs distance measurement at an extremely close distance level where D/d<0.5. On the other hand, the imaging device 1001 can often ensure a sufficient number of pixels for the pitch between dots at a level where D/d>5. Therefore, as shown in FIG. 11, the imaging device 1001 sets the recommended threshold angle for blind spots to a range of approximately -67 degrees to -46 degrees.

As described above, according to this embodiment, light corresponding to an excessively high density distribution of patterned structured light (e.g., dot pattern) is present outside the field of view angle actually used, and therefore excessively close dot pattern light does not exist within the field of view angle. This has the effect of eliminating areas where the dot pattern light received by the light receiving means is excessively close within the field of view angle to be measured in a ToF imaging device in which the optical axes of the illumination means that irradiates the dot pattern light and the light receiving means are not the same, thereby making it possible to obtain accurate distance information.

The imaging device 1001 of this embodiment has been described using as an example a tripod or other device 200 (see FIG. 6(a)) or a gripping unit 14 (see FIG. 6(b)) that is held by the user's hand as a support unit, but is not limited to this. FIG. 12 is a diagram showing another example of a support unit for the imaging device 1001. As shown in FIG. 12, for example, a fixing means 230 for a ceiling of a building may be provided as a support unit, and the imaging device 1001 may be used as a distance measuring means for an indoor space.

In addition, in the imaging device 1001 of this embodiment, in order to obtain a distance image of the entire circumference of the device, as shown in FIG. 9-1, the optical elements (ToF light receiving unit 61, luminance light receiving unit 30) are provided symmetrically in both the positive and negative Y directions, but this is not limited to this. Here, FIG. 13 is a diagram showing another arrangement example of the optical elements of the imaging device 1001. As shown in FIG. 13, the imaging device 1001 may be provided with the optical elements (ToF light receiving unit 61, luminance light receiving unit 30) on only one side, for example. As shown in FIG. 13, the imaging device 1001 may be provided with a rotating means 220, which is a rotating table rotated by, for example, an electric motor, in front of the fixing means 19 to which the device 200, such as a tripod, which functions as a support section, is attached. In this case, the imaging device 1001 obtains a distance image of the entire circumference while rotating the device itself by the rotating means 220.

In this embodiment, the light-projecting unit 21 that projects a dot pattern as the irradiation pattern has been given as an example, but this is not limited thereto, and the light-projecting unit 21 may project any patterned light, such as a random dot pattern with an irregular dot arrangement or a stripe pattern.

Second Embodiment
Next, a second embodiment will be described.

The second embodiment differs from the first embodiment in that the light projecting unit 21 is positioned closer to the ToF light receiving unit 61 than the luminance light receiving unit 30. In the following explanation of the second embodiment, the explanation of the same parts as in the first embodiment will be omitted, and only the differences from the first embodiment will be explained.

Fig. 14 is a diagram showing an example of the system configuration of a distance measuring system 3002 including an imaging device 1002 and an information processing device 4002 according to the second embodiment. In the imaging device 1002 of the first embodiment, the light projecting unit 21 and the luminance light receiving unit 30 are arranged at positions equally close to the ToF light receiving unit 61, but this is not limited thereto. As shown in Fig. 14, in the imaging device 1002 of this embodiment, the light projecting unit 21 is arranged at a position closer to the ToF light receiving unit 61 than the luminance light receiving unit 30.

Therefore, according to this embodiment, it is possible to reduce the parallax between the ToF light receiving unit 61 and the light projecting unit 21. As a result, the non-uniformity of the dot distribution (see FIG. 8-2) that occurs when the patterned light irradiated from the light projecting unit 21 is received by the ToF sensor 110 in the ToF light receiving unit 61 is reduced.

Third Embodiment
Next, a third embodiment will be described.

The third embodiment differs from the second embodiment in that it is equipped with uniform lighting that irradiates the entire surrounding area with light. In the following explanation of the third embodiment, the explanation of the same parts as in the second embodiment will be omitted, and only the differences from the second embodiment will be explained.

Fig. 15 is a diagram showing an example of the system configuration of a distance measuring system 3003 including an image capturing device 1003 and an information processing device 4003 according to the third embodiment. As shown in Fig. 15, in addition to the configuration described in the second embodiment, the image capturing device 1003 of this embodiment further includes a uniform illumination 71, which is a second light projecting unit that projects diffuse light of uniform illuminance over the entire periphery of the housing 11, in order to obtain distance information between dots in the dot pattern light projected from the light projecting unit 21.

The imaging device 1003 of this embodiment irradiates the entire surrounding area with diffused light using uniform illumination 71, but the illuminance of the illumination is low relative to the projecting unit 21 that is concentrating the light in the distance image obtained using uniform illumination 71, resulting in reduced distance measurement accuracy at long distances. However, by irradiating the entire surrounding area of the housing 11 with diffused light of uniform illuminance using uniform illumination 71, the imaging device 1003 can use information on the distance and shape of the blank areas between the dot patterns irradiated by the projecting unit 21 as interpolating information.

In this case, as shown in FIG. 15, if the light-projecting unit 21 is disposed immediately adjacent to the ToF light-receiving unit 61, as described in the second embodiment, the unevenness of the dot distribution (see FIG. 8-2) that occurs when the patterned light irradiated from the light-projecting unit 21 is received by the ToF sensor 110 in the ToF light-receiving unit 61 is reduced.

As shown in FIG. 15, the imaging device 1003 of this embodiment is configured with the ToF light receiving unit 61, the light projecting unit 21, the luminance light receiving unit 30, the uniform illumination 71, and the grip 14 arranged in this order. With this configuration, the imaging device 1003 is configured with the light projecting unit 21 and the uniform illumination 71, which tend to generate heat, spaced apart, making it easier to disperse the temperature rise of the imaging device 1003.

In this way, according to this embodiment, it is possible to obtain distance information between dots of the dot pattern light emitted from the light projection unit 21, and also to suppress the temperature rise of the imaging device 1003.

Here, FIG. 16 is a diagram showing a modified example of the imaging device 1003 according to the third embodiment. As shown in FIG. 16, the imaging device 1003 arranges the light projecting unit 21 and the uniform illumination 71 in close proximity to the ToF light receiving unit 61, at a distance equal to or closer to the ToF light receiving unit 61. The imaging device 1003 also arranges the luminance light receiving unit 30 between the grip 14 and the light projecting unit 21 and the uniform illumination 71. With this configuration, the imaging device 1003 has a configuration in which the light projecting unit 21 and the uniform illumination 71, which tend to generate heat, are separated from the grip 14, thereby reducing the risk of burns and the like due to a rise in temperature in parts that are touched by the hand.

Here, FIG. 17 is a diagram showing another modified example of the imaging device 1003 according to the third embodiment. As shown in FIG. 17, the imaging device 1003 has the light projecting unit 21 and the luminance receiving unit 30 disposed at an equally close distance or closer to the ToF light receiving unit 61, and has the uniform illumination 71 disposed between the grip 14 and the light projecting unit 21 and the luminance receiving unit 30. In other words, the imaging device 1003 has both the light projecting unit 21 and the luminance receiving unit 30 disposed close to the ToF light receiving unit 61 to minimize parallax, and has the uniform illumination 71, which is not affected by parallax, disposed the furthest from the ToF light receiving unit 61 to illuminate the entire surrounding area.

In addition, as long as the light projecting unit 21 is positioned at a position equal to or closer to the ToF light receiving unit 61 than the luminance light receiving unit 30 and the uniform illumination 71, the positioning of the luminance light receiving unit 30 and the uniform illumination 71 may be in combinations other than those shown in Figures 15 to 17.

Fig. 18 shows an example of a processing unit 5003 in the third embodiment that calculates the distance to an object based on light reception data from the ToF light receiving unit 61. As shown in Fig. 18, the processing unit 5003 realizes the functions of an image storage unit 5030, a distance calculation unit 5031, a correction value calculation unit 5032, a distance correction unit 5033, and an output unit 5034.

The image storage unit 5030 stores the light reception data (multiple phase images) from the ToF light receiving unit 61 in a storage unit such as a RAM. The distance calculation unit 5031 calculates distance information indicating the distance to the target object based on the multiple phase images stored by the image storage unit 5030. The correction value calculation unit 5032 calculates a correction value for correcting the distance information using distance information 1 obtained by the distance calculation unit 5031 by emitting light from the light projecting unit 21 and distance information 2 obtained by the distance calculation unit 5031 by emitting uniform illumination 71. The distance correction unit 5033 uses the correction value calculated by the correction value calculation unit 5032 to correct the distance information 1 obtained by emitting light from the light projecting unit 21. In this embodiment, instead of the processing unit 5001, the processing unit 5003 of the first embodiment may be used to calculate the correction value and correct the distance. In that case, as described in the first embodiment, multipath reduction can be performed by the correction value calculation unit 5012 and the distance correction unit 5013.

The output unit 5034 outputs distance information indicating the distance to the object corrected by the distance correction unit 5033 to an external device.

(Fourth embodiment)
Next, a fourth embodiment will be described.

The fourth embodiment differs from the first embodiment in that it is configured to reduce fluctuations in the light-projecting unit 21 caused by camera shake. In the following explanation of the fourth embodiment, explanations of the same parts as in the first embodiment will be omitted, and only the differences from the first embodiment will be explained.

Fig. 19 is a diagram showing an example of the system configuration of a distance measuring system 3004 including an imaging device 1004 and an information processing device 4004 according to the fourth embodiment. In the imaging device 1004 of the first embodiment, the light projecting unit 21 and the luminance light receiving unit 30 are arranged at positions equally close to the ToF light receiving unit 61, but this is not limited to the above. As shown in Fig. 19(a), the imaging device 1004 of this embodiment includes the luminance light receiving unit 30 between the ToF light receiving unit 61 and the light projecting unit 21. That is, the imaging device 1004 of this embodiment is configured such that the light projecting unit 21 is arranged near the grip portion 14.

As a result, as shown in FIG. 19(b), the fluctuation of the light-projecting unit 21 due to hand shake is smaller than the fluctuation of the ToF light-receiving unit 61 and the luminance light-receiving unit 30 due to hand shake.

When the imaging device 1004 is held by hand by gripping the gripping portion 14, camera shake may occur. In particular, when multiple images are taken and integrated while the light projection unit 21 is acquiring a single distance image (for example, the average of the results of multiple images is output, or the information is integrated after images are taken with low light source output for close distances and images with high light source output for long distances), the light projection unit 21 is most affected by camera shake.

As an example of the effect of camera shake, the following describes the case of a light projection unit 21 that projects a dot pattern.

Figure 20 is a diagram showing an example of the effects of camera shake. Distance information with a good S/N ratio is obtained only from the points where the dot pattern is projected by the light-projecting unit 21. In this case, the projected points are sparse and localized, so camera shake causes the light-projecting unit 21 to move significantly during shooting. If the projected points of the dot pattern move due to the light-projecting unit 21 moving significantly in this way, as shown in Figure 20, the result may appear as if the shape of the target to be measured has changed significantly. This may result in the shape features of the target not matching in multiple shooting results during the acquisition of a single distance image, and the quality of the integrated distance image may be reduced.

In contrast, with the ToF light receiving unit 61, even if camera shake causes fluctuations in the position of the image seen by the ToF light receiving unit 61, as long as the fluctuation in the illumination position by the light projecting unit 21 is small, the shape information of the object obtained will hardly change, and camera shake correction can be performed later based on the results of feature point extraction, etc.

The luminance light receiving unit 30 can also suppress the effects of camera shake to some extent by using general camera shake correction. Therefore, the light projecting unit 21 is the unit for which you want to suppress fluctuations caused by camera shake as a top priority.

For the above reasons, the imaging device 1004 of this embodiment has the light-projecting unit 21 positioned near the gripping portion 14, where fluctuations due to camera shake are minimal.

In this way, this embodiment can reduce fluctuations in the light-projecting unit 21 caused by camera shake.

Fifth embodiment
Next, a fifth embodiment will be described.

The fifth embodiment differs from the fourth embodiment in that it is equipped with uniform lighting that irradiates the entire surrounding area with light. In the following explanation of the fifth embodiment, the explanation of the same parts as the fourth embodiment will be omitted, and only the differences from the fourth embodiment will be explained.

Fig. 21 is a diagram showing an example of the system configuration of a distance measurement system 3005 including an image capture device 1005 and an information processing device 4005 according to the fifth embodiment. As shown in Fig. 21, in addition to the configuration described in the fourth embodiment, the image capture device 1005 of this embodiment further includes uniform illumination 71 that irradiates light over the entire surrounding area in order to obtain distance information between dots of the dot pattern light irradiated from the light projection unit 21.

The imaging device 1005 of this embodiment irradiates the entire surrounding area with uniform illumination 71, but the illuminance of the illumination is low relative to the projecting unit 21 where the light is focused in the distance image obtained by uniform illumination 71, so the distance measurement accuracy at long distances is reduced. However, by irradiating the entire surrounding area with uniform illumination 71, the imaging device 1005 can use information on the distance and shape of the blank areas between the dot patterns irradiated by the projecting unit 21 as interpolating information.

As shown in FIG. 21, the imaging device 1005 of this embodiment is configured with a ToF light receiving unit 61, a luminance light receiving unit 30, a light projecting unit 21, a uniform illumination 71, and a grip portion 14 arranged in this order.

In this way, according to the imaging device 1005 of this embodiment, by placing the light projecting unit 21 and the uniform lighting 71 at the lowest level among the optical elements, the wiring from the power supply means built into the gripping portion 14 to the light sources inside the light projecting unit 21 and the uniform lighting 71 can be minimized, and light emission delays can be minimized.

Here, FIG. 22 is a diagram showing a modified example of the imaging device 1005 according to the fifth embodiment. As shown in FIG. 22, the imaging device 1005 has the luminance light receiving unit 30 and uniform illumination 71 located immediately adjacent to the ToF light receiving unit 61 and at a distance equal to or closer to the ToF light receiving unit 61. The imaging device 1005 also has the light projecting unit 21 located between the grip 14 and the luminance light receiving unit 30 and uniform illumination 71. With this configuration, the imaging device 1005 has a configuration in which the light projecting unit 21 and uniform illumination 71, which tend to generate heat, are spaced apart, making it easier to disperse the temperature rise of the imaging device 1005.

Here, FIG. 23 is a diagram showing another modified example of the imaging device 1005 according to the fifth embodiment. As shown in FIG. 23, the imaging device 1005 has the ToF light receiving unit 61 and the uniform illumination 71 arranged in that order, and the light projecting unit 21 and the luminance light receiving unit 30 arranged next to the grip part 14. In other words, the imaging device 1005 has a configuration in which both the light projecting unit 21 and the luminance light receiving unit 30 are placed near the grip part 14. This allows the imaging device 1005 to suppress the amount of camera shake not only of the light projecting unit 21 but also of the luminance light receiving unit 30. Furthermore, the imaging device 1005 can reduce the effect of blur due to camera shake even when it is necessary to extend the exposure time of the luminance light receiving unit 30.

In addition, as long as the light projecting unit 21 is positioned at a position equal to or closer to the gripping portion 14 than the luminance receiving unit 30 and the uniform illumination 71, the positioning of the luminance receiving unit 30 and the uniform illumination 71 may be in combinations other than those shown in Figures 21 to 23.

Sixth embodiment
Next, a sixth embodiment will be described.

The sixth embodiment differs from the first to fifth embodiments in that it is configured to provide a shielding means for blocking part of the light projection range of the light projection unit 21 of the imaging device 1006.

Figure 24 is a diagram showing the overlap of patterned light from the light projection unit 21 of the imaging device 1006. When the patterned light is dot pattern light, the dot patterns are spaced apart on surface (1) where the projection ranges do not overlap, but interference between excessively close dot lights is observed at distances farther than surface (2), which is a long distance from the imaging device 1006. On surface (3), the dots overlap exactly, so the dot pattern functions without problems, but on surfaces (4) and (5), etc., interference between excessively close dot lights impairs the function of the dot pattern.

Therefore, in this embodiment, to avoid the above problem, a shielding means is provided to block part of the patterned light from the light projection unit 21 so that the light projection ranges do not overlap in the desired imaging area.

Figure 25 is a diagram showing an example of the blocking means according to the sixth embodiment. As an example of the blocking means, a light-blocking shape 11b can be added to the housing 11 as shown in Figure 25(b) for the configuration shown in Figure 25(a). This blocks a part of the patterned light projected upward from the light-projecting unit 21 (the part shown as "light-blocking range" in Figure 25(b)). By setting the light-blocking shape 11b to an arbitrary protrusion amount, the overlapping range of the patterned light can be changed, and problems associated with overlapping of the projection range of the patterned light at a desired distance can be avoided. In this configuration, the protrusion amount of the light-blocking shape 11b on both the left and right sides is less than the length at which the light distribution angle θ of the light-projecting unit 21 is 90 degrees or less. Note that when θ = 90 degrees, the patterned light does not overlap up to infinity.

Fig. 26 is a diagram showing another example of the blocking means according to the sixth embodiment. Fig. 26 shows a cross section of the periphery of the light projecting unit 21 of the imaging device 1006. As another example of the blocking means, as shown in Fig. 26, a ring-shaped member 12 can be added to the outer periphery of the lens barrel of the optical system 21a. This ring-shaped member 12 blocks part of the light projection range of the light projecting unit 21.

In this example, it is possible to adjust the range over which the patterned light is blocked by moving the ring-shaped member 12 in the direction of the optical axis of the light projecting unit 21. Also, for example, as shown in FIG. 26, if the inner circumference of the ring-shaped member 12 is female-threaded and the outer circumference of the lens barrel of the optical system 21a is male-threaded, the position of the ring moves back and forth in the direction of the optical axis as the ring rotates, making it easy to fine-tune the range over which the patterned light is blocked.

Figure 27 is a diagram showing another example of the blocking means according to the sixth embodiment. As shown in Figure 27, instead of fixing the ring-shaped member 12 directly to the outer periphery of the lens barrel of the optical system 21a, a cylindrical protrusion is provided on the outer periphery of the light-projecting unit 21 on the side of the housing 11 to which the light-projecting unit 21 is fixed, and the ring-shaped member 12 is fixed to this protrusion, and the same effect can be obtained.

Fig. 28 is a diagram showing another example of the blocking means according to the sixth embodiment. Fig. 28 is a diagram showing the light projecting unit 21 as viewed from the optical axis direction. As yet another example of the blocking means, as shown in Fig. 28, an aperture blade member 13 can be added inside the optical system 21a. This aperture blade member 13 blocks part of the light projection range of the light projecting unit 21.

The aperture blade member 13 can adjust the size of the opening 21b by opening and closing it. When the aperture blade member 13 is opened, the opening 21b becomes larger as shown in FIG. 28(a), and when the aperture blade member 13 is closed, the opening 21b becomes smaller as shown in FIG. 28(b). In this way, by opening and closing the aperture blade member 13, it is possible to adjust the size of the light blocking range at the outer edge of the projection range of the light projection unit 21.

Seventh embodiment
Next, a seventh embodiment will be described.

The seventh embodiment differs from the first to sixth embodiments in that the imaging device 1007 is a distance measuring device equipped with a processing unit 5007, and the imaging device 1007 constitutes a distance measuring system 3007. In the following explanation of the seventh embodiment, explanations of the same parts as the first to sixth embodiments will be omitted, and only differences from the first to sixth embodiments will be explained.

Fig. 29 is a diagram showing an example of the configuration of an imaging device 1007 (distance measurement system 3007) according to the seventh embodiment. As shown in Fig. 29, the imaging device 1007 is a distance measurement device including a processing unit 5007.

The processing unit 5007 is configured by, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), etc. of the imaging device 1007. The function of the processing unit 5007 is similar to that of the processing unit 5001 of the information processing device 4001 in the first embodiment.

As described above, according to this embodiment, in a ToF imaging device in which the optical axis of the illumination means that irradiates the dot pattern light and the light receiving means are not the same, it is possible to eliminate areas in which the dot pattern light received by the light receiving means is excessively close in the angle of view that is the subject of distance measurement, and to output accurate distance information.

In this embodiment, the imaging device 1003 equipped with the light projection unit 21 and uniform illumination 71 described in the third embodiment is configured to include a processing unit 5007, but this is not limited thereto, and the imaging device 1001 equipped with the light projection unit 21 described in the first embodiment may be configured to include a processing unit 5007.

Although each embodiment of the present invention has been described above, the above-mentioned embodiments are presented as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These new embodiments and modifications are included in the scope and gist of the invention, as well as in the scope of the invention and its equivalents described in the claims. Furthermore, components from different embodiments and modifications may be combined as appropriate.

REFERENCE SIGNS LIST 11 Housing 11a Step 11b Light-shielding shape 12 Ring-shaped member 13 Diaphragm blade member 14, 200, 230 Support portion 21 First light-projecting portion 21a Optical system 21b Opening 30 Second light-receiving portion 61 First light-receiving portion 71 Second light-projecting portion 1001 Imaging device, distance measuring device 3001 Distance measuring system 4001 Information processing device 5001 Processing portion

JP 2019-113530 A

Claims (11)

1. An imaging device, comprising:
A housing and
A first light projecting unit that projects structured light;
a first light receiving unit to which incident light including the structured light from the first light projecting unit is incident;
a support portion on which the imaging device is supported;
Equipped with
the first light receiving unit, the first light projecting unit, and the support unit are arranged at different positions in the first direction of the housing in the order of the first light receiving unit, the first light projecting unit, and the support unit;
1. An imaging device comprising: The first light receiving unit is capable of capturing an image of 360 degrees around the first direction.
2. The imaging device according to claim 1 . The first light-emitting unit and the first light-receiving unit are each provided in a plurality of units,
The plurality of first light emitting units and the plurality of first light receiving units are provided around an axis of the housing in the first direction.
2. The imaging device according to claim 1 . the first light receiving unit has a non-imaging region,
The non-imaging region includes the first light projecting portion and the support portion.
2. The imaging device according to claim 1 . the housing has a step between the first light emitting unit and the first light receiving unit,
The step forms a blind spot of the first light receiving unit and prevents the structured light from being directly incident on the first light receiving unit from the first light projecting unit.
2. The imaging device according to claim 1 . A second light receiving unit that captures a luminance image is further provided,
The first light projecting unit is disposed at a position equal to or closer to the first light receiving unit than the second light receiving unit.
2. The imaging device according to claim 1 . a second light projecting unit that projects diffuse light of uniform illuminance onto the entire periphery of the housing;
The first light projecting unit is disposed closer to the first light receiving unit than the second light projecting unit and the second light receiving unit.
7. The imaging device according to claim 6. A second light receiving unit that captures a luminance image is further provided,
The first light projecting unit is disposed at a position equal to or closer to the support unit than the second light receiving unit.
2. The imaging device according to claim 1 . a second light projecting unit that projects diffuse light of uniform illuminance onto the entire periphery of the housing;
The first light-projecting unit is disposed at a position at least as close to the support unit as the second light-projecting unit and the second light-receiving unit.
7. The imaging device according to claim 6. A plurality of the first light projecting units;
and a shielding means,
a light-projection range of one of the plurality of first light-projecting units includes an overlapping region that overlaps with the light-projection range of another of the first light-projecting units,
The shielding means shields at least a portion of the light emitted toward the overlap region.
2. The imaging device according to claim 1 . An imaging device according to any one of claims 1 to 10;
A processing unit that calculates a distance to an object based on the light reception data from the first light receiving unit;
A ranging system comprising:

Images