CN115248440A - TOF Depth Camera Based on Lattice Light Casting - Google Patents

Abstract

The invention provides a TOF depth camera based on dot matrix light projection, which comprises: a structured light projector for projecting structured light toward a target; the infrared camera is used for collecting an infrared speckle image and a plurality of infrared reference images; the processor module is used for generating a speckle depth map according to the phase difference of multi-frame infrared speckle images, the speckle depth map comprises the infrared speckle images and depth data, a corresponding infrared reference image is determined according to the depth data, a reference area corresponding to the speckle points in the infrared reference image is determined according to pixel coordinates of the scattered points, each sub-area in the reference area is matched with the corresponding speckle point area to determine a target sub-area with the highest correlation degree, the central offset between the target sub-area and the reference area is determined, and when the offset is larger than a preset threshold value, the speckle point area is determined to be a multipath interference point. The invention can detect mirror image multipath, remove multipath interference and realize the application of the TOF depth camera based on speckle dot matrix projection to the sweeping robot.

Description

TOF Depth Camera Based on Lattice Light Casting

technical field

The present invention relates to a depth camera, in particular, to a TOF depth camera based on dot matrix light projection.

Background technique

The Time of Flight (TOF) depth camera calculates the distance by emitting a flood beam of a specific band, and then uses the sensor to receive the reflected beam of the object in the measured space and measure the flight time of the beam in space to calculate the distance, thereby obtaining the measured space. depth image. TOF depth cameras can simultaneously obtain grayscale images and depth images, and are widely used in 3D depth vision related gesture recognition, face recognition, 3D modeling, somatosensory games, machine vision, assisted focus, security, automatic driving and other technical fields.

The traditional TOF depth camera assumes that the received beam is only reflected once in the target scene, but there is always a specular or diffuse reflection material surface in the actual scene, which will reflect the incident light in all directions. In this way, the TOF sensor receives It may be the superposition of the single reflected beam and the multiple reflected beam, which interferes with the accuracy of the TOF depth camera's distance measurement, an effect called multipath interference.

The traditional TOF depth camera usually includes a light projector and a light receiving sensor. The light projector emits a flood beam to the space to provide illumination, the light receiving sensor receives the reflected flood beam for imaging, and the depth calculation device transmits and receives the light. The phase delay of , calculates the flight time, and then obtains the distance information. There are some limitations to measuring depth in this way. For example, the interference of ambient light will affect the accuracy of the measurement, especially when the intensity of ambient light is higher than that of the directly reflected light, resulting in the signal received by the light-receiving sensor is mainly ambient light. The scene is mirrored a few more.

Specular multipath is a frequently encountered problem in the sweeping robot scene. The ground is a tile with high reflectivity. The signals received by the pixels corresponding to the points on the ground are direct reflection (primary path) and multiple reflections from objects (secondary path). superposition of optical signals. For highly reflective objects, the intensity of the primary diameter is much lower than that of the secondary diameter, causing the measured ground depth to deviate from the actual depth.

SUMMARY OF THE INVENTION

In view of the defects in the prior art, the purpose of the present invention is to provide a TOF depth camera based on dot matrix light projection.

According to the TOF depth camera based on dot matrix light projection provided by the present invention, 1. A TOF depth camera based on dot matrix light projection is characterized in that, it includes the following modules:

Structured light projector for projecting structured light to the target;

an infrared camera, configured to receive the structured light reflected by the target to generate an infrared speckle image and a plurality of infrared reference images, wherein the infrared reference images are collected and generated from a reference target at a plurality of different distances by a depth camera;

The processor module is configured to generate a speckle depth map of the target according to the phase difference of the multi-frame infrared speckle images, the speckle depth map includes the infrared speckle image and the depth data of each speckle in the infrared speckle image, and The depth data of each speckle determines the corresponding infrared reference image, and then determines the reference area corresponding to the speckle in the infrared reference image according to the pixel coordinates of the speckle, and associates each sub-area in the reference area with the corresponding reference area. The speckle area is matched to determine the target sub-area with the highest correlation, and the center offset between the target sub-area and the reference area is determined. When the offset is greater than a preset threshold, determine the The speckle area is a multipath interference point, and then the multipath interference point in the infrared speckle image is removed to generate a target speckle image.

Preferably, when determining the reference area corresponding to the speckle in the infrared reference image, the following steps are included:

Step M1: obtain the depth data of each speckle, and determine the distance between the target surface point corresponding to the speckle and the depth camera according to the depth data;

Step M2: determine the infrared reference image collected at the same distance according to the distance;

Step M3: Determine the reference area corresponding to the speckle in the corresponding infrared reference image according to the pixel coordinates of the speckle.

Preferably, when determining that the speckle is a multipath interference point, the following steps are included:

Step N1: Calculate the correlation between the speckle area and a sub-area in the upper left corner of the corresponding reference area;

Step N2: according to the order from left to right and top to bottom, each time the sub-area is moved by one pixel, the correlation between the sub-area and the speckle area is calculated, until the target sub-area with the highest correlation is determined;

Step N3: Determine the center offset between the target sub-region and the speckle area, and when the offset is greater than a preset threshold, determine that the speckle is a multipath interference point.

Preferably, the calculation method of the correlation degree r is as follows:

为参考区域的平均幅度；B_mn为散斑点区域的幅度，

为散斑点区域的平均幅度；m、n为像素坐标范围。Among them, A _mn is the amplitude of the reference area,

is the average amplitude of the reference area; B _mn is the amplitude of the speckle area,

is the average amplitude of the speckle area; m, n are the pixel coordinate range.

Preferably, the acquisition of the infrared reference image includes the following steps:

Step S101: projecting lattice light to the reference target through the structured light projector at a distance;

Step S102: generating an infrared reference image by receiving the lattice light reflected by the reference target by the infrared camera;

Step S103: Repeat steps S101 to S102 to obtain infrared reference images at different distances.

Preferably, the structured light projector includes a light source, a light source driver and a light modulator;

the light source driver is connected to the light source, and is used for driving the light source to emit light;

The light modulator is used for modulating the projected light of the light source into structured light and then projecting the structured light to a target.

Preferably, the infrared camera comprises a lens, an optical filter and an image sensor arranged along the optical path, the image sensor is provided with at least four receiving windows; the pulse width of the receiving windows is larger or smaller than that of the structured light Pulse Width;

The image sensor is configured to receive at least four light signals of the structured light through at least four receiving windows; the at least four receiving windows are sequentially arranged in time sequence, and then according to the light received by each receiving window The signal generates each of the speckle depth maps.

Preferably, the structured light is lattice light; the lattice light is distributed in the following preset shapes: linear, triangle, quadrangle, circle, hexagon, pentagon, random arrangement, spatial encoding arrangements and quasi-lattice arrangements.

Preferably, the size of the speckle area can be set as a 5×5 pixel area, and the speckle area includes a speckle;

The size of the reference area may be set to an 8×8 pixel area; the threshold value may be set to two pixels.

The TOF depth camera based on dot matrix light projection provided according to the present invention includes the following modules:

Structured light projector for projecting structured light to the target;

Infrared camera for collecting infrared speckle images;

a memory for storing multiple infrared reference images, the infrared reference images are generated by pre-collecting a reference target at multiple different distances by the depth camera,

The processor module is configured to generate a speckle depth map of the target according to the phase difference of the multi-frame infrared speckle images, the speckle depth map includes the infrared speckle image and the depth data of each speckle in the infrared speckle image, and The depth data of each speckle determines the corresponding infrared reference image, and then determines the reference area corresponding to the speckle in the infrared reference image according to the pixel coordinates of the speckle, and associates each sub-area in the reference area with the corresponding reference area. The speckle area is matched to determine the target sub-area with the highest correlation, and the center offset between the target sub-area and the reference area is determined. When the offset is greater than a preset threshold, determine the The speckle area is a multipath interference point, and then the multipath interference point in the infrared speckle image is removed to generate a target speckle image.

Compared with the prior art, the present invention has the following beneficial effects:

In the present invention, the offset of each speckle in the collected infrared speckle image is determined by the infrared reference image, and when the offset is greater than a preset threshold, the speckle area is determined to be a multipath interference point, and then The target speckle image is generated by removing the multipath interference points in the infrared speckle image, so that the mirror image multipath can be detected, and the multipath interference can be removed, so as to realize the application of the TOF depth camera based on speckle array projection on the sweeping robot.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to the provided drawings without creative work. Other features, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

1 is a schematic diagram of a module of a TOF depth camera based on dot matrix light projection in an embodiment of the present invention;

2 is a flowchart of steps for determining a reference area corresponding to a speckle in an embodiment of the present invention;

3 is a flow chart of steps for determining a speckle as a multipath interference point in an embodiment of the present invention;

FIG. 4 is a flowchart of steps for collecting an infrared reference image in an embodiment of the present invention;

5 is a schematic diagram of a module of a structured light projector according to an embodiment of the present invention;

6 is a schematic diagram of a module of an infrared camera in an embodiment of the present invention;

Figure 7 (a), (b), (c) are schematic diagrams of aperiodic arrangement of lattice light in an embodiment of the present invention;

FIG. 8 is a schematic diagram of a module of a TOF depth camera based on dot matrix light projection in a modification of the present invention.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances so that the embodiments of the invention described herein can, for example, be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

The technical solutions of the present invention will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

The TOF depth camera based on dot matrix light projection provided by the present invention aims to solve the problems existing in the prior art.

The technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

1 is a schematic diagram of a module of a TOF depth camera based on dot matrix light projection in an embodiment of the present invention. As shown in FIG. 1 , the TOF depth camera based on dot matrix light projection provided by the present invention includes the following modules:

Structured light projector for projecting structured light to the target;

an infrared camera, configured to receive the structured light reflected by the target to generate an infrared speckle image and a plurality of infrared reference images, wherein the infrared reference images are collected and generated from a reference target at a plurality of different distances by a depth camera;

The processor module is configured to generate a speckle depth map of the target according to the phase difference of the multi-frame infrared speckle images, the speckle depth map includes the infrared speckle image and the depth data of each speckle in the infrared speckle image, and The depth data of each speckle determines the corresponding infrared reference image, and then determines the reference area corresponding to the speckle in the infrared reference image according to the pixel coordinates of the speckle, and associates each sub-area in the reference area with the corresponding reference area. The speckle area is matched to determine the target sub-area with the highest correlation, and the center offset between the target sub-area and the reference area is determined. When the offset is greater than a preset threshold, determine the The speckle area is a multipath interference point, and then the multipath interference point in the infrared speckle image is removed to generate a target speckle image.

In the embodiment of the present invention, the infrared camera is an infrared detector, and the lattice light reflected by the target person is received by the infrared detector.

The infrared speckle image is collected by the depth camera at a distance of 30 to 80 cm from the target person. The depth camera adopts a TOF camera, and the light projector of the TOF camera can project a lattice light to the target.

In this embodiment of the present invention, it is possible to choose to directly delete the multipath interference points to generate the target speckle image.

In this embodiment of the present invention, an infrared reference image is used to determine the offset of each speckle in the collected infrared speckle image, and when the offset is greater than a preset threshold, it is determined that the speckle area is a multipath interference point , and then remove the multipath interference points in the infrared speckle image to generate the target speckle image, so that the mirror multipath can be detected, and the multipath interference can be removed to realize the TOF depth camera based on speckle array projection on the sweeping robot. application.

FIG. 2 is a flowchart of steps for determining a reference area corresponding to a speckle in an embodiment of the present invention. As shown in FIG. 2 , when determining the reference area corresponding to the speckle in the infrared reference image, the following steps are included:

Step M1: obtain the depth data of each speckle, and determine the distance between the target surface point corresponding to the speckle and the depth camera according to the depth data;

Step M2: determine the infrared reference image collected at the same distance according to the distance;

Step M3: Determine the reference area corresponding to the speckle in the corresponding infrared reference image according to the pixel coordinates of the speckle.

In the embodiment of the present invention, each speckle is extracted from the infrared speckle image, the distance is determined according to the depth data of the speckle, and then the corresponding infrared reference image is determined.

And according to the pixel coordinates of each scattered spot, the reference area of the corresponding position is determined in the infrared reference image.

FIG. 3 is a flowchart of steps for determining a speckle as a multipath interference point in an embodiment of the present invention. As shown in FIG. 3 , when determining the speckle as a multipath interference point, the following steps are included:

Step N1: Calculate the correlation between the speckle area and a sub-area in the upper left corner of the corresponding reference area;

Step N2: according to the order from left to right and top to bottom, each time the sub-area is moved by one pixel, the correlation between the sub-area and the speckle area is calculated, until the target sub-area with the highest correlation is determined;

Step N3: Determine the center offset between the target sub-region and the speckle area, and when the offset is greater than a preset threshold, determine that the speckle is a multipath interference point.

In this embodiment of the present invention, the size of the speckle area may be set as a 5×5 pixel area, and the pixel area includes one speckle; the size of the reference area may be set as an 8×8 pixel area; the The threshold may be set to two pixels, that is, when the center offset is greater than or equal to two pixels, the speckle is determined to be a multipath interference point.

In this embodiment of the present invention, the speckle area is traversed through each pixel in the reference area in a pixel-by-pixel method in an order from left to right and from top to bottom.

In the embodiment of the present invention, the calculation method of the correlation degree r is as follows:

为参考区域的平均幅度；B_mn为散斑点区域的幅度，

为散斑点区域的平均幅度；m、n为像素坐标范围。Among them, A _mn is the amplitude of the reference area,

is the average amplitude of the reference area; B _mn is the amplitude of the speckle area,

is the average amplitude of the speckle area; m, n are the pixel coordinate range.

In this embodiment of the present invention, the amplitude may also be represented by any physical quantity among gray value, pixel value, illuminance, luminous flux, and radiation power.

FIG. 4 is a flowchart of steps for collecting an infrared reference image in an embodiment of the present invention. As shown in FIG. 4 , the infrared reference image collection includes the following steps:

Step S101: projecting lattice light to the reference target through the structured light projector at a distance;

Step S102: generating an infrared reference image by receiving the lattice light reflected by the reference target by the infrared camera;

Step S103: Repeat steps S101 to S102 to obtain infrared reference images at different distances.

In this embodiment of the present invention, a plurality of infrared reference images at different distances may be pre-collected and stored in a memory for retrieval and use by the depth camera. The reference target can be a checkerboard or a flat plate or the like.

FIG. 5 is a schematic diagram of a module of a structured light projector in an embodiment of the present invention. As shown in FIG. 5 , the structured light projector includes a light source, a light source driver, and a light modulator;

the light source driver is connected to the light source, and is used for driving the light source to emit light;

The light modulator is used for modulating the projected light of the light source into structured light and then projecting the structured light to a target.

In the embodiment of the present invention, the light modulator adopts a diffraction grating (DOE) or a spatial light modulator (SLM).

FIG. 6 is a schematic diagram of a module of an infrared camera in an embodiment of the present invention. As shown in FIG. 6 , the infrared camera includes a lens, a filter, and an image sensor arranged along an optical path, and the image sensor is provided with at least four receivers. a window; the pulse width of the receiving window is larger or smaller than the pulse width of the structured light;

The image sensor is configured to receive at least four light signals of the structured light through at least four receiving windows; the at least four receiving windows are sequentially arranged in time sequence, and then according to the light received by each receiving window The signal generates each of the speckle depth maps.

The image sensor includes a plurality of photodetectors distributed in an array;

The lens adopts an optical imaging lens, which is used to make the direction vector of the collimated light beam entering the photodetector array through the lens to be in a one-to-one correspondence with the photodetector;

The light detector is used for receiving the collimated light beam reflected by the target object.

In the embodiment of the present invention, in order to filter background noise, a narrow-band filter is usually installed in the optical imaging lens, so that the photodetector array can only pass the incident collimated light beam with a preset wavelength. The preset wavelength may be the wavelength of the incident collimated beam, such as 950 nanometers, or may be between 50 nanometers smaller than the incident collimated beam and 50 nanometers greater than the incident collimated beam. The photodetector arrays may be arranged periodically or aperiodically. The photodetector array can be a combination of a plurality of single-point photodetectors or a sensor chip integrating a plurality of photodetectors according to the requirement of the discrete lattice light quantity. In order to further optimize the sensitivity of the photodetector, the illumination spot of a discrete lattice light on the target person may correspond to one or more photodetectors. When multiple photodetectors correspond to the same illumination spot, the signals of each detector can be connected through a circuit, so that they can be combined into a photodetector with a larger detection area.

In this embodiment of the present invention, the light detector may use a CMOS light sensor, a CCD light sensor, or a SPAD light sensor.

In the embodiment of the present invention, the structured light is lattice light; the lattice light is distributed in the following preset shapes: linear, triangle, quadrangle, circle, hexagon, pentagon, random arrangement fabrics, spatially encoded arrangements, and quasi-lattice arrangements.

Figures 7(a), (b), and (c) are schematic diagrams of aperiodic arrangement of lattice light in an embodiment of the present invention. As shown in Figure 7(a), the spatial encoding arrangement is specifically in the In the periodic arrangement, a part of the beams are defaulted to realize the spatial encoding of the arrangement position. The actual encoding that can be used is not limited to the example in Fig. 7(a); as shown in Fig. 7(b), the Random arrangement, specifically, the arrangement of collimated beams is randomly distributed, so that the similarity of the arrangement of different positions is small or close to zero, as shown in Figure 7(c), the quasi-lattice arrangement, specifically The collimated beams are arranged aperiodically at close proximity and periodically at long distances. Since the implementation of the present invention is limited by the optical system, the arrangement of the actual collimated beam in the cross-section may be distorted, such as stretching and twisting. The energy distribution of each collimated beam in the cross section can be circular, annular or elliptical or other shapes. In this arrangement as shown in 7, this arrangement is conducive to uniform sampling of non-deterministic targets and optimizes the effect of the final 3D depth map.

8 is a schematic diagram of a module of a TOF depth camera based on dot matrix light projection in a modification of the present invention. As shown in FIG. 8 , the TOF depth camera based on dot matrix light projection provided by the present invention includes the following modules:

Structured light projector for projecting structured light to the target;

an infrared camera, configured to receive the structured light reflected by the target to generate an infrared speckle image;

a memory for storing multiple infrared reference images, the infrared reference images are generated by pre-collecting a reference target at multiple different distances by the depth camera,

The processor module is configured to generate a speckle depth map of the target according to the phase difference of the multi-frame infrared speckle images, the speckle depth map includes the infrared speckle image and the depth data of each speckle in the infrared speckle image, and The depth data of each speckle determines the corresponding infrared reference image, and then determines the reference area corresponding to the speckle in the infrared reference image according to the pixel coordinates of the speckle, and associates each sub-area in the reference area with the corresponding reference area. The speckle area is matched to determine the target sub-area with the highest correlation, and the center offset between the target sub-area and the reference area is determined. When the offset is greater than a preset threshold, determine the The speckle area is a multipath interference point, and then the multipath interference point in the infrared speckle image is removed to generate a target speckle image.

In a modification of the present invention, a plurality of infrared reference images at different distances may be pre-collected and stored in a memory for retrieval and use by the processor module.

In the embodiment of the present invention, the offset of each speckle in the collected infrared speckle image is determined by using the infrared reference image, and when the offset is greater than a preset threshold, it is determined that the speckle area is multipath interference and then remove the multipath interference points in the infrared speckle image to generate the target speckle image, so that the mirror image multipath can be detected, and the multipath interference can be removed, so as to realize the TOF depth camera based on speckle array projection on the sweeping robot Applications.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above-mentioned specific embodiments, and those skilled in the art can make various variations or modifications within the scope of the claims, which do not affect the essential content of the present invention.

Claims (10)

1. A TOF depth camera based on lattice light projection, comprising the following modules:

a structured light projector for projecting structured light toward a target;

the infrared camera is used for receiving the structured light reflected by the target to generate an infrared speckle image and a plurality of infrared reference images, and the infrared reference images are generated by collecting a reference target at a plurality of different distances through the depth camera;

the processor module is used for generating a speckle depth map of a target according to the phase difference of a plurality of frames of infrared speckle images, wherein the speckle depth map comprises an infrared speckle image and depth data of each scattered spot in the infrared speckle image, determining a corresponding infrared reference image according to the depth data of each scattered spot, further determining a reference area corresponding to the speckle point in the infrared reference image according to pixel coordinates of the speckle point, matching each sub-area in the reference area with the corresponding scattered spot area to determine a target sub-area with the highest correlation degree, determining the central offset of the target sub-area and the reference area, determining the speckle point area as a multipath interference point when the offset is greater than a preset threshold, and further removing the multipath interference point in the infrared speckle image to generate the target speckle image.

2. The lattice light projection based TOF depth camera of claim 1, comprising, in determining a corresponding reference region of the speckle pattern in the infrared reference image, the steps of:

step M1: acquiring depth data of each scattered spot, and determining the distance between a target surface point corresponding to the scattered spot and a depth camera according to the depth data;

step M2: determining the infrared reference images collected at the same distance according to the distance;

step M3: and determining a corresponding reference area corresponding to the speckle point in the infrared reference image according to the pixel coordinate of the speckle point.

3. The lattice light projection based TOF depth camera of claim 1, wherein in determining the speckle pattern as a multipath interference point, comprising the steps of:

step N1: calculating the correlation degree of the scattered spot area and a sub-area of the upper left corner of the corresponding reference area;

and step N2: moving the sub-region by one pixel each time according to the sequence from left to right and from top to bottom, and then calculating the correlation degree of the sub-region and the speckle point region until a target sub-region with the highest correlation degree is determined;

and step N3: and determining the central offset of the target sub-region and the speckle point region, and determining the speckle point as a multipath interference point when the offset is greater than a preset threshold value.

4. The lattice light projection based TOF depth camera according to claim 3, wherein the correlation r is calculated as follows:

wherein A is _mn Is the amplitude of the reference region or regions,

is the average amplitude of the reference region; b _mn Is the amplitude of the speckle point area and,

the average amplitude of the speckle point area is obtained; m and n are pixel coordinate ranges.

5. The lattice light projection based TOF depth camera of claim 1, wherein said infrared reference image acquisition comprises the steps of:

step S101: projecting lattice light at a distance through the structured light projector toward the reference target;

step S102: receiving the dot matrix light reflected by the reference target by the infrared camera to generate an infrared reference image;

step S103: and repeatedly executing the step S101 to the step S102 to acquire the infrared reference images at a plurality of different distances.

6. The lattice light projection based TOF depth camera of claim 1 wherein the structured light projector comprises a light source, a light source driver, and a light modulator;

the light source driver is connected with the light source and used for driving the light source to emit light;

the light modulator is used for modulating the projected light of the light source into structured light and projecting the structured light to a target.

7. The dot matrix light projection based TOF depth camera according to claim 1, wherein the infrared camera comprises a lens, a filter and an image sensor arranged along an optical path, the image sensor being provided with at least four of the receiving windows; the pulse width of the receiving window is larger or smaller than the pulse width of the structured light;

the image sensor is used for receiving light signals of at least four structured lights through at least four receiving windows; the at least four receiving windows are sequentially arranged in time sequence, and then each speckle depth map is generated according to the optical signals from each receiving window.

8. The lattice light projection based TOF depth camera of claim 1, wherein the structured light is lattice light; the lattice light is distributed in the following preset shape: linear, triangular, quadrilateral, circular, hexagonal, pentagonal, randomly arranged, spatially encoded and quasi-lattice arrangements.

9. The dot matrix light projection based TOF depth camera of claim 1, wherein the size of the speckle point area can be set to 5 x 5 pixel area, the speckle point area comprising a scattered dot;

the size of the reference region may be set to 8 × 8 pixel regions; the threshold is set to two pixels.

10. A TOF depth camera based on lattice light projection, comprising the following modules:

a structured light projector for projecting structured light toward a target;

the infrared camera is used for receiving the structured light reflected by the target to generate an infrared speckle image;

a memory for storing a plurality of infrared reference images generated by the depth camera pre-acquiring a reference target at a plurality of different distances,

the processor module is used for generating a speckle depth map of a target according to the phase difference of a plurality of frames of infrared speckle images, wherein the speckle depth map comprises an infrared speckle image and depth data of each scattered spot in the infrared speckle image, determining a corresponding infrared reference image according to the depth data of each scattered spot, further determining a reference area corresponding to the speckle point in the infrared reference image according to pixel coordinates of the speckle point, matching each sub-area in the reference area with the corresponding scattered spot area to determine a target sub-area with the highest correlation degree, determining the central offset of the target sub-area and the reference area, determining the speckle point area as a multipath interference point when the offset is greater than a preset threshold, and further removing the multipath interference point in the infrared speckle image to generate the target speckle image.

Images