CN117589302B - Three-dimensional high-speed compression imaging method and system - Google Patents

Abstract

The present application proposes a three-dimensional high-speed compression imaging method and system, which uses a stripe projector to sequentially project N projection stripes of different phases toward a three-dimensional object to obtain an imaging image, wherein the number of cycles N of the stripe projector is set to be equal to the image compression ratio M; each imaging image reaches a spatial light modulator through an imaging lens and is encoded to obtain a corresponding encoded image, wherein M different binary masks are loaded on the spatial light modulator; the N encoded images are collected by a camera through a telecentric lens with a single exposure to obtain an encoded compressed image, and a three-dimensional compression reconstruction algorithm is used to perform three-dimensional reconstruction on the encoded compressed image to obtain a reconstructed three-dimensional high-speed image, thereby overcoming the problem that the original projection three-dimensional imaging device cannot be applied to the shooting of high-speed moving three-dimensional object scenes, and realizing the rapid imaging of three-dimensional objects.

Description

A three-dimensional high-speed compression imaging method and system

Technical Field

The present application relates to the field of compression imaging technology, and in particular to a three-dimensional high-speed compression imaging method and system.

Background Art

Three-dimensional imaging technology refers to the technology of obtaining the three-dimensional coordinates of each point on the surface of an object and other surface characteristics. It has broad application prospects in industrial manufacturing and testing, biomedical testing, public safety monitoring and consumer electronics. Fringe projection three-dimensional imaging technology is one of the important technologies. Due to its advantages of non-contact, high resolution, high efficiency and low cost, it has become one of the main methods of three-dimensional imaging.

The current fringe projection 3D imaging technology uses a projector based on a spatial light modulator to perform fringe projection and a high-speed camera to collect fringe images. By projecting sinusoidal fringes whose phase is linearly related to the projector's object coordinates, the phase of the fringes is used to encode the projector's object coordinates, and the fringe projection reconstruction algorithm is used to perform camera unwrapping and 3D image reconstruction on the collected image. However, the frame rate of the projected fringes by the spatial light modulator-based projector can reach more than 20,000fps, but the camera's acquisition frame rate is generally 10-800fps, which is much lower than the frame rate of the projected fringes by the projector. As a result, the speed of fringe projection 3D imaging is greatly restricted by the camera, which seriously limits the practical application of fringe projection 3D imaging technology in high-speed scenes.

Summary of the invention

The embodiments of the present application provide a three-dimensional high-speed compressed imaging method and system. In view of the problem that the speed of the fringe projection three-dimensional imaging technology proposed in the current technology is limited by the camera acquisition frame rate, a three-dimensional high-speed compressed imaging method is proposed, which can realize the imaging of three-dimensional high-speed scenes.

In a first aspect, the present application provides a three-dimensional high-speed compressed imaging method, comprising: a stripe projector sequentially projects N projection stripes of different phases toward a three-dimensional object to obtain an imaging image, wherein the number of cycles N of the stripe projector is set to be equal to the image compression ratio M; each imaging image reaches a spatial light modulator through an imaging lens and is encoded to obtain a corresponding encoded image, wherein M different binary masks are loaded on the spatial light modulator; the N encoded images are collected by a camera through a telecentric lens with a single exposure to obtain an encoded compressed image; and a three-dimensional compression reconstruction algorithm is used to perform three-dimensional reconstruction on the encoded compressed image to obtain a reconstructed three-dimensional high-speed image.

In a second aspect, the present application provides a three-dimensional high-speed compression imaging system, comprising: a three-dimensional high-speed compression imaging device, wherein the three-dimensional high-speed compression imaging device comprises a stripe projector, an imaging lens, a camera, a telecentric lens, a spatial light modulator and a control unit, a three-dimensional imaging space for placing a three-dimensional object is formed between the stripe projector and the imaging lens, the imaging lens, the spatial light modulator, the telecentric lens and the camera are arranged in sequence along the imaging light path, the stripe projector sequentially projects N projection stripes of different phases toward the three-dimensional object to obtain an imaging image, wherein the number of cycles N of the stripe projector is set to be the same as the image compression ratio M, each imaging image reaches the spatial light modulator through the imaging lens and is encoded to obtain a corresponding encoded image, wherein M different binary masks are loaded on the spatial light modulator, and the N encoded images are collected by the camera through a single exposure through the telecentric lens to obtain an encoded compressed image;

The computing unit is used to perform three-dimensional reconstruction on the coded compressed image by using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image.

In a third aspect, the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes the three-dimensional high-speed compression imaging method according to the above-mentioned method.

The main contributions and innovations of the present invention are as follows:

Compared with the prior art, the present application introduces an image compression acquisition module based on the original projection stripe three-dimensional imaging device, uses N projection stripes projected by a stripe projector to act on a three-dimensional object, and then uses a spatial light modulator to encode the three-dimensional object in a high-speed motion scene to obtain a coded image, which is then acquired by a camera to obtain a coded compressed image, and uses an optimized three-dimensional high-speed compression imaging reconstruction algorithm to perform three-dimensional reconstruction on the acquired coded compressed image to obtain a three-dimensional high-speed image. In this way, the limitation of imaging rate caused by the camera frame rate limitation is overcome, and the scheme can be used for fast imaging of three-dimensional objects in high-speed motion scenes.

Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

FIG1 is a schematic diagram of a framework of a three-dimensional high-speed compression imaging device according to an embodiment of the present application;

FIG2 is a schematic diagram of imaging of a three-dimensional high-speed compression imaging method according to an embodiment of the present application;

FIG3 is a schematic diagram of a reconstruction algorithm of a three-dimensional high-speed compression imaging method according to an embodiment of the present application;

FIG. 4 is a schematic flow chart of a three-dimensional high-speed compression imaging method according to an embodiment of the present application.

In the accompanying drawings: 1- three-dimensional object, 2- fringe projector, 3- imaging lens, 4- camera, 5- telecentric lens, 7- spatial light modulator, 8- control unit.

DETAILED DESCRIPTION

Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are only examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the attached claims.

It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

Since the imaging speed of the traditional projection stripe 3D imaging device is greatly restricted by the camera acquisition frame rate, the traditional projection stripe 3D imaging device cannot be well adapted to the 3D imaging of high-speed scenes. In view of this practical problem, the present invention introduces an image compression acquisition module based on the traditional projection 3D imaging device, uses N projection stripes projected by a stripe projector to act on a 3D object, and then uses a spatial light modulator to encode the 3D object in the high-speed motion scene to obtain an encoded image and collect it by the camera to obtain an encoded compressed image, and uses an optimized designed 3D high-speed compressed imaging reconstruction algorithm to perform 3D reconstruction on the collected encoded compressed image to obtain a 3D high-speed image.

Embodiment 1

As shown in FIG1 , the present solution provides a three-dimensional high-speed compression imaging device, which includes: a stripe projector 2, an imaging lens 3, a camera 4, a telecentric lens 5, a spatial light modulator 7, and a control unit 8. A three-dimensional imaging space for placing three-dimensional objects is formed between the stripe projector 2 and the imaging lens 3. The imaging lens 3, the spatial light modulator 7, the telecentric lens 5, and the camera 4 are sequentially arranged along the imaging optical path, and the control unit 8 is communicatively connected to the stripe projector 2 and the spatial light modulator 7. The three-dimensional high-speed compression imaging device provided by the present solution adds an image compression acquisition module composed of an imaging lens 3 and a spatial light modulator 7 to the traditional projection three-dimensional imaging device. The setting of the image compression acquisition module overcomes the problem that the original camera frame rate is difficult to match high-speed scene shooting.

Correspondingly, as shown in FIG4 , the present solution provides a three-dimensional high-speed compression imaging method, which is implemented based on the three-dimensional high-speed compression imaging device mentioned above, and includes the following steps:

The fringe projector 2 sequentially projects N projection fringes of different phases toward the three-dimensional object to obtain an imaging image, wherein the number of cycles N of the fringe projector 2 is set to be equal to the image compression ratio M;

Each imaging image reaches the spatial light modulator 7 through the imaging lens 3 and is encoded to obtain a corresponding encoded image, wherein the spatial light modulator 7 is loaded with M different binary masks;

N coded images are collected by the camera 4 through a single exposure through the telecentric lens 5 to obtain a coded compressed image;

The coded compressed image is three-dimensionally reconstructed using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image.

It should be noted that, each time the fringe projector 2 projects a phased projection fringe toward the three-dimensional object to obtain an imaging image, the spatial light modulator is triggered to encode once. In other words, each imaging image is encoded to obtain an encoded image. When the spatial light modulator 7 completes the encoding of the M encoded images in a single cycle, the camera 4 is triggered to collect multiple encoded images in a single cycle by single exposure to obtain a coded compressed image, and the encoding is cyclically coded in sequence until the shooting of the high-speed motion scene when the three-dimensional object is in high-speed motion is completed.

In the step of "the stripe projector 2 sequentially projects N projection stripes of different phases toward the three-dimensional object to obtain an imaging image", the stripe projector 2 is set to sequentially project projection stripes toward the three-dimensional object in a projection cycle pattern, the number of projection stripes in each projection cycle is N and the phases of the projection stripes in the same projection cycle are different from each other.

In the actual shooting process, the three-dimensional object is placed in the three-dimensional imaging space between the fringe projector 2 and the imaging lens 3 for imaging shooting.

As shown in FIG2 , in a specific embodiment, the number of cycles of the fringe projector 2 is set to 4. At this time, each projection cycle contains four projection fringes with different phases and phase shifts of 0, π/2, π, and 3π/2. The reason why the number of cycles is set to be the same as the image compression ratio in this solution is to ensure that a cycle of fringe projection is completed in each coded compressed image.

In the step of "each imaging image reaches the spatial light modulator 7 through the imaging lens 3 and is encoded to obtain a corresponding encoded image, wherein the spatial light modulator 7 is loaded with M different binary masks", the stripe projector 2 is set to trigger the spatial light modulator 7 to encode once each time it projects a projection stripe, and each imaging image in a single projection cycle corresponds to a binary code. The stripe projector 2 is controlled by the control unit 8 to realize stripe projection and control the triggering of the spatial light modulator 7.

As shown in FIG. 2 , in a specific embodiment, projection fringes of four phases are projected onto a three-dimensional object to obtain four imaging images, and each imaging image is encoded corresponding to a binary mask on the spatial light modulator 7 to obtain a corresponding encoded image.

In the step of "N coded images are collected by the camera 4 through a single exposure through the telecentric lens 5 to obtain a coded compressed image", each time the spatial light modulator 7 completes the image encoding in a projection cycle to obtain N coded images, the camera 4 is triggered to expose and collect the N coded images to obtain a coded compressed image. The triggering of the camera 4 is controlled by the control unit 8.

As shown in FIG. 2 , in a specific embodiment, the camera 4 captures four coded images to obtain one coded compressed image.

As shown in Figure 3, in the step of "the control unit 8 uses a three-dimensional compression reconstruction algorithm to perform three-dimensional reconstruction on the coded compressed image to obtain a reconstructed three-dimensional high-speed image", the coded compressed image is input into a trained three-dimensional image reconstruction model, wherein the three-dimensional image reconstruction model decodes the coded compressed image to obtain N decoded images, the N decoded images are phase unwrapped to obtain different phases, and three-dimensional reconstruction is performed based on the different phases to obtain a reconstructed three-dimensional high-speed image.

In other words, the coded compressed image of the present invention is reconstructed by a three-dimensional compression reconstruction method using a trained three-dimensional image reconstruction model.

It should be noted that the three-dimensional image reconstruction model mentioned in this scheme consists of three parts: a compressed image reconstruction algorithm, a deep learning camera unwrapping algorithm and a three-dimensional image reconstruction algorithm, wherein the compressed image reconstruction algorithm is used to decode the encoded compressed image to obtain N decoded images, and the compressed image reconstruction algorithm can be trained using existing video images without the need for additional data annotation; wherein the deep learning camera unwrapping algorithm is used to phase unwrapped N decoded images to obtain different phases, wherein the deep learning camera unwrapping algorithm uses the stripe image and the corresponding phase unwrapped image for one-to-one training, and the training model can be a U-Net model; wherein the three-dimensional image reconstruction algorithm is used to perform three-dimensional reconstruction based on different phases to obtain a reconstructed three-dimensional high-speed image, which directly calculates the three-dimensional structure through position calibration and mathematical modeling to generate a three-dimensional high-speed image.

It should be noted that this solution can be used for imaging a single three-dimensional object, and can also realize imaging of high-speed motion scenes of three-dimensional objects through circular projection imaging. Since this solution is additionally provided with a spatial light modulator, the three-dimensional high-speed compression imaging method provided by this solution can quickly capture moving three-dimensional objects.

Embodiment 2

Based on the same concept, the present application also proposes a three-dimensional high-speed compression imaging system, comprising:

A three-dimensional high-speed compression imaging device, wherein the three-dimensional high-speed compression imaging device comprises a stripe projector 2, an imaging lens 3, a camera 4, a telecentric lens 5, a spatial light modulator 7 and a control unit 8, a three-dimensional imaging space for placing a three-dimensional object is formed between the stripe projector 2 and the imaging lens 3, the imaging lens 3, the spatial light modulator 7, the telecentric lens 5 and the camera 4 are arranged in sequence along the imaging light path, the stripe projector 2 sequentially projects N projection stripes of different phases toward the three-dimensional object located in the three-dimensional imaging space to obtain an imaging image, wherein the number of cycles N of the stripe projector 2 is set to be the same as the image compression ratio M, each imaging image reaches the spatial light modulator 7 through the imaging lens 3 and is encoded to obtain a corresponding encoded image, wherein M different binary masks are loaded on the spatial light modulator 7, and the N encoded images are collected by the camera 4 through a single exposure through the telecentric lens 5 to obtain an encoded compressed image;

The computing unit is used to perform three-dimensional reconstruction on the coded compressed image by using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image.

The contents of the three-dimensional high-speed compression imaging system that are the same as those in the first embodiment will not be described in detail here.

Embodiment 3

This embodiment further provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process. The process includes the three-dimensional high-speed compression imaging method according to the first embodiment:

The fringe projector 2 sequentially projects N projection fringes of different phases toward a three-dimensional object located in a three-dimensional imaging space to obtain an imaging image, wherein the number of cycles N of the fringe projector 2 is set to be equal to the image compression ratio M;

Each imaging image reaches the spatial light modulator 7 through the imaging lens 3 and is encoded to obtain a corresponding encoded image, wherein the spatial light modulator 7 is loaded with M different binary masks;

N coded images are collected by the camera 4 through a single exposure through the telecentric lens 5 to obtain a coded compressed image;

The coded compressed image is three-dimensionally reconstructed using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image.

It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

Embodiments of the present invention can be implemented by computer software, which is executable by a data processor of a mobile device, such as in a processor entity, or implemented by hardware, or implemented by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and/or macros can be stored in any device readable data storage medium, and they include program instructions for performing specific tasks. Computer program products can include one or more computer executable components configured to perform embodiments when the program is running. One or more computer executable components can be at least one software code or a part thereof. In addition, at this point, it should be noted that any box of the logic flow in the figure can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored in physical media such as memory chips or storage blocks implemented in processors, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and data variants thereof, CDs. Physical media are non-transient media.

Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (10)

1. A three-dimensional high-speed compression imaging method, comprising: The fringe projector sequentially projects N projection fringes of different phases toward the three-dimensional object to obtain an imaging image, wherein the number of cycles N of the fringe projector is set to be equal to the image compression ratio M; Each imaging image reaches the spatial light modulator through the imaging lens and is encoded to obtain a corresponding encoded image, where M different binary masks are loaded on the spatial light modulator; N coded images are collected by the camera in a single exposure through a telecentric lens to obtain a coded compressed image; The coded compressed image is three-dimensionally reconstructed using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image. 2. The three-dimensional high-speed compression imaging method according to claim 1 is characterized in that the fringe projector projects projection fringes onto the three-dimensional object in sequence according to a projection cycle, the number of projection fringes in each projection cycle is N, and the phases of the projection fringes in the same projection cycle are different from each other. 3. The three-dimensional high-speed compressed imaging method according to claim 2 is characterized in that the fringe projector triggers the spatial light modulator to perform encoding each time it projects a projection fringe, and each imaging image in a single projection cycle corresponds to a binary mask. 4. The three-dimensional high-speed compressed imaging method according to claim 2 is characterized in that each time the spatial light modulator completes image encoding within a projection cycle to obtain N encoded images, the camera is triggered to expose and collect the N encoded images to obtain a encoded compressed image. 5. The three-dimensional high-speed compressed imaging method according to claim 1 is characterized in that the coded compressed image is input into a trained three-dimensional image reconstruction model, wherein the three-dimensional image reconstruction model decodes the coded compressed image to obtain N decoded images, the N decoded images are phase unwrapped to obtain different phases, and three-dimensional reconstruction is performed based on the different phases to obtain a reconstructed three-dimensional high-speed image. 6. The three-dimensional high-speed compression imaging method according to claim 2 is characterized in that a fringe projector cyclically projects images to obtain a plurality of three-dimensional high-speed images, which are applied to the shooting of a motion scene in which a three-dimensional object is in high-speed motion. 7. The three-dimensional high-speed compressed imaging method according to claim 5 is characterized in that the three-dimensional image reconstruction model consists of three parts: a compressed image reconstruction algorithm, a deep learning camera unwrapping algorithm and a three-dimensional image reconstruction algorithm, wherein the compressed image reconstruction algorithm is used to decode the encoded compressed image to obtain N decoded images, the deep learning camera unwrapping algorithm is used to phase unwrapping the N decoded images to obtain different phases, the three-dimensional image reconstruction algorithm is used to perform three-dimensional reconstruction based on different phases to obtain a reconstructed three-dimensional high-speed image, and the three-dimensional structure is directly calculated through position calibration and mathematical modeling to generate a three-dimensional high-speed image. 8. A three-dimensional high-speed compression imaging system, comprising: A three-dimensional high-speed compression imaging device, wherein the three-dimensional high-speed compression imaging device comprises a fringe projector, an imaging lens, a camera, a telecentric lens, a spatial light modulator and a control unit, a three-dimensional imaging space for placing a three-dimensional object is formed between the fringe projector and the imaging lens, the imaging lens, the spatial light modulator, the telecentric lens and the camera are arranged in sequence along the imaging light path, the fringe projector sequentially projects N projection fringes of different phases toward the three-dimensional object to obtain an imaging image, wherein the number of cycles N of the fringe projector is set to be the same as the image compression ratio M, each imaging image reaches the spatial light modulator through the imaging lens and is encoded to obtain a corresponding encoded image, wherein M different binary masks are loaded on the spatial light modulator, and the N encoded images are collected by the camera through a single exposure through the telecentric lens to obtain an encoded compressed image; The computing unit is used to perform three-dimensional reconstruction on the coded compressed image by using a three-dimensional compression reconstruction algorithm to obtain a reconstructed three-dimensional high-speed image. 9. The three-dimensional high-speed compressed imaging system according to claim 8 is characterized in that the coded compressed image is input into a trained three-dimensional image reconstruction model, wherein the three-dimensional image reconstruction model decodes the coded compressed image to obtain N decoded images, the N decoded images are phase unwrapped to obtain different phases, and three-dimensional reconstruction is performed based on the different phases to obtain a reconstructed three-dimensional high-speed image. 10. A readable storage medium, characterized in that a computer program is stored in the readable storage medium, the computer program includes a program code for controlling a process to execute a process, and the process includes the three-dimensional high-speed compression imaging method according to any one of claims 1 to 7.