CN114740472B - A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar - Google Patents
A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar Download PDFInfo
- Publication number
- CN114740472B CN114740472B CN202210312335.1A CN202210312335A CN114740472B CN 114740472 B CN114740472 B CN 114740472B CN 202210312335 A CN202210312335 A CN 202210312335A CN 114740472 B CN114740472 B CN 114740472B
- Authority
- CN
- China
- Prior art keywords
- dimensional
- neural network
- network model
- column vector
- scene
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 56
- 238000003062 neural network model Methods 0.000 claims abstract description 100
- 238000012549 training Methods 0.000 claims abstract description 56
- 238000000034 method Methods 0.000 claims abstract description 24
- 239000013598 vector Substances 0.000 claims description 94
- 238000001514 detection method Methods 0.000 claims description 44
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 230000001360 synchronised effect Effects 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 6
- 238000013480 data collection Methods 0.000 claims description 6
- 230000006870 function Effects 0.000 abstract description 11
- 238000009825 accumulation Methods 0.000 abstract description 6
- 238000013135 deep learning Methods 0.000 abstract description 4
- 238000004590 computer program Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
- G01S13/88—Radar or analogous systems specially adapted for specific applications
- G01S13/89—Radar or analogous systems specially adapted for specific applications for mapping or imaging
- G01S13/90—Radar or analogous systems specially adapted for specific applications for mapping or imaging using synthetic aperture techniques, e.g. synthetic aperture radar [SAR] techniques
- G01S13/904—SAR modes
- G01S13/9043—Forward-looking SAR
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/90—Lidar systems specially adapted for specific applications for mapping or imaging using synthetic aperture techniques
Landscapes
- Engineering & Computer Science (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- General Physics & Mathematics (AREA)
- Image Analysis (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
本申请涉及一种无扫描单通道太赫兹雷达前视三维成像方法和系统。所述方法包括：通过单发单收太赫兹雷达获取一维距离像，通过飞行时间相机采集三维深度图像，以大量一维距离像为输入，对应的三维深度图像为输出，对神经网络模型进行训练，得到训练好的神经网络模型；获取待检测场景的一维距离像，输入训练好的神经网络模型中，根据训练好的神经网络模型的输出得到待检测场景的三维图像。本发明仅使用单发单收的单通道太赫兹雷达，避免天线阵列的使用和波前调制，不依赖孔径积累或相对运动实现无扫描单通道前视三维成像，极大地简化系统、降低成本；提出基于深度学习的前视三维成像算法，提高了系统的成像效率，能够满足高分辨高帧率成像需求。
The present application relates to a method and system for forward-looking three-dimensional imaging of a scanless single-channel terahertz radar. The method comprises: obtaining a one-dimensional range image through a single-transmitting and single-receiving terahertz radar, collecting a three-dimensional depth image through a time-of-flight camera, taking a large number of one-dimensional range images as input and the corresponding three-dimensional depth images as output, training a neural network model to obtain a trained neural network model; obtaining a one-dimensional range image of a scene to be detected, inputting it into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the output of the trained neural network model. The present invention only uses a single-channel terahertz radar with a single transmit and single receive function, avoids the use of an antenna array and wavefront modulation, does not rely on aperture accumulation or relative motion to achieve scanless single-channel forward-looking three-dimensional imaging, greatly simplifies the system and reduces costs; proposes a forward-looking three-dimensional imaging algorithm based on deep learning, improves the imaging efficiency of the system, and can meet the requirements of high-resolution and high-frame rate imaging.
Description
Technical Field
The application relates to the technical field of terahertz radar three-dimensional imaging, in particular to a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method and system.
Background
The term terahertz is formally appearing in 1970 and refers to electromagnetic waves with the frequency of 0.1-10 THz. Terahertz waves are in the transition frequency band of electronics to optics. Compared with a microwave radar, the terahertz radar has the advantages of short wavelength, large bandwidth and extremely high spatial resolution, so that the imaging resolution is extremely high, and the target characteristics can be finely depicted.
In the fields of forward looking and gaze imaging, such as aperture coding imaging, electromagnetic vortex imaging and the like, the pattern and wavefront space of electromagnetic waves are modulated to obtain rich equivalent irradiation modes, so that the target information in echo is more abundant, and super-resolution imaging is carried out according to the pattern and wavefront space. However, the above methods either require relative motion between the radar and the target or require modulation of electromagnetic waves or wavefront space, which complicates the imaging system, and all have respective limitations. In addition, the existing radar forward-looking imaging technology uses more array systems, scanning systems and wavefront space modulation systems. The radar imaging system of the array system and the wavefront space modulation system has complex structure and high cost, while the radar imaging system of the scanning system needs long-time data accumulation process and is difficult to realize high frame rate imaging. Therefore, the prior art has the problem of poor adaptability.
Disclosure of Invention
Based on this, there is a need to provide a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method, system, computer device and storage medium capable of simplifying a radar imaging system while satisfying the requirement of high-resolution high-frame-rate imaging.
A scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method, the method comprising:
Respectively controlling a single-shot terahertz radar receiving and transmitting signal through two paths of synchronous clock signals to obtain a one-dimensional range profile of a detection scene, controlling a time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set consisting of a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
Converting a one-dimensional range profile in a training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by taking the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional range profile as output to obtain a trained neural network model;
And obtaining a one-dimensional range profile of the scene to be detected, converting the one-dimensional range profile into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
In one embodiment, the method further comprises the steps of controlling the single-shot terahertz radar to acquire signals in a trigger mode through a first clock signal, so that the single-shot terahertz radar can receive and transmit signals and store data when detecting one falling edge, and a one-dimensional range profile of a detection scene is obtained;
the method comprises the steps of controlling a time-of-flight camera to acquire signals in a trigger mode through a second clock signal, so that the time-of-flight camera acquires and stores data when detecting a falling edge, and obtaining a three-dimensional depth image corresponding to a detection scene, wherein the three-dimensional depth image comprises the abscissa and the ordinate of each point in the detection scene and depth information;
the first clock signal is kept synchronized with the second clock signal.
In one embodiment, the method further comprises the steps of converting a one-dimensional range profile in a training set into column vectors, inputting the column vectors into a pre-designed neural network model, training the neural network model by taking the column vectors obtained by flattening the three-dimensional depth image corresponding to the one-dimensional range profile as output, and obtaining a trained neural network model, wherein the neural network is a neural network of a multi-layer perceptron, the neural network of the multi-layer perceptron comprises an input layer, three fully connected layers and an output layer, the fully connected layers are used for connecting each point of input data with each point of output data, and an activation function layer is arranged behind each fully connected layer.
In one embodiment, the method further comprises the steps of converting one-dimensional range profiles in a training set into column vectors and inputting the column vectors into a pre-designed neural network model;
Flattening M multiplied by N dimensions of the three-dimensional depth image corresponding to the one-dimensional distance image in the transverse and longitudinal directions into a column vector with the length of MN;
And training the neural network model by taking the column vector with the length of MN as the output of the neural network model to obtain a trained neural network model.
In one embodiment, the method further comprises the step of recovering the column vector with the length of MN output by the trained neural network model into M multiplied by N dimensions to obtain the three-dimensional image of the scene to be detected.
A scanless single channel terahertz radar forward-looking three-dimensional imaging system, the system comprising:
the data collection module is used for respectively controlling the single-shot terahertz radar receiving and transmitting signals to obtain a one-dimensional range profile of a detection scene through two paths of synchronous clock signals, controlling the time-of-flight camera to acquire signals to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set formed by a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
The model training module is used for converting one-dimensional range profiles in a training set into column vectors, inputting the column vectors into a pre-designed neural network model, and training the neural network model by taking the column vectors obtained by flattening the three-dimensional depth image corresponding to the one-dimensional range profiles as output to obtain a trained neural network model;
The model application module is used for acquiring a one-dimensional distance image of the scene to be detected, converting the one-dimensional distance image into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
A computer device comprising a memory storing a computer program and a processor which when executing the computer program performs the steps of:
Respectively controlling a single-shot terahertz radar receiving and transmitting signal through two paths of synchronous clock signals to obtain a one-dimensional range profile of a detection scene, controlling a time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set consisting of a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
Converting a one-dimensional range profile in a training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by taking the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional range profile as output to obtain a trained neural network model;
And obtaining a one-dimensional range profile of the scene to be detected, converting the one-dimensional range profile into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
Respectively controlling a single-shot terahertz radar receiving and transmitting signal through two paths of synchronous clock signals to obtain a one-dimensional range profile of a detection scene, controlling a time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set consisting of a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
Converting a one-dimensional range profile in a training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by taking the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional range profile as output to obtain a trained neural network model;
And obtaining a one-dimensional range profile of the scene to be detected, converting the one-dimensional range profile into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
The scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method and system are characterized in that a one-dimensional range profile of a detection scene is obtained through receiving terahertz radar receiving and transmitting signals in a single mode, three-dimensional depth images are acquired through a time-of-flight camera to obtain a training set formed by one-dimensional range profile and three-dimensional depth image data pairs, a large number of one-dimensional range profiles are taken as input, corresponding three-dimensional depth images are taken as output, a neural network model is trained to obtain a trained neural network model, the one-dimensional range profile of the scene to be detected is obtained, the one-dimensional range profile of the scene to be detected is input into the trained neural network model, and a three-dimensional image of the scene to be detected is obtained according to the output of the trained neural network model. The invention only uses single-shot single-received single-channel terahertz radar, avoids the use of an antenna array and wave front modulation, realizes scanning-free single-channel forward-looking three-dimensional imaging without relying on aperture accumulation or relative movement, greatly simplifies the system and reduces the cost, and provides a forward-looking three-dimensional imaging algorithm based on deep learning, thereby improving the imaging efficiency of the system and meeting the imaging requirement of high resolution and high frame rate.
Drawings
Fig. 1 is an application scenario diagram of a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method in one embodiment;
FIG. 2 is a flow diagram of a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method in one embodiment;
FIG. 3 is a multi-layer perceptron model framework used in one embodiment;
FIG. 4 is a block diagram of a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging system in one embodiment;
FIG. 5 is a block diagram of a scanning-free single-channel terahertz radar front-view three-dimensional imaging system in one embodiment;
Fig. 6 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method provided by the application can be applied to an application environment shown in fig. 1. Under the forward-looking imaging condition, the terahertz radar system transmits a linear frequency modulation pulse waveform and acquires a one-dimensional range profile of a scene, and the time-of-flight camera is used for simultaneously acquiring depth information of the whole scene, namely a real three-dimensional image of the scene. And controlling the terahertz radar system and the time-of-flight camera to synchronously work by controlling two paths of synchronous clock signals separated from the processing terminal, so as to obtain a one-dimensional range profile-depth image pair, wherein the former serves as input for training a neural network model, and the latter serves as output. The control and processing terminal may be, but is not limited to, various personal computers, notebook computers, tablet computers, and portable wearable devices.
In one embodiment, as shown in fig. 2, a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method is provided, and the method is applied to the control and processing terminal in fig. 1 for illustration, and includes the following steps:
step 202, respectively controlling a single-shot terahertz radar receiving and transmitting signal to obtain a one-dimensional range profile of a detection scene through two paths of synchronous clock signals, controlling a time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set formed by a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene.
The invention only uses single-shot single-received single-channel terahertz radar, can avoid the use of an antenna array and wave front modulation, realizes scanning-free single-channel forward-looking three-dimensional imaging without relying on aperture accumulation or relative movement, and greatly simplifies the system and reduces the cost.
The terahertz radar transmits a linear frequency modulation pulse signal to a target scene, and the general expression form of the signal is as follows:
Where w (t) is the real envelope, The phase is modulated for the signal. For standard LFM signals, there are:
where T p denotes the pulse width of the signal and K denotes the tone frequency.
Let the received echo delay be t 0, then the echo expression is:
sr(t)＝s(t-t0)＝rect((t-t0)/Tp)exp(jπK(t-t0)2) (3)
the spectrum is in the form of:
According to the set chirp characteristics, the time domain and frequency domain responses of the matched filter can be obtained as follows:
after matched filtering, the frequency domain expression of the output signal is:
The Fourier transform of the rectangular window function is converted into a sine function, and the time domain expression of the output signal after matched filtering can be obtained according to the property of the Fourier transform, wherein the time domain expression is as follows:
sout(t)＝IFFT[Sout(f)]＝Tpsinc[πKTp(t-t0)] (7)
By observing the above equation, it can be known that the output signal has a peak at time t=t 0 according to the property of the sinc function, and the position information of the detected target, that is, the one-dimensional distance profile can be obtained by multiplying t 0 by c/2.
The time-of-flight camera may measure distances between itself and objects within the detection scene, thereby obtaining 3D information of the scene.
And 204, converting the one-dimensional range profile in the training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by taking the column vector obtained by flattening the three-dimensional depth image corresponding to the one-dimensional range profile as output to obtain a trained neural network model.
Neural network models are typically trained using back-propagation algorithms. And by adjusting the parameters, the loss function is reduced to obtain an optimal network model.
And 206, obtaining a one-dimensional range profile of the scene to be detected, converting the one-dimensional range profile into a column vector, inputting the column vector into a trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
Because the radar imaging system of the array system and the wavefront space modulation system has complex structure, the cost is higher, and aiming at the defect, the invention uses the radar system with single-shot and no wavefront modulation, thereby greatly simplifying the system and reducing the cost; the radar imaging system of the scanning system needs a long-time data accumulation process, and high-frame-rate imaging is difficult to realize, and aiming at the defect, the invention provides a forward-looking three-dimensional imaging algorithm based on deep learning. The invention realizes a scanning-free single-channel terahertz radar forward-looking three-dimensional imaging system which has the advantages of simple and small system and low cost and can meet the imaging requirement of high resolution and high frame rate.
In the scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method, a one-dimensional range profile of a detection scene is obtained through receiving terahertz radar receiving and transmitting signals in a single mode, a three-dimensional depth image is acquired through a time-of-flight camera to obtain a training set formed by one-dimensional range profile and three-dimensional depth image data pairs, a large number of one-dimensional range profiles are taken as input, corresponding three-dimensional depth images are taken as output, a neural network model is trained to obtain a trained neural network model, the one-dimensional range profile of the scene to be detected is obtained, the one-dimensional range profile of the scene to be detected is input into the trained neural network model, and a three-dimensional image of the scene to be detected is obtained according to the output of the trained neural network model. The invention only uses single-shot single-received single-channel terahertz radar, avoids the use of an antenna array and wave front modulation, realizes scanning-free single-channel forward-looking three-dimensional imaging without relying on aperture accumulation or relative movement, greatly simplifies the system and reduces the cost, and provides a forward-looking three-dimensional imaging algorithm based on deep learning, thereby improving the imaging efficiency of the system and meeting the imaging requirement of high resolution and high frame rate.
In one embodiment, the method further comprises the steps of controlling the single-shot terahertz radar to acquire signals in a trigger mode through a first clock signal, enabling the single-shot terahertz radar to transmit and receive a signal and store data when detecting one falling edge, obtaining a one-dimensional range profile of a detection scene, controlling the time-of-flight camera to acquire signals in the trigger mode through a second clock signal, enabling the time-of-flight camera to acquire and store data when detecting one falling edge, obtaining a three-dimensional depth image corresponding to the detection scene, wherein the three-dimensional depth image comprises the horizontal coordinates, the vertical coordinates and depth information of each point in the detection scene, and the first clock signal and the second clock signal are kept synchronous.
In one embodiment, the method further comprises the steps of converting one-dimensional range profiles in a training set into column vectors, inputting the column vectors into a pre-designed neural network model, training the neural network model by taking the column vectors obtained by flattening three-dimensional depth images corresponding to the one-dimensional range profiles as output, and obtaining a trained neural network model, wherein the neural network is a neural network of a multi-layer perceptron, the neural network of the multi-layer perceptron comprises an input layer, three full-connection layers and an output layer, the full-connection layers are used for connecting each point of input data with each point of output data, the number of nodes of the three full-connection layers is 1024,512,256, an activation function layer is arranged behind each full-connection layer, and an activation function used is a tanh function.
In particular, the multi-layer perceptron model framework used in the present invention is shown in FIG. 3. The multi-layer perceptron (Multilayer Perceptron, MLP) is an artificial neural network (ARTIFICIAL NEURAL NETWORK, ANN) in the form of a forward structure that maps a set of input vectors into a set of output vectors. It can be abstracted into a directed graph, which is composed of multiple well-defined layers of nodes, each layer of nodes being fully connected to the next adjacent layer of nodes. In MLP, each node of the remaining layers, except the input layer, is provided with an activation function, which is typically trained using a back propagation algorithm. Analyzing its structure, it can be known that it can connect each point of input data with each point of output data. And each distance unit for acquiring the one-dimensional distance image by the single-shot single-received terahertz radar contains information from all imaging grids of the three-dimensional scene. It can be seen that the two have some similarity in structure.
In one embodiment, the method further comprises the steps of converting one-dimensional distance images in a training set into column vectors and inputting the column vectors into a pre-designed neural network model, flattening M multiplied by N dimensions of a three-dimensional depth image corresponding to the one-dimensional distance images in the transverse and longitudinal directions into column vectors with the length of MN, and training the neural network model by taking the column vectors with the length of MN as output of the neural network model to obtain a trained neural network model.
Specifically, data acquired by the single-shot terahertz radar and the time-of-flight camera are subjected to certain preprocessing to acquire main information of the data. Then, each one-dimensional range profile is stored in the form of a column vector, each depth image (with the dimension of MxN) is flattened into a column vector (with the dimension of 1 xMN) in the same way, the two are in one-to-one correspondence, and each pair of data is a group. 90% of all data sets were randomly chosen as training set, the remaining 10% as test set. And taking the one-dimensional range profile in each group of data as an input of a training neural network, and taking a column vector generated by the depth image as an output. And by adjusting the parameters, the loss function is reduced to obtain an optimal network model.
In one embodiment, the method further comprises the step of recovering column vectors with the length of MN output by the trained neural network model into M multiplied by N dimensions to obtain a three-dimensional image of the scene to be detected.
It should be understood that, although the steps in the flowchart of fig. 2 are shown in sequence as indicated by the arrows, the steps are not necessarily performed in sequence as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.
In one embodiment, as shown in FIG. 5, a scanless single-channel terahertz radar forward-looking three-dimensional imaging system is provided, comprising a data collection module 502, a model training module 504, and a model application module 506, wherein:
the data collection module 502 is configured to control the single-shot terahertz radar transceiver signal to obtain a one-dimensional range profile of the detection scene through two paths of synchronous clock signals, control the time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtain a training set formed by a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
The model training module 504 is configured to convert a one-dimensional distance image in the training set into a column vector, input the column vector into a pre-designed neural network model, and train the neural network model by using the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional distance image as an output to obtain a trained neural network model;
The model application module 506 is configured to obtain a one-dimensional distance image of the scene to be detected, convert the one-dimensional distance image into a column vector, input the column vector into a trained neural network model, and obtain a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model.
The data collection module 502 is further configured to control the single-shot terahertz radar to collect signals in a trigger mode through a first clock signal, so that the single-shot terahertz radar can collect signals and store data when detecting a falling edge, and obtain a one-dimensional range profile of a detection scene, and control the time-of-flight camera to collect signals in the trigger mode through a second clock signal, so that the time-of-flight camera can collect and store data when detecting a falling edge, and obtain a three-dimensional depth image corresponding to the detection scene, wherein the three-dimensional depth image comprises the horizontal coordinates, the vertical coordinates and the depth information of each point in the detection scene, and the first clock signal and the second clock signal are kept synchronous.
The model training module 504 is further configured to convert a one-dimensional distance image in the training set into a column vector, input the column vector into a pre-designed neural network model, train the neural network model by using the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional distance image as an output, and obtain a trained neural network model, where the neural network is a neural network of a multi-layer perceptron, the neural network of the multi-layer perceptron includes an input layer, three full-connection layers and an output layer, the full-connection layers are used to connect each point of input data with each point of output data, and an activation function layer is located behind each full-connection layer.
The model training module 504 is further configured to convert a one-dimensional distance image in the training set into a column vector, input the column vector into a pre-designed neural network model, flatten m×n dimensions of a three-dimensional depth image corresponding to the one-dimensional distance image in a transverse direction and a longitudinal direction into a column vector with a length of MN, and train the neural network model by taking the column vector with the length of MN as an output of the neural network model to obtain a trained neural network model.
The model training module 504 is further configured to train the neural network model through a back propagation algorithm, so as to obtain a trained neural network model.
The model application module 506 is further configured to restore the column vector with the length MN output by the trained neural network model to m×n dimensions, so as to obtain a three-dimensional image of the scene to be detected.
Specific limitations regarding the scanning-free single-channel terahertz radar forward-looking three-dimensional imaging system can be found in the above description of the scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method, and are not repeated here. The modules in the scanning-free single-channel terahertz radar forward-looking three-dimensional imaging system can be all or partially realized by software, hardware and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 6. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by the processor, implements a scanless single-channel terahertz radar forward-looking three-dimensional imaging method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 6 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In an embodiment a computer device is provided comprising a memory storing a computer program and a processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the method embodiments described above.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (4)
1. A scanning-free single-channel terahertz radar forward-looking three-dimensional imaging method, which is characterized by comprising the following steps:
Respectively controlling a single-shot terahertz radar receiving and transmitting signal through two paths of synchronous clock signals to obtain a one-dimensional range profile of a detection scene, controlling a time-of-flight camera to acquire a signal to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set consisting of a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
Converting a one-dimensional range profile in a training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by taking the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional range profile as output to obtain a trained neural network model;
acquiring a one-dimensional distance image of a scene to be detected, converting the one-dimensional distance image into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model;
The method comprises the steps of respectively controlling a single-shot terahertz radar to send and receive signals through two paths of synchronous clock signals to obtain a one-dimensional distance image of a detection scene, and controlling a time-of-flight camera to acquire signals to obtain a three-dimensional depth image corresponding to the detection scene, wherein the method comprises the steps of controlling the single-shot terahertz radar to acquire signals in a trigger mode through a first clock signal so that the single-shot terahertz radar can acquire signals and save data when detecting a falling edge send and receive signal to obtain the one-dimensional distance image of the detection scene, controlling the time-of-flight camera to acquire signals in the trigger mode through a second clock signal so that the time-of-flight camera can acquire and save data when detecting a falling edge to obtain the three-dimensional depth image corresponding to the detection scene, and the three-dimensional depth image comprises the abscissa and depth information of each point in the detection scene;
The method comprises the steps of converting a one-dimensional range profile in a training set into a column vector, inputting the column vector into a pre-designed neural network model, taking the column vector obtained by flattening a three-dimensional depth image corresponding to the one-dimensional range profile as output, training the neural network model to obtain a trained neural network model, and training the neural network model to obtain the trained neural network model by taking the column vector obtained by flattening the three-dimensional depth image corresponding to the one-dimensional range profile as output, wherein the neural network is a neural network of a multi-layer perceptron, the neural network of the multi-layer perceptron comprises an input layer, three full-connection layers and an output layer, and each full-connection layer is used for connecting each point of input data with each point of output data and is provided with an activation function layer.
2. The method of claim 1, wherein converting the one-dimensional range profile in the training set into a column vector, inputting the column vector into a pre-designed neural network model, and training the neural network model by using the column vector flattened by the three-dimensional depth image corresponding to the one-dimensional range profile as an output, to obtain a trained neural network model, comprising:
Converting the one-dimensional distance image in the training set into a column vector and inputting the column vector into a pre-designed neural network model;
the three-dimensional depth image corresponding to the one-dimensional range profile is arranged in the transverse and longitudinal directions Dimension flat to lengthIs a column vector of (2);
With the length as And training the neural network model to obtain a trained neural network model by taking the column vector of the neural network model as the output of the neural network model.
3. The method according to claim 2, wherein obtaining a three-dimensional image of the scene to be detected from the column vectors output by the trained neural network model comprises:
outputting the trained neural network model to have the length of Is restored to the column vector of (2)And D, obtaining the three-dimensional image of the scene to be detected.
4. A scanless single-channel terahertz radar forward-looking three-dimensional imaging system, the system comprising:
the data collection module is used for respectively controlling the single-shot terahertz radar receiving and transmitting signals to obtain a one-dimensional range profile of a detection scene through two paths of synchronous clock signals, controlling the time-of-flight camera to acquire signals to obtain a three-dimensional depth image corresponding to the detection scene, and further obtaining a training set formed by a plurality of groups of one-dimensional range profile and three-dimensional depth image pairs through changing the gesture and the position of a target in the detection scene;
The model training module is used for converting one-dimensional range profiles in a training set into column vectors, inputting the column vectors into a pre-designed neural network model, and training the neural network model by taking the column vectors obtained by flattening the three-dimensional depth image corresponding to the one-dimensional range profiles as output to obtain a trained neural network model;
The model application module is used for acquiring a one-dimensional distance image of a scene to be detected, converting the one-dimensional distance image into a column vector, inputting the column vector into the trained neural network model, and obtaining a three-dimensional image of the scene to be detected according to the column vector output by the trained neural network model;
The data collection module is also used for controlling the single-shot terahertz radar to collect signals in a trigger mode through a first clock signal so that the single-shot terahertz radar can receive and transmit signals and store data when detecting one falling edge, and a one-dimensional range profile of a detection scene is obtained; the method comprises the steps of controlling a time-of-flight camera to acquire signals in a trigger mode through a second clock signal, so that the time-of-flight camera acquires and stores data when detecting a falling edge, and obtaining a three-dimensional depth image corresponding to a detection scene, wherein the three-dimensional depth image comprises the horizontal coordinates, the vertical coordinates and depth information of each point in the detection scene;
the model training module is also used for converting one-dimensional distance images in a training set into column vectors and inputting the column vectors into a pre-designed neural network model, and converting three-dimensional depth images corresponding to the one-dimensional distance images in the transverse and longitudinal directions Dimension flat to lengthIs a column vector of (2); with the length asAnd training the neural network model to obtain a trained neural network model by taking the column vector of the neural network model as the output of the neural network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210312335.1A CN114740472B (en) | 2022-03-28 | 2022-03-28 | A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210312335.1A CN114740472B (en) | 2022-03-28 | 2022-03-28 | A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114740472A CN114740472A (en) | 2022-07-12 |
| CN114740472B true CN114740472B (en) | 2024-12-27 |
Family
ID=82277792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210312335.1A Active CN114740472B (en) | 2022-03-28 | 2022-03-28 | A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114740472B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115327543B (en) * | 2022-08-15 | 2023-07-07 | 中国科学院空天信息创新研究院 | Multi-node time-frequency synchronization method for unmanned aerial vehicle bee colony SAR |
| CN115586542B (en) * | 2022-11-28 | 2023-03-03 | 中国人民解放军国防科技大学 | Remote terahertz single photon radar imaging method and device based on scaling training |
| CN117111093B (en) * | 2023-10-20 | 2024-02-06 | 中山大学 | Single-pixel three-dimensional imaging method and system based on neural network |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253386A (en) * | 2011-03-29 | 2011-11-23 | 西安电子科技大学 | Airborne three-dimensional synthetic aperture radar imaging system based on transmitted beam scanning |
| CN111208509A (en) * | 2020-01-15 | 2020-05-29 | 中国人民解放军国防科技大学 | Ultra-wideband radar human body target posture visualization enhancing method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5341141A (en) * | 1993-03-09 | 1994-08-23 | Hughes Missile Systems Company | Three dimensional imaging radar |
-
2022
- 2022-03-28 CN CN202210312335.1A patent/CN114740472B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102253386A (en) * | 2011-03-29 | 2011-11-23 | 西安电子科技大学 | Airborne three-dimensional synthetic aperture radar imaging system based on transmitted beam scanning |
| CN111208509A (en) * | 2020-01-15 | 2020-05-29 | 中国人民解放军国防科技大学 | Ultra-wideband radar human body target posture visualization enhancing method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114740472A (en) | 2022-07-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114740472B (en) | A forward-looking three-dimensional imaging method and system for non-scanning single-channel terahertz radar | |
| CN105652273B (en) | A kind of sparse imaging algorithm of MIMO radar based on mixing matching pursuit algorithm | |
| CN109471082B (en) | Array element defect MIMO radar angle estimation method based on signal subspace reconstruction | |
| CN107015190A (en) | Relatively prime array Wave arrival direction estimating method based on the sparse reconstruction of virtual array covariance matrix | |
| US12061257B2 (en) | Methods to maintain image quality in ultrasound imaging at reduced cost, size, and power | |
| CN103399315A (en) | High-resolution detecting and imaging method for real-aperture phased array radar | |
| Kang et al. | SAR imaging based on deep unfolded network with approximated observation | |
| CN114580535A (en) | Multi-base radar human body behavior fusion identification method, device and medium based on attention multi-view learning network | |
| Tang et al. | Through-wall human activity recognition with complex-valued range–time–Doppler feature and region-vectorization ConvGRU | |
| Feng et al. | A double-step grid-free method for sound source identification using deep learning | |
| Jaswal et al. | Range-Doppler hand gesture recognition using deep residual-3DCNN with transformer network | |
| CN111896956B (en) | Real-time microwave associated imaging device and method based on FPGA and DSP | |
| CN114740470B (en) | Microwave wavefront modulation forward-looking imaging method and device based on attribute scattering model | |
| Tang et al. | Enhanced through-the-wall radar imaging using Bayesian compressive sensing | |
| Saleem et al. | Gender classification based on gait analysis using ultrawide band radar augmented with artificial intelligence | |
| Dao et al. | Temporal rate up-conversion of synthetic aperture radar via low-rank matrix recovery | |
| Song et al. | Deep parametric imaging for bistatic SAR: Model, property and approach | |
| Tivive et al. | Multi-stage compressed sensing and wall clutter mitigation for through-the-wall radar image formation | |
| CN115586542A (en) | Remote terahertz single photon radar imaging method and device based on scaling training | |
| CN110554386B (en) | A channel compression synthetic aperture radiometer imaging method and system | |
| Aeron et al. | On sensing capacity of sensor networks for the class of linear observation, fixed SNR models | |
| Breaker et al. | Radar based continuous indoor activity recognition using deep learning | |
| CN112183534A (en) | Intelligent Joint Detection Method of Moving Target Based on Video Synthetic Aperture Radar | |
| Aeron et al. | Information theoretic bounds to sensing capacity of sensor networks under fixed SNR | |
| Xu et al. | Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |