CN116953642A - Millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network - Google Patents

Abstract

The invention discloses a millimeter wave radar gesture recognition method based on an adaptive coding VisionTransformer network, which includes: using millimeter wave radar to collect gestures to be classified; performing feature extraction on the collected gesture echo data, and extracting the extracted micro gestures. Dynamic Doppler features and pitch azimuth angle features are synthesized into three-channel RGB gesture feature maps; the denoising diffusion implicit model is used to expand the RGB gesture feature map; the expanded fitting image and the RGB gesture feature map are mixed to form a data set , and divide the data set into a training data set and a verification data set; build an adaptive coding VisionTransformer network model; use the training data set to train the adaptive coding VisionTransformer network model; input the verification data set into the trained adaptive coding VisionTransformer network classification in the model. The present invention adopts the adaptive coding VisionTransformer network model, which reduces the computational complexity, improves the recognition accuracy, and is easy to deploy under low computing power.

Description

Millimeter wave radar gesture based on adaptive coding Vision Transformer network recognition methods

Technical field

The present invention relates to the technical field of millimeter wave radar gesture recognition, and in particular to a millimeter wave radar gesture recognition method based on an adaptive coding VisionTransformer network.

Background technique

Since FMCW radar has a relatively mature technology for extracting micro-motion Doppler features and pitch azimuth angles when objects are moving from echo signals, its application in the field of gesture recognition provides additional options for gesture recognition. FMCW radar gesture recognition has many traditional solutions, which require manually separating feature sets that rely on relevant domain knowledge from the data after feature extraction, and then using appropriate machine learning models, such as SVM support vector machines and k-neighbors. Common algorithm models such as , Random Forest, etc. classify these feature vectors into different categories to achieve gesture recognition.

However, with the rapid development of deep learning, various models derived from it provide more solutions for gesture classification. At the same time, the deep learning model also reduces the complexity of gesture feature extraction. Some scholars have used three-channel synthesized gesture feature data for CNN gesture classification and achieved very good results. However, because gestures themselves are also complex, it is not feasible to use only a CNN model when facing gestures with similar features or gestures with unclear features. Therefore, some scholars have proposed using a multi-channel model that combines channel attention and spatial attention. Gesture feature recognition, such as the CNN-based enhanced multi-channel synthetic gesture feature recognition proposed by Du et al. and Spatial Attention Mechanism", IEEE Access, 2020, 8: 144610). However, the current various CNN classifications have the following problems:

First, the full-image attention method will lose the local features of the image, and its performance is not ideal when dealing with images with relatively small differences.

Second, the existing CNN algorithm requires too much computing power and is not suitable for deployment with low computing power.

Third, the correlation of the information between the pixels of the image itself is not taken into account, and the direct convolution is used to ignore the relationship between the pixels.

Contents of the invention

In order to solve the technical problems existing in the background technology, the present invention proposes a millimeter wave radar gesture recognition method based on an adaptive coding VisionTransformer network.

The present invention proposes a millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network, including:

Use millimeter wave radar to collect gestures to be classified;

Perform feature extraction on the collected gesture echo data to obtain micro-motion Doppler features and pitch azimuth angle features, and synthesize the micro-motion Doppler features and pitch azimuth angle features into an RGB gesture feature map in three channels;

The denoising diffusion implicit model is used to expand the RGB gesture feature map to obtain a fitted image;

Mix the fitting images and RGB gesture feature maps to form a data set, and divide the data set into a training data set and a verification data set;

Build an adaptive coding Vision Transformer network model;

Use the training data set to train the adaptive coding Vision Transformer network model;

Input the verification data set into the trained adaptive coding Vision Transformer network model for classification, and obtain the classification results.

Furthermore, the denoising diffusion implicit model is used to expand the RGB gesture feature map, including:

Establish the diffusion process and noise prediction model of the RGB gesture feature map; among them, the noise prediction model adopts the U-Net neural network structure;

Input the RGB gesture feature map into the noise prediction model according to the diffusion process, and train the noise prediction model according to the diffusion process to obtain the trained denoising diffusion implicit model;

The RGB gesture feature map is expanded using the trained denoising diffusion implicit model.

Further, the denoising diffusion implicit model includes a forward diffusion process and a reverse denoising generation process;

In the forward diffusion process, by continuously adding Gaussian noise to the RGB gesture feature map x ₀ , the original data distribution is changed into a normal distribution, and a pure noise image x _t is obtained;

In the reverse denoising generation process, the noise prediction model is used to quickly restore the pure noise image x _t from the normal distribution to the original data distribution to obtain a fitted image.

Further, the noise prediction model includes a generator and a discriminator; among them, the RGB gesture feature map is input into the noise prediction model according to the diffusion process, and the noise prediction model is trained according to the diffusion process, thereby obtaining a trained denoising diffusion implicit model. Specifically include:

Perform diffusion noise on the RGB gesture feature map to obtain the diffusion-noised image and the variance of its noise components;

Input the diffusion-noised image and the variance of its noise component into the generator to obtain the prediction result image;

Input the RGB gesture feature map and the prediction result image into the discriminator to obtain the quality information of the prediction result image;

According to the image quality information of the prediction results, the generator and discriminator are optimized to obtain the trained generator and discriminator;

Update the trained generator and discriminator to the fast denoising prediction model to obtain the trained denoising diffusion implicit model.

Further, the quality information of the prediction result image includes the kernel starting distance KID of the feature map corresponding to the prediction result image and the RGB gesture feature map.

Further, the discriminator includes an input layer, a resize layer, an InceptionV3 module and a calculation module.

Further, the adaptive coding Vision Transformer network model includes: image division module, patch encoding module, Transformer Encoder feature extraction module and MLP classification module;

The image dividing module is used to divide the image into multiple image patch blocks; the patch encoding module is used to encode each image patch block; where each encoded image patch block includes the position information of the image patch block and the image Information about the relationship between pixels in the patch block; the Transformer Encoder feature extraction module is used to extract the feature vector of the encoded image patch block, and the MLP classification module is used to classify based on the extracted feature vector.

Further, the MLP classification module includes a third normalization layer, a dropout layer and three fully connected layers.

Furthermore, the Transformer Encoder feature extraction module includes 8 encoder blocks connected in sequence. Each Transformer encoder block includes a first layer of normalization layer, a second layer of normalization layer, a multi-head attention layer and a multi-layer perception layer. machine;

The input of the first normalization layer serves as the input of the encoder block, the output of the first normalization layer is connected to the input of the multi-head attention layer, and the input of the first normalization layer is connected to the input of the multi-head attention layer. The output skip connection is connected to the input of the second normalized layer, the output of the second normalized layer is connected to the input of the multi-layer perceptron, and the output of the second normalized layer is connected to the multi-layer perceptron. The output of is jump-connected to serve as input to this encoder block.

Further, the verification data set is input into the trained adaptive coding Vision Transformer network model for classification, which also includes:

The t-SNE algorithm is used to display the classification effect.

In the present invention, the proposed millimeter wave radar gesture recognition method based on the adaptive coding Vision Transformer network uses millimeter wave radar to collect gestures to be classified, and performs feature extraction on the collected gesture echo data to obtain micro-motion Doppler features and pitch azimuth angle features, and combine the micro-motion Doppler features and pitch azimuth angle features into a three-channel RGB gesture feature map, and then expand the RGB gesture feature map through a denoising diffusion implicit model. After obtaining a part of the data basis This eliminates the need for subsequent data collection processes that are time consuming and have unstable results. In addition, compared with the traditional classification algorithms SVM support vector machine, k-neighborhood, random forest and convolutional neural network, the present invention adopts the adaptive coding ViT network model, completely abandoning the methods of convolutional feature extraction and manual feature vector extraction, and enhances In addition to its ability to perceive local information, the invention reduces the computational complexity, improves the recognition accuracy, and is easy to deploy under low computing power.

Description of the drawings

Figure 1 is a schematic flowchart of a millimeter wave radar gesture recognition method based on an adaptive coding Vision Transformer network in an embodiment of the present invention.

Figure 2 is a schematic structural diagram of the U-net neural network in DDIM in an embodiment of the present invention.

Figure 3 is a process diagram of DDIM generating pictures in an embodiment of the present invention.

Figure 4 is a change curve of KID during the process of generating pictures by DDIM in an embodiment of the present invention.

Figure 5 is an original image of a gesture feature map in an embodiment of the present invention.

Figure 6 is a picture when the original gesture feature map is divided into multiple patches in an embodiment of the present invention.

Figure 7 is a schematic structural diagram of the adaptive coding Vision Transformer network model in an embodiment of the present invention.

Figure 8 is a variation curve of the loss function during the training process of the adaptive coding Vision Transformer network model in an embodiment of the present invention.

Figure 9 is a confusion matrix obtained after the recognition of the data set by the trained adaptive coding Vision Transformer network model in an embodiment of the present invention.

Figure 10 shows the original data and the data distribution diagram drawn using the t-SNE algorithm after training the adaptive coding Vision Transformer network model in an embodiment of the present invention.

Figure 11 is a schematic diagram of a CNN gesture recognition model based on channel and spatial attention mechanisms in the prior art.

Figure 12 is a change curve of the loss value during the training process of the CNN model based on the channel and spatial attention mechanism in the prior art.

Figure 13 shows the confusion matrix obtained after the recognition of the data set by the CNN model based on the trained channel and spatial attention mechanism in the prior art.

Figure 14 shows the original data and the data distribution diagram drawn using the t-SNE algorithm after being trained by the CNN model based on the channel and spatial attention mechanism in the prior art.

Figure 15 shows 10 gesture feature map samples.

Detailed ways

It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

Referring to Figure 1, the present invention proposes a millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network, including:

S1. Use millimeter wave radar to collect gestures to be classified;

S2. Extract features from the collected gesture echo data to obtain micro-motion Doppler features and pitch azimuth features, and synthesize the micro-motion Doppler features and pitch azimuth features into an RGB gesture feature map in three channels;

S3. Use denoising diffusion implicit models (DDIM) to expand the RGB gesture feature map to obtain a fitted image;

S4. Mix the fitting image and the RGB gesture feature map to form a data set, and divide the data set into a training data set and a verification data set;

S5. Construct an adaptive coding Vision Transformer network model;

S6. Use the training data set to train the adaptive coding Vision Transformer network model;

S7. Input the verification data set into the trained adaptive coding Vision Transformer network model for classification, and obtain the classification results.

The present invention uses millimeter wave radar to collect gestures to be classified, and performs feature extraction on the collected gesture echo data to obtain micro-motion Doppler features and pitch azimuth features, and combines the micro-motion Doppler features and pitch azimuth features The RGB gesture feature map is synthesized according to three channels, and then the RGB gesture feature map is expanded through the denoising diffusion implicit model. On the basis of obtaining a part of the data, the subsequent data collection process that is time-consuming and has unstable results is eliminated. In addition, compared with the traditional classification algorithms SVM support vector machine, k-neighborhood, random forest and convolutional neural network, the present invention adopts the adaptive coding ViT network model, completely abandoning the methods of convolutional feature extraction and manual feature vector extraction, and enhances In addition to its ability to perceive local information, the invention reduces the computational complexity, improves the recognition accuracy, and is easy to deploy under low computing power.

In this embodiment, in S1, millimeter wave radar is used to collect gestures to be classified, which specifically includes:

The IWR1642 millimeter wave radar is used to collect multiple rounds of gestures to be recognized.

In a further embodiment, millimeter wave radar is used to collect gestures to be classified, which specifically includes:

The gestures to be classified are collected using millimeter-wave radar emitted by a radar system with dual transmitters and four receivers.

Among them, the receivers of the radar system are Rx1, Rx2, Rx3 and Rx4. Among them, Rx1 and Rx2 are placed vertically. Since there is a wavelength distance between Rx1 and Rx2, the degree of freedom of elevation angle measurement is obtained; Rx2, Rx3 and Rx4 are Spaces of half the wavelength are arranged in parallel, enabling freedom in azimuth measurement. This radar system can measure the azimuth and elevation angles of gestures at a specific time through a special arrangement of transmitting and receiving antennas. The receiving antenna has degrees of freedom in azimuth and elevation angles. Among them, in this application, short-time fast Fourier transform (FFT) is used to obtain the spectrum diagram of the gesture to analyze the micro-Doppler characteristics. Among them, the window size of FFT and the time step of non-overlapping samples are set to 256ms and 1ms respectively.

In S2, after the micro-motion Doppler features and pitch azimuth angle features are synthesized into RGB gesture feature maps in three channels, it also includes: classifying the RGB gesture feature maps to facilitate subsequent DDIM training.

In this embodiment, in S3, the denoising diffusion implicit model is used to expand the RGB gesture feature map, which specifically includes:

S31. Establish the diffusion process and noise prediction model of the RGB gesture feature map; among them, the noise prediction model adopts the U-Net neural network structure, as shown in Figure 2;

S32. Input the RGB gesture feature map into the noise prediction model according to the diffusion process, and train the noise prediction model according to the diffusion process to obtain the trained denoising diffusion implicit model;

S33. Use the trained denoising diffusion implicit model to expand the RGB gesture feature map.

In the process of expanding the RGB gesture feature map, the denoising diffusion implicit model uses the non-Markov chain process and U-Net to generate nearly identical fitting images based on the original data set to expand the data set. This increases the amount of training and at the same time alleviates the problem of large intra-class sample variance caused by differences in the gestures of the same user, so that the subsequent network can learn gesture features in a variety of situations.

In this embodiment, in S3, the denoising diffusion implicit model includes a forward diffusion process and a reverse denoising generation process;

In the forward diffusion process, by continuously adding Gaussian noise to the RGB gesture feature map x ₀ , the original data distribution is changed into a normal distribution, and a pure noise image x _t is obtained;

In the reverse denoising generation process, the noise prediction model is used to quickly restore the pure noise image x _t from the normal distribution to the original data distribution to obtain a fitted image.

Specifically, in the reverse denoising generation process, the recursive model is

In the formula, ε _θ represents the noise prediction model, σ is a real vector as an index, and is the total number of noise adding steps, α _t :=1-β _t , β _t is the parameter value of adding noise at the artificially set time t, 0＜β ₁ ＜…＜β _t-1 ＜β _t ＜…＜ β _T <1, x _t represents a given noisy observation.

Among them, the noise prediction model is T is the total number of noise adding T steps.

Specifically, the U-Net neural network includes 4 downsampling layers and 4 upsampling layers, with skip connections added between layers with the same resolution. Based on this, the diffusion model can be viewed as a denoising autoencoder without bottlenecks. Among them, the U-Net neural network structure also introduces a residual structure to complete the network transfer and the expansion and reduction of the number of channels, thereby preventing gradient disappearance and gradient explosion.

Specifically, in the process of training the denoising diffusion implicit model, Adam W is set as the optimization algorithm of the overall network, the learning rate δ is set to 10 ^-3 , the numerical stability constant ε _r is set to 10 ^-4 , and noise is added at time t The parameter value β _t is set to The trigonometric function value on, the input image size is 64×64, and finally the Tensorflow framework is used to automatically calculate the gradient of each layer and propagate backward to update the learnable parameters.

In a further embodiment, the noise prediction model includes a generator and a discriminator; the RGB gesture feature map is input into the noise prediction model according to the diffusion process, and the noise prediction model is trained according to the diffusion process, thereby obtaining the trained denoising diffusion implicit Models, specifically include:

Perform diffusion noise on the RGB gesture feature map to obtain the diffusion-noised image and the variance of its noise components;

Input the diffusion-noised image and the variance of its noise component into the generator to obtain the prediction result image;

Input the RGB gesture feature map and the prediction result image into the discriminator to obtain the quality information of the prediction result image;

Optimize the generator and discriminator based on the image quality information of the prediction results to obtain the trained generator and discriminator;

Update the trained generator and discriminator to the fast denoising prediction model to obtain the trained denoising diffusion implicit model.

Such a setting can improve the generation ability of the generator and the discriminative ability of the discriminator, so that the trained denoising diffusion implicit model can generate nearly identical fitting images based on the original data set to expand the data set.

In order to enable the denoising diffusion implicit model to generate nearly identical fitting images based on the original data set to expand the data set, in a further embodiment, the quality information of the prediction result image includes the prediction result image and RGB gesture The kernel starting distance KID of the feature map corresponding to the feature map.

Specifically, for a noise map and generated image of a batch_size size (here set to 64), the polynomial kernel formula is used to calculate the polynomial kernel relative to itself and each other, and then based on the polynomial kernel, the average kernel for the batch_size is taken. And add them to get the final KID value.

What needs to be known is that KID is used as an image evaluation standard, and its value should be as small as possible. Ideally, if KID=0, the generated image will be almost exactly the same as the original image.

Specifically, in S324, during the process of optimizing the generator and the discriminator according to the image quality information of the prediction result image, when the KID of the prediction result image drops to a certain level, the generation of the prediction result map is stopped.

In a further embodiment, the discriminator includes an input layer, a resize layer, an InceptionV3 module, and an image quality evaluation module. Among them, the resize layer is used to adjust the image size of the image output by the input layer; the InceptionV3 module is used to extract features from the adjusted image; the calculation module is used to perform KID calculation based on the extracted features to obtain the quality information of the predicted result image. .

Specifically, the resize layer changes the size of the image output by the input layer to 75×75 to adapt to the InceptionV3 module. Specifically, the IInceptionV3 module preloads weights from “imagenet” when performing feature extraction.

In one specific embodiment, in S4, the number of pictures in the data set is 1,600, and the ratio of the training set and the verification set is 3:1.

As shown in Figure 7, in this embodiment, the adaptive coding Vision Transformer network model includes: image division module, patch encoding module, Transformer Encoder feature extraction module and MLP classification module.

Among them, the image division module is used to divide the image into multiple image patch blocks, as shown in Figure 5 and Figure 6; the patch encoding module is used to encode each image patch block; among them, each encoded image patch block Including the position information of the image patch and the relationship between the pixels in the image patch; the Transformer Encoder feature extraction module is used to extract the feature vector of the encoded image patch, and the MLP classification module is used to extract the feature vector based on the extracted feature vector. sort.

With this arrangement, the adaptive coding Vision Transformer network model directly enhances its ability to perceive local information through adaptive coding, which reduces the computational complexity of the present invention, improves recognition accuracy, and is easy to deploy under low computing power.

Specifically, the encoding of the position information of the image patch can be directly done by Embedding, and the encoding of the relationship information between pixels in the image patch is learned by the multi-layer perceptron and flattened in the patch. The relationship between each pixel is output. Different from the linear encoding method, this encoding method greatly enhances the adaptive ability of the encoding. Specifically, the final encoding of the image patch block is obtained by the addition and fusion of the position encoding of the patch and the pixel encoding of the patch.

As shown in Figure 7, in a further embodiment, the Transformer Encoder feature extraction module includes 8 encoder blocks connected in sequence. Each Transformer encoder block includes a first layer of normalization layer and a second layer of normalization layer. layers, multi-head attention layers and multi-layer perceptrons.

Specifically, the input of the encoder block serves as the input of the first normalization layer, the output of the first normalization layer is connected with the input of the multi-head attention layer, and the input of the first normalization layer is connected with the multi-head attention layer. The output of the force layer is jump-connected and connected to the input of the second normalization layer. The output of the second normalization layer is connected to the input of the multi-layer perceptron, and the output of the second normalization layer is connected to the multi-layer perceptron input. The output of the layer perceptron is skip-connected as input to this encoder block to prevent exploding or vanishing gradients.

It can be seen that compared with traditional classification algorithms such as SVM support vector machine, k-neighbor, random forest, etc., as well as convolutional neural networks, the adaptive coding ViT network completely abandons the methods of convolutional feature extraction and manual feature vector extraction, directly through The form of adaptive coding and multi-head attention enhances its ability to perceive local information, reduces the computational complexity of the present invention, improves recognition accuracy, and is easy to deploy under low computing power.

In a further embodiment, the multilayer perceptron includes two fully connected layers. Specifically, the first fully connected layer of the multilayer perceptron is 128 units, and the activation function is gelu. The second fully connected layer is 64 units, and the activation function is gelu.

In a further embodiment, the MLP classifier includes a third normalization layer, a dropout layer and three fully connected layers.

Specifically, the output of the Transformer encoder is used as the input of the third normalization layer, the output of the third normalization layer is flattened and sent to the Dropout layer, and the output of the Dropout layer is sent to the three fully connected layers. Among them, among the three fully connected layers, the first fully connected layer has 2048 units, and the activation function is gelu; the second fully connected layer has 1024 units, and the activation function is gelu; the third fully connected layer has 10 units to classify 10 types of gesture data.

In this embodiment, in S6, the training data set is used to train the adaptive coding Vision Transformer network model, which specifically includes:

The training data set is input into the constructed adaptive coding Vision Transformer network model, and cross entropy is used as the loss function to train the adaptive coding Vision Transformer network model until convergence, and the trained adaptive coding Vision Transformer network model is obtained.

Specifically, the loss function optimization method of the adaptive coding Vision Transformer network model is AdamW, the learning rate is set to 0.001, the weight attenuation coefficient is set to 0.0001, and Tensorflow is used to automatically perform gradient operations to learn various parameters in the network.

In this embodiment, in S7, the verification data set is input into the trained adaptive coding VisionTransformer network model for classification, which specifically includes:

Input the verification data set into the trained adaptive coding Vision Transformer network model for classification, and obtain the classification results;

The t-SNE algorithm is used to display the classification results to facilitate observation of the classification results.

The effect of the present invention can be further explained through the following simulation:

In order to verify the superiority of the present invention in millimeter-wave radar gesture recognition, a set of experiments was conducted to compare the present invention with the traditional method. Since the data volume before and after expansion without using DDIM is too different, the verification results are highly accidental. Therefore, the difference in data volume is not compared, and only the recognition results of the subsequent adaptive coding Vision Transformer (ViT) network model and CNN network are compared.

1. Simulation conditions

The test hardware platform parameters are shown in Table 1, the software platform parameters are shown in Table 2, and the radar platform parameters are shown in Table 3:

Table 1 Hardware platform parameters

CPU AMD Ryzen 74800H Memory 16 GB Graphics card model NVIDIA GeForceGTX 1660Ti Graphics card memory 6GB

Table 2 Software platform parameters

operating system Windows 1064 bit translater Pycharm 2021.3.1 CUDA version 11.2.0 Tensorflow version 2.10 Keras version 2.10

Table 3 Hard radar platform parameters

bandwidth ~4GHz Chirp repeat time 400μs Number of chirps per frame 128 distance resolution ~4cm speed resolution 0.032m/s

2. Simulation method

(1) The existing CNN gesture recognition network based on channel and spatial attention mechanisms recognizes and classifies gesture feature maps;

(2) The method of the present invention is a millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network.

3. Simulation content and simulation results

The radar used in this application is the IWR1642 radar operating at 77Ghz. The average output power of this radar is 12dBm. The radial velocity range that the radar can sense is 2.6cm/s to 2.6m/s, and the antenna beam width is 120 degrees, which is suitable for gesture measurement. Hand movements are measured in the main lobe of the radar antenna, with an average distance from the main lobe to the radar of approximately 30 cm.

The 10 gestures to be recognized by this application are (1) sliding from left to right (2) sliding from right to left (3) sliding from lower left to upper right (4) sliding from lower right to upper left (5) sliding forward (6) Swipe backwards (7) Swing left and right (8) Double-click with fingers (9) Clench and open fists (10) Snap fingers. The corresponding gesture feature diagram is shown in Figure 15.

Simulation experiment 1 uses a CNN gesture recognition network based on channel and spatial attention mechanisms to identify and classify gesture feature maps. The confusion matrix of the classification results is shown in Figure 13. The classification accuracy of each gesture can be obtained from the confusion matrix. The total classification accuracy is 91.00%.

Simulation experiment 2 uses the adaptive coding ViT network model in the present invention to identify and classify gesture feature maps. The confusion matrix of the classification results is shown in Figure 9. The classification accuracy of each gesture can be obtained from the confusion matrix, and the total classification accuracy is 97.25 %.

Among them, the window size of FFT and the time step of non-overlapping samples are set to 256ms and 1ms respectively. During the image generation process, set the learning rate of DDIM denoising diffusion implicit model training to 0.001, the batch size to 64, and the number of original data repetitions to 40 times. In each cycle of training, the same amount of verification set data as the training set is randomly selected for verification, and the KID is calculated for the generated feature images and the feature images in the verification set, and is calculated and displayed with each iteration through the callback method in keras The KID of the image makes it easy to monitor changes in image quality throughout the process and to interrupt training when the image quality reaches a certain level. The total number of training times is set to 70, and the training time of the entire data set is about 1 hour. The process of generating images is shown in Figure 3. The decrease of KID is shown in Figure 4. The smaller the KID, the higher the image quality. Finally, nearly 1,500 gesture feature maps were generated using the network weights after training.

Compared with the existing CNN-enhanced multi-channel synthetic gesture feature recognition based on channel and spatial attention mechanisms, this invention not only significantly improves the overall accuracy, but also greatly improves the classification accuracy of gesture feature maps with unobvious features or similar features. The magnitude improvement proves that the present invention can extract more comprehensive and discriminative features to reflect the properties of gesture feature images, and is conducive to improving the classification accuracy of gesture feature images.

After the t-SNE algorithm reduces the dimensionality of the original data and training results and visualizes it, the CNN gesture recognition network based on the channel and spatial attention mechanism is used to identify and classify the gesture feature map. The results are shown in Figure 14, using the adaptive coding in the present invention. The results of the ViT network model's recognition and classification of gesture feature maps are shown in Figure 10. It can be seen that the model in the present invention has better generalization ability and stronger robustness, which is beneficial to the identification and classification of never-seen data. , and the CNN gesture recognition network based on the channel and spatial attention mechanism recognizes and classifies the gesture feature map, and the results are too concentrated.

Comparing Figure 8 and Figure 12, it can be seen that the stability of gradient descent during the training process of the present invention is also very excellent and can reduce human intervention during the training process.

The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can, within the technical scope disclosed in the present invention, implement the technical solutions of the present invention. Equivalent substitutions or changes of the inventive concept thereof shall be included in the protection scope of the present invention.

Claims (10)

1. A millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network, which is characterized by: Use millimeter wave radar to collect gestures to be classified; Perform feature extraction on the collected gesture echo data to obtain micro-motion Doppler features and pitch azimuth angle features, and synthesize the micro-motion Doppler features and pitch azimuth angle features into an RGB gesture feature map in three channels; The denoising diffusion implicit model is used to expand the RGB gesture feature map to obtain a fitted image; Mix the fitting images and RGB gesture feature maps to form a data set, and divide the data set into a training data set and a verification data set; Build an adaptive coding Vision Transformer network model; Use the training data set to train the adaptive coding Vision Transformer network model; Input the verification data set into the trained adaptive coding Vision Transformer network model for classification, and obtain the classification results. 2. The millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network according to claim 1, characterized in that the RGB gesture feature map is expanded by using a denoising diffusion implicit model, specifically including: Establish the diffusion process and noise prediction model of the RGB gesture feature map; among them, the noise prediction model adopts the U-Net neural network structure; Input the RGB gesture feature map into the noise prediction model according to the diffusion process, and train the noise prediction model according to the diffusion process to obtain the trained denoising diffusion implicit model; The RGB gesture feature map is expanded using the trained denoising diffusion implicit model. 3. The millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network according to claim 2, characterized in that the denoising diffusion implicit model includes a forward diffusion process and a reverse denoising generation process; In the forward diffusion process, by continuously adding Gaussian noise to the RGB gesture feature map x ₀ , the original data distribution is changed into a normal distribution, and a pure noise image x _t is obtained; In the reverse denoising generation process, the noise prediction model is used to quickly restore the pure noise image x _t from the normal distribution to the original data distribution to obtain a fitted image. 4. The millimeter wave radar gesture recognition method based on the adaptive coding Vision Transformer network according to claim 2, characterized in that the noise prediction model includes a generator and a discriminator; wherein the RGB gesture feature map is input into the noise according to the diffusion process. Prediction model, train the noise prediction model according to the diffusion process to obtain the trained denoising diffusion implicit model, including: Perform diffusion noise on the RGB gesture feature map to obtain the diffusion-noised image and the variance of its noise components; Input the diffusion-noised image and the variance of its noise component into the generator to obtain the prediction result image; Input the RGB gesture feature map and the prediction result image into the discriminator to obtain the quality information of the prediction result image; According to the image quality information of the prediction results, the generator and discriminator are optimized to obtain the trained generator and discriminator; Update the trained generator and discriminator to the fast denoising prediction model to obtain the trained denoising diffusion implicit model. 5. The millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network according to claim 4, characterized in that the quality information of the prediction result image includes the kernel of the feature map corresponding to the prediction result image and the RGB gesture feature map. Starting distance KID. 6. The millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network according to claim 5, characterized in that the discriminator includes an input layer, a resize layer, an InceptionV3 module and a calculation module. 7. The millimeter wave radar gesture recognition method based on the adaptive coding Vision Transformer network according to any one of claims 1 to 6, characterized in that the adaptive coding Vision Transformer network model includes: an image division module, a patch coding module, Transformer Encoder feature extraction module and MLP classification module; The image dividing module is used to divide the image into multiple image patch blocks; the patch encoding module is used to encode each image patch block; where each encoded image patch block includes the position information of the image patch block and the image Information about the relationship between pixels in the patch block; the Transformer Encoder feature extraction module is used to extract the feature vector of the encoded image patch block, and the MLP classification module is used to classify based on the extracted feature vector. 8. The millimeter wave radar gesture recognition method based on adaptive coding Vision Transformer network according to claim 7, characterized in that the MLP classification module includes a third normalization layer, a dropout layer and three fully connected layers. 9. The millimeter wave radar gesture recognition method based on adaptive encoding Vision Transformer network according to claim 7, characterized in that the Transformer Encoder feature extraction module includes 8 encoder blocks connected in sequence, each Transformer encoder block It includes the first normalization layer, the second normalization layer, the multi-head attention layer and the multi-layer perceptron; The input of the first normalization layer serves as the input of the encoder block, the output of the first normalization layer is connected to the input of the multi-head attention layer, and the input of the first normalization layer is connected to the input of the multi-head attention layer. The output skip connection is connected to the input of the second normalized layer, the output of the second normalized layer is connected to the input of the multi-layer perceptron, and the output of the second normalized layer is connected to the multi-layer perceptron. The output of is jump-connected to serve as input to this encoder block. 10. The millimeter wave radar gesture recognition method based on the adaptive coding Vision Transformer network according to claim 1, characterized in that the verification data set is input into the trained adaptive coding Vision Transformer network model for classification, and further includes: The t-SNE algorithm is used to display the classification effect.