CN115439743A - Method for accurately extracting visual SLAM static characteristics in parking scene - Google Patents

Abstract

The invention discloses a method for accurately extracting visual SLAM static characteristics in a parking scene, which uses multithreading parallelism to frame out vehicles and pedestrians to form mask masks by using a target detection model when dynamic objects such as pedestrians and vehicles exist in the parking scene; meanwhile, the common manual features of the VSLAM system at present are abandoned, an improved feature extraction model SuperPoint based on deep learning is selected for feature extraction, the accuracy of feature extraction is higher and more robust, key points and descriptors of image frames are obtained, feature points in dynamic object frames are screened and removed according to generated mask masks, feature matching and camera pose estimation are carried out by using the remaining accurate static feature points, and then tracking, drawing and loop detection threads can be executed subsequently to complete the whole SLAM work. By using the method, the probability of mismatching in the memory parking scene is reduced, the defects that the SLAM algorithm is difficult to remove dynamic characteristic points and the scene recognition precision is low can be effectively overcome, and the reliability of memory parking is improved.

Description

A Method for Accurately Extracting Static Features of Visual SLAM in Parking Scenes

technical field

The invention belongs to the field of visual SLAM and deep learning, and in particular relates to a method for visual SLAM in a parking scene using deep learning target detection to remove dynamic feature points.

Background technique

Simultaneous localization and mapping (SLAM) technology uses the robot's own sensors to capture and process the surrounding environment information without prior knowledge of the environment, and then completes the map construction and the robot's own positioning. The use of camera sensors to complete the perception of the surrounding environment is called visual SLAM. Visual sensors take advantage of their low cost and rich collection of information to become sensors commonly used in modern SLAM research.

With the continuous improvement and development of visual SLAM technology, a number of excellent open source SLAM frameworks such as ORB-SLAM2 and OpenVSLAM have emerged. The classic visual SLAM is mainly composed of several modules such as sensor data input, front-end visual odometer, back-end optimization, loop detection, and map building. ORB-SLAM2 is run in parallel by three threads of tracking, local mapping and loop detection. The traditional ORB algorithm is used for feature extraction. It has been verified that its robustness to illumination is poor. In recent years, some feature extraction algorithms based on deep learning have emerged. Feature extraction has played a pivotal role in the entire visual SLAM system, ensuring that the extracted feature points have a good representative effect on the scene. Most of the current visual SLAM can only All the feature points in the current scene of this scene are extracted, but the feature points on dynamic objects such as vehicles and pedestrians may not be matched in the next positioning, resulting in positioning failure.

Contents of the invention

In view of the shortcomings of the above-mentioned traditional visual SLAM algorithm for dynamic feature points and the accuracy and robustness of feature points, the purpose of the present invention is to provide a method for accurately extracting static features of visual SLAM in a parking scene, which can improve feature point extraction. It can effectively reduce the impact of dynamic object feature points on visual SLAM mapping and positioning, and improve the reliability of visual SLAM.

In order to solve the above technical problems, the present invention provides a method for accurately extracting static features of visual SLAM in a parking scene, comprising the following steps:

Step 1: Extract the image of the parking lot scene in front of the car, preprocess the image and input the image into the target detection network for target detection, and obtain the detection frame of the target object;

Step 2: Filter the dynamic object detection frame output in step 1, and form a mask mask, use it in combination with the feature points extracted by SuperPoint, eliminate the feature points of the dynamic object detection frame, and obtain key points and descriptors, the SuperPoint network includes key The shared encoder, key point decoder and descriptor decoder of points and descriptors, the shared encoder is used to encode the image to obtain the feature map, the key point decoder is used to obtain the coordinates of the key points in the image, and the descriptor decoder is used for Obtain the descriptor vector of key points, where the improvements to the SuperPoint network include: changing all convolutions in the encoder to depth-separable convolutions, where target detection and feature extraction use multi-threaded parallel technology, in the feature extraction Simultaneous target detection;

Step 3: If the mask represents a pedestrian, the SuperPoint network removes the feature points in the mask; if it is a car, compare the car target detection areas of adjacent frames, and the non-common points of the adjacent two target detection areas Part of the feature points are retained, and the common part of the feature points is eliminated to obtain the filtered static feature points;

Step 4: Use the SuperPoint network to extract and remove key points and descriptors in the mask, use the remaining feature points for feature matching, continue to execute the tracking module of visual SLAM, calculate the camera pose and build a map, and complete the entire SLAM work.

In step 1, the target detection network uses the YOLOv5 network, and the target detection algorithm process based on YOLOv5 is as follows: input a 608*608*3 RGB image, scale the input image to the input size of the network, and use Mosaic for data enhancement, Mosaic Randomly select 4 pictures to scale, rotate, and arrange to form a new picture, which not only greatly increases the number of pictures, but also speeds up the training speed and achieves the effect of data enhancement; the Backbone module uses the CSPDarknet53 structure and the Focus structure to extract some general features ;Transport the extracted general features to the Neck network to extract more diverse and robust features, input them to the CSP2_X and CBL structure, and after upsampling, contcat with the features output by the backbone network, enhancing the ability of feature fusion ;The final output uses CIoU_LOSS instead of the previous GIoU_LOSS as the loss function of the Bounding Box. The CIoU formula is as follows:

α表示损失平衡因子。CIou considers the size ratio of the real frame and the predicted frame, where v ∈ [0, 1] represents the normalized representation of the proportional difference between the length and width of the predicted frame and the corresponding real frame,

α represents the loss equalization factor.

In step 2, the SuperPoint network used in step 2 is before training,

The SuperPoint network is extracted in a self-supervised manner. First, a fully convolutional network-Base Detector is trained using regular geometric shapes as a data set; the detection results of unlabeled real pictures using the Base Detector network are used as pseudo-Ground Truth Keypoint (false-true Value key point), in order to make the pseudo-Ground Truth Keypoint more robust and accurate, use Homographic Adaptation technology (homographic technology) to extract features from unlabeled real pictures at different sizes to generate pseudo-labels; after generating pseudo-labels , you can put the real unlabeled pictures into the SuperPoint network for training. In the image input stage, data enhancement methods such as flipping are used.

The SuperPoint network includes three parts: a shared encoder of key points and descriptors, a key point decoder, and a descriptor decoder. Further, in step 2, the SuperPoint network detects key points and descriptors as follows:

Input an image frame of H*W*3, convert it to H*W*1 after grayscale, and then input the image to an improved and lighter shared encoder. After the encoder, the input image size is converted H _c =H/8, W _c =W/8, so as to reduce the image size;

The key point decoder performs sub-pixel convolution operation, converts the input vector from H/8*W/8*65 to H*W through the depth to space process, and finally outputs the probability that each pixel is a Keypoint;

The descriptor decoder uses the convolutional network to obtain a semi-dense descriptor, then uses the bicubic difference to obtain the remaining description, and finally obtains a descriptor of uniform length (H*W*D) through L2 normalization.

In step 2, the shared encoder of the improved SuperPoint is used. The original SuperPoint encoder uses a VGG6-like convolutional network layer, but the calculation amount and training parameters are huge. The present invention changes all convolutions of the encoder part into depth-separable convolutions. The normal convolution process is shown in Figure 3, set the input image size to H*W*3, and output an m-layer feature map, then the normal convolution kernel parameter is 3*f*f*m;

Depth separable convolution (as shown in Figure 4) is divided into two continuous processes of channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution is to convolve each channel with a separate convolution kernel, and the convolution The process is transformed into a two-dimensional plane, and finally a mid feature map (intermediate feature map) is generated. The convolution kernel parameter of this link is f*f*3, and the generated mid featuremap is convolved point by point, using 1*1* 3 convolution kernels, which have the function of data fusion, and finally output the m-layer feature map. The parameter quantity of this part is 1*3*m; the parameter quantity of the convolution kernel of channel-by-channel kernel and point-by-point convolution is 3* (f*f+m), which is an order of magnitude lower than the 3*f*f*m of direct convolution, and the time efficiency will be greatly improved; although the number of parameters learned is lower than that of ordinary convolution, the accuracy is higher. Didn't drop much.

In step 2, the loss function of the improved SuperPoint consists of two parts: key point extraction loss and descriptor detection loss:

为关键点损失，使用交叉熵损失函数，

为描述子损失，λ为平衡因子。where the loss function consists of two parts:

For the keypoint loss, use the cross-entropy loss function,

is the descriptor loss, and λ is the balance factor.

In step 2, the operating mechanism of target detection and feature extraction is as follows: using multi-thread parallel technology, target detection is performed while feature extraction is performed, and the tracking thread does not need to wait for the detection results of YOLOv5, which improves CPU utilization and improves operating efficiency.

(图7中A区域)，

(图7中B区域)，

表示第n帧中得第i个特征点。将相邻两帧检测同一动态物体目标框的交集作为最终的动态目标特征点，即D＝D_nD_n+1(图7的C区域)，将D集合中特征点最为最终动态特征点集合，这样降低了动态特征点误剔除的概率，也保留部分疑似静态特征点，剩余的绿色、蓝色和黄色区域为保留区域，增加了跟踪线程的可靠性。In step 3, remove the feature points in the detection frames of the dynamic objects (predetermined vehicles and pedestrians) in steps 1 and 2. If you directly remove the feature points in each dynamic object frame, it will be difficult to match due to too few feature points. As shown in Figure 7, the feature points of the detected dynamic target frames in two adjacent frames are

(A region in Figure 7),

(area B in Figure 7),

Indicates the i-th feature point in the n-th frame. The intersection of the same dynamic object target frame detected in two adjacent frames is used as the final dynamic target feature point, that is, D=D _n D _n+1 (C area of Figure 7), and the feature point in the D set is the final dynamic feature point set , which reduces the probability of false removal of dynamic feature points, and also retains some suspected static feature points. The remaining green, blue, and yellow areas are reserved areas, which increases the reliability of tracking threads.

In step 4, the process of calculating the camera pose is as follows: through the feature points and descriptors selected in the above process, image matching is performed, and the RANSAC random sampling consensus algorithm is used to eliminate mismatched feature points, and the matching relationship is converted into 2D points to 2D points The epipolar geometry problem. Assuming that x ₁ and x ₂ are the normalized coordinates of the corresponding matching points on the two images, R is the camera rotation matrix, and t is the translation matrix, then satisfy

x ₂ =Rx ₁ +t

Left multiply x ^T ₂ t:

即为对极约束表达式，在按照最小重投影误差即可求出相机位姿。令基础矩阵E＝t^R，本质矩阵F＝K^-TEK^-1。求解相机位姿可以转化成以下两步：求出基础矩阵E或者本质矩阵F；根据E或F，求解R，t。The left side of the equation is 0, then:

It is the epipolar constraint expression, and the camera pose can be calculated according to the minimum reprojection error. Let the fundamental matrix E=t^R, the essential matrix F=K ^-T EK ^-1 . Solving the camera pose can be transformed into the following two steps: finding the fundamental matrix E or the essential matrix F; according to E or F, solving R, t.

In step 4, the screening of key frames has a great influence on the redundancy of information and the release of computing resources. If the system is in positioning mode, the local map is occupied, or repositioning has just finished, no keyframe will be inserted.

Compared with the prior art, the present invention can at least achieve the following beneficial effects:

The present invention uses a deep learning-based SuperPoint network combined with a target detection network (such as YOLOv5) to perform key point, descriptor extraction, and dynamic target detection. Traditional solutions mostly use ORB and SURF for feature extraction, but for parking lot scene changes, lighting The feature extraction effect is not good when the intensity changes are obvious, but the improved SuperPoint used in the present invention first makes the network model more lightweight, and finally makes the feature extraction more robust to different scene changes, and the extracted feature points are more uniform and reasonable.

Description of drawings

Fig. 1 is a schematic flowchart of a method for accurately extracting static features of visual SLAM in a parking scene provided by an embodiment of the present invention.

Fig. 2 is a flowchart of an improved lightweight SuperPoint core provided by an embodiment of the present invention.

Figure 3 is a schematic diagram of ordinary convolution.

Figure 4 is a schematic diagram of depth-separable convolution, where (a) is a schematic diagram of channel-by-channel convolution, and (b) is a schematic diagram of point-by-point convolution.

Fig. 5 is a detection effect diagram of YOLOv5 in the parking lot scene provided by the embodiment of the present invention.

Fig. 6 is an effect diagram of SuperPoint feature extraction provided by an embodiment of the present invention.

Fig. 7 is a schematic diagram of dynamic feature screening provided by an embodiment of the present invention (C is the removal area).

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions will be clearly and completely described below in conjunction with the present application.

The traditional visual SLAM feature extraction is based on the assumption of a static environment, but for each parking space in the parking lot environment, there are not always cars parked, so it is necessary to remove the feature points of some dynamic objects on the parking space to ensure these Dynamic features are not preserved on the final map. The dynamic feature points proposed in the prior art generally use traditional feature extraction algorithms such as ORB, SURF, etc., but the effect of these feature points on different scene changes is very different, and it is not very robust. The present invention uses depth-based The learned SuperPoint network is combined with the YOLOv5 algorithm to extract key points, descriptors, and dynamic object detection, and to improve SuperPoint to make the network model more lightweight, and finally make the feature extraction more robust to different scene changes, and the extracted feature points are more uniform Reasonable. The method provided by the present invention is described in detail below.

As shown in Figures 1 to 6, the present invention discloses a method for accurately extracting static features of visual SLAM in a parking scene, comprising the following steps:

Stepl: Extract the image of the parking lot scene in front of the car, preprocess the image, input the image into the target detection network for target detection, and obtain the detection frame of the target object.

In some embodiments of the present invention, the target detection network used is the YOLOV5 network model. It can be understood that in other embodiments, other target detection networks can also be used.

Use a monocular camera to collect images of the parking lot scene in front of the car in real time, and after image preprocessing operations (such as filtering, image enhancement, etc.), a H*W*3 (H, W represent the length and width of an image respectively) The number of pixels) of the RGB image is input into the YOLOV5 network model, the input image is scaled to the input size of the network, and the data is enhanced using Mosaic. Mosaic randomly selects 4 pictures to scale, rotate, and arrange to form a new picture. It not only greatly increases the number of pictures, but also can speed up the training speed and achieve the effect of data enhancement; the backbone network extracts image features and generates feature maps; the Backbone module uses the CSPDarknet53 structure and Focus structure to extract some general features; the extracted general features are sent to More diverse and robust features are extracted from the Neck network, input to the CSP2_X and CBL structures, and after upsampling, they are spliced with the features output by the backbone network to enhance the ability of feature fusion; the final output uses CIoU_LOSS to replace the previous ones GIoU_LOSS is used as the loss function of Bounding Box, and the CIoU formula is as follows:

α表示损失平衡因子。In the formula, CIou considers the size ratio of the real frame and the predicted frame, CIoU(B _pre , B _GT ) represents the CIOU intersection ratio between the predicted frame and the real target frame; Iou(B _pre , B _Gr ) represents the predicted frame and Intersection and union ratio between real target frames; B _pre represents the predicted frame; B _GT represents the real target detection frame; ρ(B _pre , B _GT ) represents the center point distance between the predicted frame and the real frame; v represents the predicted frame and the real frame The aspect ratio similarity of , v∈[0,1]; w ^GT represents the width of the real frame; h ^GT represents the height of the real frame; GT represents the information of the real frame; w, h represent the width and height of the predicted frame, respectively;

α represents the loss equalization factor.

Step2: Screen the dynamic object detection frame output in step 1, and form a mask mask, and use it in combination with the feature points extracted by SuperPoint to eliminate the feature points of the dynamic object detection frame.

In some embodiments of the present invention, the dynamic objects in the dynamic object detection frame include: vehicles, pedestrians, and a few animals.

At the same time, another thread conducts the feature extraction process. The collected RGB image with a size of H*W*3 is grayscaled and then input into the lightweight SuperPoint network. The SuperPoint network uses a self-supervised method for extraction. First, a full volume is trained using regular geometric shapes as a data set. Product network Base Detector; unmarked real pictures using the detection results of the Base Detector network as false ground truth key points (pseudo Ground TruthKeypoint), in order to make false ground truth key points more robust and accurate, using homography technology ( HomographicAdaptation technology), which extracts features from unlabeled real pictures at different sizes to generate pseudo-labels; after generating pseudo-labels, the real unlabeled pictures can be put into the SuperPoint network for training. In the image input stage, data enhancement methods such as flipping are used.

The SuperPoint network includes shared encoders for key points and descriptors, key point decoders, and descriptor decoders. The shared encoders are used to encode images to obtain feature maps. The key point decoders are used to obtain the coordinates of key points in the image and describe The sub-decoder is used to obtain the descriptor sub-vectors of the keypoints.

Specifically, input an image frame of H*W*3, grayscale it and convert it into H*W*1, and then input the image into an improved and lighter shared encoder, in which all The ordinary convolution operation is transformed into a depthwise separable convolution. For ordinary convolution, set the output feature map to have m layers, then the convolution kernel parameter is 3*f*f*m (f represents the convolution kernel size, m represents the final output channel number), after the depth separable convolution After the product, the amount of parameters becomes 3*(f*f+m), which is an order of magnitude lower than the 3*f*f*m of direct convolution, and the time efficiency will be greatly improved; although the amount of parameters learned is compared with ordinary convolution It has decreased, but the accuracy has not decreased too much.

In some embodiments of the present invention, depthwise separable convolution includes two continuous processes of channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution is to convolve each channel with a separate convolution kernel. The convolution process is transformed into a two-dimensional plane, and finally an intermediate feature map is generated. The convolution kernel parameter in this link is f*f*3, and the generated intermediate feature map is convolved point by point, using 1*1*3 volume The accumulation kernel has the function of data fusion, and finally outputs the m-layer feature map. The parameter amount of this part is 1*3*m; the parameter amount of the convolution kernel of channel-by-channel kernel and point-by-point convolution is 3*(f *f+m), which is an order of magnitude lower than the 3*f*f*m of direct convolution, and the time efficiency will be greatly improved; although the number of parameters learned is lower than that of ordinary convolution, the accuracy is not. Dropped too much.

After the encoder, the input image size is converted to H _c = H/8, W _c = W/8, thereby reducing the image size; the key point decoder performs sub-pixel convolution operation, through depth to space (the depth dimension The data is moved to the space dimension) process converts the input vector from H/8*W/8*65 to H*W vector, and the H*W vector is subjected to softmax operation, and then dimensionally transformed by Resahpe, and the final output of each pixel is The probability of a key point (KeyPoint), expressed in vector form, is the key point coordinates where the key point is screened by the probability threshold; the descriptor detector uses a convolutional network to obtain a semi-dense descriptor, and then uses the bicubic difference to obtain the remaining description , and finally obtain a descriptor of uniform length (H*W*D) through L2 normalization.

The loss function of the improved SuperPoint consists of two parts: key point extraction loss and descriptor detection loss:

为关键点损失，

为为翻转后图像的关键点损失，使用交叉熵损失函数，

为描述子损失，

表示图像经编码网络模型后对关键点的响应，尺寸为

表示原图经过翻转后经编码网络模型后对关键点的响应；D表示图像经编码网络模型后对描述子的响应；D′表示翻转后的图像经编码网络模型后对描述子的响应；Y表示关键点坐标标签；Y′表示翻转后图像的关键点坐标标签；S表示表示原图像和翻转后图像组成的图像对；λ为平衡因子。where the loss function consists of two parts:

is the keypoint loss,

For the keypoint loss of the flipped image, use the cross-entropy loss function,

is the descriptor loss,

Indicates the response of the image to key points after being encoded by the network model, and the size is

Indicates the response of the original image to the key points after being flipped and encoded by the network model; D indicates the response of the image to the descriptor after being encoded by the network model; D′ indicates the response of the flipped image to the descriptor after being encoded by the network model; Y Indicates the key point coordinate label; Y' indicates the key point coordinate label of the flipped image; S represents the image pair composed of the original image and the flipped image; λ is the balance factor.

Step3: If the mask represents pedestrians, the SuperPoint network removes the feature points in the mask; if it is a car, compare the car target detection areas of adjacent frames, the non-common of two adjacent target detection areas Part of the feature points are retained, and the common part of the feature points is eliminated to obtain the filtered static feature points.

The target detection frame and deep features use multi-thread parallel technology to perform target detection while feature extraction, and the tracking thread does not need to wait for the detection results of the target detection network, which improves CPU utilization and improves operating efficiency.

(如图7中A区域)，

(如图7中B区域)，

表示第n帧中的第i个特征点，p表示第n帧中的特征点的总数，q表示第n+1帧中的特征点的总数。将相邻两帧检测同一动态物体目标框的交集作为最终的动态目标特征点，即D＝D_nD_n+1(如图7中的C区域)，将D集合中特征点作为最终动态特征点集合，将D集合中的特征点进行剔除，这样降低了动态特征点误剔除的概率，也保留了部分疑似静态特征点，剩余的A、B区域为保留区域，增加了跟踪线程的可靠性，将筛选后的静态特征点进行保存，用于后续的特征匹配与位姿计算。Eliminate the feature points in the detection frame of dynamic objects (such as vehicles and pedestrians) in steps 1 and 2. If you directly remove the feature points in each dynamic object frame, it will be difficult to match due to too few feature points. Therefore, in some of the embodiments of the present invention, the feature points of the detected dynamic target frames of two adjacent frames are respectively

(A region in Figure 7),

(as in area B in Figure 7),

Represents the i-th feature point in the nth frame, p represents the total number of feature points in the nth frame, and q represents the total number of feature points in the n+1th frame. The intersection of the same dynamic object target frame detected in two adjacent frames is used as the final dynamic target feature point, that is, D=D _n D _n+1 (as shown in C area in Figure 7), and the feature point in the D set is used as the final dynamic feature Point collection, the feature points in the D set are eliminated, which reduces the probability of dynamic feature points being mistakenly eliminated, and also retains some suspected static feature points. The remaining A and B areas are reserved areas, which increases the reliability of the tracking thread. , and save the filtered static feature points for subsequent feature matching and pose calculation.

Step4: Use the SuperPoint network to extract and eliminate key points and descriptors in the mask mask, use the remaining feature points for feature matching, continue to execute the Tracking module of visual SLAM, use the minimum reprojection error to calculate the camera pose and build a map, and complete The whole SLAM job.

Image matching is carried out through the feature points and descriptors screened out in the above process. In some embodiments of the present invention, the RANSAC random sampling consensus algorithm is used to eliminate the mismatching feature points, and convert them into 2D points to 2D points according to the matching relationship. Epipolar geometry problems. Assuming that x ₁ and x ₂ are the normalized coordinates of the corresponding matching points on the two images, R is the camera rotation matrix, and t is the translation matrix, then satisfy

x ₂ =Rx ₁ +t

Left multiply x ^T ₂ t:

即为对极约束表达式，按照最小重投影误差即可求出相机位姿。令基础矩阵E＝t^R，本质矩阵F＝K^-TEK^-1，K-相机内参矩阵，E-基本矩阵，T表示矩阵的转置运算，t表示平移矩阵，R表示旋转矩阵。求解相机位姿可以转化成以下两步：求出基础矩阵E或者本质矩阵F；根据E或F，求解R，t。若系统处于定位模式、局部地图被占用或者刚刚结束重定位，则不插入关键帧。在不满足上面三种情况的前提下，若当前帧匹配的内点数超过设定的阈值，可将当前帧设置成关键帧，跟踪线程完成，接着继续进行建图和回环检测，最终完成整个地图的建立。The left side of the equation is 0, then:

It is the epipolar constraint expression, and the camera pose can be calculated according to the minimum reprojection error. Let the basic matrix E=t^R, the essential matrix F=K ^-T EK ^-1 , K-camera internal reference matrix, E-basic matrix, T represents the transpose operation of the matrix, t represents the translation matrix, and R represents the rotation matrix. Solving the camera pose can be transformed into the following two steps: finding the fundamental matrix E or the essential matrix F; according to E or F, solving R, t. If the system is in positioning mode, the local map is occupied, or just finished repositioning, no keyframe will be inserted. Under the premise that the above three conditions are not satisfied, if the number of inliers matched by the current frame exceeds the set threshold, the current frame can be set as a key frame, the tracking thread is completed, and then continue to build the map and loop detection, and finally complete the entire map of establishment.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A method for accurately extracting visual SLAM static features under a parking scene, characterized in that, comprising the following steps: Step 1: Extract the image of the parking lot scene in front of the car, preprocess the image and input the image into the target detection network for target detection, and obtain the detection frame of the target object; Step 2: Screen the dynamic object detection frame output in step 1, and form a mask mask, use it in combination with the feature points extracted by the SuperPoint network, eliminate the feature points of the dynamic object detection frame, and obtain key points and descriptors, the SuperPoint network includes The shared encoder, key point decoder and descriptor decoder of key points and descriptors, the shared encoder is used to encode the image to obtain the feature map, the key point decoder is used to obtain the coordinates of the key points in the image, and the descriptor decoder is used To obtain the descriptor vector of the key point, the improvement of the SuperPoint network includes: changing all the convolutions in the encoder to depth separable convolutions, wherein the target detection and feature extraction use multi-threaded parallel technology, and feature extraction while performing target detection; Step 3: If the mask represents a pedestrian, the SuperPoint network removes the feature points in the mask; if it is a car, compare the car target detection areas of adjacent frames, and the non-common points of the adjacent two target detection areas Part of the feature points are retained, and the common part of the feature points is eliminated to obtain the filtered static feature points; Step 4: Use the SuperPoint network to extract and remove key points and descriptors in the mask, use the remaining feature points for feature matching, continue to execute the tracking module of visual SLAM, calculate the camera pose and build a map, and complete the entire SLAM work. 2. the method for accurately extracting visual SLAM static feature under a kind of parking scene according to claim 1, is characterized in that, in step 1, target detection network adopts YOLOv5 network, and the process of target detection comprises: Input an RGB image, scale the input image to the input size of the network, and perform data enhancement; The backbone network extracts image features and generates feature maps. The Backbone module uses CSPDarknet53 structure and Focus structure to extract general features; the extracted general features are sent to the Neck network to extract more diverse and robust features, which are input to CSP2_X and CBL structure, and after upsampling, splicing with the features output by the backbone network; the final output uses CIoU_LOSS as the loss function of the Bounding Box. 3. the method for accurately extracting visual SLAM static features under a kind of parking scene according to claim 1, is characterized in that, before the SuperPoint network that adopts in step 2 is trained, The SuperPoint network is extracted in a self-supervised manner, first using regular geometric shapes as a data set to train a fully convolutional network; Utilizing the detection results of the full convolutional network for the unmarked real picture as a false truth key point, and using the homography technique to extract features from the unmarked real picture under different sizes to generate a pseudo-label; After the pseudo-label is generated, the real unlabeled image can be put into the SuperPoint network for training. 4. the method for accurately extracting visual SLAM static feature under a kind of parking scene according to claim 1, is characterized in that, in step 2, SuperPoint network detects key point and describes sub-process as follows: Input an image frame of H*W*3, convert it to H*W*1 after grayscale, and then input the image to an improved and lighter shared encoder. After the encoder, the input image size is converted _Hc =H/8, _Wc =W/8; The key point decoder performs sub-pixel convolution operation, and converts the input vector from H/8*W/8*65 to H*W, and finally outputs the probability that each pixel is a key point; The descriptor decoder uses a convolutional network to obtain a semi-dense descriptor, then uses the bicubic difference to obtain the remaining description, and finally obtains a uniform-length descriptor through L2 normalization. 5. The method for accurately extracting static features of visual SLAM in a parking scene according to claim 1, wherein the depth-separable convolution in step 2 includes two consecutive convolutions by channel and point by point. The channel-by-channel convolution is to convolve each channel with a separate convolution kernel, transform the convolution process into a two-dimensional plane, and finally generate an intermediate feature map. The convolution kernel parameter in this link is f* f*3, the generated intermediate feature map is convolved point by point, using 1*1*3 convolution kernel, and finally output m layer feature map, the parameter amount of this part is 1*3*m; then the channel-by-channel kernel and The parameter amount of the convolution kernel of point-by-point convolution is 3*(f*f+m). 6. the method for accurately extracting visual SLAM static features under a kind of parking scene according to claim 1, is characterized in that, in step 2, the loss function of the improved SuperPoint network is made of key point extraction loss and descriptor detection loss two Partial composition:

In the formula,

is the keypoint loss,

is the keypoint loss of the flipped image,

is the descriptor loss, χ represents the response of the image to key points after being encoded by the network model; χ′ represents the response to key points after the original image is flipped and encoded by the network model; D represents the response of the image to the descriptor after being encoded by the network model Response; D' represents the response of the flipped image to the descriptor after being encoded by the network model; Y represents the key point coordinate label; Y' represents the key point coordinate label of the flipped image; S represents the original image and the flipped image. Image pair; λ is the balance factor. 7. The method for accurately extracting visual SLAM static features under a parking scene according to claim 1, wherein the method of removing feature points in step 3 is: The feature points of the detected dynamic target frame in two adjacent frames are respectively

Represents the i-th feature point in the nth frame, and the intersection of the same dynamic object target frame detected in two adjacent frames is used as the final dynamic target feature point, that is, D=D _n D _n+1 , and the feature points in the D set As the final set of dynamic feature points. 8. The method for accurately extracting static features of visual SLAM in a parking scene according to claims 1-7, wherein the process of calculating the camera pose in step 4 is as follows: the selected feature points and descriptors are Image matching, eliminate mismatched feature points, and transform it into a 2D point-to-2D point epipolar geometric problem according to the matching relationship, assuming that x ₁ and x ₂ are the normalized coordinates of the corresponding matching points on the two images, and R is the camera rotation matrix , t is the translation matrix, and T represents the transposition operation of the matrix, then it satisfies x ₂ =Rx ₁ +t Left multiply x ^T ₂ t^:

The left side of the equation is 0, then:

That is, the epipolar constraint expression, and then the camera pose can be calculated according to the minimum reprojection error. 9. The method for accurately extracting visual SLAM static features in a parking scene according to claim 8, wherein the RANSAC random sampling consensus algorithm is used to eliminate mismatched feature points. 10. The method for accurately extracting static features of visual SLAM in a parking scene according to claim 8, characterized in that, when building a map in step 4, if the system is in positioning mode, the local map is occupied or just finished position, no keyframe is inserted.

Images