CN114964236A - Mapping and vehicle positioning system and method for underground parking lot environment - Google Patents

Abstract

The invention discloses a mapping and vehicle positioning system and method for an underground parking lot environment. In the system, a front-view camera is used to obtain the original image of the area in front of the vehicle; an inverse perspective transformation module is used to obtain a top view and input it to semantic features detection module; the semantic feature detection module is used to obtain the semantically segmented image of the top view, and obtain the semantic features and input them to the mapping module and the positioning module; the odometer module is used to obtain the pose of the vehicle; the mapping module is based on the pose of the vehicle, Project the semantic features from the vehicle body coordinate system to the global coordinate system to obtain a global semantic map; the positioning module is used to obtain the global coordinate points of the current semantic features of the vehicle, and match the global coordinate points with the global semantic map to obtain a global semantic map. The current semantic localization result of the vehicle. The invention can realize long-term stable positioning of the vehicle in the underground parking lot environment, more robustly cope with environmental changes, and at the same time, the cost is low.

Description

A mapping and vehicle positioning system and method for underground parking lot environment

technical field

The invention relates to the technical field of vehicle positioning, in particular to a mapping and vehicle positioning system and method for an underground parking lot environment.

Background technique

Autonomous parking is a specific application in the field of autonomous driving. In this task, vehicles often need to navigate autonomously in narrow, crowded, low-light parking lots without GPS signals. Therefore, accurate vehicle positioning is crucial. A large number of positioning schemes have emerged in the past decade, including vision-based positioning schemes, visual-inertial navigation-based positioning schemes, and lidar-based positioning schemes. In order to save costs, a lot of research is based on visual positioning. Traditional visual positioning schemes mostly use geometric features such as sparse points, line segments or planes in the environment. Among them, corner points are widely used in visual odometry. The general process of these schemes is to estimate the location of map points through feature point matching. , build a map and estimate the camera pose based on this map.

In recent years, the solution of mapping and positioning based on ORB features has attracted much attention in academia and industry. For example, the invention patent application with publication number CN113808203A discloses a navigation and positioning based on LK optical flow method and ORB-SLAM2. method, this method adds the GPU-based LK optical flow algorithm before ORB-SLAM2, and uses the number of optical flow tracking feature points as the judgment condition for whether the current frame is a key frame, and for non-key frames, it will not enter the third stage of ORBSLAM2. This prevents non-key frame extraction of feature points and subsequent calculations, thereby speeding up ORBSLAM2's processing of Tracking threads, enhancing the real-time performance of the algorithm, and does not affect its robustness. It is suitable for autonomous driving of cars and AGV logistics trolleys positioning and navigation. However, this localization method is easily affected by changes in illumination, viewing angle, and appearance of the environment, and cannot effectively localize for a long time. Especially the environment of underground parking lot presents a huge challenge to traditional visual positioning schemes such as ORB-SLAM. On the one hand, the underground indoor parking lot is mainly composed of walls, ground and columns. This weak texture structure makes the detection and matching of feature points very unstable, and the positioning is also prone to tracking loss. On the other hand, the parking lot environment changes caused by the entry and exit of different vehicles in different time periods. Long-term vehicle relocation is almost impossible for traditional visual positioning solutions.

SUMMARY OF THE INVENTION

Aiming at the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is: how to provide a vehicle that can realize long-term stable positioning of vehicles in an underground parking lot environment, cope with environmental changes more robustly, and at the same time have a lower cost A mapping and vehicle positioning system and method for an underground parking lot environment.

In order to solve the above-mentioned technical problems, the present invention adopts the following technical solutions:

A mapping and vehicle positioning system for an underground parking lot environment, comprising a front-view camera, an inverse perspective transformation module, a semantic feature detection module, an odometer module, a mapping module and a positioning module;

The forward-looking camera is used to obtain the original image of the area in front of the vehicle and input it to the inverse perspective transformation module;

The inverse perspective transformation module is configured to perform inverse perspective transformation on the original image input by the front-view camera to obtain a top view and input it to the semantic feature detection module;

The semantic feature detection module is used to obtain the semantically segmented image of the top view through the convolutional neural network, and obtain the semantic features and input them to the mapping module and the positioning module;

The odometer module is used to obtain the pose of the vehicle and input it to the mapping module and the positioning module;

The mapping module projects semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a global semantic map;

The positioning module is used for acquiring the global coordinate point of the current semantic feature of the vehicle, and matching the global coordinate point with the global semantic map to obtain the current semantic positioning result of the vehicle.

Preferably, the odometer module includes an inertial measurement unit and two wheel speed sensors;

The mapping module includes a local map module and a global map module;

The local map module projects semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and input it to the global map module;

The global map module is used to generate a global semantic map after adding loopback detection and global optimization to the local map.

A mapping and vehicle positioning method for an underground parking lot environment, using the above-mentioned mapping and vehicle positioning system for an underground parking lot environment, wherein the mapping and vehicle positioning method includes a mapping method and a vehicle positioning method;

The mapping method includes the following steps:

Step A1) The forward-looking camera obtains the original image of the area in front of the vehicle and inputs it to the inverse perspective transformation module;

Step A2) The inverse perspective transformation module performs inverse perspective transformation on the original image input by the front-view camera to obtain a top view and input it to the semantic feature detection module;

Step A3) The semantic feature detection module obtains the image after the semantic segmentation of the overhead view through the convolutional neural network, and obtains the semantic feature and inputs it to the mapping module;

Step A4) The odometer module obtains the pose of the vehicle and inputs it to the mapping module;

Step A5) The mapping module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a global semantic map;

Step A6) completes mapping;

The vehicle positioning method includes the following steps:

Step S1) The forward-looking camera obtains the original image of the area in front of the current position of the vehicle and inputs it to the inverse perspective transformation module;

Step S2) The inverse perspective transformation module performs inverse perspective transformation on the original image input by the front-view camera to obtain a top view and inputs it to the semantic feature detection module;

Step S3) the semantic feature detection module obtains the semantically segmented image of the overhead view through the convolutional neural network, and obtains the semantic feature and inputs it to the positioning module;

Step S4) The odometer module obtains the current posture of the vehicle and inputs it to the positioning module;

Step S5) The positioning module projects the semantic feature from the vehicle body coordinate system to the global coordinate system based on the current pose of the vehicle provided by the odometer module, so as to obtain the global coordinate point of the current semantic feature of the vehicle;

Step S6) Match the global coordinate points obtained in step S5) with the global semantic map obtained in the mapping method to obtain the current semantic positioning result of the vehicle, and complete the vehicle positioning.

Preferably, the formula for inverse perspective transformation is:

In the formula: π _c ( ) is the projection model of the forward-looking camera, [R _c t _c ] is the extrinsic parameter matrix transformed from the forward-looking camera coordinate system to the vehicle body coordinate system, [uv] is the pixel coordinate of the semantic feature, [x ^v y ^v ] is the coordinate of the semantic feature in the vehicle body coordinate system, and λ is the scale factor.

Preferably, the convolutional neural network of the semantic feature detection module uses the parking lot picture set collected by the forward-looking camera for training and classification, and the classification categories of the convolutional neural network include lane lines, parking lines, guide lines, and speed bumps. , passable areas, obstacles and walls, and the semantic features obtained by the semantic feature detection module through the convolutional neural network include lane lines, stop lines, guide lines and speed bumps.

Preferably, the formula for projecting semantic features from the vehicle body coordinate system to the global coordinate system is:

In the formula: [x ^w, y ^w z ^w ] is the coordinate of the semantic feature in the global coordinate system, R _o is the rotation matrix transformed from the vehicle body coordinate system to the global coordinate system, and t _o is the transformation from the vehicle body coordinate system to the global coordinate system. The translation vector for the coordinate system transformation.

Preferably, the odometer module includes an inertial measurement unit and two wheel speed sensors;

The mapping module includes a local map module and a global map module;

The local map module projects semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and input it to the global map module;

The global map module is used to generate a global semantic map after adding loopback detection and global optimization to the local map;

In step A5), the local map module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and input it to the global map module. The global map module generates a global semantic map after adding loopback detection and global optimization to the local map.

Preferably, in step A5), the method for adding loopback detection to the local map by the global map module is: using the data-based registration method to match the current local map with the previously generated local map, if the matching result satisfies the set value , it proves that a loopback occurs, and the calculated relative pose is used to optimize the pose graph to eliminate the accumulated error.

Preferably, in step A5), the method for performing global optimization on the local map by the global map module includes constraining the measurement value of the odometer module of two consecutive frames of local maps, and the relative data obtained based on the data registration method during loop closure detection. The pose is used as a constraint between loopback frames.

Preferably, in step A5), the method that the global map module performs global optimization on the local map is the Gauss-Newton method, and the objective function is:

为从里程计模块获取的相对位姿测量值，L为回环帧对的集合，

为基于数据配准法得到的第i帧和第j帧之间的相对位姿，做测量值使用，T_i,j为第i帧和第j帧带有累计误差的相对位姿估计值。In the formula: χ is the pose set, T _{t, t+1} is the relative pose estimation value of the front-view camera at frame t and frame t+1,

is the relative pose measurement obtained from the odometer module, L is the set of loopback frame pairs,

For the relative pose between the i-th frame and the j-th frame obtained based on the data registration method, it is used as a measurement value, and T _i,j is the relative pose estimation value of the i-th frame and the j-th frame with accumulated errors.

Compared with the prior art, the present invention has the following advantages:

1. The present invention is different from the geometric features in the environment used in the traditional visual positioning scheme. The present invention utilizes the semantic features in the environment. For the parking lot environment, it mainly includes lane lines, guide lines, parking lines and speed bumps. Semantic features Compared with geometric features, which can exist stably for a long time and remain robust under changes in illumination, perspective, and environment, these semantic features are detected by convolutional neural networks, and the semantic features are used to build a global semantic map, and then use the global semantic map to conduct vehicle analysis. position. Therefore, the scheme can cope with environmental changes more robustly than the traditional positioning scheme, and can maintain a stable and accurate use state for a long time.

2. The sensors used in the present invention are only a forward-looking camera, an IMU (Inertial Measurement Unit) and two wheel speed sensors, wherein the IMU and the wheel speed sensors form an odometer module, which provides the relative pose of the vehicle during mapping and positioning Therefore, the use cost of the present invention is very low, and the mass production vehicle can also be easily configured.

3. The framework of the present invention mainly includes two parts, mapping and positioning. As the name implies, mapping is to establish a global semantic map of the parking lot environment; the front-view camera obtains the original image of the area in front of the vehicle, and after inverse perspective transformation, the top view is obtained, and then the convolutional neural network is input to obtain the semantically segmented image, and obtain the lane. Semantic features such as lines, parking lines, guide lines and speed bumps; then based on the pose provided by the odometer module, the semantic features are projected to the global coordinate system. In view of the data drift of the odometer module, the present invention also utilizes loopback detection and global Optimization is done to eliminate these accumulated errors, and finally by saving these feature points, a global semantic map of the parking lot is established. After the global semantic map is generated, the vehicle entering the parking lot obtains the image through the forward-looking camera, inverse perspective transformation, semantic feature detection and the pose provided by the odometer module to obtain the global coordinate point of the semantic feature, and then uses the ICP algorithm (based on data registration) to obtain the global coordinate points of the semantic feature. method) and the built global semantic map to match and correct the vehicle pose, and finally obtain accurate vehicle positioning results.

Description of drawings

FIG. 1 is a system block diagram of a mapping and vehicle positioning system for an underground parking lot environment according to the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments.

As shown in Figure 1, a mapping and vehicle positioning system for an underground parking lot environment includes a forward-looking camera, an inverse perspective transformation module, a semantic feature detection module, an odometer module, a mapping module and a positioning module;

The front-view camera is used to obtain the original image of the area in front of the vehicle and input it to the inverse perspective transformation module;

The inverse perspective transformation module is used to inversely transform the original image input by the front-view camera to obtain a top view and input it to the semantic feature detection module;

The semantic feature detection module is used to obtain the semantically segmented image of the top view through the convolutional neural network, and obtain the semantic feature input to the mapping module and the positioning module;

The odometer module is used to obtain the pose of the vehicle and input it to the mapping module and the positioning module;

Based on the pose of the vehicle provided by the odometer module, the mapping module projects the semantic features from the vehicle body coordinate system to the global coordinate system to obtain a global semantic map;

The localization module is used to obtain the global coordinate point of the current semantic feature of the vehicle, and match the global coordinate point with the global semantic map to obtain the current semantic localization result of the vehicle.

In this embodiment, the odometer module includes an inertial measurement unit and two wheel speed sensors;

The mapping module includes a local map module and a global map module;

The local map module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and input it to the global map module;

The global map module is used to generate a global semantic map after adding loopback detection and global optimization to the local map.

A mapping and vehicle positioning method for an underground parking lot environment, using the above-mentioned mapping and vehicle positioning system for an underground parking lot environment, the mapping and vehicle positioning method includes a mapping method and a vehicle positioning method;

The mapping method includes the following steps:

Step A1) The front-view camera obtains the original image of the area in front of the vehicle and inputs it to the inverse perspective transformation module;

Step A2) inverse perspective transformation module obtains a top view after inverse perspective transformation is performed on the original image input by the front-view camera and is input to the semantic feature detection module;

Step A3) The semantic feature detection module obtains the image after the semantic segmentation of the overhead view through the convolutional neural network, and obtains the semantic feature and inputs it to the mapping module;

Step A4) The odometer module obtains the pose of the vehicle and inputs it to the mapping module;

Step A5) the mapping module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a global semantic map;

Step A6) completes mapping;

The vehicle positioning method includes the following steps:

Step S1) the front-view camera obtains the original image of the area in front of the current position of the vehicle and inputs it to the inverse perspective transformation module;

Step S2) inverse perspective transformation module performs inverse perspective transformation on the original image input by the front-view camera to obtain a top view and input to the semantic feature detection module;

Step S3) the semantic feature detection module obtains the semantically segmented image of the overhead view through the convolutional neural network, and obtains the semantic feature and inputs it to the positioning module;

Step S4) the odometer module obtains the current pose of the vehicle and inputs it to the positioning module;

Step S5) the positioning module projects the semantic feature from the vehicle body coordinate system to the global coordinate system based on the current pose of the vehicle provided by the odometer module, to obtain the global coordinate point of the current semantic feature of the vehicle;

Step S6) Match the global coordinate points obtained in step S5) with the global semantic map obtained in the mapping method to obtain the current semantic positioning result of the vehicle, and complete the vehicle positioning.

In this embodiment, the internal and external parameters of the front-view camera have been calibrated offline, and the original image obtained by the front-view camera is projected onto the ground through inverse perspective transformation. The formula of inverse perspective transformation is:

In the formula: π _c ( ) is the projection model of the forward-looking camera, [R _c t _c ] is the extrinsic parameter matrix transformed from the forward-looking camera coordinate system to the vehicle body coordinate system, [uv] is the pixel coordinate of the semantic feature, [x ^v y ^v ] is the coordinate of the semantic feature in the vehicle body coordinate system, and λ is the scale factor.

In this embodiment, the semantic feature detection module uses the convolutional neural network U-Net for semantic segmentation, and the convolutional neural network uses the parking lot image set collected by the front-view camera for training classification, and the classification categories of the convolutional neural network include: Lane lines, stop lines, guide lines, speed bumps, passable areas, obstacles and walls, and the semantic features obtained by the semantic feature detection module through the convolutional neural network include lane lines, stop lines, guide lines and speed bumps. Because the semantic features of lane lines, stop lines, guide lines and speed bumps are highly recognizable and stable, they are suitable for the mapping and positioning of the present invention.

In this embodiment, using the pose provided by the odometer module, the semantic feature is projected from the vehicle body coordinate system to the global coordinate system by the following formula, and the global coordinate points of the semantic feature are saved to generate a local map, and the local map range is 30m; , the formula for projecting semantic features from the vehicle body coordinate system to the global coordinate system is:

In the formula: [x ^w, y ^w z ^w ] is the coordinate of the semantic feature in the global coordinate system, R _o is the rotation matrix transformed from the vehicle body coordinate system to the global coordinate system, and t _o is the transformation from the vehicle body coordinate system to the global coordinate system. The translation vector for the coordinate system transformation.

In this embodiment, in step A5), the local map module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and input it to the global map module ; The global map module generates a global semantic map after adding loopback detection and global optimization to the local map.

In this embodiment, in step A5), the global map module adds the method of loopback detection to the local map as follows: using the data-based registration method to match the current local map with the previously generated local map, if the matching result satisfies the setting value, it proves that a loopback occurs, and the calculated relative pose is used to optimize the pose graph to eliminate the accumulated error.

In this embodiment, in step A5), the method for the global map module to perform global optimization on the local map includes constraining the measured value of the odometer module of two consecutive local maps, and the relative position obtained based on the data registration method during loop closure detection. The pose is used as a constraint between loopback frames.

In the present embodiment, in step A5), the method that the global map module performs global optimization on the local map is the Gauss-Newton method, and the objective function is:

为从里程计模块获取的相对位姿测量值，L为回环帧对的集合，

为基于数据配准法得到的第i帧和第j帧之间的相对位姿，做测量值使用，T_i,j为第i帧和第j帧带有累计误差的相对位姿估计值。In the formula: χ is the pose set, T _{t, t+1} is the relative pose estimation value of the front-view camera at frame t and frame t+1,

is the relative pose measurement obtained from the odometer module, L is the set of loopback frame pairs,

For the relative pose between the i-th frame and the j-th frame obtained based on the data registration method, it is used as a measurement value, and T _i,j is the relative pose estimation value of the i-th frame and the j-th frame with accumulated errors.

Compared with the prior art, the present invention is different from the geometric features in the environment used by the traditional visual positioning scheme. The present invention utilizes the semantic features in the environment. For the parking lot environment, it mainly includes lane lines, guide lines, stop lines and deceleration. Compared with geometric features, semantic features can exist stably for a long time, and are still robust when lighting, perspective and environment change. These semantic features are detected by convolutional neural networks, and the semantic features are used to build a global semantic map, and then use the global semantic map. The semantic map locates the vehicle. Therefore, the scheme can cope with environmental changes more robustly than the traditional positioning scheme, and can maintain a stable and accurate use state for a long time. The sensors used in the present invention are only a forward-looking camera, an IMU (inertial measurement unit) and two wheel speed sensors, wherein the IMU and the wheel speed sensors form an odometer module, which provides the relative pose of the vehicle during mapping and positioning, so The use cost of the present invention is very low, and the mass production vehicle can also be easily configured. The framework of the present invention mainly includes two parts, mapping and positioning. As the name implies, mapping is to establish a global semantic map of the parking lot environment; the front-view camera obtains the original image of the area in front of the vehicle, and after inverse perspective transformation, the top view is obtained, and then the convolutional neural network is input to obtain the semantically segmented image, and obtain the lane. Semantic features such as lines, parking lines, guide lines and speed bumps; then based on the pose provided by the odometer module, the semantic features are projected to the global coordinate system. In view of the data drift of the odometer module, the present invention also utilizes loopback detection and global Optimization is done to eliminate these accumulated errors, and finally by saving these feature points, a global semantic map of the parking lot is established. After the global semantic map is generated, the vehicle entering the parking lot obtains the image through the forward-looking camera, inverse perspective transformation, semantic feature detection and the pose provided by the odometer module to obtain the global coordinate point of the semantic feature, and then uses the ICP algorithm (based on data registration) to obtain the global coordinate points of the semantic feature. method) and the built global semantic map to match and correct the vehicle pose, and finally obtain accurate vehicle positioning results.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that those technical solutions of the present invention are modified or equivalently replaced without departing from the present technology. The purpose and scope of the solution should be included in the scope of the claims of the present invention.

Claims (10)

1. A mapping and vehicle positioning system aiming at an underground parking lot environment is characterized by comprising a forward-looking camera, an inverse perspective transformation module, a semantic feature detection module, a mileometer module, a mapping module and a positioning module;

the front-view camera is used for acquiring an original image of an area in front of the vehicle and inputting the original image to the inverse perspective transformation module;

the inverse perspective transformation module is used for performing inverse perspective transformation on an original image input by the front-view camera to obtain a top view and inputting the top view to the semantic feature detection module;

the semantic feature detection module is used for obtaining an image subjected to top view semantic segmentation through a convolutional neural network, obtaining semantic features and inputting the semantic features into the image building module and the positioning module;

the odometer module is used for acquiring pose of a vehicle and inputting the pose of the vehicle into the mapping module and the positioning module;

the mapping module projects semantic features from a vehicle body coordinate system to a global coordinate system based on the vehicle pose provided by the odometer module to obtain a global semantic map;

the positioning module is used for acquiring a global coordinate point of the current semantic features of the vehicle and matching the global coordinate point with a global semantic map so as to obtain a current semantic positioning result of the vehicle.

2. The mapping and vehicle positioning system for an underground parking garage environment of claim 1, wherein said odometer module comprises one inertial measurement unit and two wheel speed sensors;

the map building module comprises a local map module and a global map module;

the local map module projects semantic features from a vehicle body coordinate system to a global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and inputs the local map to the global map module;

and the global map module is used for generating a full local semantic map after adding loop detection and global optimization to the local map.

3. The mapping and vehicle positioning method for the underground parking lot environment is characterized in that the mapping and vehicle positioning system for the underground parking lot environment according to claim 1 is adopted, and the mapping and vehicle positioning method comprises a mapping method and a vehicle positioning method;

the mapping method comprises the following steps:

step A1) the front camera acquires an original image of the area in front of the vehicle and inputs the original image to the inverse perspective transformation module;

step A2), the inverse perspective transformation module carries out inverse perspective transformation on the original image input by the front-view camera to obtain a top view and inputs the top view to the semantic feature detection module;

step A3), the semantic feature detection module obtains an image after top view semantic segmentation through a convolutional neural network, and obtains semantic features to input the semantic features to the image building module;

step A4) the odometer module acquires the pose of the vehicle and inputs the pose to the mapping module;

step A5), the mapping module projects semantic features from a vehicle body coordinate system to a global coordinate system based on the vehicle pose provided by the odometer module to obtain a global semantic map;

step A6) completing map building;

the vehicle positioning method comprises the following steps:

step S1), the front-view camera acquires an original image of an area in front of the current position of the vehicle and inputs the original image to the inverse perspective transformation module;

step S2), the inverse perspective transformation module carries out inverse perspective transformation on the original image input by the front-view camera to obtain a top view and inputs the top view to the semantic feature detection module;

step S3), the semantic feature detection module obtains an image after top view semantic segmentation through a convolutional neural network, and obtains semantic features to input the semantic features to the positioning module;

step S4) the odometer module acquires the current pose of the vehicle and inputs the pose to the positioning module;

step S5), the positioning module projects the semantic features from the vehicle body coordinate system to the global coordinate system based on the current pose of the vehicle provided by the odometer module so as to obtain global coordinate points of the current semantic features of the vehicle;

step S6), the global coordinate points obtained in the step S5) are matched with the global semantic map obtained in the mapping method, so that the current semantic positioning result of the vehicle is obtained, and the vehicle positioning is completed.

4. The mapping and vehicle positioning method for underground parking garage environment according to claim 3, wherein the formula of inverse perspective transformation is:

in the formula: pi _c (. R) is a forward-looking camera projection model _c t _c ]For the extrinsic matrix transformed from the front-view camera coordinate system to the vehicle body coordinate system, [ u v ]]Pixel coordinates, [ x ] of semantic features ^v y ^v ]The coordinate of the semantic features under the vehicle body coordinate system is shown, and lambda is a scale factor.

5. The mapping and vehicle positioning method for underground parking lot environment according to claim 3, wherein the convolutional neural network of the semantic feature detection module performs training classification using the parking lot image set collected by the forward-looking camera, and the classification category of the convolutional neural network comprises lane lines, stop lines, guide lines, speed bumps, passable areas, obstacles and walls, and the semantic features acquired by the semantic feature detection module through the convolutional neural network comprise lane lines, stop lines, guide lines and speed bumps.

6. The mapping and vehicle positioning method for an underground parking lot environment according to claim 3, wherein the formula for projecting the semantic features from the vehicle body coordinate system to the global coordinate system is as follows:

in the formula: [ x ] of ^w ’ y ^w z ^w ]For coordinates of semantic features in a global coordinate system, R _o For a rotation matrix transformed from the vehicle body coordinate system to the global coordinate system, t _o Is a translation vector converted from the vehicle body coordinate system to the global coordinate system.

7. The mapping and vehicle locating method for an underground parking lot environment according to claim 6, wherein the odometer module comprises one inertial measurement unit and two wheel speed sensors;

the map building module comprises a local map module and a global map module;

the local map module projects semantic features from a vehicle body coordinate system to a global coordinate system based on the pose of the vehicle provided by the odometer module to obtain a local map and inputs the local map to the global map module;

the global map module is used for generating a full local semantic map after adding loop detection and global optimization to the local map;

in the step A5), the local map module projects semantic features from a vehicle body coordinate system to a global coordinate system based on the vehicle pose provided by the odometer module to obtain a local map and inputs the local map to the global map module; and the global map module adds loop detection and global optimization to the local map to generate a full local semantic map.

8. The mapping and vehicle positioning method for underground parking lot environment according to claim 7, wherein in step A5), the method for the global map module to add loop detection to the local map is as follows: and matching the current local map with the previously generated local map by using a data registration-based method, if the matching result meets a set value, proving that loop appears, and optimizing the pose graph by using the relative pose obtained by calculation to eliminate accumulated errors.

9. The mapping and vehicle positioning method for underground parking lot environment according to claim 8, wherein in step a5), the global map module global optimization method for local maps comprises the steps of constraining the measurement values of the odometer module of two frames of continuous local maps, and constraining loop frames based on the relative poses obtained by data registration method during loop detection.

10. The mapping and vehicle positioning method for the underground parking lot environment according to claim 9, wherein in step a5), the global map module performs global optimization on the local map by using gauss-newton method, and the objective function is:

in the formula: x is a pose set, T _t,t+1 Relative pose estimation values of the forward-looking cameras of the t frame and the t +1 frame,

for relative pose measurements taken from the odometer module, L is a set of loop frame pairs,

the relative pose between the ith frame and the jth frame obtained based on a data registration method is used as a measured value T _i,j And carrying out relative pose estimation values with accumulated errors for the ith frame and the jth frame.

Images