CN117664101A - A lidar-based semantic slam mapping method for airport unmanned vehicles - Google Patents

Abstract

The invention relates to a laser radar-based airport unmanned vehicle semantic SLAM map construction method, which solves the technical problem of how to design a map construction method based on SLAM, thereby providing a basis for realizing functions of autonomous mapping, navigation, obstacle avoidance, cruising, monitoring and the like; the method comprises the steps of obtaining a three-dimensional SLAM of a current vehicle through a laser radar, making a matching dimensional point cloud with a three-dimensional point cloud of a camera, combining a high-precision grid map of the laser SLAM with rich environment information of a visual SLAM, and fusing the high-precision grid map with the grid map of the laser SLAM, so that semantic SLAM mapping of an airport environment is achieved.

Description

A lidar-based semantic slam mapping method for airport unmanned vehicles

Technical field

The present invention relates to the technical field of airport unmanned vehicles, and specifically to a lidar-based semantic slam mapping method for airport unmanned vehicles.

Background technique

With the rapid development of my country's civil aviation industry, air travel has become more and more popular, which has put forward higher requirements for civil aviation travel services. With the increasing popularity of technologies such as artificial intelligence and big data, airport unmanned vehicles have become more and more widely used. Airport unmanned vehicles are highly autonomous and highly adaptable, and can operate in complex unstructured environments. To carry out autonomous movement, due to the established on-site environment of the airport, the positioning and planning of unmanned vehicles in the established environment are the research focus of technicians in this field.

Simultaneous localization and mapping (SLAM) is a method that can effectively estimate the current position of a robot and construct its current spatial model. It has been widely used in emerging fields such as photogrammetry, robotics, autonomous driving, and AR.

Currently, in view of the problems of large hub airport services such as large space, complex personnel and many service items, how to design a map construction method based on SLAM to provide a basis for realizing functions such as independent surveying and mapping, navigation, obstacle avoidance, cruising and monitoring, and then Providing FOD detection and emergency alarms in the airport can not only reduce labor costs, but also improve the service efficiency and quality of the airport.

Contents of the invention

In order to solve the technical problem of how to design a map construction method based on SLAM to provide a basic technical problem for realizing functions such as autonomous surveying and mapping, navigation, obstacle avoidance, cruising and monitoring, the present invention provides a lidar-based semantic slam for airport unmanned vehicles. Mapping method.

The present invention provides a lidar-based semantic slam mapping method for airport unmanned vehicles, which includes the following steps:

The first step: build a three-dimensional map by scanning with two monocular cameras, match the marked two-dimensional map with the three-dimensional map, and construct a three-dimensional semantic map;

The second step: Match the features of the three-dimensional point cloud acquired by the lidar with the semantic point cloud output in the first step. If the preset threshold is reached, it is determined to be true and the semantic information is added to the three-dimensional point cloud of the lidar. In the cloud, output a three-dimensional point cloud with semantic information;

The third step is to construct a raster map through the combined data of lidar, odometry, and GPS+IMU;

The fourth step is to fuse the raster map with the three-dimensional map with semantic information output in the second step.

Preferably, in the third step:

For data fusion through the LIO-SAM algorithm, it is first necessary to determine the machine position and attitude through GPS+IMU as calibration, and then combine the target data detected by the lidar with the calibration data for coordinate transformation and uniform mapping to the machine coordinate system, and the machine surroundings can be obtained The coordinate value of environmental information;

By calculating the inertial measurement unit and odometer data, the robot's pose in the world coordinate system can be obtained. Combined with the distance and angle information of each laser scanning point obtained by the lidar, all lasers at the current moment can be calculated through the triangle theorem. Scan the collection, which records the position of each laser scanning point in the world coordinate system; divide the environment into small grids, each grid represents an area in the map; then, judge each grid based on the lidar data Whether it is occupied by obstacles is represented by the binary form of the raster map, in which the occupied grid is 1 and the unoccupied grid is 0. In this way, the lidar data is converted into a raster map and has the information of each obstacle. Coordinate points.

Preferably, the first step is:

Images are collected through two monocular cameras, the SA algorithm is used for image segmentation, the BM algorithm is used for multi-camera depth estimation, the position of the object in the image is obtained, and a semantic point cloud map is generated.

By integrating lidar, GPS, camera and other technologies, this method can realize functions such as independent surveying and mapping, navigation, obstacle avoidance, cruising and monitoring. It can also provide FOD detection and emergency alarm in the airport, while reducing labor costs. It can also improve the service efficiency and quality of the airport.

Panoramic segmentation is used to extract semantic information from the image, and semantic information is added to the raster map established by laser slam to establish a semantic map.

To solve the problem that laser SLAM cannot provide semantic information, the semantic map construction method based on panoramic segmentation can construct a semantic map. In this method, two monocular cameras are first needed to scan and build a three-dimensional map, and then the existing two-dimensional map is semantically annotated, and the annotated two-dimensional map is matched with the three-dimensional map to construct a three-dimensional semantic map. It is mainly divided into three parts: semantic extraction, raster map construction and semantic map construction. Airport unmanned vehicles use vision sensors, lidar, odometry and IMU sensors. The semantic extraction part uses a panoramic segmentation algorithm to perform panoramic segmentation on the complete spliced image to obtain the type and position of the target object in the image. The grid map construction part uses data from sensors such as lidar, odometry, and inertial measurement units. After preprocessing, the map of the airport's unmanned vehicle environment is constructed to obtain a high-precision environmental grid map. The semantic map construction part combines the information of the semantic extraction part and the information of the raster map construction part. Based on the established environmental raster map, the specific coordinates and depth of the obstacles in the raster map are used to correspond to the output of the semantic extraction part. The obstacle depth is corrected to establish a high-precision semantic map containing object semantic information.

The beneficial effect of the present invention is that it uses hardware such as cameras, lidar, and IMU to transform existing airport vehicles. The camera uses a monocular camera in the visible light band. This camera is cheap and has rich color information and texture information. However, in the airport environment, it is easily affected by natural light and the final image has exposure problems. Lidar uses solid-state lidar and mechanical lidar. Lidar is less affected by light, so it is planned to use lidar and camera fusion to provide a priori information about the surrounding environment for unmanned vehicles and use it for subsequent mapping and positioning.

The semantic SLAM mapping scheme for airport unmanned vehicles disclosed by the present invention: First, it is necessary to use cameras, lidar, IMU, GPS, and odometer combined with a semantic parsing module. The cameras are deployed on the airport unmanned vehicles, and the cameras are used to map environmental objects. Recognition, segmentation, two-dimensional semantic parsing, and depth estimation of environmental objects through multiple cameras. Obtain the 3D SLAM of the current vehicle through lidar and match the 3D point cloud with the camera's 3D point cloud, then combine it with the high-precision raster map of laser SLAM and the rich environmental information of visual SLAM, and combine it with the raster map of laser SLAM Fusion is performed to achieve semantic SLAM mapping of the airport environment.

Further features and aspects of the invention will become apparent from the following description of specific embodiments with reference to the accompanying drawings.

Description of drawings

Figure 1 is a flow chart of the semantic slam mapping method for airport unmanned vehicles based on lidar;

Figure 2 is a comparison diagram of edge detection using the Segment Anything algorithm and the Sim-T algorithm;

Figure 3 shows the mapping results of airport unmanned vehicles using various algorithms in the airport;

Figure 4 is the relative translation error of each algorithm in Figure 3 after returning to the starting point, the unit is meters;

Figure 5 is the actual measurement result of semantic composition of an airport tractor, where Figure (a) is a point cloud map and Figure (b) is semantic composition;

Figure 6 is the actual measurement result of semantic composition of the airport corridor bridge, where Figure (a) is a point cloud map and Figure (b) is semantic composition.

Detailed ways

The present invention will be further described in detail with specific embodiments below with reference to the accompanying drawings.

As shown in Figure 1, the semantic slam mapping system for airport unmanned vehicles based on lidar is as follows:

1. Camera system semantic point cloud module

Multiple cameras are placed on both sides above the front of the unmanned vehicle, at the same horizontal line, with a fixed baseline. The multi-camera system consists of two left and right cameras, which need to be calibrated in advance through conventional calibration technology to obtain the internal parameters of the camera to correct the camera. In the fixed road section of the airport, the SA (segment anything) algorithm is used to collect images of vehicles under ideal parameters and perform real-time panoramic segmentation and semantic acquisition of the collected images. In a multi-camera system, the common area depth map is estimated based on the disparity maps of the left and right views, and the 3D point cloud is reconstructed based on the 3D reconstruction algorithm. Then the semantic information of the two-dimensional image is integrated into the three-dimensional point cloud to construct a semantic point cloud.

2. Lidar and camera fusion module

The lidar is placed above the front side of the unmanned vehicle, and the three-dimensional point cloud obtained by the lidar is matched with the semantic point cloud output by the camera system. When the preset threshold is reached, it is determined to be true and the semantic information is added to the laser In the three-dimensional point cloud of the radar, a three-dimensional point cloud with semantic information is output.

3. Raster map building module

This is accomplished through the combined data of lidar, odometer, and GPS+IMU, including the following steps: data collection, multi-source data fusion, local map update, and global map update.

3.1 Data collection

Lidar uses the ToF method. Starting from a certain point, it measures in all directions from multiple angles to obtain the distance and angle information between the laser reflection point and the lidar. The obtained point cloud data also needs to be down-sampled using voxel filtering to reduce the size of the point cloud. Computational quantity; Inertial Measurement Unit (IMU) is composed of an accelerometer, a gyroscope and a magnetometer. By integrating the acceleration, the attitude, position, speed and acceleration relative to the world coordinate system are measured in combination, combined with GPS. Real-time positioning information can avoid offset errors caused by slippage and other issues; the odometer can use the rotation information of the vehicle's tire rolling to estimate the vehicle's moving distance.

3.2 Multi-source data fusion

The fusion of multi-sensor data needs to ensure the time and space synchronization of each sensor information. Time synchronization adopts the soft synchronization method, using timestamps to match information from different sensors. The interpolation method is used to unify each sensor data to the sensor data with a longer scanning period. After obtaining the sensor data at the same time, the data is fused through the LIO-SAM algorithm. Spatial synchronization is to unify the information of each sensor into the world coordinate system. First, the machine position and attitude need to be determined through GPS+IMU as calibration, and then the target data detected by the lidar is combined with the calibration data for coordinate transformation and unified mapping to machine coordinates. Under the system, the coordinate values of the machine's surrounding environment information can be obtained.

3.3 Update local map

By calculating the inertial measurement unit and odometer data, the robot's pose in the world coordinate system can be obtained. Combined with the distance and angle information of each laser scanning point obtained by the lidar, all lasers at the current moment can be calculated through the triangle theorem. Scan collection, which records the position of each laser scanning point in the world coordinate system. Divide the environment into small grids, each grid represents an area on the map. Then, based on the lidar data, it is determined whether each grid is occupied by obstacles. The binary form of the raster map can be used, in which the occupied grid is 1 and the unoccupied grid is 0. In this way, the lidar data is converted into a raster map with the coordinate points of each obstacle.

3.4 Update global map

Continuous lidar scans are processed using algorithms, new lidar data are integrated with existing maps, and obstacle information in the map is updated.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes.

Claims (3)

1. The airport unmanned vehicle semantic slam map building method based on the laser radar is characterized by comprising the following steps of:

the first step: a three-dimensional map is established through scanning of two monocular cameras, and the marked two-dimensional map is matched with the three-dimensional map to construct a three-dimensional semantic map;

and a second step of: performing feature matching on the three-dimensional point cloud obtained by the laser radar and the point cloud with the semantics output in the first step, judging to be true if the three-dimensional point cloud reaches a preset threshold value, adding semantic information into the three-dimensional point cloud of the laser radar, and outputting the three-dimensional point cloud with the semantic information;

thirdly, constructing a grid map through combined data of a laser radar, an odometer and a GPS+IMU;

and fourthly, fusing the grid map with the three-dimensional map with the semantic information output in the second step.

2. The method for semantic slam mapping of an airport unmanned vehicle based on laser radar according to claim 1, wherein in the third step:

the LIO-SAM algorithm is used for data fusion, the position and the gesture of the machine are determined through the GPS and the IMU to be used as calibration, and then the target data detected by the laser radar are combined with calibration data to be subjected to coordinate transformation and uniformly mapped to a machine coordinate system, so that coordinate values of surrounding environment information of the machine can be obtained;

the pose of the robot under the world coordinate system can be obtained through calculation of the inertial measurement unit and the odometer data, the distance and angle information of each laser scanning point obtained by the laser radar are combined, all laser scanning sets at the current moment can be obtained through calculation through a triangle theorem, and the position of each laser scanning point under the world coordinate system is recorded in the sets; dividing the environment into small grids, each grid representing an area in the map; then, it is determined whether each cell is occupied by an obstacle from the laser radar data, expressed using a binary form of a grid map in which occupied cells are 1 and unoccupied cells are 0, so that the laser radar data is converted into a grid map and has coordinate points of each obstacle.

3. The method for constructing the semantic slam map of the airport unmanned vehicle based on the laser radar according to claim 2, wherein the process of the first step is as follows:

the method comprises the steps of acquiring images through two monocular cameras, performing image segmentation through an SA algorithm, performing multi-camera depth estimation through a BM algorithm, obtaining the position of an object in the images, and generating a semantic point cloud map.