CN115439621A - A 3D map reconstruction and target detection method for a coal mine inspection robot - Google Patents

Abstract

The invention discloses a three-dimensional map reconstruction and target detection method for a coal mine underground inspection robot, which comprises the following steps: step 1, acquiring a point cloud data set containing selected scene information; step 2, point cloud data is subjected to point cloud registration by adopting a Map-to-Map method to obtain an established three-dimensional Map; step 3, positioning the routing inspection route of the routing inspection robot in real time according to the three-dimensional map; step 4, in the process of the inspection robot advancing, a fusion algorithm is utilized to carry out target detection; and 5, optimally matching the target detection results obtained in the step 4, realizing classification of the detection results, and preferentially outputting the detection results as the obstacle targets. The invention can also carry out effective high-precision map construction for the situation of complex environmental results; meanwhile, partial redundant data can be removed, the algorithm time complexity is reduced, and the image building efficiency is improved; the fusion algorithm takes time domain information into consideration, fuses the multi-frame detection results, and can effectively reduce false detection rate and missed detection rate caused by shielding.

Description

A 3D map reconstruction and target detection method for a coal mine inspection robot

technical field

The invention belongs to the technical field of inspection robots, and relates to a three-dimensional map reconstruction and target detection method for an inspection robot in underground coal mines.

Background technique

Since 2016, the annual procurement of inspection robots has tended to increase exponentially. Most companies have realized the necessity and trend of replacing manpower with intelligent inspection robots. As industry segmentation and scenarios become more and more clear, customers According to their own personalized customization is more and more diverse, in the period of rising trend, the demand will inevitably increase every year in the future. At present, coal mines contain a wide variety of manually driven fuel vehicles or electric vehicles for tasks such as material transportation, patrol inspection, and personnel connection, which have problems such as low operating efficiency, large consumption of human resources, and high operating costs. At the same time, traditional inspections are done manually, which is labor-intensive, low in efficiency, low in detection quality, difficult to guarantee personal safety, heavy in the inspection workload, and restricted by harsh environments, so that the quality and quantity of the inspection work cannot be guaranteed. Therefore, unmanned inspection robots are the best choice to replace traditional human inspections.

The core problem of unmanned inspection robots is positioning. Inaccurate positioning will cause major mistakes in decision-making and control. The goal of the path planning of the inspection robot is to use the appropriate path planning method to find the optimal path without collision from the starting point to the end point. In a controllable structured environment, the inspection robot has a certain degree of autonomy and can perceive the surrounding environment. Detect obstacles and plan a collision-free path to navigate to your destination. Literature [1] describes the SLAM (Simultaneous Localization and Mapping) path planning problem of inspection robots. Literature [2] compares two SLAM schemes to obtain choices in different scenarios. Literature [3] studied the use of probabilistic methods to optimize map positioning accuracy and reduce computational complexity. Literature [4] uses sparse pose adjustment to solve the matrix direct solution problem in nonlinear optimization. After studying the literature [5], it is concluded that the Gauss-Newton method can be used to solve the scan matching problem in map positioning, but it has high requirements for sensors. Finally, through the research of literature [6], the Cartographer algorithm is analyzed, and the cumulative error is reduced by using closed-loop detection on the local subgraph and the global graph at the same time.

At the same time, the obstacle detection of traditional unmanned inspection robots cannot meet the needs of environmental perception. Therefore, the method of multi-sensor fusion is proposed to improve the accuracy, real-time and robustness of obstacle detection. Lu Feng et al. used the convolutional neural network (Faster R-CNN) algorithm model to train the actual robot to collect data, and selected the center point set of the lower edge of the matrix frame in the image target detection to match with the radar target detection target data point set. The closest point matching (ICP) algorithm removes the target pseudo point pairs, realizes the data fusion of the image and the 3D point cloud, and effectively improves the accuracy and efficiency of the algorithm. Li Yanfang et al. used lidar to emit laser beams and receive target echoes, extract the closest echo points for cluster analysis, and judge whether the echo points are obstacles; image detection uses YOLO network training data sets to generate target frames, and through radar targets The detected detection frame is fused with the detection frame of the camera target detection, and the percentage of the overlapping area of the bounding frame is used as the criterion for judging obstacles, realizing the decision-making fusion of the lidar and the camera, and improving the detection accuracy of vehicles and pedestrians by about 5%.

To sum up, how to build high-precision maps and obstacle inspections is a hot research issue in current unmanned inspections. The main problems in the above-mentioned existing technologies are as follows: the construction of high-precision maps is limited by sensors and algorithms, and it is difficult to perform mapping and positioning operations for environments with complex structures; Most of the redundant data leads to imprecise estimation of the position of obstacles in the real world.

The related documents are as follows:

[1] Chen Zhuo, Su Weihua, Wei Ning, et al. Implementation of Mobile Robot SLAM and Path Planning under ROS Framework [J]. Medical and Health Equipment, 2017,38(02):109-113.

[2] Luan Jianing, Zhang Wei, Sun Wei, et al. High-precision positioning algorithm based on fusion of two-dimensional code vision and laser radar [J]. Computer Applications, 2021, 41(05): 1484-1491.

[3] Liu Liwei, Zhu Xukang, Li Xiuhua, et al. Research on 2D SLAM Algorithm Map Evaluation for Low-cost Mobile Robots [J]. Computer Simulation, 2021,38(04):291-295+320.

[4] Zhao Ruoyu, Guan Zhiwei, Tong Minyong, et al. Functional optimization design of SLAM system based on single-line lidar [J]. China Automotive, 2021(02):4-9+43.

[5] Han Wenhua. Research on mobile robot dynamic environment navigation based on single-line laser radar [D]. Harbin Institute of Technology, 2019.

[6] Ji Xingliang. Research on Synchronous Positioning and Composition Algorithm Based on 3D Laser Point Cloud Segmentation and Matching [D]. University of Electronic Science and Technology of China, 2020.

Contents of the invention

The purpose of the present invention is to provide a three-dimensional map reconstruction and target detection method for a coal mine underground inspection robot to solve the technical problems of how to construct a high-precision map in a complex structural environment and how to accurately estimate obstacles in a high-precision map.

In order to achieve the above object, the technical solution adopted in the present invention is:

A method for three-dimensional map reconstruction and target detection of a coal mine inspection robot, comprising the following steps:

Step 1: Select and build a map scene, collect and process the point cloud data of the selected scene, and obtain a point cloud dataset containing the information of the selected scene;

Step 2, use the Map-to-Map method to perform point cloud registration on the point cloud data of the scene obtained in step 1, and obtain the established 3D map; specifically include the following sub-steps:

Step 21, obtaining continuous n frames of point cloud data from the point cloud data set obtained in step 1, establishing a local submap Submap for the n frames of point cloud data, and using it as the current latest local submap;

Step 22, reacquire continuous n frames of point cloud data from the remaining point cloud data in the point cloud data set, and establish a local submap Submap for the n frames of point cloud data;

Step 23, use the global method in the cartographer algorithm to perform loopback detection in the backend process, if there is a loopback, go to step 24; otherwise discard the local subgraph established in step 22, and return to step 22;

Step 24, matching the local subgraph established in step 22 with the current latest local subgraph to obtain the matched local subgraph, and updating the matched local subgraph to the current latest local subgraph; wherein, the matching is Refers to the superposition of two local subgraphs;

Step 25, return to step 22, until all the point cloud data in the point cloud data set obtained in step 1 are acquired, then the current latest local submap obtained at this time is the required three-dimensional map;

Step 3, according to the three-dimensional map generated in step 2, perform real-time positioning on the inspection route of the inspection robot;

Step 4, during the traveling process of the inspection robot, use the fusion algorithm to detect the target;

Step 5, perform optimal matching on the target detection results obtained in step 4, realize the classification of the detection results, and select the best output as the obstacle target.

Further, in the step 1, the 16-line laser radar is used to collect the laser point cloud by using the reserved command to generate the bag file containing the selected scene information; then the bag file is subjected to noise reduction filtering processing to obtain the processed bag The file is the point cloud dataset containing the selected scene information.

Further, in the step 23, the backend refers to the optimization operation part of the cartographer algorithm; the occurrence of the loop means that the similarity between the local subgraph established in step 22 and the current latest local subgraph is greater than a certain weight value.

Further, in the step 24, the superposition of the two local sub-images refers to discarding redundant point cloud frame data in the two local sub-images, and performing two local sub-images after discarding the redundant data. Splicing results in a new local subgraph.

Further, said step 3 includes the following sub-steps:

Step 31, using the IMU inertial sensor to collect pose information; output as csv file information by command;

Step 32, select the coordinate system of the IMU inertial sensor as the positioning operation coordinate system;

Step 33, use the csv file generated in step 31 and the 3D map generated in step 2, and cooperate with the Cartographer-SLAM algorithm to locate part of the source code to perform the route positioning operation of the inspection robot.

Further, said step 4 includes the following sub-steps:

Step 41, performing noise reduction processing on the point cloud data collected in step 1 through statistical filtering, and outputting the point cloud information after noise reduction;

Step 42, use the RealsenseD435i camera to collect image information for the scene selected in step 1, then input the image information to the fusion algorithm CenterNet network for target detection, and output the position of the center point of the target and the target category C, which is known from the KIITI dataset;

Step 43, input the point cloud data output in step 41 to the fusion algorithm CenterPiont network for target detection, output the target 3D detection frame, distance and category C, and extract the center point of the 3D target detection frame.

7. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot according to claim 1, wherein the step 5 includes the following sub-steps:

步骤43得到的目标3D检测框中心点位置表示为

n为选取的中心点的个数；Step 51, respectively select the position of the center point of the target obtained in step 42 and the position of the center point of the target 3D detection frame obtained in step 43 to form point pairs as the input of the improved KNN algorithm, and obtain n×n point pairs in total; wherein, step 42 The obtained target center point position is expressed as

The position of the center point of the target 3D detection frame obtained in step 43 is expressed as

n is the number of selected center points;

Step 52, calculating the Euclidean distance between two center points in each point pair selected in step 51;

对应的n个点对的欧式距离最小的点对作为最优点对，共得到n个最优点对，对每个最优点对进行融合作为检测到的障碍物目标，共得到n个障碍物目标。Step 53, in the Euclidean distance of all point pairs obtained in step 52, select

The point pair with the smallest Euclidean distance of the corresponding n point pairs is used as the optimal point pair, and a total of n optimal point pairs are obtained, and each optimal point pair is fused as the detected obstacle target, and a total of n obstacle targets are obtained.

The beneficial effects of the present invention are:

(1) Mapping part: Based on the Cartographer algorithm, a Map to Map method is proposed to match local subgraph information, thereby making loopback detection and improving its matching degree. At the same time, redundant information between data frames is removed, and the map is improved. accuracy and mapping efficiency. It is more suitable for mapping of mobile robots. After the algorithm is improved, it can also construct effective high-precision maps for complex environmental results; at the same time, it can remove some redundant data, reduce the time complexity of the algorithm, and improve the efficiency of map construction;

(2) Target detection part: Based on multi-sensor fusion, target detection and tracking are performed on the original data of the image and point cloud respectively, and the improved KNN algorithm is used to optimally match the detection results, and the projection of the point cloud to the image is used to correct the mismatch The center points of the image detection are further matched, and finally the fusion result is selected and output. Sensor fusion improves data reliability and target detection accuracy; at the same time, the fusion algorithm considers time domain information and fuses multi-frame detection results, which can effectively reduce the false detection rate and missed detection rate caused by occlusion.

Description of drawings

Figure 1 shows the back-end loopback detection error due to the front-end matching error in the prior art;

Figure 2 is a comparison of the effect of Cartographer algorithm before and after the improvement of the map (point cloud registration map);

Figure 3 is a comparison of the mapping effect before and after improvement;

Figure 4 is a comparison of CPU usage before and after improvement under the same server;

Figure 5 is the positioning effect of back-end closed-loop detection under RVIZ;

Fig. 6 is a schematic diagram of an inspection robot;

Figure 7 is the fusion algorithm framework;

Fig. 8 is a block diagram of target center point matching structure;

Fig. 9 is a connection relationship diagram between a center point set and an image detection center point;

Figure 10 is the image detection test results; where (a) is the target occlusion scene, (b) is the scene with uneven illumination, and (c) is the scene with a long distance;

Figure 11 is point cloud target detection; among them, (a) is the target occlusion scene, (b) is the scene with uneven illumination, and (c) is the scene with a long distance;

Figure 12 is the detection results of lidar in different scenes (detection regression 3D frame); among them, (a) is the target occlusion scene, (b) is the scene with uneven illumination, and (c) is the scene with a long distance;

Figure 13 is the fusion detection result of point cloud and image detection; where (a) is the image detection result, and (b) is the fusion algorithm detection result.

The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments.

detailed description

1. SLAM mapping and positioning

1.1 Graph optimization algorithm

The Cartographer algorithm belongs to the graph-optimized SLAM algorithm. The graph-optimized SLAM adjusts the front-end pose through the back-end loop detection, and finally obtains the current robot position and pose closest to the real value. The graph optimization SLAM problem is decomposed into two tasks:

(1) Construct the graph, the pose of the robot is regarded as the vertex, and the relationship between the poses is regarded as the edge (front-end - the accumulation of sensor information);

(2) Optimize the graph, adjust the robot pose vertices to meet the constraints of the edges (back-end);

1.1.1 Cartographer Algorithm

The Cartographer algorithm is an open source algorithm proposed by Google in 2016. A new loop detection method based on lidar point cloud data is proposed. This method can reduce the amount of calculation, meet the needs of large-scale mapping, and optimize large-scale data in real time. The whole algorithm is divided into two parts: Local SLAM (front-end matching), and GlobalSLAM (back-end closed loop). The present invention improves the front end of the Local and cooperates with the back-end closed-loop link of the Cartographer algorithm to improve the accuracy of the reconstructed map.

The original algorithm adopts the Scan-to-Map registration method in the SLAM point cloud registration in the front-end point cloud registration. After each frame of laser scan (radar frame data) data is obtained, it is matched with the currently recently established Map. Insert the laser scan data of this frame into the Map. The disadvantage of this method is that when the environment structure is similar, the collected point cloud data frames will have similar repetitions. When used for back-end closed-loop detection, it will lead to failure to close the loop , and thus failed to build the map.

As shown in Figure 1, due to the front-end matching error, the back-end detection cannot be closed.

1.1.2 Point cloud registration

Therefore, based on the Cartographer algorithm, a new lidar point cloud registration method Map-to-Map is proposed for point cloud registration. After obtaining certain frames of point cloud data, a Submap (local submap) is established. After each local submap is established, it is matched with the most recently established Submap, thereby avoiding the matching of point cloud data frames with similar environmental structures, resulting in subsequent Terminal closed loop detection error. The specific process is shown in Figure 2.

2. Target detection

Before target detection, joint calibration of lidar and depth camera is required. Joint calibration is the basis of data fusion. Through joint calibration, the transformation matrix of camera coordinate system and lidar coordinate system is obtained, and the data of radar and camera are fused to obtain The data information is more comprehensive and accurate. Figure 6 is the joint calibration diagram.

The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot of the present invention comprises the following steps:

Step 1. Select and build a map scene, collect and process the point cloud data of the selected scene, and obtain the point cloud dataset containing the information of the selected scene: through the 16-line laser radar (model is robsense), use the reserved command (rosbag record+topic name) Collect the laser point cloud and generate a bag file containing the selected scene information; then perform noise reduction filter processing (statistical filter algorithm) on the bag file to obtain the processed bag file;

Step 2, use the Map-to-Map method to perform point cloud registration on the point cloud data of the scene obtained in step 1, and obtain the established 3D map; as shown in Figure 2, the specific operations are as follows:

Step 21, obtaining continuous n frames of point cloud data from the point cloud data set obtained in step 1, establishing a local submap Submap for the n frames of point cloud data, and using it as the current latest local submap;

Step 22, reacquire continuous n frames of point cloud data from the remaining point cloud data in the point cloud data set, and establish a local submap Submap for the n frames of point cloud data;

Step 23, utilize the global (global) mode in the cartographer algorithm, carry out loopback detection in the backend process, if loopback occurs, then perform step 24; otherwise discard the local subgraph established in step 22, and return to step 22;

Described back end refers to: cartographer algorithm map construction optimization operation part; Described loopback refers to that the similarity between the local subgraph established in step 22 and the current latest local subgraph is greater than a certain weight, and the weight can be set by yourself;

Step 24, matching the local subgraph established in step 22 with the current latest local subgraph to obtain the matched local subgraph, and updating the matched local subgraph to the current latest local subgraph; wherein, the matching is Refers to superimposing two local submaps (that is, discarding the redundant point cloud frame data in the two local submaps, and splicing the two local submaps after discarding the redundant data to obtain a new local submap);

Among them, in the matching process of two local sub-images, step 22 establishes the point cloud data of the local sub-image and inserts the pose conversion formula when inserting the current latest local sub-image as follows:

表示包含场景信息的点云数据集中的每个点云的姿态，

为点云在x,y方向的平移，

为平面的旋转量；

为点云在当前局部子图中的位姿；p为障碍物存在的概率值，可自行设定。in,

Represents the pose of each point cloud in the point cloud dataset containing scene information,

is the translation of the point cloud in the x and y directions,

is the rotation amount of the plane;

is the pose of the point cloud in the current local submap; p is the probability value of the existence of obstacles, which can be set by yourself.

Step 25, return to step 22 until all point cloud data in the point cloud data set obtained in step 1 are acquired, then the current latest local submap obtained at this time is the required 3D map.

Using the Map-to-Map method in step 2 for point cloud registration to construct a 3D map can avoid the existing method: every time a frame is obtained containing the selected scene information data, it is put into the established sub-graph, The matching of point cloud data frames with similar environmental structures leads to back-end closed-loop detection errors.

Step 3, according to the three-dimensional map generated in step 2, perform real-time positioning on the inspection route of the inspection robot; the specific operation is as follows:

Step 31, use the IMU inertial sensor, connect to the computer, and collect the pose information; output the csv file information through the command;

Step 32, select the coordinate system of the IMU inertial sensor as the positioning operation coordinate system;

Step 33, use the csv file generated in step 31 and the 3D map generated in step 2 to coordinate with the Cartographer-SLAM algorithm to locate part of the source code (enter the absolute path of the file in the source code and modify the parameters of the configuration file) to locate the route of the inspection robot operate. The results in Figure 5 are obtained (the solid line in the figure is the trajectory of the positioning operation).

Step 4. During the inspection robot's travel, use the fusion algorithm to detect the target. The specific operation is as follows:

In step 41, the point cloud data collected in step 1 is subjected to noise reduction processing through statistical filtering, and the point cloud information after noise reduction is output.

Step 42, use the RealsenseD435i camera to collect image information for the scene selected in step 1, then input the image information to the fusion algorithm CenterNet network for target detection, and output the position of the center point of the target and the target category C, which is known from the KIITI dataset;

Step 43, input the point cloud data output in step 41 to the fusion algorithm CenterPiont network for target detection, output the target 3D detection frame, distance and category C, and extract the center point of the 3D target detection frame.

Specifically, as shown in Figure 6, the inspection robot is 95cm high and the chassis is 25cm high. Both the laser radar and the camera are installed on the inspection robot, the camera is located directly below the laser radar, and the IMU inertial sensor is installed on the chassis of the inspection robot.

Step 5 and Step 4 respectively output the position of the center point of the image target and the position of the point cloud target. In these center points, there are many detection situations, including correctly detected and tracked targets, wrongly detected targets, etc. Use the improved KNN algorithm to optimally match the target detection results obtained in step 4, realize the classification of the detection results, and select the optimal output as the obstacle target. The specific operations are as follows:

步骤43得到的目标3D检测框中心点位置表示为

n为选取的中心点的个数；In step 51, the position of the center point of the target obtained in step 42 and the position of the center point of the target 3D detection frame obtained in step 43 are respectively selected to form point pairs as the input of the improved KNN algorithm, and a total of n×n point pairs are obtained. Wherein, the target central point position obtained in step 42 is expressed as

The position of the center point of the target 3D detection frame obtained in step 43 is expressed as

n is the number of selected center points;

Step 52, calculating the Euclidean distance between two center points in each point pair selected in step 51;

对应的n个点对的欧式距离最小的点对作为最优点对，共得到n个最优点对，对每个最优点对进行融合，此处融合是指将该点对中的两个点合并为一个点(任选其一作为点对的中心点位置)，作为检测到的障碍物目标，共得到n个障碍物目标。Step 53, in the Euclidean distance of all point pairs obtained in step 52, select

The point pair with the smallest Euclidean distance of the corresponding n point pairs is used as the optimal point pair, and a total of n optimal point pairs are obtained, and each optimal point pair is fused. Fusion here refers to merging two points in the point pair is a point (choose one of them as the center point position of the point pair), and as the detected obstacle target, a total of n obstacle targets are obtained.

In the process of center point fusion, the current frame image target point and the center point of the target detection frame in the point cloud realize data fusion. The black point is the target detection center point set in the point cloud data, and the white point is the center point of the image detection result ( See Figure 9), the two are matched by the closest distance in step 6, and merged into the same target center point, and the connection relationship is shown in Figure 8.

In order to verify the feasibility and effectiveness of the present invention, carried out following test:

1. SLAM mapping and positioning

In order to prove that the improved Cartographer algorithm has a better mapping effect in the case of similar environmental structures, this experiment site chooses a simulated coal mine underground to collect laser point cloud data, and uses the improved algorithm before and after to construct the map. Comparing the mapping time efficiency, mapping effect and mapping accuracy (the mapping operation is performed through the collected and processed bag files, the mapping efficiency is shown in Figure 4), which shows that the improved algorithm is better than the pre-improved algorithm.

(1) Contrast chart of mapping effect:

Figure 3 is a comparison of the effect of map building before and after improvement:

Before improvement: When the environment structure is similar, there is a loopback detection error, resulting in low accuracy of the map construction;

After improvement: For situations with complex and similar environmental structures, loopback detection can still be performed to achieve the effect of closed loop

(2) Comparison of mapping efficiency

By comparing the CPU occupancy rate (under the same server) during the map building process before and after the improved algorithm, the map building efficiency before and after the improved algorithm can be obtained.

In this experiment, a quad-core processor was used for the map building experiment. It can be seen from Figure 4 that the CPU occupancy rate before improvement is higher than that after improvement, and the CPU occupancy rate after improvement is lower, and it tends to increase with time. Yu stable. Therefore, under the same experimental environment, the improved Cartographer algorithm is more suitable for mobile robot mapping than before.

(3) Back-end closed-loop positioning

Through the proposed new lidar point cloud registration method Map-to-Map. After acquiring certain frames of point cloud data, establish a Submap (local submap), perform point cloud registration, and remove redundant data at the same time. It is transmitted to the back end for closed-loop detection and positioning. Figure 5 shows the positioning effect of back-end closed-loop detection under RVIZ:

Since the above steps have been improved in point cloud matching, the obtained 3D map effect can achieve better results than before. Then the 3D map can be used as the input of the positioning operation to ensure the accuracy requirements, so it can be obtained from the RVIZ software. It can be seen (trajectory line depicted in the figure) that the effect is better.

2. Target detection

①After the training and processing of the KITTI dataset, select 3 data sets of different vehicle driving scenarios for testing, and mark the categories and target locations. The image detection test results are shown in Figure 9;

②Camera-dependent CenterNet image target detection has great limitations. In Figure 10, the target occlusion scene and the scene with uneven illumination have obvious omissions of occluded vehicles. The accuracy is greatly reduced, resulting in a large number of missed targets. The test results show that camera-based target detection is difficult to meet the requirements of open-air automatic driving mining trucks.

③ Figure 11(a), (b), and (c) are the detection results of lidar in different scenarios. From the detection frame in Figure 11(a), it can be known that the image detection did not detect the target, while the lidar detection to the target. It can be seen from Fig. 11(b) that the target position detected by lidar deviates from the actual target position. Since the point cloud data is discretely distributed in form, the position and interval of data points are irregularly distributed in three-dimensional space, resulting in Points of the same type are difficult to select, and the target position is not precise. As shown in Figure 11(b), due to the irregularity of the point cloud data itself, the positions of the red detection frame and the green detection frame deviate in different scenarios. To obtain more accurate target location information, the image detection results are fused with the point cloud detection results to solve this problem.

④In different test scenarios, there are obvious missing detections in image detection. In point cloud target detection, distant and blurred targets can be effectively located and captured. As shown in Figure 12(b), the fusion algorithm Compared with the single image detection method, the number of occluded vehicle detections increases by 7. Obviously, in the case of occlusion and uneven illumination, the fusion algorithm effectively reduces the missed detection rate.

in conclusion:

(1) Mapping positioning: This paper studies the Graph-slam (graph optimization) mapping algorithm--Cartographer algorithm, and analyzes the point cloud registration method Scan to Map (frame and local submap) method of the algorithm, which is similar to the environment structure In some cases, there are loopback detection errors, resulting in low accuracy of map construction. At the same time, there are redundant data information between frame data, and the efficiency of map construction is low. Aiming at such problems, a new LiDAR point cloud registration method--Map toMap (local submap and local submap) registration method is proposed on the basis of the algorithm, and the point cloud data is obtained to construct a local submap. Local submaps are registered with local submaps to perform loop closure detection. Improve its matching degree, remove redundant information between data frames at the same time, improve map accuracy and map construction efficiency. It is more suitable for mapping of mobile robots.

(2) Obstacle detection: This invention proposes a radar and camera decision-making layer fusion method, using 16-line laser radar and Realsense D435 depth camera to obtain point cloud data, and projecting point cloud data and image data through joint calibration of laser radar and camera To the same coordinate system, combined with the target detection algorithm, the obstacles in the image are returned to the target center point (including depth, size, category), and the obstacles in the point cloud are returned to the target center point set. Use the KNN (K-NearestNeighbor) method to fuse the center point of the image and point cloud data to achieve optimal matching of the detection results, and finally output a fusion result with high reliability.

Claims (7)

1. A coal mine underground inspection robot three-dimensional map reconstruction and target detection method, is characterized in that, comprises the steps: Step 1: Select and build a map scene, collect and process the point cloud data of the selected scene, and obtain a point cloud dataset containing the information of the selected scene; Step 2, use the Map-to-Map method to perform point cloud registration on the point cloud data containing scene information obtained in step 1, and obtain the established 3D map; specifically include the following sub-steps: Step 21, obtaining continuous n frames of point cloud data from the point cloud data set obtained in step 1, establishing a local submap Submap for the n frames of point cloud data, and using it as the current latest local submap; Step 22, reacquire continuous n frames of point cloud data from the remaining point cloud data in the point cloud data set, and establish a local submap Submap for the n frames of point cloud data; Step 23, use the global method in the cartographer algorithm to perform loopback detection in the backend process, if there is a loopback, go to step 24; otherwise discard the local subgraph established in step 22, and return to step 22; Step 24, matching the local subgraph established in step 22 with the current latest local subgraph to obtain the matched local subgraph, and updating the matched local subgraph to the current latest local subgraph; wherein, the matching is Refers to the superposition of two local subgraphs; Step 25, return to step 22, until all the point cloud data in the point cloud data set obtained in step 1 are acquired, then the current latest local submap obtained at this time is the required three-dimensional map; Step 3, according to the three-dimensional map generated in step 2, perform real-time positioning on the inspection route of the inspection robot; Step 4, during the traveling process of the inspection robot, use the fusion algorithm to detect the target; Step 5, perform optimal matching on the target detection results obtained in step 4, realize the classification of the detection results, and select the best output as the obstacle target. 2. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot as claimed in claim 1, wherein in the step 1, the 16-line laser radar is used to collect laser point clouds using reserved instructions to generate The bag file containing the selected scene information; after that, the bag file is subjected to noise reduction and filtering processing, and the processed bag file is obtained, which is the point cloud dataset containing the selected scene information. 3. coal mine underground inspection robot three-dimensional map reconstruction and target detection method as claimed in claim 1, it is characterized in that, in described step 23, described back end refers to cartographer algorithm map building optimization operation part; Described occurrence loop is It means that the similarity between the local subgraph established in step 22 and the latest local subgraph is greater than a certain weight. 4. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot as claimed in claim 1, wherein in the step 24, the superposition of the two local sub-graphs refers to discarding the two local sub-graphs The redundant point cloud frame data, and splicing the two local submaps after discarding the redundant data to obtain a new local submap. 5. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot as claimed in claim 1, wherein said step 3 comprises the following sub-steps: Step 31, using the IMU inertial sensor to collect pose information; output as csv file information by command; Step 32, select the coordinate system of the IMU inertial sensor as the positioning operation coordinate system; Step 33, use the csv file generated in step 31 and the 3D map generated in step 2, and cooperate with the Cartographer-SLAM algorithm to locate part of the source code to perform the route positioning operation of the inspection robot. 6. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot according to claim 1, wherein the step 4 includes the following sub-steps: Step 41, performing noise reduction processing on the point cloud data collected in step 1 through statistical filtering, and outputting the point cloud information after noise reduction; Step 42, use the RealsenseD435i camera to collect image information for the scene selected in step 1, then input the image information to the fusion algorithm CenterNet network for target detection, and output the position of the center point of the target and the target category C, which is known from the KIITI dataset; Step 43, input the point cloud data output in step 41 to the fusion algorithm CenterPiont network for target detection, output the target 3D detection frame, distance and category C, and extract the center point of the 3D target detection frame. 7. The three-dimensional map reconstruction and target detection method of the coal mine underground inspection robot according to claim 1, wherein the step 5 includes the following sub-steps: Step 51, respectively select the position of the center point of the target obtained in step 42 and the position of the center point of the target 3D detection frame obtained in step 43 to form point pairs as the input of the improved KNN algorithm, and obtain n×n point pairs in total; wherein, step 42 The obtained target center point position is expressed as

The position of the center point of the target 3D detection frame obtained in step 43 is expressed as

n is the number of selected center points; Step 52, calculating the Euclidean distance between two center points in each point pair selected in step 51; Step 53, in the Euclidean distance of all point pairs obtained in step 52, select the point pair with the minimum Euclidean distance of n point pairs corresponding to P _yi as the optimal point pair, and obtain n optimal point pairs altogether, for each optimal point Fusion is carried out as the detected obstacle target, and a total of n obstacle targets are obtained.

Images