CN117308950A - Method for realizing autonomous navigation of robot by BIM - Google Patents

Abstract

The invention relates to a method of using BIM to realize autonomous navigation of robots. First, the BIM model is used to generate a navigation map, and then the scene is optically scanned through three-dimensional scanning to obtain a point cloud image of the scene and semantic segmentation is performed to obtain a semantic map. The robot uses the semantic map to realize autonomy. navigation. The method of using BIM model to map conversion and semantic understanding to realize autonomous navigation of robots can overcome the obstacles of existing robot platforms in realizing automation and other related fields, and increase the autonomy of robots.

Description

Methods for realizing autonomous navigation of robots using BIM

Technical field

The present invention relates to the technology of autonomous navigation of external robots, and specifically to a method for realizing autonomous navigation of robots by utilizing BIM model to map conversion and semantic understanding.

Background technique

The field of industrial automation has made great progress in recent years, but there is still a need for robot solutions that are easier to use, versatile, and user-friendly to meet the different needs of various industries. Traditional industrial robots often require complex programming and integration, which creates challenges for small-scale operations and ad hoc automation scenarios. Additionally, the navigation, initial mapping, and reporting capabilities of existing robots may not meet the needs of different industries and tasks.

The present invention aims to address these challenges by introducing a new artificial intelligence multi-purpose mobile robot platform. The platform aims to overcome key barriers to robotic automation to enhance robot autonomy, namely:

1. Navigation limitations: Existing robotic solutions often involve detailed mapping processes that require navigation in complex environments, which may affect operational efficiency. These solutions are not suitable for establishing temporary automation solutions and will affect the robot's ability to navigate autonomously and adapt to different environments.

2. The challenge of establishing an initial navigation map: The deployment process of many robots requires manual scanning of the site and determining the initial position setting of the robot, which is time-consuming and error-prone. Current robots cannot use maps and lidar data to perform accurate initial mapping of the environment, further complicating the deployment process.

3. Insufficient reporting: Many bot solutions are limited in functionality such as providing reports, preventing users from getting relevant information and feedback about the bot’s operation and performance. Also due to the lack of a relatively comprehensive reporting function, this will hinder the subsequent continuous improvement of the robot and prevent users from fully utilizing the potential of robot automation operations.

Contents of the invention

The present invention aims to provide a method for realizing robot autonomous navigation using BIM model to map conversion and semantic understanding, so as to overcome the obstacles of existing robot platforms in realizing automation and other related fields, and increase the autonomy of robots.

The technical solution of the present invention is a method for realizing robot autonomous navigation using BIM, which includes the following steps:

S1. Generate a navigation map from the BIM model, including using the BIM model data to generate a two-dimensional navigation map and a model point cloud map, and forming the boundary data corresponding to the object in the model point cloud map according to the object boundaries in the BIM model; and according to the objects in the BIM model The information in obtains the semantic information of the object;

S2. Optically scan the site corresponding to the BIM model through three-dimensional scanning and obtain the site point cloud map. Use the above boundary data and the model point cloud map to group the site point cloud maps into object groups and use semantic information to perform semantic segmentation on the site point cloud map to form site semantics. map;

S3. Output the two-dimensional navigation map and semantic map to the robot. After receiving the navigation instructions, the robot conducts autonomous navigation based on the two-dimensional navigation map and semantic map.

Preferably, the step of using the BIM model to generate a two-dimensional navigation map in step S1 is: exporting the BIM model data as a two-dimensional map, and modifying the exported two-dimensional map; exporting the modified two-dimensional map as a graphic. The file is then converted into an executable ROS map.

Preferably, the two-dimensional map is an AutoCad format document. The two-dimensional map in AutoCad format is edited to generate a graphic file, and finally the PGM and YAML files of the ROS map are generated.

Preferably, the step of performing semantic segmentation on the on-site point cloud image to obtain a semantic map in step S2 is:

S2-1. The on-site point cloud image obtained by three-dimensional scanning is grouped into objects. The grouping method is to judge the geometric and spatial information composed of each point in the on-site point cloud image through an algorithm based on proximity examples and analyze it to determine whether it belongs to the corresponding Objects to implement object grouping;

S2-2. Form a mapping relationship between the semantic information of the object obtained from the BIM model and the corresponding object in the on-site point cloud map;

S2-3. Repeat steps S2-1 and S2-2 to form a mapping relationship between the objects in the on-site point cloud map and the semantic models of the BIM model to obtain the on-site semantic map.

Preferably, the method of grouping objects in the on-site point cloud map also includes encoding the geometric information in the point cloud data in the object grouping and inputting it into a neural network for processing to identify useful geometric features; the neural network performs feature encoding on the geometric information. To filter the noise information in the on-site point cloud image; then decode the feature encoding to complete the semantic segmentation, and form a mapping relationship with the semantic information of the corresponding object.

Preferably, the following steps are also included:

S4. Update the semantic map, use the on-site point cloud map to identify and classify the geometric and spatial features of the object, perform semantic segmentation and then update the semantic map and BIM model.

Preferably, the following steps are also included:

S5. Based on the report of on-site information, compare the on-site point cloud image obtained by 3D scanning with the point cloud image generated by the BIM model, and report based on the comparison results.

Compared with the existing technology, the beneficial effects of the present invention are as follows: the invention has a wide range of commercial applications, especially in construction sites and factory environments. include:

Construction sites: The operational efficiency of construction sites can be significantly improved through the capabilities of semantic segmentation and BIM models. By providing accurate as-built information in real time, project managers and field engineers can make more informed decisions about resource allocation, scheduling and progress tracking. In addition, semantic segmentation can help autonomous navigation systems accurately identify and avoid obstacles in complex construction sites, further improving the efficiency of on-site operations. The resulting BIM model can be used to report construction status in real time, allowing potential problems to be identified early and dealt with promptly, reducing the possibility of costly delays and rework.

Factory environment: The invention can also significantly improve the efficiency and productivity of factory environments. By leveraging semantic segmentation technology to identify and locate different objects and machinery on the factory floor, robotic systems can navigate their environment more safely and efficiently. Additionally, integrated BIM-to-map automated creation allows factory managers to create accurate 3D models of facilities, providing valuable insights into the layout and positioning of equipment. It can also quickly and accurately simulate factories, creating digital models to further optimize automation implementation. These models can be used to optimize the placement of machines and materials, shortening production times and reducing costs.

Construction environment: In addition to defect inspection or patrolling, the invention can enhance the robot's ability to navigate in the construction environment, perform specific tasks and report related issues and findings in a human-like manner.

The invention's potential to improve efficiency and quality across a variety of industries therefore highlights the technology's broad applicability and its value as a versatile and powerful tool.

Description of drawings

Figure 1 is a work flow chart of the present invention;

Figure 2 is a workflow diagram for generating navigation diagrams from BIM models in the present invention;

Figure 3 is a workflow diagram for converting three-dimensional scanned point clouds into semantic maps in the present invention;

Figure 4 is a top view of the braking mechanism of the present invention;

Implementation

The following will give a clear and complete description of the concept, specific structure and technical effects of the present invention in conjunction with the embodiments and drawings, so as to fully understand the purpose, solutions and effects of the present invention. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of this application can be combined with each other.

The main purpose of this system is to launch a new artificial intelligence multi-purpose mobile robot platform by overcoming existing challenges such as navigation, initial mapping and reporting deficiencies. The overall workflow diagram of this system is shown in Figure 1. BIM files first extract semantic information through boundary generation and conversion of BIM to model point cloud images. Once objects are located in the model point cloud, semantic annotation is performed and each object is assigned a corresponding category to generate a semantic map. Then, the semantic map is matched with the segmented point cloud output by semantic segmentation. Then, the location of the semantic graph can be found. Additionally, these positions will be input into the autonomous navigation system, including building initial navigation maps, navigation and semantic position-based reporting.

In establishing the initial navigation map, the semantic map can provide additional information for the robot to establish the initial navigation map. Since there are many similar different areas in the map, it is difficult to identify the initial position based only on the shape information of the area. Precisely because the autonomous navigation system is placed in different locations, the navigation map initialized each time may also be different. Building an initial navigation map helps obtain the correct map position and obtain the current position of the autonomous navigation system.

During the navigation process, semantic maps can help the autonomous navigation system better understand the surrounding environment and objects in the environment, thereby helping the autonomous navigation system navigate in the environment. Semantic maps can help autonomous navigation systems identify objects in the environment and differentiate between them. This can help autonomous navigation systems maneuver around obstacles and avoid collisions. Additionally, it can be used to plan and execute tasks in the environment. For example, if an autonomous navigation system is tasked with picking up an object from a specified location, a semantic map can provide information about the object's location and path to reach it. Therefore, semantic maps can help autonomous navigation systems navigate, giving them a better understanding of the environment and objects in it. This improves the efficiency and accuracy of navigation and increases safety and reliability.

In semantic location-based reporting, semantic maps aid reporting by providing more accurate and relevant information about the location of events or incidents (such as defect inspections). By using semantic localization, bots can provide users with more detailed information about where an event occurred, helping users better understand the context and significance of the event. In addition, semantic localization can help bots identify patterns and trends in events occurring in specific locations, which is useful for developing deeper and more insightful coverage. Therefore, semantic localization can improve the quality and accuracy of coverage.

The following is a detailed description of each work process of the present invention:

BIM, Building Information Model, is a model that digitizes the physical and functional characteristics of a building or structure. It involves using computer-aided design (CAD) software to create a virtual model of a building that contains data about its components, systems and materials.

BIM-generated maps include two parts, BIM-to-3D model world map and BIM-to-2D map. BIM-generated maps are mainly used for Gazebo and RVIZ simulations, which can help users simulate in the virtual world and obtain visualization results. Developers can use this solution to start programming and modeling without a real-world environment. In recent years, BIM has become increasingly popular in the construction industry as more and more companies realize the benefits of using BIM technology. In the future, BIM mapping methods will be widely used in the construction and manufacturing industries.

The workflow of BIM map generation is shown in Figure 2. First, select the 2D map in Revit and generate its .dwg file. Then import the file into AutoCAD. Different Revit models will have CAD drawings in different formats. The user needs to delete all marks, dimensions and other useless parts of the drawing. For example, parts that can be passed by autonomous navigation systems or people should have corresponding passages, while walls or boundaries that cannot be passed by autonomous navigation systems or people should be connected by uninterrupted lines, etc. When the CAD file modification is completed, export the PNG or JPG file from AutoCAD. The format of the ROS map is PGM, so it also needs to be converted. Finally, when the YAML file containing the PGM map is generated, it will become an executable ROS map. The YAML file should contain some necessary parameters, such as the path of the ROS map, resolution, original 2D pose, occupancy threshold, idle threshold and negate parameters, etc. After starting navigation in RVIZ, the user needs to initialize the position and attitude. Users can also send ROS messages to set initial location and destination.

In the traditional process of using ROS simulation, users need to perform SLAM (Simultaneous Localization and Mapping) in the environment and draw a map that can be used for RVIZ. Therefore, both the usage environment and the autonomous navigation system must be ready. However, sometimes we need to simulate and derive feasibility results before the building is built. If we switch to a BIM-to-map approach, we do not need to set up the environment and draw the map, but can directly perform simulations in the virtual environment created by the BIM model.

The benefits of using BIM mapping include increased efficiency, improved accuracy, reduced errors and rework, and better communication among stakeholders. BIM can also help reduce costs and shorten project times through more efficient scheduling and coordination. BIM maps can work with computer vision to accomplish certain tasks. Specifically, maps generated from BIM files can be automatically linked with information provided by BIM, which is important data that can be used for reporting purposes.

By using lidar to scan the site and obtain 3D data accordingly, the system can automatically segment important building structures such as walls, columns, beams, pipes and HVAC systems. This process involves identifying and classifying individual components in point cloud data based on geometric and spatial characteristics. Advanced deep learning algorithms such as ResPointNet++ can be used to improve the efficiency and accuracy of semantic segmentation. The segmented components are then converted into corresponding BIM elements to generate detailed and accurate as-built BIM models. This automated process simplifies the creation of BIM models to better manage building information, enhance maintenance planning and improve facility management efficiency.

The specific workflow is shown in Figure 2 and described in detail as follows:

Step 1: Input data: First, obtain the on-site point cloud image through lidar, and input the on-site point cloud image data into the neural network. Each point in the on-site point cloud map is a three-dimensional space point, which contains information such as XYZ coordinates, intensity values, and reflectivity data.

Step 2 Grouping: Since the on-site point cloud map can contain multiple objects, it is best to group close points before segmenting the on-site point cloud map to separate different objects. For grouping, it is mainly based on geometric and spatial information from on-site point cloud images. Based on an empirically set threshold, if these detected points are too far away from the relevant area, they should be classified into other object groups.

Step 3 Feature Encoding: This step involves encoding the geometric information of the scene point cloud map so that it can be processed through the neural network, inputting the scene point cloud map data with spatial information to the neural network, and utilizing multiple neural network layers to capture the scene Local geometric features in point cloud images. Additionally, the points are normalized by taking samples of a set size. Based on the feature encoding process, the encoding features can be used to compress the local geometric information in the on-site point cloud image and pass the useful information to the decoder, which can further reduce the noise caused by the capture device and the surrounding environment.

Step 4 Feature Decoding: After encoding the features, they need to be decoded to generate semantic segmentation of the point cloud. This is achieved by applying a feature decoding process, which uses a decoding network to generate a set of per-point labels corresponding to different object categories in the scene. The decoding network can exploit additional information such as RGB images or other sensor data to improve segmentation accuracy.

Step 5 Output result: The output result is semantic segmentation of the on-site point cloud image obtained by lidar. Semantic segmentation is a color-coded map of a scene point cloud, where each point is assigned a label that corresponds to a specific object category in the scene, such as buildings, roads, or trees. This semantic segmentation can be used as input for subsequent applications such as object detection, autonomous navigation and other related applications.

By applying feature encoding and decoding processes, the proposed method can process and segment large, complex, and irregularly shaped live point clouds with varying densities. The resulting semantic segmentation can improve accuracy and efficiency in a wide range of applications, including self-driving cars, robotics, and urban planning.

a) Use BIM for semantic annotation

Based on the technology of generating navigation maps from BIM models, the model point cloud images in the BIM model are marked and grouped into objects. When grouping and labeling, measure the distance between each point and all objects, and divide the point to the object with the smallest distance to it. The labeled points are used to form a semantic map, which is useful for the next stage of semantic localization.

b) Use lidar semantic map for global semantic positioning

The workflow of semantic positioning is shown in Figure 3. Before semantic positioning, semantic information must be obtained in the step of converting the BIM model into a navigation map. Then, 3D lidar is used to perform 3D scanning, and the on-site point cloud map is formed in the scan. Then, semantic segmentation is performed to group and label the field point cloud. Then, point matching is performed based on distance and semantic information respectively, and the semantic information is matched with the scanned on-site point cloud. Although the neighbor distance algorithm can be used to find the correspondence between points and objects, further utilizing semantic information can filter out false matches and further ensure the correctness of the correspondence between points and objects. Then, point-object correspondences are found by minimizing spatial and semantic distances in the optimization transformation step. Finally, after performing the optimization transformation, the semantic information can be aligned with the on-site point cloud map of the 3D lidar scan to provide semantic localization.

c) Use semantic positioning for navigation

Navigation using semantic positioning refers to using semantic information to assist navigation and path planning. This may involve identifying landmarks, points of interest, or other semantic information to help guide users to their destination. Semantic positioning can be used in a range of navigation applications, including GPS navigation systems, indoor navigation systems and augmented reality navigation. In each case, the aim is to provide users with a clear and easy-to-understand route to their destination, and to use semantic information to guide navigation.

In indoor navigation systems, semantic localization can be used to identify key features of indoor environments, such as rooms, elevators, or corridors. This semantic information can then be used to provide users with clear directions to their destination, taking into account the semantic context of the indoor environment.

d) Report based on positioning information

Reporting using semantically positioned information is the process of presenting or summarizing information in a way that takes into account its specific three-dimensional location, context, and semantics. Generate reports based on the above process that highlight key features, objects, or changes in a specific three-dimensional environment based on semantic tags or attributes, or use semantic data related to on-site point clouds obtained by lidar to provide positioning information about specific topics or events.

In the context of robotics, semantic localization information can be used to generate reports about the location and semantics of objects or obstacles in a warehouse or manufacturing facility. This may involve identifying key objects based on semantic tags or attributes and presenting this information in a way that is relevant and meaningful to the robot control system.

The above are only preferred embodiments of the present invention. The present invention is not limited to the above-mentioned embodiments. As long as the technical effects of the present invention are achieved by the same means, any modification can be made within the spirit and principles of the present disclosure. Any modifications, equivalent substitutions, improvements, etc. shall be included in the scope of protection of this disclosure. All belong to the protection scope of the present invention. Various modifications and changes may be made to the technical solutions and/or implementations within the scope of the present invention.

Claims (7)

1. The method for realizing autonomous navigation of the robot by using BIM is characterized by comprising the following steps: the method comprises the following steps:

s1, generating a navigation map by using a BIM model, wherein the navigation map comprises a two-dimensional navigation map and a model point cloud map generated by using BIM model data, and boundary data corresponding to an object in the model point cloud map is formed according to the boundary of the object in the BIM model; acquiring semantic information of the object according to the information of the object in the BIM;

s2, carrying out optical scanning on a field corresponding to the BIM model through three-dimensional scanning, obtaining a field point cloud picture, carrying out object grouping on the field point cloud picture by utilizing the boundary data and the model point cloud picture, and carrying out semantic segmentation on the field point cloud picture by using semantic information to form a field semantic map;

and S3, outputting the two-dimensional navigation map and the semantic map to the robot, and performing autonomous navigation according to the two-dimensional navigation map and the semantic map after the robot receives the navigation instruction.

2. The method for realizing autonomous navigation of a robot by using BIM according to claim 1, wherein: the step of generating the two-dimensional navigation map by using the BIM model in the step S1 is as follows: the BIM model data are exported as a two-dimensional map, and the exported two-dimensional map is modified; the modified two-dimensional map is exported as a graphic file and converted to an executable ROS map.

3. The method for realizing autonomous navigation of a robot by using BIM according to claim 2, wherein the two-dimensional map is an AutoCad format document, the two-dimensional map in the AutoCad format is edited to generate a graphic file, and finally PGM and YAML files of the ROS map are generated.

4. The method for realizing autonomous navigation of a robot by using BIM according to claim 1, wherein: in the step S2, the step of performing semantic segmentation on the field point cloud image to obtain a semantic map includes:

s2-1, performing object grouping on the field point cloud image obtained by three-dimensional scanning, wherein the grouping method is to judge geometrical and spatial information formed by each point in the field point cloud image through an algorithm based on a proximity example for analysis so as to judge that the point cloud image belongs to a corresponding object and realize object grouping;

s2-2, forming a mapping relation between semantic information of the object obtained from the BIM model and the corresponding object in the field point cloud picture;

s2-3, repeating the steps S2-1 and S2-2 to form a mapping relation between the object of the field point cloud picture and semantic information of the BIM model, and then obtaining the field semantic map.

5. The method for achieving autonomous navigation of a robot using BIM according to claim 4, wherein: the method for grouping the on-site point cloud images further comprises the steps of encoding geometric information in point cloud data in the object groups and inputting the geometric information into a neural network for processing to identify useful geometric features; the neural network performs feature coding on the geometric information to filter noise information in the field point cloud picture; and then, decoding the feature codes to complete semantic segmentation, and forming a mapping relation with semantic information of the corresponding object.

6. The method for achieving autonomous robot navigation using BIM according to claim 1, wherein: the method also comprises the following steps:

s4, updating the semantic map, identifying and classifying geometric and spatial features of the object by using the on-site point cloud picture, and updating the semantic map after semantic segmentation.

7. The method for realizing autonomous navigation of a robot by using BIM according to claim 1, wherein: the method also comprises the following steps:

s5, comparing the field point cloud image obtained by three-dimensional scanning with the point cloud image generated by the BIM based on the report of the field information, and reporting according to the comparison result.