CN117733854A - Operation and maintenance management method and device for composite robot - Google Patents

Abstract

The present invention relates to the field of computer technology and provides a method and device for operation and maintenance management of a composite robot. The method includes using a laser radar sensor to obtain distance information of the environment, constructing an environment map based on the distance information; and based on the emergency level and emergency level of each task. According to the current location of each robot, the corresponding task is assigned to the corresponding robot; and the corresponding starting point and end point are designated for the robot. Based on the environment map, starting point and end point, a path planning algorithm is used to plan the traveling route; the robot follows the traveling route Marching, during the marching process, the robot's motion trajectory is adjusted according to the real-time position of the robot and the location of obstacles. The present invention centrally manages multiple robots and multiple tasks, and takes into account the current location of the robot and the emergency level of the task, thereby planning a suitable path for the robot to improve the efficiency of the robot and ensure that emergency tasks are completed. Deal with it promptly.

Description

A compound robot operation and maintenance management method and device

Technical field

The present invention relates to the field of computer technology, and in particular to a compound robot operation and maintenance management method and device.

Background technique

With the continuous advancement of industrial technology in modern society, industrial robots are widely used in various industries. Especially in the automobile manufacturing industry, large-scale industrial robots are put into use in OEMs, parts factories, and production lines to improve production efficiency and product quality.

Traditional operation and maintenance task management systems often face the cost of hiring and training personnel to perform tasks. It is difficult to optimize manual task allocation, leading to a decrease in efficiency. It is difficult for personnel to adapt to environmental changes in real time, which can easily lead to task failure. Some operation and maintenance tasks may be harmful to personnel. Safety poses risks. At this time, it is necessary to use robots to perform corresponding tasks. In the existing technology, a robot often performs complex and fixed tasks, but the robot's execution of tasks is often accompanied by positional movement, and some tasks require emergency processing. When the robot is far away from the task end position, the processing will be delayed and the task processing efficiency of the robot will also be affected.

In view of this, overcoming the shortcomings of the prior art is an urgent problem to be solved in this technical field.

Contents of the invention

The technical problem to be solved by this invention is that the existing technology often involves a complex and fixed task for a robot, but the robot's execution of tasks is often accompanied by positional movement, and some tasks require emergency processing. When the robot is far away from the end point of the task, it will This results in untimely processing and also affects the robot's task processing efficiency.

The present invention adopts the following technical solutions:

In a first aspect, the present invention provides a composite robot operation and maintenance management method, including:

Use lidar sensors to obtain distance information of the environment and build an environment map based on the distance information;

According to the urgency level of each task and the current location of each robot, the corresponding task is assigned to the corresponding robot;

According to the task corresponding to the robot, specify the corresponding starting point and end point for the robot, and use the path planning algorithm to plan the traveling route based on the environment map, starting point and end point;

The robot travels according to the travel route. During the travel process, the camera is used to collect image data of the environment, and obstacles in the environment are identified from the image data. The robot's movement is adjusted based on the real-time position of the robot and the location of the obstacles. Trajectory to achieve robot obstacle avoidance.

Preferably, the corresponding tasks are assigned to the corresponding robots according to the emergency level of each task and the current location of each robot, specifically including:

If the robots closest to the end of each task are different, each task will be assigned to the robot closest to the end of the corresponding task;

If there is a situation where a robot is closest to the end point of multiple tasks, the reciprocal of the distance between the robot and the end point of the corresponding task is used as the first base value, and the product of the emergency level of the task corresponding to the robot and the first base value is used as the corresponding task. The allocation base value is the maximum sum of the allocation base values of all tasks as the objective function, and one task corresponds to one robot as the constraint function; combine the objective function and constraint function to establish a task decision-making model;

Solve the task decision-making model to obtain the task allocation method.

Preferably, the obstacle in the environment is identified from the image data, and the movement trajectory of the robot is adjusted according to the real-time position of the robot and the location of the obstacle, specifically including:

Identify whether there are obstacles in the direction of travel within the preset range from the robot's real-time position;

If there is an obstacle, target recognition is performed on the obstacle. If the obstacle is identified as a moving animal, multiple image data are combined to analyze the moving direction and speed of the obstacle. According to the moving direction and speed of the obstacle, and The robot's own traveling speed determines whether there is a risk of collision with the obstacle; if there is a risk of collision, adjust the robot's traveling direction to the opposite direction of the obstacle to avoid the obstacle; if there is no risk of collision The risk of collision will cause the robot to follow the established travel route;

If it is recognized that the obstacle is a fixed object, the robot's direction will be adjusted to either side of the obstacle to avoid the obstacle.

Preferably, based on the environment map, starting point and end point, a path planning algorithm is used to plan the traveling route, which specifically includes:

The first constraint function is that the time required to reach the end point from the starting point does not exceed the preset maximum time, and the second constraint function is that the energy consumed from the starting point to the end point does not exceed the preset maximum energy;

Taking one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point, and the minimum number of direction adjustments required from the starting point to the end point as the planning goal, generate the corresponding objective function;

Combine the objective function, the first constraint function and the second constraint function to establish a path planning model;

Solve the path planning model to obtain the traveling route.

Preferably, the method further includes:

Calculate the remaining energy situation of the robot before the first travel route planning based on the travel route that has been planned for each robot, and determine the charging emergency of the robot based on the remaining energy situation;

Compare the emergency level of the robot's corresponding task with the charging emergency level, and according to the comparison result, selectively move the robot along the first travel route or charge the robot; wherein, the first travel route The route planned for the first route of travel.

Preferably, the emergency level of the robot's corresponding task is compared with the charging emergency level, and based on the comparison result, the robot is selectively moved along the first travel route or the robot is charged, which specifically includes:

If the charging urgency of the robot is higher than the emergency level of the corresponding task, the robot is charged; wherein the range of the urgency level of the task is within the first interval, and the range of the charging urgency of the robot is within the second interval. , the minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the first interval is greater than the maximum value of the second interval;

Otherwise, the robot is moved along the first traveling route.

Preferably, charging the robot specifically includes:

The planning goals are to generate the corresponding goals with the smallest distance between the target charging device and the robot, the fastest charging of the robot by the target charging device, and the highest balance between the number of robots using each target charging device for charging. function;

Use the objective function to establish a charging decision model, solve the charging decision model, and obtain the target charging equipment for charging the robot. With the target charging equipment as the end point, perform route planning for the robot so that the robot Arrive at the target charging equipment location for charging.

Preferably, the method further includes:

If the travel route planning of the corresponding task fails, the robot corresponding to the task will be charged;

If the emergency level of the task is higher than the preset level, the robot corresponding to the task closest to the end of the task will be called from the task with a lower emergency level than the task; the task of the called party will be called from the task than the called one. The task is transferred from a task with a lower emergency level until the emergency level of the corresponding task is not higher than the preset level.

Preferably, each robot communicates with the central control platform via wireless, and the central control platform also communicates with the control terminal via wireless. The control terminal has a positioning function, and the environment map is displayed in the control terminal and displayed in the environment map. The actual traveling path of the robot, identifying obstacles in the environment from the image data, and adjusting the movement trajectory of the robot based on the real-time position of the robot and the location of the obstacles, also includes:

During the movement of the robot, the surrounding environment point cloud data of the location where the robot passes is collected. Based on the surrounding environment point cloud data, it is judged whether each path location belongs to an indoor area or an outdoor area. The surrounding environment point cloud data is analyzed to obtain the indoor and outdoor dividing lines. , establish a building map based on the indoor and outdoor dividing lines;

Match the planned travel route of the robot with the building map to obtain the buildings that the robot needs to inspect;

If the robot detects an obstacle ahead while traveling, the robot's motion trajectory will be adjusted to bypass the obstacle; if the obstacle is identified as a fixed object, the robot will record the time it starts to bypass the obstacle until it returns. The obstacle bypass path along the planned travel route is based on the starting position and end position of the obstacle bypass path, and is matched with the planned travel route to obtain the original planned path, and it is judged whether the original planned path belongs to the interior of the corresponding building. area;

If the original planned path belongs to the indoor area of the corresponding first building, a blind spot mark is displayed on the environmental map of the control terminal corresponding to the location of the first building, so that the controller can identify the blind spot through the environmental map, and carry the control terminal to perform manual inspection of the first building;

When the controller clicks on the blind spot mark and chooses to perform blind spot inspection, the controller is navigated according to the real-time positioning information of the control terminal; wherein the end point of the navigation route displayed in the environmental map is the The starting position of the obstacle avoidance path;

After identifying that the controller reaches the starting position of the obstacle bypassing path based on the real-time positioning information of the control terminal, the controller's action trajectory is recorded, and after identifying that the controller reaches the end position of the obstacle bypassing path, When the action trajectory of the controller is matched with the first building, it is determined whether the action trajectory has entered the indoor area of the first building;

If it is determined that the action trajectory has entered the indoor area of the first building, the action trajectory will be displayed in the environment map, and a prompt window will be displayed around the action trajectory, and the controller will be asked in the prompt window Whether the action trajectory can be used by the robot, and provide a confirmation button and a cancel button for the controller to choose;

If the controller selects the confirmation button, the next time the robot performs path planning and obtains the buildings to be inspected including the first building, the action trajectory will be used to replace the first building in the route. A partial path of a building to facilitate subsequent inspection of the first building by the robot.

In a second aspect, the present invention also provides a composite robot operation and maintenance management device, which includes an environment perception and navigation module and a core management module;

The environment perception and navigation module is used to obtain distance information of the environment using a lidar sensor and construct an environment map based on the distance information;

The core management module is used to assign corresponding tasks to corresponding robots according to the emergency level of each task and the current location of each robot; specify the corresponding starting point and end point for the robot according to the task corresponding to the robot, and use the path planning algorithm to plan a travel route based on the environment map, the starting point and the end point; and enable the robot to move along the travel route. During the movement, a camera is used to collect image data of the environment, obstacles in the environment are identified from the image data, and the robot's motion trajectory is adjusted according to the robot's real-time position and the location of the obstacle to achieve obstacle avoidance of the robot.

In a third aspect, the present invention also provides a composite robot operation and maintenance management device for implementing the composite robot operation and maintenance management method described in the first aspect. The device includes:

At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor for The composite robot operation and maintenance management method described in the first aspect is executed.

In a fourth aspect, the present invention also provides a non-volatile computer storage medium that stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors to complete the first step. The composite robot operation and maintenance management method described in the aspect.

The present invention centrally manages multiple robots and multiple tasks, and takes into account the current location of the robot and the emergency level of the task, thereby planning a suitable path for the robot to improve the efficiency of the robot and ensure that emergency tasks are completed. Deal with it promptly.

Description of drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of the first composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 2 is a schematic flow chart of the second composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 3 is a schematic flow chart of the third composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 4 is a schematic flow chart of the fourth composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 5 is a schematic flowchart of the fifth composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 6 is a schematic flow chart of the sixth composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 7 is a schematic flowchart of the seventh composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 8 is a schematic diagram of the architecture between the control terminal, the robot and the central control platform in a compound robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 9 is a schematic flowchart of the eighth compound robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 10 is a schematic diagram of the first composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 11 is a schematic diagram of the second composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 12 is a schematic diagram of the third composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 13 is a schematic diagram of the fourth composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 14 is a schematic diagram of the fifth composite robot operation and maintenance management method provided by an embodiment of the present invention;

Figure 15 is a schematic module diagram of a compound robot operation and maintenance management device provided by an embodiment of the present invention;

Figure 16 is a module schematic diagram of yet another compound robot operation and maintenance management device provided by an embodiment of the present invention;

Figure 17 is a schematic architectural diagram of a compound robot operation and maintenance management device provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

The terms "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, features defined by "first," "second," etc. may explicitly or implicitly include one or more of such features. In the description of this application, unless otherwise stated, "plurality" means two or more.

In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Example 1:

Embodiment 1 of the present invention provides a composite robot operation and maintenance management method, as shown in Figure 1, including:

In step 201, use a lidar sensor to obtain distance information of the environment, and construct an environment map based on the distance information;

In step 202, corresponding tasks are assigned to corresponding robots based on the urgency level of each task and the current location of each robot; the urgency of the task is analyzed in advance by those skilled in the art based on task requirements.

In step 203, a corresponding starting point and end point are specified for the robot according to the task corresponding to the robot. Based on the environment map, starting point and end point, a path planning algorithm is used to plan a traveling route.

In step 204, the robot travels according to the travel route. During the travel process, the camera is used to collect image data of the environment, and obstacles in the environment are identified from the image data. According to the real-time position of the robot and the location of the obstacles, , adjust the robot's motion trajectory to achieve the robot's obstacle avoidance.

In actual use, the starting point is usually the charging station or docking point of the robot (i.e., the current location), the end point is the task end point, and the path planning algorithm can be the Dijkstra algorithm (referred to as: Dijkstra algorithm) .

This embodiment centrally manages multiple robots and multiple tasks, and takes into account the current location of the robot and the emergency level of the task, thereby planning an appropriate path for the robot to improve the efficiency of the robot and ensure emergency tasks. were dealt with in a timely manner.

In actual application scenarios, the corresponding tasks are assigned to the corresponding robots based on the emergency level of each task and the current location of each robot, as shown in Figure 2, which specifically includes:

In step 301, if the robots closest to the end points of each task are different, each task is assigned to the robot closest to the end point of the corresponding task.

In step 302, if there is a situation where a robot is closest to multiple task end points, the reciprocal of the distance between the robot and the corresponding task end point is used as the first base value, and the relationship between the emergency level of the robot's corresponding task and the first base value is used. The product of is used as the allocation base value of the corresponding task, the maximum sum of the allocation base values of all tasks is used as the objective function, and one task corresponds to one robot as the constraint function; combined with the objective function and constraint function, a task decision-making model is established.

Solve the task decision-making model to obtain the task allocation method.

In an optional implementation, the objective function is expressed in the form of a formula:

Among them, N is the total number of tasks, is the jth task, M is the total number of robots,/> is the i-th robot, is the distance between the j-th task and the i-th robot,/> is the urgency level of the j-th task, Indicates whether to assign the j-th task to the i-th robot. If the j-th task is assigned to the i-th robot, then , otherwise,/> . Make decisions with the goal of maximizing the value of the objective function.

Among them, the obstacle in the environment is identified from the image data, and the movement trajectory of the robot is adjusted according to the real-time position of the robot and the location of the obstacle, as shown in Figure 3, specifically including:

In step 401, identify whether there are obstacles in the traveling direction within a preset range from the real-time position of the robot; the preset range is obtained based on empirical analysis by those skilled in the art. The traveling direction is an angular range set by those skilled in the art, such as identifying whether there are obstacles within a 30° angle formed by taking the front of the robot as the center and within 1.5m from the robot.

In step 402, if there is an obstacle, target identification is performed on the obstacle. If the obstacle is identified as a moving animal, multiple image data are combined to analyze the moving direction and moving speed of the obstacle. According to the moving direction of the obstacle, and the moving speed of the robot, as well as the robot's own moving speed, to determine whether there is a risk of collision with the obstacle; if there is a risk of collision, adjust the robot's moving direction to the opposite direction of the obstacle to avoid the obstacle. objects; if there is no risk of collision, let the robot travel along the established route.

In step 403, if the obstacle is identified as a fixed object, the robot's direction is adjusted to either side of the obstacle to avoid the obstacle. The movable animals include humanoids, animals, traveling vehicles and other robots, etc. The fixed objects are fixed objects, which can be obtained through object recognition. All objects except humanoids, animals, traveling vehicles and other robots can be regarded as fixed objects. ; In actual use, when the obstacle is recognized to be a moving animal or humanoid, the opponent will be reminded to avoid it through voice in advance. If a collision risk is still detected after the preset time, the direction of travel will be adjusted.

When it is recognized that the obstacle is another robot and there is a risk of collision, the central control system is notified through the wireless module. The central control system will, according to the tasks corresponding to each robot, let the one with a lower emergency level of the task avoid the one with a higher emergency level. one party.

This embodiment analyzes the types of obstacles to determine whether there is a risk of collision, and then selects a corresponding avoidance method, thereby making obstacle avoidance more accurate, improving the safety of the robot's obstacle avoidance, and improving the efficiency of the robot.

In actual use, the path planning algorithm is used to plan the travel route based on the environment map, starting point and end point, as shown in Figure 4, which specifically includes:

In step 501, the first constraint function is that the time required to reach the end point from the starting point does not exceed the preset maximum time, and the second constraint function is that the energy consumed from the starting point to the end point does not exceed the preset maximum energy; where, The preset maximum time and the preset maximum energy are obtained by those skilled in the art based on empirical analysis. In actual use, the preset maximum energy may be the current power of the robot.

In step 502, one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point, and the minimum number of direction adjustments required from the starting point to the end point is used as the planning goal, and a corresponding objective function is generated; Considering that in actual use, the robot may need to adjust its direction when it reaches the end point, and the robot usually needs to stop first and then deflect, which takes longer than the robot moving forward. When the robot needs to adjust its direction, When the number of direction adjustments is large, it will also affect the robot's traveling efficiency. Therefore, this embodiment also takes the number of direction adjustments required from the starting point to the end point into consideration, thereby optimizing the objective function and making the final planned travel route optimal. When there are multiple planning objectives, the objective function is actually a weighted summation between multiple expressions representing the corresponding planning objectives. The positivity and negativity of each weight and the size of the value are obtained by those skilled in the art based on empirical analysis. .

In step 503, a path planning model is established by combining the objective function, the first constraint function and the second constraint function.

In step 504, the path planning model is solved to obtain the traveling route.

In a preferred embodiment, as shown in Figure 5, the method further includes:

In step 601, based on the travel routes that have been planned for each robot, the remaining energy situation of the robot before the first travel route planning is calculated, and the charging emergency level of the robot is determined based on the remaining energy situation; the remaining energy situation That is, the remaining power. The smaller the remaining power, the higher the charging urgency. Determining the charging urgency of the robot according to the remaining energy situation may be: dividing a plurality of remaining energy intervals, each remaining energy interval corresponds to a charging emergency, and the remaining energy interval corresponding to the remaining energy situation has a charging emergency. The degree is the charging urgency of the robot.

In step 602, the emergency level of the robot's corresponding task is compared with the charging emergency level, and based on the comparison result, the robot is selectively moved along the first travel route or the robot is charged; wherein, The first traveling route is the route obtained by planning the first traveling route.

This embodiment compares the urgency of the task with the charging urgency of the robot, and determines whether to charge the robot or continue to let the robot perform the task based on the comparison result, thereby ensuring the normal progress of the task and ensuring that the robot can be charged in time.

In an optional implementation, the emergency level of the robot's corresponding task is compared with the charging emergency level, and based on the comparison result, the robot is selectively moved along the first travel route, or the robot is implemented Charging, specifically including:

If the charging urgency of the robot is higher than the emergency level of the corresponding task, the robot is charged; wherein the range of the urgency level of the task is within the first interval, and the range of the charging urgency of the robot is within the second interval. , the minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the first interval is greater than the maximum value of the second interval.

Otherwise, the robot is moved along the first traveling route.

The first interval and the second interval are both obtained by analysis by those skilled in the art based on the work requirements of the robot. The minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the first interval is greater than the maximum value of the second interval to ensure that when the robot has sufficient power, no matter how urgent the task is, There is no need to charge, and when the emergency level of the task is high, even if the robot's power is low, the task will be executed first. The robot's power to meet the task execution requirements is determined by the constraints of the robot's path planning (i.e. predetermined Assume the maximum energy) limit is achieved.

In actual use, charging equipment, such as charging piles, can also be set up around it. The robot travels to the charging equipment to charge. When there are multiple charging equipment, the robot is charged, as shown in Figure 6. Specifically, include:

In step 701, the planning goal is to have the smallest distance between the target charging device and the robot, the fastest charging of the robot by the target charging device, and the highest balance between the number of robots using each target charging device for charging. , generate a corresponding objective function; the balance degree may be the mean square error or variance of the number of robots charged using each target charging device.

In step 702, use the objective function to establish a charging decision model, solve the charging decision model, and obtain the target charging equipment for charging the robot.

In step 703, with the target charging equipment as the end point, route planning is performed for the robot so that the robot reaches the target charging equipment location for charging.

In some cases, when the remaining power of the corresponding robot cannot meet the requirements of the path planning corresponding to the task, that is, when the above-mentioned second constraint function cannot be satisfied, the task's travel route planning may also fail. In order to deal with this situation , this embodiment also provides a preferred implementation, as shown in Figure 7, that is, the method further includes:

In step 801, if there is a failure in the travel route planning of the corresponding task, the robot corresponding to the task is charged; the reason for the failure in the travel route planning may be that the power of the robot cannot meet the planning requirements of the corresponding path.

In step 802, if the emergency level of the task is higher than the preset level, the robot corresponding to the closest distance to the end point of the task is selected from the tasks with a lower emergency level than the task to execute the task.

In step 803, the called party's task is called from tasks with a lower emergency level than the called party's task until the corresponding task emergency level is not higher than the preset level. The preset level can be obtained by those skilled in the art based on empirical analysis. For example, there are 9 emergency levels preset, and the default level is level 5. When the emergency level of task A is level 8, which is higher than level 5, the corresponding robot will be transferred to task A from the tasks of levels 1 to 7. If the emergency level of the called party B is level 6 and is still higher than level 5, then the corresponding robot will be called from levels 1 to 5 and given to the called party B at level 6, and so on, until The emergency level of the last called party is not higher than level 5.

This embodiment sequentially transfers robots from tasks with a low emergency level to tasks with a high emergency level, thereby ensuring that when the corresponding robot is low on power, priority is given to ensuring that tasks with a high emergency level can be executed.

In actual use, robots may also need to conduct inspections inside buildings, such as inside factories. However, in some scenarios, the entire factory may be repaired or maintained, or some entrances to the factory may be closed for maintenance. In this case, the robot may not be able to inspect the interior of the factory normally. In order to solve this problem, this embodiment also provides the following preferred implementation methods, which specifically include:

Each robot communicates with the central control platform through wireless communication, and the central control platform also communicates with the control terminal through wireless communication, as shown in Figure 8. Among them, the number of control terminals and the number of robots are analyzed by those skilled in the art based on demand, and It is not absolutely a one-to-one relationship. The control terminal has a positioning function, an environment map is displayed in the control terminal, and the actual traveling path of the robot is displayed in the environment map, as shown in Figure 9. From the Obstacles in the environment are identified in the image data, and the robot's movement trajectory is adjusted based on the real-time position of the robot and the location of the obstacles, including:

In step 901, during the movement of the robot, the surrounding environment point cloud data of the location where the robot passes is collected, and each path location is judged to belong to an indoor area or an outdoor area based on the surrounding environment point cloud data, and the surrounding environment point cloud data is analyzed to obtain The indoor and outdoor dividing lines are used to establish a building map based on the indoor and outdoor dividing lines; as shown in Figure 10. Among them, the judgment that each path location belongs to an indoor area or an outdoor area based on the surrounding environment point cloud data is obtained by identifying objects in the surrounding environment point cloud data. For example, when it is recognized that the surrounding areas are shelves, tables and chairs, or have continuous If there is a certain wall, etc., it is judged to be an indoor area. If it is recognized that there are trees, street lights, etc. in the surrounding area, or if there is an open space through scanning (that is, the distance between the point cloud data in a certain direction and the robot is far), then It is determined to be an outdoor area, and one direction is an indoor area and the other direction is an outdoor area, and when relatively close walls are detected between the two opposite directions, the current location is considered to be the entrance and exit of the corresponding building. Wherein, the building map is superimposed on the environment map and displayed in the control terminal.

In step 902, the planned travel route of the robot is matched with the building map to obtain each building that the robot needs to inspect; as shown in Figure 11, the robot's travel route enters buildings 1, 2, and 3 , 4, 5, 7, and 8, then the buildings that need to be inspected are buildings 1, 2, 3, 4, 5, 7, and 8.

In step 903, if the robot detects an obstacle ahead while traveling, the robot's motion trajectory is adjusted to bypass the obstacle; if it is identified that the obstacle is a fixed object (i.e., a fixed object), Through object recognition, all objects except humans, animals, driving vehicles and other robots can be regarded as fixed objects), then the obstacle path that the robot takes from the beginning to bypass the obstacle until it returns to the planned travel route is recorded. The starting position and end position of the obstacle bypassing path are matched with the planned traveling route to obtain the original planned path, and it is judged whether the original planned path belongs to the indoor area of the corresponding building; as shown in Figure 12, when building 5 exits When the outer ring is surrounded due to maintenance or other reasons, resulting in the robot being unable to enter the interior of building 5, the robot will bypass the obstacle. The obstacle path is as marked in Figure 12, where the position marked as the starting point is the starting position of the obstacle path. The position marked as the end point is the end position of the obstacle bypass path, and the dotted line part is the original planned path. Since the original planned path passes through building 5, the original planned area belongs to the indoor area of building 5.

In step 904, if the original planned path belongs to the indoor area of the corresponding first building, a blind spot mark is displayed on the environment map of the control terminal corresponding to the location of the first building to facilitate the control personnel to pass through the environment map. Identify the blind area and carry out manual inspection of the first building with the control terminal.

In step 905, when the controller clicks on the blind spot mark and chooses to perform blind spot inspection, the controller is navigated according to the real-time positioning information of the control terminal; wherein, the navigation route displayed in the environmental map The end point of is the starting position of the obstacle bypass path, that is, the starting point marked in Figure 12; as shown in Figure 13, the question mark icon is used as the blind spot mark, and when the controller clicks on the blind spot mark, an option pops up next to the blind spot mark window. In the options window, the controller is reminded that "this area is blocked to form a blind spot. Should manual inspection be performed on this area?" and provides the "Perform Inspection" option button and the "Cancel" option button. When the controller selects "Perform Inspection" When "check", it is considered that the blind spot inspection is selected, a navigation route is generated from the current position of the controller to the starting position of the obstacle bypass path, and is displayed on the environment map.

In step 906, after identifying that the controller has arrived at the starting position of the obstacle bypass path based on the real-time positioning information of the control terminal, the action trajectory of the controller is recorded, and after identifying that the controller has arrived at the bypass path, When reaching the end position of the obstacle path, the action trajectory of the controller is matched with the first building, and it is determined whether the action trajectory has entered the indoor area of the first building.

In step 907, if it is determined that the action trajectory has entered the indoor area of the first building, the action trajectory is displayed in the environment map, and a prompt window is displayed around the action trajectory. The window asks the controller whether the action trajectory described can be used by the robot, and provides a confirmation button and a cancel button for the controller to choose; as shown in Figure 14, the dotted line position is the action trajectory of the controller, and the prompt window displays "Already Artificially generate a new action trajectory and whether the action trajectory can be reused by the robot." Among them, since most robots do not have the function of autonomously going up and down steps, the controller comprehensively considers factors such as whether there are steps in the action trajectory and whether the path width allows the robot to pass, so as to determine whether the action trajectory can be reused for the robot.

In step 908, if the controller selects the confirmation button, the next time the robot performs path planning and obtains the buildings to be inspected including the first building, the action trajectory will be used to replace the route. A partial path passing through the first building is provided to facilitate subsequent inspection of the first building by the robot. For example, in subsequent use, when the robot reaches the starting position of the obstacle bypassing path, it will travel along the action trajectory until it reaches the end position of the obstacle bypassing path, and then proceed according to the planned path.

If the controller selects the cancel button, the action path will not be reused for the robot. When planning the path for the robot next time, the first building will be avoided and the first building will not be inspected.

Example 2:

Based on the composite robot operation and maintenance management method described in Embodiment 1, this embodiment also provides a composite robot operation and maintenance management device, as shown in Figure 15, including an environment perception and navigation module and a core management module.

The environment perception and navigation module is used to obtain distance information of the environment using a lidar sensor and construct an environment map based on the distance information.

The core management module is used to assign corresponding tasks to corresponding robots according to the emergency level of each task and the current location of each robot; according to the tasks corresponding to the robots, specify corresponding starting points and end points for the robots, based on the environment map , starting point and end point, use the path planning algorithm to plan the traveling route; make the robot travel according to the traveling route, during the traveling process, use the camera to collect image data of the environment, and identify obstacles in the environment from the image data , according to the real-time position of the robot and the location of obstacles, adjust the robot's motion trajectory to achieve obstacle avoidance of the robot.

The composite robot operation and maintenance management device can also be called a composite robot-based operation and maintenance task management system. In a preferred embodiment, as shown in Figure 16, the system includes a core management module, a data center module, an intelligent module, a security module, and a core management module. and authentication module, user interface module, multi-robot collaboration module, maintenance and remote support module, and environment perception and navigation module.

Among them, the core management module is used to coordinate and control composite robot operation and maintenance tasks to ensure that all modules work together to achieve efficient task management; the data center module is used to store all task-related data, including task history, sensor data, and robot status. and performance data; the intelligence module is used to learn and optimize task allocation strategies from historical data by interacting with the data center, and supports automated decision-making to improve the intelligence and adaptive capabilities of the system; the security and authentication module is responsible for ensuring System security, including user authentication, data encryption, permission management and blockchain security; user interface module is used to provide operation and maintenance personnel and administrators with an easy-to-use interface to monitor and manage the robot system; maintenance and remote support The module is used to support maintenance personnel in remote diagnosis and repair to reduce maintenance costs and downtime; the multi-robot collaboration module is used to support collaborative work between multiple robots to cope with complex tasks and improve overall efficiency; environment perception and navigation module are responsible for the real-time positioning, path planning and obstacle avoidance of the robot. This system architecture will provide a comprehensive solution for the operation and maintenance task management of composite robots, improving the autonomy, efficiency and reliability of the robot system to cope with changing task requirements.

In an optional implementation, the core management module includes: a task management unit, responsible for task allocation, robot control and task decision-making; a status monitoring and communication unit, responsible for monitoring the status of the robot, including real-time status and sensor data, and also responsible for Communication within and outside the system; user interface and data management unit, used to provide an interactive interface between users and the system, and responsible for the storage and management of task-related data; security and authentication unit, used to ensure the security of the system, including users Authentication, data encryption and rights management; maintenance and remote support unit to support remote diagnosis and repair to reduce maintenance costs and reduce downtime. This unit supports remote diagnosis and repair to reduce maintenance costs and downtime, and it works in conjunction with the condition monitoring and fault detection unit to ensure the stability and maintainability of the robot system.

The data center module includes: data storage unit, used for persistent storage of task-related data, including task history, robot status, sensor data, map data, etc.; data management unit, responsible for data storage, backup, recovery and cleaning; data analysis and The processing unit is used to analyze and process task data to extract useful information and insights; the data access and query unit is used to allow other system modules or users to query and access data; the data security and privacy unit is responsible for the security and privacy protection. The units in these data centers work together to manage and maintain task-related data while supporting data analysis and data access. This helps improve system reliability, performance and decision support capabilities. Data centers also need to consider data backup and recovery strategies. , to cope with data loss or system failure.

The intelligent module includes: a data analysis unit, which is responsible for processing and analyzing data collected from robots, sensors and tasks; an automated decision-making unit, which uses the results of data analysis to automate decision-making on task allocation, resource scheduling and robot control; an adaptive task allocation unit, Responsible for adaptive task allocation, adjusting the task allocation strategy based on real-time conditions and the performance of the robot; the learning and optimization unit, using machine learning technology to continuously improve task allocation and decision-making strategies; the fault detection and prediction unit, used to monitor the status of the robot and performance, and predict potential failures; a user feedback and improvement unit that allows users and administrators to provide feedback to improve system performance. These units work together to enable the system to automatically adapt to changing task requirements and environmental conditions. They use data analysis and machine learning to improve the efficiency of task allocation and decision-making, thereby achieving automation and intelligence of the system, which helps improve Performance, reliability and autonomy of robotic systems.

The security and authentication module includes: the authentication and access control unit, which is responsible for user authentication and determining the user's access rights to system resources; the data protection and encryption unit, which is responsible for the protection and encryption of data while managing the system's security policy; auditing and The monitoring unit is responsible for recording system activities, monitoring threats and providing threat detection and defense; the security awareness and training unit is used for security training and improving security awareness of users and administrators. This unit focuses on security training and increasing security awareness for users and administrators. It provides training courses, educational materials, and communicates security policies to help users better understand and adhere to security best practices.

The user interface module includes: a user interaction interface unit, which is used to focus on the direct interaction between users and the system, including interface design, user interface interaction and user input processing; a feedback and notification unit, which is responsible for providing feedback and notifications to users, including operation results, errors Messages and warnings; the Internationalization and Accessibility Support unit, which focuses on multilingual support and accessibility to ensure that the system can adapt to different cultures and user ability levels; the User Documentation and Support unit, which provides user manuals, online help, Resources such as troubleshooting guides to support users in better understanding and using the system and obtaining relevant information. The division of the units into more functional chunks helps simplify the design and maintenance of user interfaces while improving the user-friendliness and accessibility of the system. The four units cover key aspects of user interaction with the system, including Interface design, feedback mechanisms, internationalization and documentation support.

The maintenance and remote support module includes: a fault diagnosis and maintenance unit, which focuses on the health status and maintenance of the system, including problem diagnosis, error log analysis, maintenance tasks and system updates; a remote support and monitoring unit, which allows remote support teams to access the system and monitor performance , and respond quickly to problems; the data protection and recovery unit is responsible for data protection and disaster recovery plans to ensure the security and recoverability of system data; the user support and knowledge management unit provides user support, including problem solving and assistance, and the knowledge base and documentation required by maintainers. Each unit is divided into more functional blocks to help improve system maintainability and user support efficiency. These four units cover problem diagnosis, maintenance, remote support, monitoring, data protection, recovery, user support and Key aspects such as knowledge management.

The multi-robot collaboration module includes: a task allocation and coordination unit, which is responsible for assigning tasks to robots, coordinating their activities, and ensuring the successful completion of tasks; a perception and decision-making unit, which focuses on the robot's perception capabilities and decision-making, ensuring that the robot can understand the environment and Take appropriate actions; the communication and collaboration unit allows robots to communicate, share information and work collaboratively to support collaborative execution of tasks; the maintenance and monitoring unit focuses on the robot's status monitoring, maintenance and fault handling to ensure that the robot is functioning properly Maintain high availability in collaborative tasks. It helps to simplify the design and management of multi-robot collaborative systems, while emphasizing key aspects such as task allocation, perception, decision-making, communication and maintenance. Each unit has specific responsibilities to ensure the collaborative operation and task execution of the system.

The environment perception and navigation module includes: sensor unit for acquiring visual information and generating environment maps, detecting the direction and acceleration of the robot, measuring the distance between the robot and obstacles, sound detection and environment perception; environment modeling and map generation unit , used to build an environment map while estimating the robot's position on the map, integrating data from different sensors into a consistent environment map; a path planning and decision-making unit, used to determine how the robot moves from its current location to its target location, Taking into account the environment map and obstacles; the control and execution unit is used to perform path planning and decision-making, control the robot's motors, wheels or leg actuators to achieve navigation and ensure that the robot avoids collisions and maintains stability during navigation; The positioning and self-positioning unit is used to obtain global position information and estimate the accurate position of the robot on the map, usually working in conjunction with the SLAM algorithm; the user interface unit provides the display of the robot status, navigation progress and interaction options. These units work together to realize the robot's environment perception and navigation. The sensor unit is responsible for obtaining environmental information, the environment modeling and map generation unit creates maps, the path planning and decision-making unit determines actions, and the control and execution unit executes these actions, positioning and self- The positioning unit ensures that the robot knows its position on the map, and the user interface unit provides information feedback and user interaction. These units jointly enable the robot to navigate and operate in complex environments.

The system described in this embodiment is used to execute the compound robot operation and maintenance management method, which specifically includes:

1. Task allocation and coordination, used to define tasks and goals, assign tasks to appropriate robots, formulate task scheduling algorithms to optimize task distribution and collaborative work, monitor task execution in real time, and coordinate robots to avoid conflicts and overlapping tasks.

2. Environmental perception and navigation, used to obtain environmental information, build a real-time environment map, ensure that the robot can understand the surrounding environment, determine the best path for the robot, consider obstacles and target locations, and decide the robot's actions based on perceived information and path planning. Such as avoiding obstacles and navigating to a target location.

3. Communication and collaboration, used to establish communication protocols and interfaces so that robots can share information and work collaboratively, implement collaborative algorithms to ensure that robots perform tasks collaboratively, and assist each other in solving problems, and provide remote access functions to allow remote support teams Monitor and intervene in the operation of the robot.

4. Maintenance and monitoring, implement maintenance and status monitoring units to monitor the health status and performance of the robot, record and analyze error logs in real time to quickly diagnose and solve problems, and develop preventive maintenance plans to minimize unplanned downtime. time, and the data protection and recovery unit ensures the security and recoverability of system data.

Specifically, it includes task allocation and coordination, by defining tasks, allocating robots, formulating task scheduling algorithms, and monitoring and coordinating task execution in real time to avoid conflicts and task overlaps. At the same time, the environment perception and navigation module uses sensors to obtain environmental information, build a real-time map, determine the best path, and decide the robot's actions based on the perception information and path planning to ensure safe navigation and obstacle avoidance. The communication and collaboration module establishes communication protocols, implements collaboration algorithms, supports information sharing between robots, and provides remote access functions to ensure robots work collaboratively. The maintenance and monitoring module implements maintenance and status monitoring, records error logs, and formulates preventive maintenance plans while ensuring data security and ensuring efficient and stable operation of the system.

In this embodiment, by integrating sensors, environment modeling, path planning and decision-making units, the system can achieve efficient autonomous navigation. The robot can quickly perceive the surrounding environment, build an accurate map, plan the best path, and make intelligent decisions to avoid obstacles. This helps improve the robot's navigation efficiency, reduces the risk of collisions, and saves time and resources. And the system provided can promptly sense changes in the environment and take appropriate actions, such as the emergence of obstacles or changes in environmental conditions. The path planning and decision-making unit can quickly adjust the robot's actions to adapt to new situations, thereby improving the robot's performance in complex and dynamic environments. adaptability and adaptability. This helps ensure task completion and safety. In this embodiment, the operation and maintenance tasks based on the composite robot can be performed more efficiently and reliably. Task allocation and coordination ensure the smooth completion of the task. Environmental perception and navigation enable the robot to understand its surrounding environment and navigate to the target. Communication and collaboration promote the interaction between robots. Cooperation, maintenance and monitoring ensure the health of the system and the security of data, helping to improve operation and maintenance efficiency and system maintainability.

Example 3:

As shown in Figure 17, it is a schematic architectural diagram of a compound robot operation and maintenance management device according to an embodiment of the present invention. The compound robot operation and maintenance management device in this embodiment includes one or more processors 21 and memories 22 . Among them, a processor 21 is taken as an example in FIG. 17 .

The processor 21 and the memory 22 may be connected through a bus or other means. In FIG. 17 , the connection through a bus is taken as an example.

As a non-volatile computer-readable storage medium, the memory 22 can be used to store non-volatile software programs and non-volatile computer executable programs, such as the compound robot operation and maintenance management method in Embodiment 1. The processor 21 executes the composite robot operation and maintenance management method by running non-volatile software programs and instructions stored in the memory 22 .

Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 22 optionally includes memory located remotely relative to the processor 21, and these remote memories may be connected to the processor 21 through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

The program instructions/modules are stored in the memory 22, and when executed by the one or more processors 21, the compound robot operation and maintenance management method in the above-mentioned Embodiment 1 is executed.

It is worth noting that the information interaction and execution process between the above-mentioned devices and modules and units in the system are based on the same concept as the processing method embodiments of the present invention. For specific content, please refer to the description in the method embodiments of the present invention. , which will not be described again here.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: Read memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. A compound robot operation and maintenance management method, which is characterized by including: Use lidar sensors to obtain distance information of the environment and build an environment map based on the distance information; Assign corresponding tasks to the corresponding robots according to the urgency level of each task and the current location of each robot; According to the task corresponding to the robot, specify the corresponding starting point and end point for the robot, and use the path planning algorithm to plan the traveling route based on the environment map, starting point and end point; The robot travels according to the travel route. During the travel process, the camera is used to collect image data of the environment, and obstacles in the environment are identified from the image data. The robot's movement is adjusted based on the real-time position of the robot and the location of the obstacles. Trajectory to achieve robot obstacle avoidance. 2. The compound robot operation and maintenance management method according to claim 1, characterized in that, according to the emergency level of each task and the current location of each robot, the corresponding task is assigned to the corresponding robot, specifically including: If the robots closest to the end of each task are different, each task will be assigned to the robot closest to the end of the corresponding task; If there is a situation where a robot is closest to the end point of multiple tasks, the reciprocal of the distance between the robot and the end point of the corresponding task is used as the first base value, and the product of the emergency level of the task corresponding to the robot and the first base value is used as the corresponding task. The allocation base value is the maximum sum of the allocation base values of all tasks as the objective function, and one task corresponds to one robot as the constraint function; combine the objective function and constraint function to establish a task decision-making model; Solve the task decision-making model to obtain the task allocation method. 3. The compound robot operation and maintenance management method according to claim 1, characterized in that, obstacles in the environment are identified from the image data, and the robot is adjusted according to the real-time position of the robot and the location of the obstacles. Movement trajectory, specifically including: Identify whether there are obstacles in the direction of travel within the preset range from the robot's real-time position; If there is an obstacle, target recognition is performed on the obstacle. If the obstacle is identified as a moving animal, multiple image data are combined to analyze the moving direction and speed of the obstacle. According to the moving direction and speed of the obstacle, and The robot's own traveling speed determines whether there is a risk of collision with the obstacle; if there is a risk of collision, adjust the robot's traveling direction to the opposite direction of the obstacle to avoid the obstacle; if there is no risk of collision; The risk of collision will cause the robot to follow the established travel route; If it is recognized that the obstacle is a fixed object, the robot's direction will be adjusted to either side of the obstacle to avoid the obstacle. 4. The compound robot operation and maintenance management method according to claim 1, characterized in that, based on the environment map, starting point and end point, a path planning algorithm is used to plan the traveling route, which specifically includes: The first constraint function is that the time required to reach the end point from the starting point does not exceed the preset maximum time, and the second constraint function is that the energy consumed from the starting point to the end point does not exceed the preset maximum energy; Taking one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point, and the minimum number of direction adjustments required from the starting point to the end point as the planning goal, generate the corresponding objective function; Combine the objective function, the first constraint function and the second constraint function to establish a path planning model; Solve the path planning model to obtain the traveling route. 5. The compound robot operation and maintenance management method according to claim 4, characterized in that the method further includes: Calculate the remaining energy situation of the robot before the first travel route planning based on the travel route that has been planned for each robot, and determine the charging emergency of the robot based on the remaining energy situation; Compare the emergency level of the robot's corresponding task with the charging emergency level, and according to the comparison result, selectively move the robot along the first travel route or charge the robot; wherein, the first travel route The route planned for the first route of travel. 6. The compound robot operation and maintenance management method according to claim 5, characterized in that the emergency level of the robot's corresponding task is compared with the charging emergency level, and based on the comparison result, the robot is selectively moved according to the Travel on the first traveling route, or charge the robot, specifically including: If the charging urgency of the robot is higher than the emergency level of the corresponding task, the robot is charged; wherein the range of the urgency level of the task is within the first interval, and the range of the charging urgency of the robot is within the second interval. , the minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the first interval is greater than the maximum value of the second interval; Otherwise, the robot is moved along the first traveling route. 7. The compound robot operation and maintenance management method according to claim 5, characterized in that charging the robot specifically includes: The planning goals are to generate the corresponding goals with the smallest distance between the target charging device and the robot, the fastest charging of the robot by the target charging device, and the highest balance between the number of robots using each target charging device for charging. function; Use the objective function to establish a charging decision model, solve the charging decision model, and obtain the target charging equipment for charging the robot. With the target charging equipment as the end point, perform route planning for the robot so that the robot Arrive at the target charging equipment location for charging. 8. The compound robot operation and maintenance management method according to claim 5, characterized in that the method further includes: If the travel route planning of the corresponding task fails, the robot corresponding to the task will be charged; If the emergency level of the task is higher than the preset level, the robot corresponding to the task closest to the end of the task will be called from the task with a lower emergency level than the task; the task of the called party will be called from the task than the called one. The task is transferred from a task with a lower emergency level until the emergency level of the corresponding task is not higher than the preset level. 9. The compound robot operation and maintenance management method according to claim 1, characterized in that each robot communicates with the central control platform through wireless, and the central control platform also communicates with the control terminal through wireless, and the control terminal has a positioning function. , display the environment map in the control terminal, and display the actual traveling path of the robot in the environment map. Obstacles in the environment are identified from the image data. According to the real-time position of the robot and the location of the obstacles, Adjusting the robot's motion trajectory also includes: During the movement of the robot, the surrounding environment point cloud data of the location where the robot passes is collected. Based on the surrounding environment point cloud data, it is judged whether each path location belongs to an indoor area or an outdoor area. The surrounding environment point cloud data is analyzed to obtain the indoor and outdoor dividing lines. , establish a building map based on the indoor and outdoor dividing lines; Match the planned travel route of the robot with the building map to obtain the buildings that the robot needs to inspect; If the robot detects an obstacle ahead while traveling, the robot's motion trajectory will be adjusted to bypass the obstacle; if the obstacle is identified as a fixed object, the robot will record the time it starts to bypass the obstacle until it returns. The obstacle bypass path along the planned travel route is based on the starting position and end position of the obstacle bypass path, and is matched with the planned travel route to obtain the original planned path, and it is judged whether the original planned path belongs to the interior of the corresponding building. area; If the original planned path belongs to the indoor area of the corresponding first building, a blind spot mark is displayed on the environmental map of the control terminal corresponding to the location of the first building, so that the controller can identify the blind spot through the environmental map, and carry the control terminal to perform manual inspection of the first building; When the controller clicks on the blind spot mark and chooses to perform blind spot inspection, the controller is navigated according to the real-time positioning information of the control terminal; wherein, the end point of the navigation route displayed in the environmental map is the The starting position of the obstacle avoidance path; After identifying that the controller reaches the starting position of the obstacle bypassing path based on the real-time positioning information of the control terminal, the controller's action trajectory is recorded, and after identifying that the controller reaches the end position of the obstacle bypassing path, When the action trajectory of the controller is matched with the first building, it is determined whether the action trajectory has entered the indoor area of the first building; If it is determined that the action trajectory has entered the indoor area of the first building, the action trajectory will be displayed in the environment map, and a prompt window will be displayed around the action trajectory, and the controller will be asked in the prompt window Whether the action trajectory can be used by the robot, and provide a confirmation button and a cancel button for the controller to choose; If the controller selects the confirmation button, the next time the robot performs path planning and obtains the buildings to be inspected including the first building, the action trajectory will be used to replace the first building in the route. A partial path of a building to facilitate subsequent inspection of the first building by the robot. 10. A compound robot operation and maintenance management device, characterized by including: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor for Implement the compound robot operation and maintenance management method described in any one of claims 1-9.