CN117742337A - Inspection route control method and device - Google Patents

Abstract

The invention relates to the technical field of computers, and provides a method and a device for controlling a routing inspection route. The method comprises the following steps: acquiring distance information of an environment by using a laser radar sensor, and constructing an environment map according to the distance information; acquiring image data of an environment by using a camera, and identifying obstacles in the environment from the image data; and designating a corresponding starting point and a corresponding terminal point in the environment map, and planning to obtain a routing inspection route by using a path planning algorithm based on the environment map, the obstacle, the starting point and the terminal point so that the routing inspection device can inspect according to the routing inspection route. The invention uses the path planning algorithm to plan the inspection route, is more flexible than the prior art that uses the fixed inspection route, and can quickly adapt to the real-time environment change, thereby having higher efficiency.

Description

Inspection route control method and device

Technical field

The invention relates to the field of computer technology, and in particular to an inspection route control method and device.

Background technique

Inspection systems are widely used in many fields, such as industrial production, building safety, environmental monitoring, etc. However, there are some problems with the existing inspection system, as follows:

In existing inspection systems, robots usually operate along fixed inspection routes. This fixed route cannot adapt to real-time environmental changes, resulting in low inspection efficiency and waste of resources.

Existing charging strategies are usually based on fixed time intervals or simple energy level rules. This simple strategy cannot be flexibly adjusted according to the robot's inspection tasks and energy requirements, resulting in inaccurate energy management and the risk of inspection interruption.

There are also challenges in improving inspection efficiency when multiple robots work together. The current inspection system lacks effective task allocation and collaboration mechanisms and cannot fully utilize the advantages of multiple robots, resulting in redundant operations and waste of time during the inspection process. Generally speaking, the inspection route control methods in the existing technology have problems of low efficiency and inaccurate energy management.

In view of this, overcoming the shortcomings of the prior art is an urgent problem to be solved in this technical field.

Contents of the invention

The technical problem to be solved by this invention is that in the prior art, operations are usually performed along a fixed inspection route. This fixed route cannot adapt to real-time environmental changes, resulting in low efficiency of the inspection route control method.

The present invention adopts the following technical solutions:

In a first aspect, the present invention provides an inspection route control method, including:

Use a lidar sensor to obtain distance information of the environment, and construct an environmental map based on the distance information; use a camera to collect image data of the environment, and identify obstacles in the environment from the image data;

Specify the corresponding starting point and end point in the environment map, and use the path planning algorithm to plan the inspection route based on the environment map, obstacles, starting point and end point, so that the inspection device can perform inspection according to the inspection route. .

Preferably, when planning the inspection route for a single inspection device, the inspection route is planned using a path planning algorithm based on the environmental map, obstacles, starting point and end point, specifically including:

The first constraint function is that the time required to reach the end point from the starting point does not exceed the preset maximum time, the second constraint function is that the energy consumed from the start point to the end point does not exceed the preset maximum energy, and the path from the starting point to the end point is not Passing through obstacles is the third constraint function;

Taking one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point, and the minimum number of direction adjustments required from the starting point to the end point as the planning goal, generate the corresponding objective function;

Combine the objective function, the first constraint function, the second constraint function and the third constraint function to establish a path planning model;

Solve the path planning model to obtain the inspection route.

Preferably, when planning inspection routes for multiple inspection devices, one or more end points are specified; based on the environmental map, obstacles, starting points and end points, the inspection route is planned using a path planning algorithm, specifically include:

The time required for each inspection device to reach the corresponding end point from the corresponding starting point does not exceed the preset maximum time as the first constraint function, and the energy consumed by each inspection device from the corresponding starting point to the corresponding end point does not exceed the preset maximum energy as the third constraint function. The second constraint function is that the path of each inspection device from the corresponding starting point to the corresponding end point does not pass through obstacles as the third constraint function, and the fourth constraint function is that each end point can be reached by the corresponding inspection device;

The total path of all inspection devices from the corresponding starting point to the corresponding end point is the shortest, the total time of all inspection devices from the corresponding starting point to the corresponding end point is the shortest, and the total number of direction adjustments required by all inspection devices from the corresponding starting point to the corresponding end point is the least. One or more of them are used as planning objectives to generate corresponding objective functions;

Combine the objective function, the first constraint function, the second constraint function, the third constraint function and the fourth constraint function to establish a path planning model;

Solve the path planning model to obtain the inspection routes of each inspection device.

Preferably, an inspection route planning is performed every preset time interval; wherein, the end point of the previous inspection route planning is used as the starting point of the next inspection route planning, and the next preset inspection location is the starting point of the next inspection route planning. end.

Preferably, the method further includes:

Calculate the remaining energy of the inspection device before the first inspection route is planned based on the inspection route that has been planned for each inspection device, and determine the charging urgency of the inspection device based on the remaining energy condition;

Based on the first inspection route planned for the inspection device, the task urgency of the inspection task corresponding to the first inspection route is obtained; the task urgency is compared with the charging urgency, and according to the comparison result , selectively inspect the inspection device according to the first inspection route, or charge the inspection device.

Preferably, the task urgency is compared with the charging urgency, and based on the comparison result, the inspection device is selectively inspected according to the first inspection route, or the inspection device is charged. , specifically including:

If the charging urgency of the patrol inspection device is higher than the task urgency of the corresponding patrol inspection task, the patrol inspection device is charged; wherein the range of the task urgency of the patrol inspection task is within the first interval, and the patrol inspection The range of the charging urgency of the device is within a second interval, the minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the second interval is greater than the maximum value of the first interval;

Otherwise, the inspection device is used to perform inspection according to the first inspection route.

Preferably, if the patrol inspection device is charged, the patrol inspection device is excluded from the patrol inspection route planning, and the patrol inspection route planning is performed again for other patrol inspection devices.

Preferably, charging the inspection device specifically includes:

The planning is based on the minimum distance between the target charging equipment and the inspection device, the fastest charging of the inspection device by the target charging equipment, and the highest balance between the number of inspection devices using each target charging equipment for charging. Goal, generate the corresponding objective function;

Use the objective function to establish a charging decision model, solve the charging decision model, and obtain the target charging equipment for charging the inspection device. With the target charging equipment as the end point, perform route planning for the inspection device, Make the inspection device arrive at the target charging equipment location for charging.

Preferably, each inspection device communicates with the central control platform via wireless, and the central control platform also communicates with the control terminal via wireless. The control terminal has a positioning function, and the environment map is displayed in the control terminal, and the environment map is displayed on the control terminal. The actual traveling path of the inspection device is displayed in , and the method further includes:

During the movement of the inspection device, the surrounding environment point cloud data of the location passed by the inspection device is collected. Based on the surrounding environment point cloud data, it is judged that each path location belongs to the indoor area or outdoor area, and the surrounding environment point cloud data is analyzed to obtain the indoor The outdoor dividing line is used to establish a building map based on the indoor and outdoor dividing line;

Match the planned inspection route of the inspection device with the building map to obtain each building that the inspection device needs to inspect;

If the patrol inspection device detects an obstacle ahead during travel, the motion trajectory of the patrol inspection device is adjusted to bypass the obstacle; if the obstacle is identified as a fixed object, the patrol inspection device records the time from the start of the bypass. The obstacle bypassing path that is passed through the obstacle until returning to the planned inspection route is based on the starting position and end position of the obstacle bypassing path, and is matched with the planned inspection route to obtain the original planned path, and the original planning is judged Whether the path belongs to the indoor area of the corresponding building;

If the original planned path belongs to the indoor area of the corresponding first building, a blind spot mark is displayed on the environmental map of the control terminal corresponding to the location of the first building, so that the controller can identify the blind spot through the environmental map, and carry the control terminal to perform manual inspection of the first building;

When the controller clicks on the blind spot mark and chooses to perform blind spot inspection, the controller is navigated according to the real-time positioning information of the control terminal; wherein, the end point of the navigation route displayed in the environmental map is the The starting position of the obstacle avoidance path;

After identifying that the controller reaches the starting position of the obstacle bypassing path based on the real-time positioning information of the control terminal, the controller's action trajectory is recorded, and after identifying that the controller reaches the end position of the obstacle bypassing path, When the action trajectory of the controller is matched with the first building, it is determined whether the action trajectory has entered the indoor area of the first building;

If it is determined that the action trajectory has entered the indoor area of the first building, the action trajectory will be displayed in the environment map, and a prompt window will be displayed around the action trajectory, and the controller will be asked in the prompt window Whether the action trajectory can be used by the inspection device, and provide a confirmation button and a cancel button for the controller to choose;

If the controller selects the confirmation button, the next time the inspection device is routed and the buildings required to be inspected are obtained, including the first building, the action trajectory will be used to replace all the objects passing through in the route. A partial path of the first building is provided to facilitate subsequent inspection of the first building by the inspection device.

In a second aspect, the present invention provides an inspection route control device, including an environment modeling module and a path planning module;

The environment modeling module is used to use a lidar sensor to obtain distance information of the environment, and construct an environment map based on the distance information; use a camera to collect image data of the environment, and identify obstacles in the environment from the image data;

The path planning module is used to specify the corresponding starting point and end point in the environment map. Based on the environment map, obstacles, starting point and end point, use the path planning algorithm to plan the inspection route, so that the inspection device can follow the specified path. Conduct inspections along the inspection routes described above.

In a third aspect, the present invention also provides an inspection route control device for implementing the inspection route control method described in the first aspect. The device includes:

At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the processor to execute the inspection route control method described in the first aspect.

In a fourth aspect, the present invention also provides a non-volatile computer storage medium that stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors to complete the first step. The inspection route control method described in the aspect.

By using a path planning algorithm to plan the inspection route, the present invention is more flexible than the fixed inspection route used in the prior art, and can quickly adapt to real-time environmental changes, thereby having higher efficiency.

Description of drawings

In order to explain the technical solutions of the embodiments of the present invention more clearly, the drawings required to be used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic flow chart of the first inspection route control method provided by an embodiment of the present invention;

Figure 2 is a schematic flow chart of the second inspection route control method provided by an embodiment of the present invention;

Figure 3 is a schematic flow chart of a third inspection route control method provided by an embodiment of the present invention;

Figure 4 is a schematic flowchart of the fourth inspection route control method provided by an embodiment of the present invention;

Figure 5 is a schematic flowchart of the fifth inspection route control method provided by an embodiment of the present invention;

Figure 6 is a schematic diagram of the architecture between the control terminal, the inspection device and the central control platform in an inspection route control method provided by an embodiment of the present invention;

Figure 7 is a schematic flowchart of the sixth inspection route control method provided by an embodiment of the present invention;

Figure 8 is a schematic diagram of the first inspection route control method provided by an embodiment of the present invention;

Figure 9 is a schematic diagram of the second inspection route control method provided by the embodiment of the present invention;

Figure 10 is a schematic diagram of the third inspection route control method provided by the embodiment of the present invention;

Figure 11 is a schematic diagram of the fourth inspection route control method provided by an embodiment of the present invention;

Figure 12 is a schematic diagram of the fifth inspection route control method provided by the embodiment of the present invention;

Figure 13 is a module schematic diagram of an inspection route control device provided by an embodiment of the present invention;

Figure 14 is a module schematic diagram of another inspection route control device provided by an embodiment of the present invention;

Figure 15 is a schematic flowchart of the seventh inspection route control method provided by the embodiment of the present invention;

Figure 16 is a schematic structural diagram of an inspection route control device provided by an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention and are not intended to limit the present invention.

The terms "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, features defined by "first," "second," etc. may explicitly or implicitly include one or more of such features. In the description of this application, unless otherwise stated, "plurality" means two or more.

In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

Example 1:

In the prior art, operations are usually performed along fixed inspection routes, resulting in low efficiency of the inspection route control method. In order to solve this problem, Embodiment 1 of the present invention provides an inspection route control method, as shown in Figure 1. include:

In step 201, a lidar sensor is used to obtain distance information of the environment, and an environmental map is constructed based on the distance information; a camera is used to collect image data of the environment, and obstacles in the environment are identified from the image data; in actual use , the obstacle recognition can be obtained by using deep learning and convolutional neural network recognition.

In step 202, the corresponding starting point and end point are specified in the environment map. Based on the environment map, obstacles, starting point and end point, a path planning algorithm is used to plan the inspection route, so that the inspection device can follow the inspection route. Conduct inspections along inspection lines.

In actual use, the starting point is usually the charging station or docking point of the robot (i.e., the current location), and the end point is a certain location in the inspection area, which is obtained from the inspection task. The path planning algorithm can be Dije Stella's algorithm (referred to as: Dijkstra's algorithm).

This embodiment uses a path planning algorithm to plan the inspection route, which is more flexible than the fixed inspection route used in the existing technology, and can quickly adapt to real-time environmental changes, thereby achieving higher efficiency.

The inspection route control method described in this embodiment is mainly used for inspection robots. In actual use, there are two different application scenarios: a single robot performing inspections and multiple robots performing inspections. This embodiment also targets Two different application scenarios provide the following two optional implementation methods, including:

In the scenario where a single robot performs inspection, that is, when planning an inspection route for a single inspection device, the inspection route is planned using a path planning algorithm based on the environment map, obstacles, starting point and end point, as shown in FIG2 , specifically including:

In step 301, the first constraint function is that the time required to reach the end point from the starting point does not exceed the preset maximum time, and the second constraint function is that the energy consumed from the starting point to the end point does not exceed the preset maximum energy. The path to the end point that does not pass through obstacles is the third constraint function; wherein the preset maximum time and the preset maximum energy are obtained by those skilled in the art based on empirical analysis. In actual use, the preset maximum energy may be the current power of the inspection robot.

In step 302, one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point, and the minimum number of direction adjustments required from the starting point to the end point is used as the planning goal, and a corresponding objective function is generated; Among them, since the robot usually needs to adjust its direction of travel to avoid obstacles, etc., in actual use, the robot usually needs to stop first and then deflect, which takes longer than the time required for the robot to move forward. When the number of required direction adjustments is large, it will also affect the robot's traveling efficiency. Therefore, this embodiment also takes the number of required direction adjustments from the starting point to the end point into consideration, thereby optimizing the objective function so that the final planned patrol is The inspection route is optimal. When there are multiple planning objectives, the objective function is actually a weighted summation between multiple expressions representing the corresponding planning objectives. The positivity and negativity of each weight and the size of the value are obtained by those skilled in the art based on empirical analysis. .

In step 303, a path planning model is established by combining the objective function, the first constraint function, the second constraint function and the third constraint function.

In step 304, the path planning model is solved to obtain the inspection route.

In a scenario where multiple robots perform inspections, that is, when planning inspection routes for multiple inspection devices, one or more end points are designated; the path is used based on the environment map, obstacles, starting points and end points. The planning algorithm plans the inspection route, as shown in Figure 3, which includes:

In step 401, the time required for each inspection device to reach the corresponding end point from the corresponding starting point does not exceed the preset maximum time as the first constraint function, and the energy consumed by each inspection device from the corresponding starting point to the corresponding end point does not exceed the preset maximum time. Let the maximum energy be the second constraint function, the path from the corresponding starting point to the corresponding end point of each inspection device without passing through obstacles is the third constraint function, and the fourth constraint function is that each end point can be reached by the corresponding inspection device; Among them, when multiple end points are specified, it does not mean that each robot corresponds to a fixed end point, but that each robot can arbitrarily correspond to one of the multiple end points. In the end, it only needs to be reached by a corresponding robot at each end point. . The preset maximum energy corresponding to each inspection device may be different. For example, the current power of each inspection device is used as the preset maximum energy.

In step 402, the total path of all patrol inspection devices from the corresponding starting point to the corresponding end point is the shortest, the total time of all patrol inspection devices from the corresponding starting point to the corresponding end point is the shortest, and the required adjustment direction of all patrol inspection devices from the corresponding starting point to the corresponding end point is One or more of the minimum total times are used as planning goals to generate corresponding objective functions; where the total path length refers to the sum of the path lengths of multiple robots from their respective starting points to their respective end points, and the total time It refers to the sum of the time it takes for multiple robots to reach their respective end points from their respective starting points. The total number of required direction adjustments refers to the number of times that multiple robots need to adjust their directions from their respective starting points to reach their respective end points. sum.

In step 403, a path planning model is established by combining the objective function, the first constraint function, the second constraint function, the third constraint function and the fourth constraint function.

In step 404, the path planning model is solved to obtain the inspection routes of each inspection device.

This embodiment solves the lack of effective tasks for multi-robot collaborative work in the existing technology by sharing multiple endpoints for multiple robots in a multi-robot inspection scenario, and integrating the respective planning goals of each robot as the overall planning goal. The allocation and collaboration mechanism cannot fully utilize the advantages of multiple robots, resulting in redundant operations and time waste during the inspection process.

In actual use, inspections often need to be carried out continuously, so there are also the following preferred implementation methods, which specifically include: planning an inspection route at a preset time interval; wherein, the end point of the previous inspection route plan is used as the next inspection route. The starting point of the inspection route planning is the next preset inspection location as the end point of the next inspection route planning. Wherein, the preset time is obtained by those skilled in the art based on empirical analysis.

In a preferred embodiment, as shown in Figure 4, the method further includes:

In step 501, based on the inspection routes that have been planned for each inspection device, the remaining energy situation of the inspection device before the first inspection route planning is calculated, and based on the remaining energy situation, the inspection device is determined. Charging urgency; the inspection device is an inspection robot, and the remaining energy is the remaining power. The smaller the remaining power, the higher the charging urgency. Determining the charging emergency degree of the inspection device according to the remaining energy situation may be: dividing a plurality of remaining energy intervals, each remaining energy interval corresponds to a charging emergency degree, and the remaining energy interval corresponding to the remaining energy situation corresponds to The charging urgency is the charging urgency of the inspection device.

In step 502, based on the first inspection route planned for the inspection device, the task urgency of the inspection task corresponding to the first inspection route is obtained; the task urgency is compared with the charging urgency. Compare, and according to the comparison result, selectively inspect the inspection device according to the first inspection route, or charge the inspection device. The urgency of the inspection task is analyzed in advance by those skilled in the art based on the inspection requirements.

It can be understood that each end point corresponds to a corresponding inspection task. When the end point of the corresponding robot is determined using inspection path planning, the inspection task corresponding to the robot is also determined. Therefore, after inspection route planning, Compare task urgency and charging urgency. If the patrol inspection device is charged, the patrol inspection device will be excluded from the patrol inspection route planning, and the patrol inspection route planning will be carried out again for other patrol inspection devices.

This embodiment compares the task urgency with the charging urgency of the inspection device, and determines whether to charge the robot or continue to let the robot perform the inspection task based on the comparison result, thereby ensuring the normal progress of the inspection task while also ensuring The robot can be recharged in time.

In an optional embodiment, the task emergency level is compared with the charging emergency level, and based on the comparison result, the inspection device is selectively inspected according to the first inspection route, or the inspection device is inspected according to the first inspection route. Charging the patrol inspection device specifically includes: if the charging urgency of the patrol inspection device is higher than the task urgency of the corresponding patrol inspection task, charging the patrol inspection device; wherein, the task urgency of the patrol inspection task is The range is within the first interval, and the range of the charging urgency of the inspection device is within the second interval. The minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the second interval is greater than the first interval. The maximum value; otherwise, the inspection device is inspected according to the first inspection route.

The first interval and the second interval are both obtained by analysis by those skilled in the art based on inspection requirements. The minimum value of the second interval is less than the minimum value of the first interval, and the maximum value of the second interval is greater than the maximum value of the first interval to ensure that when the power of the inspection device is sufficient, no matter how urgent the inspection task is, Regardless of the degree, there is no need to charge, and when the inspection device is about to be powered off, the inspection device will be charged regardless of the urgency of the inspection task.

In actual use, charging equipment, such as charging piles, can also be set up around the perimeter. The inspection device travels to the charging equipment for charging. When there are multiple charging devices, the inspection device performs charging, as shown in Figure 5 shown, specifically including:

In step 601, the distance between the target charging equipment and the inspection device is the smallest, the target charging equipment charges the inspection device the fastest, and the number of inspection devices using each target charging equipment for charging is determined. The highest degree of balance is the planning target, and the corresponding objective function is generated; wherein, the degree of balance between the number of inspection devices using each target charging equipment for charging can be: the inspection devices that are charging under each target charging equipment The smaller the variance or standard deviation of the quantity, the higher the degree of balance.

In step 602, use the objective function to establish a charging decision model, solve the charging decision model, and obtain the target charging equipment for charging the inspection device.

In step 603, with the target charging equipment as the end point, route planning is performed for the inspection device so that the inspection device reaches the target charging equipment location for charging.

In actual use, the inspection device may also need to inspect the inside of a building, such as the inside of a factory. However, in some scenarios, the entire factory may be repaired or maintained, or some entrances to the factory may be closed for maintenance. In this case, the inspection device may not be able to perform inspections inside the factory normally. In order to solve this problem, this embodiment also provides the following preferred implementation methods, which specifically include:

Each inspection device communicates with the central control platform through wireless, and the central control platform also communicates with the control terminal through wireless, as shown in Figure 6. The number of control terminals and the number of inspection devices are determined by those skilled in the art according to needs. According to the analysis, it is not absolutely a one-to-one relationship. The control terminal has a positioning function, an environment map is displayed in the control terminal, and the actual traveling path of the inspection device is displayed in the environment map, as shown in Figure 7 , the method also includes:

In step 701, during the traveling process of the inspection device, the surrounding environment point cloud data of the location passed by the inspection device is collected, and each path location is judged to belong to an indoor area or an outdoor area based on the surrounding environment point cloud data, and the surrounding environment point cloud is The data is analyzed to obtain the indoor and outdoor dividing lines, and a building map is established based on the indoor and outdoor dividing lines; as shown in Figure 8. Among them, the judgment that each path location belongs to an indoor area or an outdoor area based on the surrounding environment point cloud data is obtained by identifying objects in the surrounding environment point cloud data. For example, when it is recognized that the surrounding areas are shelves, tables and chairs, or have continuous If there is a certain wall, etc., it is judged to be an indoor area. If it is recognized that there are trees, street lights, etc. in the surrounding area, or if the scan shows that there is an open space (that is, the distance between the point cloud data in a certain direction and the inspection device is far) , then it is judged to be an outdoor area, and one direction is an indoor area and the other direction is an outdoor area, and when a relatively close wall is detected between the two opposite directions, the current location is considered to be the entrance and exit of the corresponding building. . Wherein, the building map is superimposed on the environment map and displayed in the control terminal.

In step 702, the planned inspection route of the inspection device is matched with the building map to obtain each building that the inspection device needs to inspect; as shown in Figure 9, the inspection route of the inspection device enters If you go to buildings 1, 2, 3, 4, 5, 7, and 8, the buildings that need to be inspected are buildings 1, 2, 3, 4, 5, 7, and 8.

In step 703, if the patrol inspection device detects an obstacle ahead during travel, the motion trajectory of the patrol inspection device is adjusted to bypass the obstacle; if it is identified that the obstacle is a fixed object (i.e., it is not fixed). moving objects (obtained through object recognition, except for humans, animals, traveling vehicles and other inspection devices), then the inspection device is recorded from the time it starts to bypass the obstacle until it returns to the planned inspection route. Based on the starting position and end position of the obstacle bypassing path, the obstacle bypassing path is matched with the planned inspection route to obtain the original planned path, and it is judged whether the original planned path belongs to the indoor area of the corresponding building; such as As shown in Figure 10, when the outer circle of building 5 is surrounded for maintenance or other reasons, and the inspection device cannot enter the interior of building 5, the inspection device will bypass the obstacle. The bypass path is as marked in Figure 10, where The position marked as the starting point is the starting position of the obstacle bypass path, the position marked as the end point is the end position of the obstacle bypass path, and the dotted line part is the original planned path. Since the original planned path passes through building 5, the original planned area belongs to the building. 5 indoor areas.

In step 704, if the original planned path belongs to the indoor area of the corresponding first building, a blind spot mark is displayed on the environment map of the control terminal corresponding to the location of the first building to facilitate the control personnel to pass through the environment map. Identify the blind area and carry out manual inspection of the first building with the control terminal.

In step 705, when the controller clicks on the blind spot mark and chooses to perform blind spot inspection, the controller is navigated according to the real-time positioning information of the control terminal; wherein, the navigation route displayed in the environmental map The end point of is the starting position of the obstacle bypass path, that is, the starting point marked in Figure 10; as shown in Figure 11, the question mark icon is used as the blind spot mark, and when the controller clicks on the blind spot mark, an option pops up next to the blind spot mark window. In the options window, the controller is reminded that "this area is blocked to form a blind spot. Should manual inspection be performed on this area?" and provides the "Perform Inspection" option button and the "Cancel" option button. When the controller selects "Perform Inspection" When "check", it is considered that the blind spot inspection is selected, a navigation route is generated from the current position of the controller to the starting position of the obstacle bypass path, and is displayed on the environment map.

In step 706, after it is recognized that the controller has arrived at the starting position of the obstacle bypass path based on the real-time positioning information of the control terminal, the action trajectory of the controller is recorded, and after it is recognized that the controller has arrived at the bypass path, When reaching the end position of the obstacle path, the action trajectory of the controller is matched with the first building, and it is determined whether the action trajectory has entered the indoor area of the first building.

In step 707, if it is determined that the action trajectory has entered the indoor area of the first building, the action trajectory is displayed in the environment map, and a prompt window is displayed around the action trajectory. The window asks the controller whether the action trajectory described can be used by the inspection device, and provides a confirmation button and a cancel button for the controller to choose; as shown in Figure 12, the dotted line position is the action trajectory of the controller, which is displayed in the prompt window "A new action trajectory has been artificially generated. Can this action trajectory be reused by the inspection device?" Among them, since most inspection devices do not have the function of autonomously going up and down steps, the controller comprehensively considers factors such as whether there are steps in the action trajectory and whether the path width allows the inspection device to pass, so as to determine whether the action trajectory can be reused. To inspect the device.

In step 708, if the controller selects the confirmation button, the next time the inspection device is routed and the buildings to be inspected include the first building, the action trajectory is used to replace the first building. A part of the route passing through the first building is carried out to facilitate subsequent inspection of the first building by the inspection device. For example, in subsequent use, when the inspection device reaches the starting position of the obstacle bypassing path, it will travel along the action trajectory until it reaches the end position of the obstacle bypassing path, and then proceed according to the planned path.

If the controller selects the cancel button, the action path will not be reused for the inspection device. When planning the path for the inspection device next time, the first building will be avoided and the first building will not be inspected.

Example 2:

On the basis of Embodiment 1, this embodiment also provides an inspection route control device, as shown in Figure 13, including an environment modeling module and a path planning module;

The environment modeling module is used to obtain distance information of the environment using a lidar sensor, construct an environment map based on the distance information, collect image data of the environment using a camera, and identify obstacles in the environment from the image data.

The path planning module is used to specify the corresponding starting point and end point in the environment map. Based on the environment map, obstacles, starting point and end point, use the path planning algorithm to plan the inspection route, so that the inspection device can follow the specified path. Carry out inspections along the inspection routes described above.

In an optional implementation, as shown in Figure 14, the inspection route control device also includes a control execution module, a real-time monitoring module, a multi-robot collaboration module, anomaly detection and early warning module, an intelligent charging and energy management module and a communication module. Module, specifically:

The environment modeling module is used for map construction and obstacle detection in the inspection area.

The path planning module is used to generate the optimal inspection route so that the robot can efficiently cover the inspection area; wherein, the path planning module includes a map data management module, a starting point and end point selection module, a constraint setting module, and a path generation module. module, path evaluation and optimization module, and visual display module.

The control execution module is used to realize the autonomous navigation and inspection task execution of the robot; among them, the control execution module includes a communication module and a motion control module.

The real-time monitoring module is used to monitor the robot's actions and status in real time and provide feedback information. It obtains the robot's position, speed and sensor data through sensors, and can detect abnormal behavior or faults of the robot in a timely manner and provide corresponding alarms and feedback.

The multi-robot collaboration module is used to achieve collaborative work among multiple inspection robots. It can coordinate the inspection tasks of multiple robots, avoid conflicts and duplication, and improve inspection efficiency. The collaborative algorithm can be based on the principle of distributed systems and implemented through communication and coordination mechanisms.

The anomaly detection and early warning module is used to promptly detect and handle abnormal situations that occur during the inspection process. Through the analysis of sensor data and pattern recognition technology, abnormal events (such as obstacle movement, robot failure, etc.) can be detected and trigger corresponding early warning mechanism so that appropriate measures can be taken.

Intelligent charging and energy management module is used to optimize the energy consumption and charging strategy of the inspection robot. Based on the robot's energy consumption model and the location information of the charging facility, the optimal charging strategy is selected to extend the robot's working time and improve inspection efficiency.

The communication module is used for two-way communication with the motion control module to realize status monitoring and instruction transmission. The communication module can use wireless communication technology (such as Wi-Fi, Bluetooth, etc.) to realize real-time interaction with the robot. The motion control module is used to realize the robot's Autonomous navigation and precise control, which includes trajectory tracking algorithms (used to make the robot move according to a predetermined trajectory) and sensor data processing to ensure that the robot can accurately follow the generated inspection route.

The environment modeling module includes lidar sensors, camera sensors, and deep learning and convolutional neural network modules. The lidar sensor is used to obtain distance information in the environment. It is calculated by emitting a laser beam and measuring the time difference between the laser beam and the object. The distance between the object and the robot. LiDAR can provide high-precision distance data and is used to build a three-dimensional map or point cloud data of the environment. The camera sensor is used to obtain image information in the environment. It can capture visible light images of the scene and Convert it into numerical data. These image data can be used to identify and analyze obstacles, target objects, etc. in the environment. The camera can use different computer vision algorithms to process images, such as feature extraction, target detection, image segmentation, etc. Deep learning and convolutional neural network modules are used to achieve efficient obstacle detection and classification. By training deep learning models and volumes Accumulative neural network can achieve efficient obstacle detection and classification. These models can learn features and patterns in images and use them to identify and classify different types of obstacles, such as walls, furniture, people, etc. Deep learning and convolutional neural network modules often require large amounts of training data and computing resources for training and inference.

Comprehensive use of lidar sensors, camera sensors, and deep learning and convolutional neural network modules can achieve comprehensive perception and modeling of the environment, including distance information, image information, and obstacle detection and classification. This information can provide important basis for the robot's path planning, obstacle avoidance and other decisions.

The map data management module is responsible for managing and maintaining the map data of the inspection area, including the map topology, location information, obstacle locations, etc. The starting point and end point selection module is used to select the starting point and end point of the inspection task. The starting point is usually The robot's charging station or docking point ends at a certain location in the inspection area; the constraint setting module is used to set constraints for path planning, such as time limits, energy consumption limits, etc. These constraints can affect the results of the path planning algorithm. ;The path generation module is used to use the path planning algorithm to generate the optimal inspection route based on the map data, starting point and end point, and constraints. The path planning algorithm used is the Dijkstra algorithm, which generates the optimal inspection route based on the map data and constraints. route, the algorithm will consider factors such as path length, time, and obstacle avoidance; the path evaluation and optimization module is used to evaluate the quality of the generated inspection route. The evaluation indicators can include route length, coverage, energy consumption, etc., optimization algorithm Genetic algorithms, etc. can be used; the visual display module is used to display the generated optimal inspection route in a visual way to facilitate users to view and adjust.

The real-time monitoring module includes a data interaction module, a real-time data processing and analysis module, and a feedback information generation module. The data interaction module is used to interact with the camera sensor and motion control module of the inspection robot. It can be responsible for receiving image data from the camera sensor. , and send control instructions to the motion control module to realize data exchange and instruction transmission with the robot; the real-time data processing and analysis module is used to process and analyze the robot's position, speed, and sensor data information, which can be processed through the data interaction module Obtain, and after real-time processing and analysis, obtain real-time feedback on the current status of the robot and environmental conditions. For example, the robot's position and speed information can be used for path planning and motion control, and sensor data can be used for obstacle detection and environment perception. The feedback information generation module generates feedback information based on the results of the real-time data processing and analysis module and sends it to the robot. Or the operator, this feedback information can include the working status of the robot, environmental changes, anomaly detection, etc. The generation of feedback information can be based on preset rules and algorithms, and judgment and inference can be made based on the actual situation.

By comprehensively utilizing the data interaction module, real-time data processing and analysis module, and feedback information generation module, real-time monitoring and feedback of the inspection robot can be achieved. In this way, the robot's status and environmental conditions can be understood in time, and corresponding measures can be taken as needed, such as adjusting the path, changing the working mode, etc.

The multi-robot collaboration module includes a task allocation module and a communication coordination module. The task allocation module is used to allocate inspection tasks to appropriate robots. It can make task allocation decisions based on factors such as the nature of the task, the capabilities and location of the robot, and task allocation. The module can select the most appropriate robot to perform the task based on considerations such as the urgency of the task, priority, and the availability and efficiency of the robot. The task allocation module may need to take into account workload balancing among robots to ensure that the task can be efficiently performed. Completed, the communication coordination module is used to realize communication and coordination between robots. It can be responsible for establishing communication channels between robots so that they can exchange information, share task status and transfer instructions with each other. The communication coordination module can use wireless communication technology or Other communication means realize real-time data transmission and communication between robots. Through the communication coordination module, robots can share task progress, avoid conflicts, work collaboratively, and improve overall efficiency and collaboration capabilities.

By comprehensively utilizing the task allocation module and the communication coordination module, task allocation, communication and coordination between multiple robots can be achieved. This can make full use of the capabilities of multiple robots, improve the efficiency and coverage of inspection tasks, and flexibly adjust task allocation and collaboration strategies as needed.

The anomaly detection and early warning module includes an anomaly detection module and an early warning and alarm system. The anomaly detection module is used to analyze sensor data and robot behavior to detect possible faults or abnormal situations. It can monitor and analyze sensor data in real time to identify problems related to the robot. Patterns or abnormal behaviors that are inconsistent with normal operating conditions. The anomaly detection module can use various techniques, such as statistical analysis, machine learning, or artificial intelligence algorithms, to determine whether anomalies exist. For example, if the robot's sensor data shows excessive temperature, abnormal speed or position deviation, etc., it may indicate the existence of a fault or abnormal situation. The early warning and alarm system is used to generate alarms and notify relevant personnel to take appropriate measures in the event of an abnormality. When the detection module detects abnormal conditions, the early warning and alarm system will trigger the corresponding alarm mechanism. This can include sending alarm messages to operators, managers or other relevant personnel to alert them to abnormal conditions through sound, light signals or emails and to take appropriate measures in a timely manner. Warning and alarm systems can also automatically trigger emergency stops or safety measures on the robot to avoid further damage or danger.

Through the anomaly detection and early warning module, the abnormal situation of the robot system can be monitored in real time, and measures can be taken in a timely manner to ensure the safe operation of the robot and the smooth completion of the task.

The intelligent charging and energy management module includes a charging planning module and a charging pile allocation module. The charging planning module is used to plan the optimal charging strategy based on the robot's inspection tasks and remaining energy. It will consider the robot's inspection route and task priority. Factors such as level and energy consumption determine when, where and how to charge. The charging planning module can use algorithms and optimization techniques, such as dynamic programming, genetic algorithms or deep learning, to decide on the best charging strategy. For example, it can decide whether to perform a quick charge in the middle of the mission or a full charge after the mission is over based on the robot's estimated running time and energy consumption. The charging pile allocation module is used to determine the optimal charging pile location and allocation strategy to Reduce charging time and energy consumption, it can select the most suitable charging pile for the robot to charge based on factors such as the robot's location, charging needs and the availability of charging piles. The charging pile distribution module can take into account the distance between charging piles, charging speed and load condition of the charging piles to minimize charging time and energy consumption.

Through the intelligent charging and energy management module, the robot's charging strategy can be effectively planned, reducing charging time and energy consumption, and improving the robot's work efficiency and endurance. This ensures that the robot continues to operate during inspection tasks and reduces reliance on human intervention.

It should be noted here that the inspection route control method in Embodiment 1 is applicable to this embodiment.

The patrol route control device described in this embodiment is used to perform the patrol route control method, as shown in Figure 15, which specifically includes:

In step 801, the system is initialized. Specifically, the inspection route control device is started and the system is initialized and set, including loading map data, configuring the starting point and end point, and setting constraints.

In step 802, environment modeling is performed, specifically: map construction and obstacle detection are performed through the environment modeling module, and distance information in the environment is obtained using lidar sensors. Camera sensors acquire image information in the environment, and deep learning and convolutional neural network modules enable efficient obstacle detection and classification.

In step 803, path planning, specifically: using the path planning module to generate the optimal inspection route. The map data management module is responsible for managing and maintaining the map data of the inspection area. The starting point and end point selection module is used to select the starting point and end point of the inspection task. The constraint setting module is used to set the constraints of path planning. The path generation module uses path planning. The algorithm generates the optimal inspection route. The path evaluation and optimization module is used to evaluate and optimize the quality of the route. The visual display module displays the generated optimal inspection route in a visual way.

In step 804, control is executed, specifically: the control execution module is used to realize autonomous navigation and inspection task execution of the robot. The communication module and the motion control module conduct two-way communication to realize status monitoring and instruction transmission. The motion control module is responsible for realizing the autonomous navigation and precise control of the robot.

In step 805, real-time monitoring is performed. Specifically, the real-time monitoring module is used to monitor the robot's actions and status in real time and provide feedback information. The data interaction module interacts with the camera sensor and motion control module of the inspection robot. The real-time data processing and analysis module processes and analyzes the robot's position, speed, and sensor data information.

In step 906, multi-robot collaboration is performed. Specifically, the multi-robot collaboration module is used to realize collaborative work among multiple inspection robots. The task allocation module allocates inspection tasks to appropriate robots, and the communication coordination module realizes communication and coordination between robots.

In step 807, abnormality detection and early warning, specifically: using the abnormality detection and early warning module to promptly discover and handle abnormal situations that occur during the inspection process. The anomaly detection module analyzes sensor data and robot behavior to detect possible failures or anomalies, and the early warning and alarm system generates alerts and notifies relevant personnel to take appropriate measures.

In step 808, intelligent charging and energy management, specifically: use the intelligent charging and energy management module to optimize the energy consumption and charging strategy of the inspection robot. The charging planning module plans the optimal charging strategy based on the robot's inspection tasks and remaining energy. The charging pile distribution module determines the optimal charging pile location and distribution strategy to reduce charging time and energy consumption.

Example 3:

As shown in Figure 16, it is a schematic structural diagram of the inspection route control device according to the embodiment of the present invention. The inspection route control device of this embodiment includes one or more processors 21 and a memory 22 . Among them, a processor 21 is taken as an example in FIG. 16 .

The processor 21 and the memory 22 may be connected through a bus or other means. In FIG. 16 , the connection through a bus is taken as an example.

As a non-volatile computer-readable storage medium, the memory 22 can be used to store non-volatile software programs and non-volatile computer executable programs, such as the inspection route control method in Embodiment 1. The processor 21 executes the inspection route control method by running non-volatile software programs and instructions stored in the memory 22 .

Memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 22 optionally includes memory located remotely relative to the processor 21, and these remote memories may be connected to the processor 21 through a network. Examples of the above-mentioned networks include but are not limited to the Internet, intranets, local area networks, mobile communication networks and combinations thereof.

The program instructions/modules are stored in the memory 22, and when executed by the one or more processors 21, the inspection route control method in the above-mentioned Embodiment 1 is executed.

It is worth noting that the information interaction and execution process between the above-mentioned devices and modules and units in the system are based on the same concept as the processing method embodiments of the present invention. For specific content, please refer to the description in the method embodiments of the present invention. , which will not be described again here.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The storage medium can include: Read memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. The method for controlling the routing inspection route is characterized by comprising the following steps of:

acquiring distance information of an environment by using a laser radar sensor, and constructing an environment map according to the distance information; acquiring image data of an environment by using a camera, and identifying obstacles in the environment from the image data;

and designating a corresponding starting point and a corresponding terminal point in the environment map, and planning to obtain a routing inspection route by using a path planning algorithm based on the environment map, the obstacle, the starting point and the terminal point so that the routing inspection device can inspect according to the routing inspection route.

2. The method according to claim 1, wherein when performing routing for a single routing device, the routing algorithm is used to route the routing based on the environment map, the obstacle, the starting point and the destination, and the routing method specifically includes:

taking the time required for reaching the end point from the start point not exceeding the preset maximum time as a first constraint function, taking the energy consumed for reaching the end point from the start point not exceeding the preset maximum energy as a second constraint function, and taking the path from the start point to the end point as a third constraint function without passing through an obstacle;

Generating a corresponding objective function by taking one or more of the shortest path from the starting point to the end point, the shortest time from the starting point to the end point and the minimum number of times of adjusting the direction required for reaching the end point from the starting point as a planning target;

establishing a path planning model by combining the objective function, the first constraint function, the second constraint function and the third constraint function;

and solving the path planning model to obtain the routing inspection route.

3. The inspection route control method according to claim 1, wherein one or more terminal points are designated when performing inspection route planning for a plurality of inspection devices; the method for planning and obtaining the routing inspection route by using a path planning algorithm based on the environment map, the obstacle, the starting point and the end point comprises the following steps:

taking the time required by each inspection device to reach the corresponding terminal point from the corresponding starting point as a first constraint function, taking the energy consumed by each inspection device to reach the corresponding terminal point from the corresponding starting point as a second constraint function, taking the path of each inspection device to reach the corresponding terminal point from the corresponding starting point as a third constraint function without passing through an obstacle, and taking the path of each inspection device to reach the corresponding terminal point as a fourth constraint function;

Generating a corresponding objective function by taking one or more of the shortest total path of all the inspection devices from the corresponding starting point to the corresponding end point, the shortest total time of all the inspection devices from the corresponding starting point to the corresponding end point and the smallest total number of times of adjustment of all the inspection devices from the corresponding starting point to the corresponding end point as a planning target;

establishing a path planning model by combining the objective function, the first constraint function, the second constraint function, the third constraint function and the fourth constraint function;

and solving the path planning model to obtain the inspection route of each inspection device.

4. The inspection route control method according to claim 3, wherein the inspection route planning is performed once every predetermined time; the final point of the last routing inspection route is used as the starting point of the next routing inspection route, and the next preset routing inspection position is used as the final point of the next routing inspection route.

5. The inspection route control method according to claim 4, characterized in that the method further comprises:

according to the routing inspection routes which are planned for the routing inspection devices, calculating the residual energy condition of the routing inspection devices before the first routing inspection route is planned, and determining the charging emergency degree of the routing inspection devices according to the residual energy condition;

Acquiring the task emergency degree of the patrol task corresponding to the first patrol route based on the first patrol route planning as the first patrol route planned by the patrol device; and comparing the task emergency degree with the charging emergency degree, and selectively carrying out inspection on the inspection device according to the first inspection route or charging the inspection device according to the comparison result.

6. The method according to claim 5, wherein comparing the task emergency level with the charging emergency level, selectively inspecting the inspection device according to the first inspection route or charging the inspection device according to the comparison result, comprises:

if the emergency degree of charging of the inspection device is higher than the emergency degree of a task corresponding to the inspection task, charging is carried out on the inspection device; the range of the task emergency degree of the inspection task is in a first section, the range of the charging emergency degree of the inspection device is in a second section, the minimum value of the second section is smaller than the minimum value of the first section, and the maximum value of the second section is larger than the maximum value of the first section;

otherwise, the inspection device performs inspection according to the first inspection route.

7. The method according to claim 5, wherein if the inspection device is charged, the inspection device is excluded from the inspection route planning, and the inspection route planning is performed again for other inspection devices.

8. The method according to claim 5, wherein the charging the inspection device comprises:

the method comprises the steps of taking the minimum distance between target charging equipment and the inspection device, the fastest charging of the inspection device by the target charging equipment and the highest balance among the quantity of the inspection devices charged by using each target charging equipment as a planning target, and generating a corresponding target function;

and establishing a charging decision model by using the objective function, solving the charging decision model to obtain target charging equipment for charging the inspection device, and carrying out route planning on the inspection device by taking the target charging equipment as an end point so as to enable the inspection device to reach the position of the target charging equipment for charging.

9. The inspection route control method according to claim 1, wherein each inspection device communicates with a central control platform by wireless, the central control platform also communicates with a control terminal by wireless, the control terminal has a positioning function, an environment map is displayed in the control terminal, and an actual travel path of the inspection device is displayed in the environment map, the method further comprising:

In the advancing process of the inspection device, collecting surrounding environment point cloud data of the path positions of the inspection device, judging that each path position belongs to an indoor area or an outdoor area according to the surrounding environment point cloud data, analyzing the surrounding environment point cloud data to obtain an indoor and outdoor boundary, and building a building map according to the indoor and outdoor boundary;

matching the routing inspection route of the routing inspection device with a building map to obtain each building required to be inspected by the routing inspection device;

if the patrol device detects that an obstacle exists in front in the travelling process, the motion track of the patrol device is adjusted to bypass the obstacle; if the obstacle is identified to be a fixed object, recording an obstacle detouring path which is passed by the inspection device from the start of detouring the obstacle to the return of the planned inspection route, and based on the starting position and the end position of the obstacle detouring path, matching the obstacle detouring path with the planned inspection route to obtain an original planned path, and judging whether the original planned path belongs to an indoor area of a corresponding building;

if the original planning path belongs to an indoor area of a corresponding first building, displaying a blind area mark at the position of the environment map of the control terminal corresponding to the first building, so that a control person can identify the blind area through the environment map and carry the control terminal to carry out manual inspection on the first building;

When a control person clicks the blind area mark and selects to carry out blind area inspection, navigating the control person according to the real-time positioning information of the control terminal; the end point of the navigation route displayed in the environment map is the starting position of the obstacle detouring path;

after the control personnel arrive at the starting position of the obstacle detouring path according to the real-time positioning information of the control terminal, recording the action track of the control personnel, and when the control personnel arrive at the end position of the obstacle detouring path after being identified, matching the action track of the control personnel with the first building, and judging whether the action track enters an indoor area of the first building;

if the action track is judged to enter the indoor area of the first building, displaying the action track in an environment map, displaying a prompt window around the action track, inquiring a control person in the prompt window whether the action track can be used by the patrol device, and providing a confirmation button and a cancel button for the control person to select;

if the control personnel select the confirmation button, the path planning is carried out on the inspection device next time, and when the building to be inspected comprises the first building, the action track is used for replacing part of the paths passing through the first building in the running route so as to conveniently realize the follow-up inspection of the first building by the inspection device.

10. A patrol route control device, characterized by comprising:

at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor for performing the routing inspection route control method of any one of claims 1-9.