CN111300372A - Air-ground cooperative intelligent inspection robot and inspection method - Google Patents

Abstract

The invention relates to an air-ground cooperative intelligent inspection robot which comprises a robot platform and an unmanned aerial vehicle, wherein the robot platform comprises a vehicle body, wheels, a driving assembly, a mechanical arm, an environment sensing assembly, a communicator, a robot controller and a power supply assembly, and the communicator is used for realizing communication connection between the unmanned aerial vehicle and a base station. The inspection method comprises the following steps: positioning and drawing an air-ground cooperative multi-robot: the method comprises the following steps of perception positioning calculation, map creation, multi-information fusion positioning, air-ground cooperative tracking and control: the unmanned aerial vehicle self-service landing control system comprises an unmanned aerial vehicle flight control design, a robot platform trajectory tracking control and an unmanned aerial vehicle self-service landing control. The invention ensures that the robot can execute given navigation and routing inspection tasks in all directions and all weather, integrates environment sensing, dynamic decision, behavior control and alarm devices by applying the technologies of Internet of things, artificial intelligence and the like, has the capabilities of autonomous sensing, walking, protection, interactive communication and the like, can complete basic, repeated and dangerous security work, and reduces the security operation cost.

Description

Space-ground collaborative intelligent inspection robot and inspection method

technical field

The invention belongs to the technical field of robots, and in particular relates to an air-ground collaborative intelligent inspection robot and an inspection method.

Background technique

In recent years, major explosion accidents in the chemical industry are mainly caused by the leakage of hazardous chemicals, and the main reason for leakage accidents is insufficient inspection. In order to reduce the incidence of safety accidents in the chemical industry, inspection work has become one of the essential tasks in the chemical industry. At present, maintenance and inspection operations are mainly done manually. Due to the uneven quality of personnel, there are usually problems such as weak safety awareness, unimplemented safety responsibilities, and inadequate safety supervision work. It is difficult to ensure the reliability and accuracy of inspection work. In addition, the chemical industry usually covers a large area, and includes a large number of pressure vessels and nearly 100 kilometers of pressure transmission pipelines. The working environment of inspection operations is very complex, and it is difficult to complete all inspection tasks only by humans.

With the advancement of intelligent operation and maintenance, the demand for robot inspection is increasing. The chemical industry has gradually used robots to perform inspection tasks instead of humans, which can not only greatly reduce labor costs, but also ensure the efficiency and reliability of inspections. Although there are many advantages to applying robots to safety inspections in the chemical industry, inspection robots at this stage are generally single-robot inspection equipment with low intelligence, including unmanned vehicles and drones.

At this stage, inspection robots include unmanned vehicles and drones, which mainly have the following shortcomings:

1. Lack of multi-robot collaborative technology for unmanned vehicles and drones:

A single unmanned vehicle inspection equipment is limited by its own working principle, and the requirements for the smoothness of the road surface are too high. , the flight time and load capacity are limited, it cannot carry multiple sensors, and it cannot fly over long distances.

2. Lack of autonomous mobile platforms that can carry drones:

At this stage, when the inspection drones perform tasks, operators still need to take them to the designated location by means of transportation. This inspection method still has shortcomings such as low efficiency and waste of human and material resources.

3. Lack of autonomy:

At this stage, when UAVs perform inspection tasks, operators still need to remotely control UAVs on site to collect images of chemical environment and equipment at close range. This inspection method lacks autonomy and low inspection efficiency.

4. Lack of explosion-proof performance:

At this stage, unmanned vehicles and drones usually do not have explosion-proof performance. High-speed brushless motors and high-energy batteries will generate ultra-high currents. Once they come into contact with flammable gas, they will explode, and potential fire, explosion and other hidden dangers have been It's hard to imagine the consequences.

5. Lack of environmental awareness and understanding with a high degree of intelligence:

At this stage, the UAV has not fully realized the intelligence of environmental image acquisition and recognition. When the one-key return operation is activated in an emergency, it is easy to collide with or entangle with the obstacles on the return path, which will affect the chemical industry. The safety of raw materials, chemical products and inspection drones brings huge hidden dangers.

6. Lack of simultaneous mapping and positioning of dynamic environments:

At this stage, most of the SLAM navigation technologies of robots are only aimed at indoor environments, and can only be tested and operated in a relatively controllable environment. Outdoor patrol robots usually operate in more complex environments, and require extremely high positioning and navigation technology for mobile robots, especially the land-air cooperation system equipped with UAVs, which is subject to changes and uncertainties in the outdoor environment. There is no mature solution for SLAM technology.

7. Lack of high-speed and high-precision path planning technology:

The difficulty of path optimization is to ensure the search speed while ensuring the optimal path as possible. The current path planning technology usually does not incorporate the kinematic constraints of the robot into the global path planning, and cannot ensure real-time performance while the robot is actually operating; The path planning of air-ground coordination is one of the difficulties to ensure mutual cooperation and constraints under the premise that the respective paths are feasible; the technical difficulty of local path planning is to track the global path as much as possible while completing the robot's obstacle avoidance based on real-time sensor data.

8. Lack of accident early warning capability:

There is no mature accident early warning technology at this stage, so it is not possible to predict the occurrence of accidents in advance. In the algorithm of system fault diagnosis and accident prediction, how to predict accidents based on the fusion of heterogeneous data based on discrete event system theory is one of the difficulties.

SUMMARY OF THE INVENTION

One object of the present invention is to provide an open-ground collaborative intelligent inspection robot, which is applied to the safety inspection work in the chemical industry.

To achieve the above object, the technical scheme adopted in the present invention is:

An air-ground collaborative intelligent inspection robot includes a robot platform and an unmanned aerial vehicle, wherein the robot platform includes a vehicle body, wheels and driving components arranged at the bottom of the vehicle body, and a vehicle body arranged on the vehicle body. : a robotic arm, an environment perception component, a communicator, a robot controller and a power supply component, the communicator realizes communication connection between the drone and the base station.

Preferably, the robotic arm includes a base disposed on the vehicle body, a joint assembly rotatably connected to the base, a mechanical claw rotatably connected to the joint assembly, and a drive to drive each A rotary drive member for component rotation, the joint assembly includes one or more connecting joints that are rotatably connected end to end.

Further preferably, the joint assembly includes a first connecting joint rotatably connected with the base, a second connecting joint rotatably connected with the first connecting joint, and a second connecting joint rotatably connected with the first connecting joint. A third connection joint rotatably connected to the joint, a fourth connection joint rotatably connected to the third connection joint, and a fifth connection joint rotatably connected to the fourth connection joint, the The mechanical claw is rotatably connected with the fifth connecting joint.

Further preferably, the mechanical claw comprises a claw body rotatably connected to the joint assembly, a pair of claw hands connected to the claw body, and a pair of claw hands that drive the claw hands to perform a grasping action. A claw-hand drive assembly, the claw-hand drive assembly includes a worm arranged on the claw body, a turbine connected at one end of the claw and matched with the worm, and a claw that drives the worm to rotate Hand drive.

Preferably, the robot platform further includes a platform body provided on the vehicle body for taking off, landing and charging the UAV, and the platform body is connected with the power supply assembly.

Preferably, the environment perception component includes a lidar, a sensor component, and a camera and a camera component.

Further preferably, the sensor assembly includes a gas concentration sensor and a humidity temperature sensor.

Further preferably, the camera and camera components include a visible light high-definition camera, an infrared camera, and a monocular camera.

Preferably, the robot platform further includes a touch screen display arranged on the vehicle body.

Further preferably, a microphone and a speaker are integrated in the touch screen display.

Preferably, the communicator is integrated with inertial navigation equipment and GPS equipment.

Preferably, the UAV includes a fuselage, an UAV control assembly, and mounted on the fuselage: a landing gear, a propeller group, a UAV drive and power supply assembly, and a camera and sensing assembly.

Further preferably, the UAV control assembly includes a flight control module, a data transmission and an image transmission module, and the flight control module includes a flight control sealing box arranged on the fuselage, and a flight control sealing box arranged on the The flight controller, the power manager and the parameter adjustment interface in the flight control sealed box, the flight controller is respectively connected with the power manager and the parameter adjustment interface; the data transmission and image transmission modules include setting The digital and image transmission airtight box on the fuselage, the digital and image transmission UAV terminal arranged in the digital and image transmission airtight box, and the digital transmission and image transmission unmanned aerial vehicle The data transmission and the image transmission ground terminal connected with the man-machine terminal are connected with the flight controller.

Further preferably, the unmanned aerial vehicle drive and power supply assembly includes a power explosion-proof box, a battery arranged in the power supply explosion-proof box, an electronic governor and a hub, a motor seat, and a battery installed on the motor seat. motor, the battery is connected with the power manager, the battery is connected with the electronic governor through the hub, and the input end of the electronic governor is connected with the The parameter adjustment interface is connected with each other, and the output end of the electronic speed governor is connected with the motor.

Further preferably, the imaging and sensing assembly includes a camera, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

Further preferably, the propeller group is provided with four groups, and each group of the propeller group is provided with two propellers, and the two propellers are arranged up and down.

One object of the present invention is to provide an intelligent inspection method for space-ground coordination.

To achieve the above object, the technical scheme adopted in the present invention is:

An open-ground collaborative intelligent inspection method, comprising:

1) Air-ground collaborative multi-robot positioning and mapping: including perceptual positioning calculation, map creation, and multi-information fusion positioning, including:

Perception and positioning calculation uses sensor components to collect surrounding environment information, and analyzes and processes effective environmental perception and detection data after processing the collected data;

Map creation includes modeling and scanning the environment, collecting data from sensor components and creating local 3D point cloud maps in their respective reference coordinate systems, initially aligning the motion trajectories of the UAV and the robot platform, and extracting the two local maps. In the ground plane part, the two local maps are optimized for map alignment, and the motion trajectories and local maps obtained by the UAV and the robot platform are globally optimized and adjusted;

Multi-information fusion positioning includes the fusion calculation of relative position information and absolute position information such as GPS global coordinates, prior map and current perception data registration, to obtain robot position and attitude information,

2) Air-ground collaborative tracking and control: including UAV flight control design, robot platform trajectory tracking control, and UAV self-landing control, including:

The design of the UAV flight control system includes establishing the dynamic model equation for the UAV's six degrees of freedom, analyzing the UAV actuator model composed of the motor model and the propeller aerodynamic model, and using the actual measured tension and torque. The curve calculates the aerodynamic parameters related to the UAV; according to the kinematic and dynamic model of the UAV, the model is divided into two parts: attitude and position, and the motion control method is divided into two parts: position control and attitude control. The attitude of the UAV is called a target point, and the path control of the UAV is the collection of many target points in the space. The UAV needs to arrive at the planned target point in sequence according to the order of the target points;

The trajectory tracking control of the robot platform uses the DDPG algorithm to control the robot platform to track the planned path according to the state information of the robot platform and the feedback information of the environment;

The autonomous landing control of the UAV includes flying over the unmanned vehicle through the planned path, searching for the visual sign through the visual guidance system, detecting the visual sign, starting the automatic guidance and landing procedure, and realizing the autonomous landing of the UAV.

Preferably, the method further includes detection and early warning of accidents, including establishing a mapping relationship between failures and sensor events and a mapping relationship between accidents and sensor event sequences according to possible failures and accidents in the actual system, and according to the established mapping relationship , build a system diagnostic device based on the state tree. The diagnostic device performs online real-time fault detection and accident prediction by observing system events. When a system failure is detected, the diagnostic device issues a system warning; the diagnostic device calculates the probability of an accident in real time. The probability exceeds the threshold set by the system, and a warning is issued.

Due to the application of the above-mentioned technical solutions, the present invention has the following advantages compared with the prior art:

The invention builds an all-weather autonomous navigation multi-type robot platform to ensure that the robot can perform a given navigation and inspection task in an all-round and all-weather manner. Decision-making, behavior control and alarm devices have the capabilities of autonomous perception, autonomous walking, autonomous protection, and interactive communication, which can help humans complete basic, repetitive, and dangerous security work, promote security service upgrades, and reduce security operating costs. Function comprehensive intelligent equipment.

Description of drawings

Accompanying drawing 1 is the structural representation of this embodiment;

2 is a schematic structural diagram of the robot platform in this embodiment (blanking platform main body);

3 is a schematic structural diagram of the robotic arm in the present embodiment;

4 is a schematic structural diagram of the mechanical claw in this embodiment;

5 is a schematic structural diagram of an unmanned aerial vehicle in this embodiment;

6 is a schematic block diagram of the structure of this embodiment;

7 is a schematic diagram of the relationship between the inspection system in this embodiment;

Accompanying drawing 8 is the work flow chart that realizes image perception;

FIG. 9 is a schematic block diagram of the multi-robot positioning and mapping of the air-ground cooperative in this embodiment;

FIG. 10 is a schematic block diagram of sensing and positioning calculation in this embodiment;

11 is a schematic block diagram of map creation in this embodiment;

12 is a schematic block diagram of multi-information fusion positioning in this embodiment;

Accompanying drawing 13a, 13b is the UAV rotor dynamics model in this embodiment;

14 is a schematic block diagram of the UAV tracking control system in this embodiment;

15 is a schematic diagram of the state machine of the UAV controller in this embodiment;

16 is a schematic block diagram of the design of the DDPG trajectory tracking controller in this embodiment;

17 is a schematic diagram of the tracking error design in this embodiment: P is the point at the specified distance L in front of the trolley, q is the trajectory target point, and pq is perpendicular to L;

18 is a schematic diagram of the coordinated inspection process of the drone/unmanned vehicle in this embodiment;

19 is a schematic diagram of extracting event information from sensor data in this embodiment;

FIG. 20 is a schematic block diagram of an accident prediction-related model and architecture in this embodiment;

FIG. 21 is a schematic block diagram of video recognition of human action behavior in this embodiment.

In the above drawings: 1. Robot platform; 10. Car body; 11. Wheels; 12. Robot arm; 120, Base; 121, Mechanical claw; 1210, Claw body; ; 122, the first joint; 123, the second joint; 124, the third joint; 125, the fourth joint; 126, the fifth joint; 130, the lidar; 131, the gas concentration sensor; 132, the wet temperature sensor; 133, visible light high-definition camera; 134, infrared camera; 135, rotatable structure; 14, communicator; 15, touch screen display; 16, robot controller; 17, power supply assembly; 18, platform body; 2, UAV; 20, fuselage; 21, landing gear; 22, propeller; 23, camera.

Detailed ways

The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. The indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation or a specific orientation. construction and operation, and therefore should not be construed as limiting the invention. Furthermore, the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed to indicate or imply relative importance.

In the description of the present invention, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense, unless otherwise expressly specified and limited, for example, it may be a fixed connection or a detachable connection Connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood in specific situations.

As shown in Figures 1 and 2, an air-ground collaborative intelligent inspection robot includes a robot platform 1 and an unmanned aerial vehicle 2. The robot platform 1 and the UAV 2 will be described in detail below.

The robot platform 1 includes a vehicle body 10 , wheels and drive components arranged at the bottom of the vehicle body 10 , a robotic arm 12 , an environment perception component, a communicator 14 , a touch screen display 15 , a robot controller 16 and Power supply assembly 17 . in:

The wheels and driving components are driven by four motors and reducers to drive the wheels 11 to rotate, and the robot controller 16 is used to control the four wheels 11 respectively, and realize steering through differential speed.

As shown in FIG. 3 : the robotic arm 12 includes a base 120 disposed on the vehicle body 10 , a joint assembly rotatably connected to the base 120 , a mechanical claw 121 rotatably connected to the joint assembly, and a rotation for driving each component to rotate driver. The joint assembly includes one or more connecting joints that are rotatably connected end to end. In this embodiment, the joint assembly includes a first connection joint 122 rotatably connected to the base 120 , a second connection joint 123 rotatably connected to the first connection joint 122 , and rotatably connected to the second connection joint 123 The third connecting joint 124, the fourth connecting joint 125 rotatably connected with the third connecting joint 124, the fifth connecting joint 126 rotatably connecting with the fourth connecting joint 125, the mechanical claw 121 and the fifth connecting joint 126 rotatably connected. For example, a motor is used as the rotating driving part, and 6 rotating joints are driven by 6 motors to realize the movement of 6 degrees of freedom in the mechanical arm 12, and drive the mechanical claw 121 to perform the grasping operation.

The robotic arm 12 can adjust the landing position of the drone 2 so that the drone 2 can be charged wirelessly; the robotic arm 12 can also replace various sensors for the drone 2; the robotic arm 12 can also perform in a dangerous environment Handling tasks such as grasping, turning, pressing, for example, performing valve switching operations.

As shown in FIG. 4 : the mechanical claw 121 includes a claw body 1210 rotatably connected to the fifth connecting joint 126 in the joint assembly, a pair of claw hands 1211 connected to the claw body 1210 , and a pair of claw hands 1211 that drive the claw hands 1211 to perform grasping action the gripper drive assembly. In this embodiment, the gripper drive assembly includes a worm 1212 disposed on the gripper body 1210, a turbine 1213 connected to one end of the gripper 1210 and matched with the worm 1212, and a gripper driver for driving the worm 1212 to rotate. The gripper 1211 is provided with serrations to help ensure the reliability of grasping. The gripper driver can use, for example, a motor to drive the mechanical gripper 121 to realize the gripping function.

Environment perception components include lidar 130, sensor components, and camera and camera components. in:

The lidar 130 is a radar system that emits a laser beam to detect characteristic quantities such as the position and velocity of a target. The robot platform 1 uses the lidar 130 to detect the dynamic environment, and collects parameters such as target distance, orientation, height, speed, attitude, and shape. The obstacle information around the robot platform 1 is detected by the lidar 130 , and after the original data of the lidar 130 is acquired, data preprocessing needs to be performed. Since the initial laser data is cluttered, including some abnormal data obtained from measurement errors and accidental errors, it needs to be filtered, usually by low-pass filtering or Gaussian filtering. The amount of raw data is determined by the resolution of the sensor, which is usually huge. In order to facilitate practical application, it is necessary to downsample the data. The laser radar data after data filtering and downsampling will be used in obstacle recognition. , object detection and other related sensing functions.

The sensor assembly includes a gas concentration sensor 131 and a humidity temperature sensor 132 . The gas concentration sensor 131 can detect the gas composition and concentration in the air to determine whether there is leakage of hazardous gases, such as toxic gases (carbon monoxide, vinyl chloride, hydrogen sulfide, etc.) and flammable gases (hydrogen, methane, ethane, etc.) Once it is found that the ambient temperature and humidity or the concentration of harmful and combustible gases exceeds the safety threshold, it will be immediately reported to the staff for processing. The humidity and temperature sensor 132 can acquire the temperature and humidity of the environment of the actual inspection area.

The camera and camera components include a visible light high-definition camera 133, an infrared camera 134, and a monocular camera. The infrared camera 134 is mainly used for night patrol, and shoots video images at night. In order to visually perceive and understand the chemical production environment, the monocular camera can obtain and process the image information of the working environment. The inspection requirements of the chemical environment usually fall into the following categories: whether the protective equipment of the staff is fully worn; whether the appearance of the production equipment is in good condition; whether the key indicator lights are normal, etc.

Aiming at these problems, the existing methods usually use the template matching method for identification, but this method has poor generalization. If the matching target in the original image rotates or changes in size, the effect of this method is poor. In this embodiment, the method of deep learning is used to identify images. First, image samples are collected for objects in the work scene, and corresponding labels are set. Then, a deep neural network is built to train the samples, and the network can be trained on the images. Recognition, classification and other functions, as shown in Figure 8.

The vehicle body 10 is provided with a rotatable structure 135 , for example, the lidar 130 , the visible light high-definition camera 133 , and the infrared camera 134 can all be arranged on the rotatable structure 135 .

The communicator 14 is a wireless communicator, and the communicator 14 is mainly responsible for information transmission with the UAV 2 and the base station to ensure smooth information transmission. The communicator 14 is integrated with inertial navigation equipment and GPS equipment.

The touch screen display 15 provides the user with a human-computer interaction interface, so that the user can modify the control parameters and collect relevant monitoring data. Through the human-computer interaction interface and high-intelligence robot system, the system operation process is simplified to reduce labor costs. The touch screen display 15 is integrated with a microphone and a speaker, so that the robot platform 1 has the functions of sound data collection and audio playback.

The power supply assembly 17 includes an explosion-proof box, a battery arranged in the explosion-proof box, and a motor. For the chemical industry, explosion-proof capability is essential. The ultra-high current of the robot is easy to generate electric sparks during the working process. Once it comes into contact with flammable gas, it will cause the danger of explosion. Such hidden dangers will be unimaginable consequences. During the inspection process, if there is a flammable gas leakage, it is very likely to ignite the flammable gas. The explosion-proof performance is the most important technical feature of the system. The explosion-proof box structure is adopted to ensure its explosion-proof performance.

In addition, the robot platform 1 further includes a platform main body 18 arranged on the vehicle body 10 for taking off, landing and charging the UAV 1 , and the platform main body 18 is connected to the power supply assembly 17 . The platform main body 18 provides a mounting and take-off and landing platform for the UAV 2 , the platform main body 18 is circular, and the circular platform main body 18 is more suitable for the random error of the landing position of the UAV 2 . At the same time, the platform main body 18 also provides the wireless charging function for the UAV 2. After the UAV 2 performs an inspection task, it must be charged in time. Therefore, after the UAV 2 returns to the take-off and landing platform, the platform main body 18 uses wireless charging. function to charge it. The high-power wireless power supply technology provided by the platform main body 18 does not require any physical connection, and completes the charging operation through a non-radiative wireless energy transmission method, which can completely free the drone 2 from manual operation during charging, greatly improving the unmanned aerial vehicle. The automation level of machine inspection.

As shown in Figures 5 and 6, the UAV 2 includes a fuselage 20, a UAV control assembly, and arranged on the fuselage 20: a landing gear 21, a propeller group, UAV drive and power components, and cameras and sensors components.

The UAV control assembly includes a flight control module, a data transmission and a picture transmission module, and the flight control module includes a flight control sealing box arranged on the fuselage 20, a flight controller, a power manager and an adjustment box arranged in the flight control sealing box. parameter interface, the flight controller is connected with the power manager and the parameter adjustment interface respectively. The interior of the flight controller has a barometer, an inertial navigation system and an attitude stabilization system. The data transmission and image transmission module includes a digital transmission and image transmission sealed box set on the fuselage 20, a digital transmission and image transmission UAV terminal set in the digital transmission and image transmission sealed box, and a digital transmission and image transmission unmanned aerial vehicle terminal. The ground terminal of data transmission and image transmission is connected to the human-machine terminal, and the drone terminal of data transmission and image transmission is connected to the flight controller.

The UAV drive and power supply components include a power explosion-proof box, a battery arranged in the power explosion-proof box, an electronic governor and a hub, a motor base, and a motor arranged on the motor base. The battery is connected to the power manager, and the battery passes through the hub. It is connected with the electronic governor, the input end of the electronic governor is connected with the parameter adjustment interface, and the output end of the electronic governor is connected with the motor.

The camera and sensor assembly includes a camera 23, a sensor sealing box, a sensor control board arranged in the sensor sealing box, and a gas sensor connected to the sensor control board.

The propeller group is provided with four groups, and each propeller group is provided with two propellers 22, and the two propellers 22 are arranged up and down. The design of the double-propeller structure can provide stronger flight power for the UAV to carry more sensing components.

In addition, a robot platform can carry multiple drones, and the style of the robot platform is not limited, such as sensor components, the installation position of the manipulator, the increase/decrease of wheels, and the similar explosion-proof design. These changes also belong to this The scope of protection applied for.

The following is a detailed description of the method of collaborative intelligent inspection of open space:

1: Coordinated multi-robot positioning and mapping in open space:

It mainly includes high-precision and low-latency perception and positioning calculation, creation of multi-robot maps for space-ground coordination in chemical environment, and multi-information fusion positioning in highly dynamic chemical environment. Specifically:

As shown in Figure 10: Perceptual positioning calculation is realized by three parts, including sensor unit, clock synchronization device and computer unit.

The sensor unit uses industrial cameras (color and grayscale), 3D LiDAR, inertial navigation unit, GPS and other equipment to collect information on the surrounding environment of the robot; the raw data generated by each sensor is sent to the computer unit after being synchronized by the clock; the computer unit collects After the sensor data is received, the data is preprocessed, and the processing content includes point cloud noise filtering, normal vector analysis, feature point extraction, feature description operations, etc. The computer units are installed on the UAV and robot platforms respectively. And chemical inspection data are sent back to the ground workstation for comprehensive analysis and processing.

The creation of multi-robot maps for open-air collaboration in the chemical environment mainly solves the problems of map holes and lack of perspective caused by the limited viewing angle of a single robot, and is the fundamental guarantee for building a high-precision full-coverage environment map. This embodiment performs three-dimensional reconstruction of the geometric model of the chemical factory environment where the robot is located according to the perception data sent by the drone and the computer unit on the robot platform.

As shown in Figure 11: The map creation steps include:

1. The unmanned aerial vehicle and the robot platform conduct modeling and scanning of the chemical plant area environment in a remote control mode. The movement trajectories of the two are in a following state, and the sensor data of their respective computer units are collected and local 3D point clouds are performed in their respective reference coordinate systems. creation of maps;

2. Use the GPS position information on the UAV and the robot platform to initially align the motion trajectories of the two;

3. Extract the ground plane part of the two local maps, and then optimize the map alignment of the two local maps based on the surface-surface nearest point iteration algorithm;

4. Based on the optimization theory, the global optimization and adjustment of the motion trajectories and local maps obtained by the UAV and the robot platform are carried out to obtain a more accurate chemical plant environment model.

As shown in Figure 12: Multi-information fusion positioning introduces the extended Kalman filter framework, and the relative position information such as inertial navigation integration, wheel odometer, laser odometer and GPS global coordinates, prior map and current perception data registration are absolutely The position information is effectively fused and calculated to obtain the position and attitude information of the robot with high precision and low delay.

2. Air-ground collaborative tracking and control:

It mainly includes UAV flight control system design, robot platform trajectory tracking control, and UAV self-landing control. Specifically:

UAV flight control system design:

Establish a dynamic model equation for the six degrees of freedom of the UAV, analyze the UAV actuator model composed of the motor model and the propeller aerodynamic model, and use the actually measured tension and torque curves to calculate the UAV-related gas. Dynamic parameters. The model can calculate the tension and torque received by each actuator at this time according to the input air flow velocity and the speed and angular velocity of the four propeller groups of the UAV, and realize the simulation verification of the model, as shown in Figure 13a and 13b. .

According to the kinematics and dynamics model of the UAV, the model can be divided into two parts: attitude and position. The same motion control method is divided into two parts: position control and attitude control. One position and one UAV attitude are called a target. The path control of the UAV is a collection of many target points in the space. The UAV needs to arrive at the planned target points in sequence according to the order of the target points, as shown in Figure 14.

In the attitude control of the drone, the active disturbance rejection controller is used to control the altitude, yaw, pitch, and roll motion parameters of the drone. The UAV can complete the tracking of the target trajectory. In order to enable the UAV to achieve flexible maneuverability in complex environments, it is necessary to design a state machine controller for the controller, which can automatically adjust the flight attitude and position of the robot according to the different environmental states of the UAV. As shown in Figure 15.

Robot platform trajectory tracking control:

Trajectory tracking control using reinforcement learning, a method of learning a controller without knowledge of control and mechanics. Reinforcement learning emphasizes interaction with the environment and is a dynamic learning process. Deep Deterministic Policy Gradient (DDPG) is a deep reinforcement learning algorithm, which inherits the characteristics of policy gradient algorithm and actor-critic algorithm, and uses DDPG algorithm to control the wheeled robot according to the state information of wheeled robot and the information of environmental feedback. The robot tracks the planned path.

First, the driving error is calculated from the actual position of the robot and the target point on the planned trajectory through the error function, and the error information is transmitted to the DDPG network. The DDPG network perceives the environment through the state description, and makes optimal decisions based on the current environmental state information. Set the reward function to guide its own learning, and finally achieve high-precision tracking of the planned path, as shown in Figure 16. The error function can be designed by using the lateral error of the robot. The robot obtains the current position through the perception system, and the path planning system obtains the preset trajectory position, and calculates the distance between the hypothetical point p in front of the robot and the target q as the value of the error function, using Lateral error continuously guides the robot to walk along the trajectory.

Before the DDPG algorithm is used, pre-training of the network is required, and simulation training in the real environment is usually performed on the simulator. In the traditional DDPG algorithm, too few training samples will make the training efficiency very low, and the network will not be able to converge quickly. By improving the strategy of returning the samples in the algorithm to the training experience pool, when the number of samples is small, network training is not performed. Let the robot continue to explore and fill in the number of samples to speed up the training; at the same time, in order to solve the complex environment, the cost of robot exploration is too high, and a large number of trial and error processes in the early stage consume a lot of useless work. In this example, the transfer learning method is used to pre-train first. For the DDPG network in a simple environment, the trained network is placed in a complex environment, and the complexity of the environment is gradually increased, so that the network can obtain the ability to generate motion strategies in a complex environment.

UAV autonomous landing control:

The robot platform (unmanned vehicle UGV), as the carrier of the unmanned aerial vehicle (UAV), can realize the automatic take-off and automatic vision-guided landing of the unmanned aerial vehicle. In normal inspection tasks, the unmanned vehicle performs inspection work according to the preset inspection road. In some scenarios, the unmanned vehicle cannot directly reach the inspection points. At this time, the drone can be used to reach these inspection points. , using the drone as the extra eye of the unmanned vehicle to transmit the inspection information to the robot controller of the unmanned vehicle; after the drone has completed the information collection task, it can automatically land under the guidance of the visual sign. On the main body of the platform of the unmanned vehicle, and complete the subsequent inspection tasks. The autonomous landing of the UAV needs to control the adjustment of the height and attitude of the UAV, which requires the design of visual signs with directionality and certain specifications. The drone first flies over the unmanned vehicle through the planned path of the planning system, and then searches for the visual mark through the visual guidance system. Once the visual mark is detected, the automatic guidance and landing procedure is started to realize the autonomous landing of the drone. The controller that controls the landing process of the drone can use a fuzzy controller to increase the smoothness of the landing process, as shown in Figure 18.

Three: Accident detection and early warning:

In this embodiment, a discrete event system model will be dynamically established for the operation of the chemical plant, and sensor data will be analyzed with the relevant theory of discrete event systems. For the data of different structures in the system, such as people's action behavior, temperature, gas concentration, etc., according to the data structure and data format, the sensor data dictionary is established, the sensor data packet analysis method is established, and the comprehensive logic judgment, deep learning and other methods are used to extract all the data in the data. The required feature information is combined with information theory knowledge to establish a strong mapping relationship between feature information and event information, as shown in Figure 19.

According to the possible faults and accidents of the actual system, the mapping relationship between faults and sensor events and the mapping relationship between accidents and sensor event sequences are established, that is, the system fault is modeled as a fault event in the system, and the system accident is modeled as a series of event sequences. Combination, according to the established mapping relationship, a state tree-based system diagnostic device is constructed, and the diagnostic device can perform online real-time fault detection and accident prediction by observing system events. When a system failure is detected, the diagnostic device issues a system warning; for accident prediction, the diagnostic device calculates the probability of an accident in real time, and issues a warning when the probability exceeds the threshold set by the system, as shown in Figure 20.

Aiming at the safety of personnel in the production process of chemical enterprises, video surveillance technology is used to realize the detection and tracking of personnel and the identification of specific behaviors. A pedestrian detection and tracking algorithm that combines deep features and artificial features can be used to meet the real-time requirements of the system and achieve the optimal personnel detection accuracy. At the same time, combined with the deep learning-based item detection algorithm, the personnel in the production area can be detected. Whether wearing a helmet, protective goggles, whether wearing long-sleeved work clothes, etc. is automatically identified, and when the identification and analysis of the specific behavior of the person is carried out, the face recognition method based on neural network is used to identify the tracking target, and the tracking information is classified multiple times and carried out. Fusion, analyzes the relationship between individual behavior and motion trajectories, and the relationship between individual behavior and group behavior, establishes an automaton model of individual and group behavior, and uses relevant knowledge of discrete event systems to identify behavior, as shown in Figure 21.

In addition, similar transformations to the accident detection and early warning methods in this embodiment also belong to the protection scope of the present application.

The present invention can achieve the following beneficial effects:

1. Collaboration between unmanned vehicles and drones:

The invention designs a wheeled robot platform equipped with an unmanned aerial vehicle, which can meet the loading, take-off and landing requirements for the inspection unmanned aerial vehicle, get rid of the limitation of requiring staff to carry the unmanned aerial vehicle to the scene, and greatly improve the The autonomous control capability of man and machine; the wheeled robot platform can carry drones, and can carry large high-energy batteries, which can be charged when the power of the drone is low; the wheeled robot platform can also carry different types of sensor modules, including Visible light cameras, infrared cameras, lidars, satellite navigation system receivers and other devices to make up for the lack of UAV load capacity; UAVs can select the required sensor modules on the platform, and use robotic arms to replace sensors for UAVs The module greatly improves the inspection capability and efficiency of UAVs.

2. Good explosion-proof performance:

The invention uses the explosion-proof box isolation method to isolate the instantaneous high-voltage current from the outside, so that the inspection robot has explosion-proof performance.

3. Environmental perception and understanding of high intelligence:

The invention integrates a variety of sensors to perceive and understand the chemical production environment, and proposes to train the deep neural network by collecting picture samples on the spot, so as to realize the identification and detection of people or equipment in the chemical production environment. method, the detection results are more reliable and the model generalization is better.

4. Synchronous mapping and positioning of dynamic environment:

Facing the complex and changeable chemical environment, the present invention introduces the air-ground collaborative multi-robot SLAM method, breaks through the full-coverage environment modeling technology in a large-scale chemical plant area, overcomes the multi-sensor fusion positioning technology, and realizes the inspection robot in the mixed chemical environment. High-precision modeling and safe and reliable positioning.

5. High-speed and high-precision path planning technology:

The invention applies the hybrid algorithm to the solution of path optimization, which makes the algorithm more efficient, incorporates kinematic constraints into the global planning, makes the trajectory more reasonable and easy to track, can complete the space-ground coordinated path planning, and ensures that the respective paths are feasible. At the same time cooperate with each other.

6. Accident early warning capability:

Aiming at the accidents that may occur in the production process of chemical enterprises, the invention designs a set of fusion heterogeneous data accident prediction algorithms based on discrete event system theory. Complete the safety inspection work in the chemical operation environment, supervise the safety problems of the production process of the chemical enterprise, greatly reduce the safety accident, and ensure the safety of the production property and the life of the personnel of the chemical enterprise.

The above-mentioned embodiments are only intended to illustrate the technical concept and characteristics of the present invention, and the purpose thereof is to enable those who are familiar with the art to understand the content of the present invention and implement them accordingly, and cannot limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be included within the protection scope of the present invention.

Claims (10)

1. an air-ground collaborative intelligent inspection robot is characterized in that: comprising a robot platform, an unmanned aerial vehicle, and the robot platform comprises a vehicle body, a wheel and a drive assembly that are arranged at the bottom of the vehicle body, and a On the vehicle body: a mechanical arm, an environment perception component, a communicator, a robot controller and a power supply component, and the communicator realizes communication connection between the drone and the base station. 2 . The space-ground cooperative intelligent inspection robot according to claim 1 , wherein the mechanical arm comprises a base set on the vehicle body and a joint assembly rotatably connected to the base. 3 . , a mechanical claw rotatably connected to the joint assembly and a rotary drive member that drives each component to rotate, the joint assembly includes one or more connecting joints rotatably connected end to end in turn, and the mechanical claw includes A claw body rotatably connected to the joint assembly, a pair of claw hands connected to the claw body, and a claw hand driving assembly for driving the claw hands to perform a grasping action, the claw hands The drive assembly includes a worm screw disposed on the claw body, a worm wheel connected to one end of the claw hand and matched with the worm screw, and a claw hand driving member for driving the worm screw to rotate. 3 . The air-ground cooperative intelligent inspection robot according to claim 1 , wherein the robot platform further comprises a platform body arranged on the vehicle body for taking off, landing and charging the drone. 4 . , the platform main body is connected with the power supply assembly. 4 . The air-ground cooperative intelligent inspection robot according to claim 1 , wherein the environment perception component includes a lidar, a sensor component, and a camera and a camera component. 5 . 5. The air-ground cooperative intelligent inspection robot according to claim 1, wherein the drone comprises a fuselage, a drone control assembly, and on the fuselage: a landing gear, The propeller group, the unmanned aerial vehicle driving and power supply assembly, and the camera and sensing assembly, the propeller group is provided with four groups, and the propeller group in each group is provided with two propellers, and the two propellers are arranged up and down. 6 . The air-ground collaborative intelligent inspection robot according to claim 5 , wherein the UAV control assembly includes a flight control module, a data transmission and a picture transmission module, and the flight control module includes a The flight control sealing box on the fuselage, the flight controller, the power manager and the parameter adjustment interface arranged in the flight control sealing box, the flight controller is respectively connected with the power manager, The parameter adjustment interface is connected; the data transmission and image transmission module includes a digital transmission and image transmission sealing box arranged on the body, and a digital transmission and image transmission sealing box arranged in the digital transmission and image transmission. The image transmission drone terminal, the data transmission and image transmission ground terminal connected with the data transmission and image transmission drone terminal, the data transmission and image transmission drone terminal and the flight controller connected. 7. The air-ground collaborative intelligent inspection robot according to claim 6, wherein the drone drive and power supply components comprise a power explosion-proof box, a battery arranged in the power supply explosion-proof box, an electronic regulator Speeder and hub, motor base, motor arranged on the motor base, the battery is connected to the power manager, and the battery is connected to the electronic governor through the hub The input end of the electronic governor is connected with the parameter adjustment interface, and the output end of the electronic governor is connected with the motor. 8 . The air-ground cooperative intelligent inspection robot according to claim 5 , wherein the camera and sensing components comprise a camera, a sensor sealing box, a sensor control board arranged in the sensor sealing box, A gas sensor connected to the sensor control board. 9. An air-ground cooperative intelligent inspection method, characterized in that: it adopts the air-ground cooperative intelligent inspection robot according to any one of claims 1 to 8, comprising: 1) Air-ground collaborative multi-robot positioning and mapping: including perceptual positioning calculation, map creation, and multi-information fusion positioning, including: Perception and positioning calculation uses sensor components to collect surrounding environment information, and analyzes and processes effective environmental perception and detection data after processing the collected data; Map creation includes modeling and scanning the environment, collecting data from sensor components and creating local 3D point cloud maps in their respective reference coordinate systems, initially aligning the motion trajectories of the UAV and the robot platform, and extracting the two local maps. In the ground plane part, the two local maps are optimized for map alignment, and the motion trajectories and local maps obtained by the UAV and the robot platform are globally optimized and adjusted; Multi-information fusion positioning includes the fusion calculation of relative position information and absolute position information such as GPS global coordinates, prior map and current perception data registration, to obtain robot position and attitude information, 2) Air-ground collaborative tracking and control: including UAV flight control design, robot platform trajectory tracking control, and UAV self-landing control, including: The design of the UAV flight control system includes establishing the dynamic model equation for the UAV's six degrees of freedom, analyzing the UAV actuator model composed of the motor model and the propeller aerodynamic model, and using the actual measured tension and torque. The curve calculates the aerodynamic parameters related to the UAV; according to the kinematic and dynamic model of the UAV, the model is divided into two parts: attitude and position, and the motion control method is divided into two parts: position control and attitude control. The attitude of the UAV is called a target point, and the path control of the UAV is the collection of many target points in the space. The UAV needs to arrive at the planned target point in sequence according to the order of the target points; The trajectory tracking control of the robot platform uses the DDPG algorithm to control the robot platform to track the planned path according to the state information of the robot platform and the feedback information of the environment; The autonomous landing control of the UAV includes flying over the unmanned vehicle through the planned path, searching for the visual sign through the visual guidance system, detecting the visual sign, starting the automatic guidance and landing procedure, and realizing the autonomous landing of the UAV. 10. The air-ground collaborative intelligent inspection method according to claim 9, characterized in that: the method further comprises detection and early warning of accidents, including establishing fault-to-sensor events according to possible faults and accidents in the actual system According to the mapping relationship between the accident and the sensor event sequence, a state tree-based system diagnostic device is constructed. The diagnostic device performs online real-time fault detection and accident prediction by observing system events. When a system failure is detected When the error occurs, the diagnostic device issues a system warning; the diagnostic device calculates the probability of an accident in real time, and issues a warning when the probability exceeds the threshold set by the system.

Images