CN116125996B - Safety monitoring method and system for unmanned vehicle - Google Patents

Abstract

The present invention proposes a safety monitoring method and system for unmanned vehicles, including: establishing a communication relationship between multiple unmanned vehicles and a background server in a preset area; , to sense the environment inside the unmanned vehicle to obtain the first sensing data; to sense the environment outside the unmanned vehicle to obtain the second sensing data; the background server determines how many According to the type of unmanned vehicle, the corresponding level of safety monitoring is performed. Realize the safety monitoring of multiple unmanned vehicles in the preset area. At the same time, based on the type of multiple unmanned vehicles, the corresponding level of safety monitoring is performed according to the type, which reduces the load on the background server and improves the utilization of monitoring resources. utilization rate.

Description

A safety monitoring method and system for an unmanned vehicle

technical field

The invention relates to the technical field of automotive electronics, in particular to a safety monitoring method and system for an unmanned vehicle.

Background technique

An unmanned vehicle is a type of smart car, which mainly relies on the on-board sensor system to perceive the road environment, automatically plan the driving route and control the vehicle to reach the predetermined destination. It uses on-board sensors to perceive the surrounding environment of the vehicle, and controls the steering and speed of the vehicle based on the road, vehicle position and obstacle information obtained from the perception, so that the vehicle can drive safely and reliably on the road. In the prior art, most of the safety monitoring is performed on a single unmanned vehicle, and it is impossible to realize the safety monitoring on multiple unmanned vehicles in the preset area. If multiple unmanned vehicles in the preset area are monitored Performing the same level of security monitoring during security monitoring will cause a huge load on the background server and waste resources.

Contents of the invention

The present invention aims to solve one of the technical problems in the above-mentioned technologies at least to a certain extent. For this reason, the first purpose of the present invention is to propose a safety monitoring method for unmanned vehicles, which can realize the safety monitoring of multiple unmanned vehicles in a preset area, and at the same time, based on the types of multiple unmanned vehicles, The corresponding level of security monitoring is performed according to the type, which reduces the load of the background server and improves the utilization rate of monitoring resources.

The second purpose of the present invention is to propose a safety monitoring system for unmanned vehicles.

In order to achieve the above purpose, the embodiment of the first aspect of the present invention proposes a safety monitoring method for unmanned vehicles, including:

Establish a communication relationship between multiple unmanned vehicles in the preset area and the background server;

When multiple unmanned vehicles are driving in the preset area, the environment perception of the interior of the unmanned vehicle is performed to obtain the first perception data; the environment perception of the exterior of the unmanned vehicle is obtained to obtain the second perception data;

The background server determines the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and performs corresponding level of safety monitoring according to the types.

According to some embodiments of the present invention, environment perception is performed on the interior of the unmanned vehicle to obtain the first perception data, including:

Obtain the image of the control panel of the vehicle-mounted terminal inside the unmanned vehicle and the image of the scene inside;

Check the image quality of the control panel image and the scene image, and perform parameter adjustment processing to obtain the target control panel image and the target scene image when it is determined that the image quality is not up to standard;

Analyze the image of the target control panel to determine the operation information of the vehicle's control nodes, vehicle state parameter information, and vehicle-machine equipment operation information;

Analyze the image of the target scene to determine the behavior characteristics of passengers;

The first sensing data is determined according to the operation information of the control node, the vehicle state parameter information, the operation information of the on-board equipment, and the behavior characteristics of passengers.

According to some embodiments of the present invention, the image quality of the control panel image is checked, and when it is determined that the image quality is not up to standard, parameter adjustment processing is performed to obtain the target control panel image, including:

Based on the gradient algorithm, the Sobel operator is used to calculate the first gradient value of the control panel image in the horizontal direction and the second gradient value in the vertical direction;

Querying the preset first gradient value-second gradient value-sharpness data table according to the first gradient value and the second gradient value to determine the sharpness value;

When it is determined that the sharpness value is less than the preset sharpness threshold, it means that the image quality is not up to standard, determine the difference between the preset sharpness threshold and the sharpness value, query the difference-focus correction value data table according to the difference, and determine the focus correction value , and perform parameter adjustment processing according to the focal length correction value to obtain a target control panel image.

According to some embodiments of the present invention, environment perception is performed on the outside of the unmanned vehicle to obtain second perception data, including:

Obtain external environment images and radar information of unmanned vehicles;

Determining the obstacle information around the unmanned vehicle and the collision risk with each obstacle according to the external environment image and radar information, and determining the second sensing data according to the obstacle information around the unmanned vehicle and the collision risk with each obstacle.

According to some embodiments of the present invention, the background server determines the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and performs corresponding levels of safety monitoring according to the types, including:

The background server inputs the first perception data into the pre-trained data classification model for classification, and outputs several first classification data;

Inputting the second perception data into the pre-trained data classification model for classification, and outputting several second classification data;

performing data fusion on the first classification data and the second classification data of the same data type as a set of fusion data, and then obtaining several sets of fusion data;

Input several sets of fusion data into the corresponding single identification model, and output the first analysis result; after determining that at least one of the several first analysis results indicates the existence of risk data, then determine that the unmanned vehicle corresponding to the fusion data is the first A type of vehicle, and implement a level of security monitoring.

According to some embodiments of the present invention, also include:

When it is determined that all of the several first analysis results indicate that there is no risk data, select the target fusion data from several sets of fusion data;

Perform feature extraction on all fusion data except the target fusion data to obtain a feature vector, and convert the feature vector to the type corresponding to the target fusion data to obtain a converted feature vector;

Matching the target fusion data and each conversion feature vector to determine the correlation between the target fusion data and each conversion feature vector;

determining a target set according to the association relationship;

generating a data system according to the target set;

inputting the data system into the composite identification model, and outputting the second analysis result;

When it is determined that there is risk data according to the second analysis result, it is determined that the unmanned vehicle corresponding to the fusion data is the second type of vehicle, and a secondary safety monitoring is performed; when it is determined that there is no risk data according to the second analysis result, then it is determined that the fusion data The corresponding unmanned vehicle is the third type of vehicle, and performs three-level security monitoring.

According to some embodiments of the present invention, a single recognition model is trained, and the method includes:

Obtain sample fusion data;

Based on the data layer of the single identification model, the sample fusion data is preprocessed to determine the risk factor;

Based on the index layer of the single identification model, the data analysis of the risk factors is carried out, and several indicators are determined and screened to obtain the risk indicators;

Based on the model parameter layer of the single identification model, various risk indicators are combined to obtain multiple combination results, and the combination result with the highest risk probability prediction accuracy is selected as the target combination result;

The output layer based on the single recognition model outputs the prediction result of the sample fusion data. When the prediction result is consistent with the real result corresponding to the sample fusion data, it means that the training is completed.

According to some embodiments of the present invention, determining the target set according to the association relationship includes:

Taking the target fusion data as a key node, and converting the feature vector as an associated node of the key node;

Constructing a screening system according to key nodes, associated nodes and associated relationships; the screening system includes a distance value from each associated node to a key node;

Filter out the associated nodes whose distance value is less than the preset distance value as the target associated node;

Determine the target set according to the target associated nodes and key nodes.

According to some embodiments of the present invention, the compound recognition model is trained, and the method includes:

Determine the big data of unmanned vehicles, and construct the map text of the driving scene based on the big data;

Extract each node in the map text, determine the relationship between each node, and construct a knowledge map of the data link corresponding to the driving scene;

Determine the risk indicator information for each node;

Extract the associated feature vectors in the knowledge map, perform feature fusion according to the associated feature vectors and risk index information, and obtain the fusion vector;

The composite recognition model is trained based on the fused vectors.

In order to achieve the above purpose, the embodiment of the second aspect of the present invention proposes a safety monitoring system for unmanned vehicles, including:

Create a module for establishing the communication relationship between multiple unmanned vehicles and the background server in the preset area;

The sensing module is used to sense the environment inside the unmanned vehicle to obtain the first sensing data when multiple unmanned vehicles are driving in the preset area; to sense the environment outside the unmanned vehicle to obtain the second sensory data;

The background server is used to determine the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and perform corresponding level of safety monitoring according to the types.

The present invention proposes a safety monitoring method and system for unmanned vehicles, the beneficial effects of which are:

1. Realize the safety monitoring of multiple unmanned vehicles in the preset area. At the same time, based on the type of multiple unmanned vehicles, the corresponding level of safety monitoring is performed according to the type, which reduces the load on the background server and improves the monitoring. resource utilization.

2. Based on the environment perception of the interior and exterior of the unmanned vehicle, the first perception data and the second perception data are obtained, so as to realize the accurate perception of the environment where the unmanned vehicle is located and improve the comprehensiveness of the acquired data.

3. When distinguishing the types of unmanned vehicles, the identification of single data based on a single identification model, and the identification of composite data based on a composite identification model are convenient for accurate and comprehensive judgment of the first type of vehicle, the second type of vehicle and the third type. types of vehicles, and implement the corresponding level of security monitoring.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and appended drawings.

The technical solutions of the present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the attached picture:

Fig. 1 is a flow chart of a safety monitoring method for an unmanned vehicle according to an embodiment of the present invention;

FIG. 2 is a flowchart of a method for obtaining first sensing data according to an embodiment of the present invention;

Fig. 3 is a block diagram of a safety monitoring system for an unmanned vehicle according to an embodiment of the present invention.

Detailed ways

The preferred embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described here are only used to illustrate and explain the present invention, and are not intended to limit the present invention.

As shown in Figure 1, the embodiment of the first aspect of the present invention proposes a safety monitoring method for unmanned vehicles, including steps S1-S3:

S1. Establish the communication relationship between multiple unmanned vehicles and the background server in the preset area;

S2. When multiple unmanned vehicles are driving in the preset area, perform environment perception on the interior of the unmanned vehicle to obtain first perception data; perform environment perception on the outside of the unmanned vehicle to obtain second perception data;

S3. The background server determines the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and performs corresponding level of safety monitoring according to the types.

The working principle of the above technical solution: establish the communication relationship between multiple unmanned vehicles in the preset area and the background server, so that the background server can obtain the initial monitoring information of multiple unmanned vehicles in the preset area. The initial monitoring information is based on Acquired by the perception module installed on the unmanned vehicle. The preset area is the planned operating area of the unmanned vehicle.

In this embodiment, the first sensing data is the data acquired through environmental sensing of the interior of the unmanned vehicle. Exemplarily, the first sensing data is control node operation information, vehicle state parameter information, vehicle-machine equipment operation information, and passenger behavior characteristics.

In this embodiment, the second sensing data is the data acquired through environmental sensing of the outside of the unmanned vehicle. For example, the obstacle information around the unmanned vehicle and the collision risk with each obstacle determined based on the external environment image and radar information of the unmanned vehicle.

Based on the first sensing data and the second sensing data, it is convenient to conduct a comprehensive perception of the interior and exterior of the unmanned vehicle, to obtain comprehensive sensing data, and to facilitate the background server to accurately determine the type of each unmanned vehicle.

In this embodiment, the types of unmanned vehicles include the first type of vehicles, the second type of vehicles and the third type of vehicles, which respectively implement the first-level safety monitoring, the second-level safety monitoring and the third-level safety monitoring.

In this embodiment, the monitoring requirements of the first-level security monitoring, the second-level security monitoring, and the third-level security monitoring are successively reduced, and the load on the background server caused by the corresponding first-level security monitoring is the largest, followed by the second-level security monitoring. Level 3 security monitoring is the lowest. Exemplarily, when the first type of vehicle is performing level one safety monitoring, the monitoring interval for acquiring the data of environment awareness acquisition for the first type of vehicle is 1s. When the secondary safety monitoring is performed on the second type of vehicle, the monitoring interval for acquiring the data of environment perception acquisition on the second type of vehicle is 2s. When three-level security monitoring is performed on the third type of vehicle, the monitoring interval for acquiring the data of environment perception acquisition on the third type of vehicle is 3s.

The beneficial effect of the above-mentioned technical solution: realize the security monitoring of multiple unmanned vehicles in the preset area, and at the same time, based on the types of multiple unmanned vehicles, perform the corresponding level of security monitoring according to the type, reducing the background server. load, improving the utilization of monitoring resources.

As shown in FIG. 2, according to some embodiments of the present invention, environment perception is performed on the inside of the unmanned vehicle to obtain the first perception data, including steps S21-S25:

S21. Obtain the control panel image and the internal scene image of the vehicle-mounted terminal inside the unmanned vehicle;

S22. Check the image quality of the control panel image and the scene image, and when it is determined that the image quality is not up to standard, perform parameter adjustment processing to obtain the target control panel image and the target scene image;

S23. Analyzing the image of the target control panel to determine the operation information of the control node of the vehicle, the vehicle state parameter information and the operation information of the vehicle equipment;

S24. Analyzing the image of the target scene to determine the behavior characteristics of the passengers;

S25. Determine the first sensing data according to the operation information of the control node, the vehicle state parameter information, the operation information of the on-board equipment, and the behavior characteristics of the passengers.

The working principle of the above technical solution: In this embodiment, the image of the control panel of the vehicle-mounted terminal is, for example, the acquired image of the central control panel inside the unmanned vehicle.

In this embodiment, the internal scene images are images of the front row and the rear row inside the unmanned vehicle, including images of passengers riding.

In this embodiment, the target control panel image is a control panel image whose image quality meets the standard obtained by performing parameter adjustment processing on the control panel image.

In this embodiment, the target scene image is a scene image obtained by performing parameter adjustment processing on the scene image so that the image quality meets the standard.

In this embodiment, the parameter adjustment processing includes sharpness adjustment processing.

In this embodiment, the control node operation information is the operation status information of the nodes in the control software carried by the unmanned vehicle, and the nodes include hardware drive nodes and human-computer interaction nodes. The vehicle state parameter information includes brake pedal parameters, accelerator pedal parameters, steering wheel torque parameters, etc. of the vehicle. The operation information of the on-board equipment includes whether each equipment is running normally and the command response.

In this embodiment, the image of the target scene is analyzed to determine the behavior characteristics of the passengers; it is convenient to determine and collect the behavior characteristics of the passengers. Behavioral characteristics include whether seat belts are worn, whether facial expressions are normal, etc.

The beneficial effect of the above technical solution: obtain the control panel image and the internal scene image of the vehicle-mounted terminal inside the unmanned vehicle; check the image quality of the control panel image and the scene image, and perform parameter adjustment processing when it is determined that the image quality is not up to standard, A target control panel image and a target scene image are obtained; it is convenient to determine a target control panel image and a target scene image that achieve image quality. The first sensing data is determined according to the operation information of the control node, the vehicle state parameter information, the operation information of the on-board equipment, and the behavior characteristics of passengers. It is convenient to comprehensively collect the internal information of unmanned vehicles, improve the comprehensiveness of the first perception data obtained, and at the same time collect the behavior characteristics of passengers, which is convenient to add user factors and improve the level of safety monitoring.

According to some embodiments of the present invention, the image quality of the control panel image is checked, and when it is determined that the image quality is not up to standard, parameter adjustment processing is performed to obtain the target control panel image, including:

Based on the gradient algorithm, the Sobel operator is used to calculate the first gradient value of the control panel image in the horizontal direction and the second gradient value in the vertical direction;

Querying the preset first gradient value-second gradient value-sharpness data table according to the first gradient value and the second gradient value to determine the sharpness value;

When it is determined that the sharpness value is less than the preset sharpness threshold, it means that the image quality is not up to standard, determine the difference between the preset sharpness threshold and the sharpness value, query the difference-focus correction value data table according to the difference, and determine the focus correction value , and perform parameter adjustment processing according to the focal length correction value to obtain a target control panel image.

The working principle of the above technical solution: in this embodiment, the gradient algorithm includes Tenengrad gradient algorithm or Laplacian gradient algorithm. The greater the value of the determined first gradient value and the second gradient value, the greater the definition value representing the image of the control panel.

In this embodiment, the preset first gradient value-second gradient value-sharpness data table is obtained from multiple experiments, and a comparative query is performed based on the first gradient value and the second gradient value to determine the sharpness.

Beneficial effects of the above technical solution: Based on obtaining the control panel image whose definition value reaches the standard, that is, the target control panel image, it is beneficial to improve the accuracy of analyzing the target control panel image.

In one embodiment, the method for obtaining the target scene image is the same as the method for obtaining the target control panel image.

According to some embodiments of the present invention, environment perception is performed on the outside of the unmanned vehicle to obtain second perception data, including:

Obtain external environment images and radar information of unmanned vehicles;

Determining the obstacle information around the unmanned vehicle and the collision risk with each obstacle according to the external environment image and radar information, and determining the second sensing data according to the obstacle information around the unmanned vehicle and the collision risk with each obstacle.

The working principle of the above technical solution: Based on the analysis of the external environment image and radar information, it can determine which obstacles exist around, as well as the position and motion state of the obstacle, and the distance value from the unmanned vehicle, and analyze the surrounding area of the unmanned vehicle. The obstacle information and the risk of collision with each obstacle are used as the second sensing data.

The beneficial effect of the technical solution above is that it is convenient to accurately determine the external environment perception information of the unmanned vehicle, that is, the second perception data.

According to some embodiments of the present invention, the background server determines the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and performs corresponding levels of safety monitoring according to the types, including:

The background server inputs the first perception data into the pre-trained data classification model for classification, and outputs several first classification data;

Inputting the second perception data into the pre-trained data classification model for classification, and outputting several second classification data;

performing data fusion on the first classification data and the second classification data of the same data type as a set of fusion data, and then obtaining several sets of fusion data;

Input several sets of fusion data into the corresponding single identification model, and output the first analysis result; after determining that at least one of the several first analysis results indicates the existence of risk data, then determine that the unmanned vehicle corresponding to the fusion data is the first A type of vehicle, and implement a level of security monitoring.

Working principle of the above technical solution: In this embodiment, both the first sensing data and the second sensing data include different types of data. Types include image, text, etc.

In this embodiment, the several first classification data include image data, text data, etc.; the several second classification data include image data, text data, etc.

In this embodiment, the first classification data and the second classification data of the same data type are fused together as a set of fusion data, and then several sets of fusion data are obtained; for example, the image data corresponding to the first classification data is combined with the second classification data The image data corresponding to the binary classification data are fused.

In this embodiment, the single recognition model is a model that recognizes a single data type, and the example single recognition model is an image recognition model, which only recognizes image type data.

In this embodiment, several sets of fusion data are respectively input into the corresponding single recognition model, and the first analysis result is output; for example, the fusion data A is input into the single recognition model A for recognition, and the fusion data B is input into the single recognition model B For identification, on the one hand, based on the identification of fusion data, the efficiency of data identification is improved, and at the same time, the accuracy of identification of corresponding data is improved based on a specific single identification model.

In this embodiment, it is a recognition result based on a corresponding single recognition model for a certain set of fused data, that is, it is judged whether there is risk data in the set of fused data. The risk data includes whether the control information of the vehicle is correct, whether the passengers are unwell, whether the collision risk is extremely high based on the current control instructions, and so on.

Beneficial effects of the above technical solution: each set of fusion data corresponds to the fusion of the same type of internal perception data and external perception data of the unmanned vehicle, which facilitates the subsequent improvement of the processing efficiency of the same type of data. Input several sets of fusion data into the corresponding single recognition model, and output the first analysis result. On the one hand, based on the recognition of the fusion data, the efficiency of data recognition is improved, and at the same time, the recognition of the corresponding data is improved based on the specific single recognition model. accuracy. When it is determined that at least one of the several first parsing results indicates that there is risk data, it is determined that the unmanned vehicle corresponding to the fused data is a first-type vehicle, and a first-level safety monitoring is performed. It is convenient to accurately determine the unmanned vehicle as the first type of vehicle, and perform first-level safety monitoring.

According to some embodiments of the present invention, also include:

When it is determined that all of the several first analysis results indicate that there is no risk data, select the target fusion data from several sets of fusion data;

Perform feature extraction on all fusion data except the target fusion data to obtain a feature vector, and convert the feature vector to the type corresponding to the target fusion data to obtain a converted feature vector;

Matching the target fusion data and each conversion feature vector to determine the correlation between the target fusion data and each conversion feature vector;

determining a target set according to the association relationship;

generating a data system according to the target set;

inputting the data system into the composite identification model, and outputting the second analysis result;

When it is determined that there is risk data according to the second analysis result, it is determined that the unmanned vehicle corresponding to the fusion data is the second type of vehicle, and a secondary safety monitoring is performed; when it is determined that there is no risk data according to the second analysis result, then it is determined that the fusion data The corresponding unmanned vehicle is the third type of vehicle, and performs three-level security monitoring.

The working principle of the above technical solution: In this embodiment, the target fusion data is a certain type of preset data, and it is assumed that the type of the target fusion data is A, which is used as standardized data.

In this embodiment, the feature vector represents key data of the fused data. Convert the feature vector to the type corresponding to the target fusion data to obtain the converted feature vector; for example, convert the fusion data B and C into data of type A.

In this embodiment, the converted feature vector is consistent with the feature vector of the target fusion data, which is convenient for data analysis, and the target fusion data and each converted feature vector are matched to determine the relationship between the target fusion data and each converted feature vector; The type of target fusion data realizes the unification of all types of data and establishes a comprehensive and accurate relationship.

In this embodiment, the target set is determined according to the association relationship; the target set includes converted feature vectors with a high degree of correlation with the target fusion data, and target fusion data. That is, the converted feature vectors with low correlation with the target fusion data are eliminated, which reduces the amount of data processing and facilitates the subsequent improvement of the data processing rate.

In this embodiment, the data system is a complete data model constructed based on the integration of data resources based on the target set as a whole. Realize efficient logical relationship processing of data based on the data system.

In this embodiment, the second analysis result is the recognition result obtained by inputting the data system into the composite recognition model, and it is judged whether risk data exists in the data system.

In this embodiment, the composite recognition model includes recognition rules for complex relationships, not a single type of data recognition.

The beneficial effects of the above technical solution: Based on the composite identification model, it is convenient to comprehensively and accurately determine the risk data after various fusion data are combined, and it is convenient to accurately identify the second type of vehicle and the third type of vehicle. Based on a single identification model to identify from a single aspect, based on a composite identification model to identify from a composite aspect, it is convenient to accurately and comprehensively judge the first type of vehicle, the second type of vehicle and the third type of vehicle, and perform corresponding levels of safety monitoring.

According to some embodiments of the present invention, a single recognition model is trained, and the method includes:

Obtain sample fusion data;

Based on the data layer of the single identification model, the sample fusion data is preprocessed to determine the risk factor;

Based on the index layer of the single identification model, the data analysis of the risk factors is carried out, and several indicators are determined and screened to obtain the risk indicators;

Based on the model parameter layer of the single identification model, various risk indicators are combined to obtain multiple combination results, and the combination result with the highest risk probability prediction accuracy is selected as the target combination result;

The output layer based on the single recognition model outputs the prediction result of the sample fusion data. When the prediction result is consistent with the real result corresponding to the sample fusion data, it means that the training is completed.

The working principle of the above technical solution: In this embodiment, the risk factors include factors affecting vehicle control safety and factors affecting passenger safety.

In this embodiment, the risk indicators include standardized indicators formed by data analysis of risk factors based on the indicator layer.

The beneficial effect of the above technical solution: based on different types of sample fusion data, the corresponding single recognition model is trained respectively, and the training process includes training the data layer, index layer, model parameter layer, and output layer of the single recognition model, It is convenient to obtain accurate model parameters of a single recognition model, and accurately train a single recognition model.

According to some embodiments of the present invention, determining the target set according to the association relationship includes:

Taking the target fusion data as a key node, and converting the feature vector as an associated node of the key node;

Constructing a screening system according to key nodes, associated nodes and associated relationships; the screening system includes a distance value from each associated node to a key node;

Filter out the associated nodes whose distance value is less than the preset distance value as the target associated node;

Determine the target set according to the target associated nodes and key nodes.

The working principle of the above technical solution: In this embodiment, the associated nodes whose distance value is smaller than the preset distance value are screened out as the target associated nodes, that is, nodes with a high degree of association.

In this embodiment, key nodes serve as central nodes and leading nodes.

The screening system is a relationship topology diagram generated based on key nodes, associated nodes, and associated relationships. It visually displays the associated relationship between key nodes and associated nodes, and can determine the distance value from each associated node to the key node.

The beneficial effect of the above technical solution: based on the construction of the screening system, it is convenient to clearly display the key nodes, associated nodes and associated relationships, and to quantify the distance value from each associated node to the key node in the screening system; thus to facilitate the accurate determination of the target key node, and then Accurately determine the target set.

According to some embodiments of the present invention, the compound recognition model is trained, and the method includes:

Determine the big data of unmanned vehicles, and construct the map text of the driving scene based on the big data;

Extract each node in the map text, determine the relationship between each node, and construct a knowledge map of the data link corresponding to the driving scene;

Determine the risk indicator information for each node;

Extract the associated feature vectors in the knowledge map, perform feature fusion according to the associated feature vectors and risk index information, and obtain the fusion vector;

The composite recognition model is trained based on the fused vectors.

The working principle of the above technical solution: In this embodiment, the map text includes various driving scenes constructed from the big data of unmanned vehicles.

In this embodiment, the composite identification model is a risk conduction fusion model, based on the overall identification data system, to more accurately identify the overall risk data.

In this embodiment, when training the composite identification model, the model is obtained based on the training of the entire driving scene to judge the overall driving risk.

In this embodiment, the composite recognition model is a deep neural network model.

In this embodiment, each node corresponds to fusion data of the same type of data in the internal perception data and external perception data of the unmanned vehicle, and the fusion data is converted into a corresponding converted feature vector that matches the target fusion data.

In this embodiment, in the knowledge graph, each driving scene corresponds to a data link, and the data link is the relationship link of each node.

In this embodiment, the risk indicator information is the dynamic risk parameters and static risk parameters of the subject corresponding to the node. The subject of each node is a passenger or a vehicle. The dynamic risk parameters of the passenger include changes in various body movements, and the static risk parameters include the relative fixed position of the passenger, such as being in the back row or the front row. The dynamic risk parameters of the vehicle include changes in the control parameters and execution parameters of the vehicle. The static risk parameter is a fixed parameter of the vehicle, such as a device in a fixed position.

In this embodiment, the associated feature vector in the knowledge graph is extracted to represent the associated relationship of each node in the data chain.

In this embodiment, the fusion vector is obtained by fusion based on the association relationship between each node and the risk index information of each node, and the fusion vector represents the logical relationship between each node and the overall risk output result.

Beneficial effects of the above technical solution: Based on the extraction of associated feature vectors in the knowledge map, feature fusion is performed according to the associated feature vectors and risk index information to obtain fusion vectors; the composite recognition model is trained based on the fusion vectors. It is convenient to obtain an accurate compound recognition model. In the training process, construct the map text of the driving scene based on big data, extract each node in the map text, and determine the association relationship between each node, and construct the knowledge map of the data link corresponding to the driving scene; The knowledge map of the data link; it is convenient to accurately determine the actual driving scene corresponding to the unmanned vehicle, and realize the accurate judgment of the logical relationship of each node.

According to some embodiments of the present invention, generating a data system according to the target set includes:

Dimension fields for data analysis, description fields for describing dimensions and summary fields for statistics are extracted according to the target set;

Calculate and modify the summary field, and modify the statistical parameters;

Create a description script according to the description field, and establish a running program in the description script, and model the data system according to the running program and corrected statistical parameters;

During the modeling process, the dimension fields are analyzed and processed by cross-reference technology, and a data system is generated according to the analysis and processing results.

The working principle and beneficial effect of the above technical solution: construct a data system, extract dimension fields for data analysis, description fields for describing dimensions and summary fields for statistics according to the target set; the data system includes dimension fields, description fields and summary fields. The fields defined as dimensions are processed through cross-reference, which can quickly sneak into any dimension and each other to obtain the information we need most. The description field contains additional information related to the dimension. It is convenient to accurately determine the data system, and better display various data and relationships of the target set.

According to some embodiments of the present invention, the background server updates the type of the unmanned vehicle based on the safety monitoring information when performing the corresponding level of safety monitoring according to the type, and executes the new type when it is determined that the type of the unmanned vehicle changes. Corresponding level of security monitoring.

The beneficial effect of the above technical solution: it is convenient for the background server to update the type of the unmanned vehicle according to the safety monitoring information, and when the type of the unmanned vehicle is determined to be changed, the security monitoring of the corresponding level of the new type is performed to realize the rational allocation of monitoring resources, At the same time to ensure the safety and accuracy of monitoring.

According to some embodiments of the present invention, also include:

The background server receives the control verification information of the control terminal and sends it to each unmanned vehicle, and verifies the control verification information based on the following formula:

为控制验证信息在对应的无人驾驶车辆中的明文信息集；/>

为对应的无人驾驶车辆中的解密方法；/>

为控制验证信息；/>

为验证结果；当/>

时，验证结果为通过；当/>

时，验证结果为不通过；/>

为对应的无人驾驶车辆中的权限信息集；/>

为集合计数函数，/>

为预设界值，取值在0到1区间范围内。In the above formula,

To control the verification information in the corresponding plaintext information set in the unmanned vehicle; />

is the decryption method in the corresponding unmanned vehicle; />

To control authentication information; />

To verify the result; when />

, the verification result is passed; when />

, the verification result is failed; />

is the permission information set in the corresponding unmanned vehicle; />

is a set counting function, />

It is a preset limit value, and the value is in the range of 0 to 1.

When it is determined that the verification is passed, determine that the unmanned vehicle that has passed the verification is the target unmanned vehicle, and control the target unmanned vehicle to execute the control instructions included in the control verification information;

When it is determined that the verification fails, an error message is sent to the control terminal.

The working principle and beneficial effects of the above technical solution: the background server receives the control verification information of the control terminal and sends it to each unmanned vehicle for verification. Drive the vehicle, control the target unmanned vehicle to execute the control instructions included in the control verification information; when it is determined that the verification fails, send an error message to the control terminal. Each unmanned vehicle only passes the verification of specific control verification information, and executes the control instructions included in the control verification information, which improves the security of the control terminal for the control of unmanned vehicles and avoids the wrong sending of control instructions, resulting in Safety hazards caused by non-corresponding unmanned vehicle execution. The preset limit value can be set according to the security requirements.

As shown in Figure 3, the embodiment of the second aspect of the present invention proposes a safety monitoring system for unmanned vehicles, including:

Create a module for establishing the communication relationship between multiple unmanned vehicles and the background server in the preset area;

The sensing module is used to sense the environment inside the unmanned vehicle to obtain the first sensing data when multiple unmanned vehicles are driving in the preset area; to sense the environment outside the unmanned vehicle to obtain the second sensory data;

The background server is used to determine the types of multiple unmanned vehicles according to the first sensing data and the second sensing data, and perform corresponding level of safety monitoring according to the types.

The working principle of the above technical solution: establish the communication relationship between multiple unmanned vehicles in the preset area and the background server, so that the background server can obtain the initial monitoring information of multiple unmanned vehicles in the preset area. The initial monitoring information is based on Acquired by the perception module installed on the unmanned vehicle. The preset area is the planned operating area of the unmanned vehicle.

In this embodiment, the first sensing data is the data acquired through environmental sensing of the interior of the unmanned vehicle. Exemplarily, the first sensing data is control node operation information, vehicle state parameter information, vehicle-machine equipment operation information, and passenger behavior characteristics.

In this embodiment, the second sensing data is the data acquired through environmental sensing of the outside of the unmanned vehicle. For example, the obstacle information around the unmanned vehicle and the collision risk with each obstacle determined based on the external environment image and radar information of the unmanned vehicle.

Based on the first sensing data and the second sensing data, it is convenient to conduct a comprehensive perception of the interior and exterior of the unmanned vehicle, to obtain comprehensive sensing data, and to facilitate the background server to accurately determine the type of each unmanned vehicle.

In this embodiment, the types of unmanned vehicles include the first type of vehicles, the second type of vehicles and the third type of vehicles, which respectively implement the first-level safety monitoring, the second-level safety monitoring and the third-level safety monitoring.

In this embodiment, the monitoring requirements of the first-level security monitoring, the second-level security monitoring, and the third-level security monitoring are successively reduced, and the load on the background server caused by the corresponding first-level security monitoring is the largest, followed by the second-level security monitoring. Level 3 security monitoring is the lowest. Exemplarily, when the first type of vehicle is performing level one safety monitoring, the monitoring interval for acquiring the data of environment awareness acquisition for the first type of vehicle is 1s. When the secondary safety monitoring is performed on the second type of vehicle, the monitoring interval for acquiring the data of environment perception acquisition on the second type of vehicle is 2s. When three-level security monitoring is performed on the third type of vehicle, the monitoring interval for acquiring the data of environment perception acquisition on the third type of vehicle is 3s.

The beneficial effect of the above-mentioned technical solution: realize the security monitoring of multiple unmanned vehicles in the preset area, and at the same time, based on the types of multiple unmanned vehicles, perform the corresponding level of security monitoring according to the type, reducing the background server. load, improving the utilization of monitoring resources.

Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention also intends to include these modifications and variations.

Claims (9)

1. A method of safety monitoring of an unmanned vehicle, comprising:

establishing communication relations between a plurality of unmanned vehicles and a background server in a preset area;

when a plurality of unmanned vehicles run in a preset area, performing environment sensing on the interior of the unmanned vehicles to obtain first sensing data; performing environment sensing on the outside of the unmanned vehicle to obtain second sensing data;

the background server determines types of a plurality of unmanned vehicles according to the first perception data and the second perception data, and performs corresponding-level security monitoring according to the types;

The background server determines types of a plurality of unmanned vehicles according to the first sensing data and the second sensing data, and executes corresponding-level safety monitoring according to the types, and the method comprises the following steps:

the background server inputs the first perception data into a pre-trained data classification model for classification, and outputs a plurality of first classification data;

inputting the second perception data into a pre-trained data classification model for classification, and outputting a plurality of second classification data;

performing data fusion on the first classified data and the second classified data of the same data type to obtain a group of fusion data, and further obtaining a plurality of groups of fusion data;

respectively inputting a plurality of groups of fusion data into corresponding single recognition models, and outputting a first analysis result; and if at least one of the first analysis results is determined to indicate that the risk data exists, determining that the unmanned vehicle corresponding to the fusion data is a first type vehicle, and executing primary safety monitoring.

2. The method of claim 1, wherein the environmental awareness of the interior of the unmanned vehicle to obtain the first awareness data comprises:

Acquiring a control panel image and an internal scene image of a vehicle-mounted terminal in an unmanned vehicle;

checking the image quality of the control panel image and the scene image, and performing parameter adjustment processing when the image quality is determined to be unqualified, so as to obtain a target control panel image and a target scene image;

analyzing the target control panel image, and determining control node operation information, vehicle state parameter information and operation information of vehicle equipment of the vehicle;

analyzing the target scene image to determine the behavior characteristics of the passengers;

and determining first perception data according to the control node operation information, the vehicle state parameter information, the operation information of the vehicle-mounted equipment and the behavior characteristics of passengers.

3. The method for safety monitoring of an unmanned vehicle according to claim 2, wherein the step of checking the image quality of the control panel image, and performing the parameter adjustment process when it is determined that the image quality does not reach the standard, to obtain the target control panel image, comprises:

calculating a first gradient value of the control panel image in the horizontal direction and a second gradient value of the control panel image in the vertical direction by utilizing a Sobel operator based on a gradient algorithm;

inquiring a preset first gradient value-second gradient value-definition data table according to the first gradient value and the second gradient value to determine a definition value;

And when the definition value is determined to be smaller than the preset definition threshold, the image quality is not up to standard, the difference value between the preset definition threshold and the definition value is determined, the difference value-focal length correction value data table is inquired according to the difference value, the focal length correction value is determined, and parameter adjustment processing is carried out according to the focal length correction value, so that the target control panel image is obtained.

4. The method of claim 1, wherein the environmental awareness of the exterior of the unmanned vehicle to obtain the second awareness data comprises:

acquiring an external environment image and radar information of an unmanned vehicle;

and determining obstacle information around the unmanned vehicle and collision risks with all the obstacles according to the external environment image and the radar information, and determining second perception data according to the obstacle information around the unmanned vehicle and the collision risks with all the obstacles.

5. The method for safety monitoring of an unmanned vehicle according to claim 1, further comprising:

selecting target fusion data from a plurality of groups of fusion data when all the first analysis results are determined to indicate that no risk data exists;

extracting features of other fusion data except the target fusion data to obtain feature vectors, and converting the feature vectors to the types corresponding to the target fusion data according to the feature vectors to obtain converted feature vectors;

Matching the target fusion data with each conversion feature vector, and determining the association relationship between the target fusion data and each conversion feature vector;

determining a target set according to the association relation;

generating a data system according to the target set;

inputting the data system into the composite recognition model, and outputting a second analysis result;

when the risk data are determined to exist according to the second analysis result, determining that the unmanned vehicle corresponding to the fusion data is a second type vehicle, and executing secondary safety monitoring; and when the risk data are not determined to exist according to the second analysis result, determining that the unmanned vehicle corresponding to the fusion data is a third type vehicle, and executing three-level safety monitoring.

6. A method of safety monitoring of an unmanned vehicle as claimed in claim 1, wherein the training of the single recognition model comprises:

acquiring sample fusion data;

preprocessing sample fusion data based on a data layer of a single recognition model, and determining risk factors;

carrying out data analysis on the risk factors based on an index layer of the single recognition model, determining a plurality of indexes, and screening to obtain risk indexes;

combining various risk indexes based on a model parameter layer of a single recognition model to obtain various combination results, and screening out a combination result with highest risk probability prediction accuracy as a target combination result;

And outputting a predicted result of the sample fusion data based on the output layer of the single recognition model, and indicating that training is completed when the predicted result is consistent with a real result corresponding to the sample fusion data.

7. The method of safety monitoring of an unmanned vehicle according to claim 5, wherein determining a set of targets from the association relationship comprises:

taking the target fusion data as a key node, and converting the feature vector as an associated node of the key node;

constructing a screening system according to the key nodes, the associated nodes and the associated relation; the screening system comprises a distance value from each associated node to a key node;

screening out the associated nodes with the distance value smaller than the preset distance value as target associated nodes;

and determining a target set according to the target associated node and the key node.

8. The method of safety monitoring of an unmanned vehicle of claim 5, wherein the composite recognition model is trained, the method comprising:

determining big data of the unmanned vehicle, and constructing a map text of a driving scene according to the big data;

extracting each node in the map text, determining the association relation among each node, and constructing a knowledge map of a data chain corresponding to the driving scene;

Determining risk index information of each node;

extracting associated feature vectors in the knowledge graph, and carrying out feature fusion according to the associated feature vectors and risk index information to obtain fusion vectors;

and training the composite recognition model based on the fusion vector.

9. A safety monitoring system for an unmanned vehicle, comprising:

the building module is used for building communication relations between a plurality of unmanned vehicles in a preset area and a background server;

the sensing module is used for sensing the environment in the unmanned vehicles when a plurality of unmanned vehicles run in a preset area to obtain first sensing data; performing environment sensing on the outside of the unmanned vehicle to obtain second sensing data;

the background server is used for determining the types of the unmanned vehicles according to the first perception data and the second perception data and executing corresponding-level safety monitoring according to the types;

the background server determines types of a plurality of unmanned vehicles according to the first sensing data and the second sensing data, and executes corresponding-level safety monitoring according to the types, and the method comprises the following steps:

the background server inputs the first perception data into a pre-trained data classification model for classification, and outputs a plurality of first classification data;

Inputting the second perception data into a pre-trained data classification model for classification, and outputting a plurality of second classification data;

performing data fusion on the first classified data and the second classified data of the same data type to obtain a group of fusion data, and further obtaining a plurality of groups of fusion data;

respectively inputting a plurality of groups of fusion data into corresponding single recognition models, and outputting a first analysis result; and if at least one of the first analysis results is determined to indicate that the risk data exists, determining that the unmanned vehicle corresponding to the fusion data is a first type vehicle, and executing primary safety monitoring.

Images