CN117585015A - Automatic driving optimization methods, control methods, devices, electronic equipment and media - Google Patents

Abstract

The application discloses an automatic driving optimization method, a control method, an automatic driving optimization device, electronic equipment and a medium, wherein the automatic driving optimization method is applied to a cognitive decision model, the cognitive decision model comprises a cognitive layer and a decision layer, and the automatic driving optimization method comprises the following steps: acquiring in-and-out monitoring data of a training sample vehicle and a training sample scene label, and acquiring target human driving experience scene cognitive characteristics corresponding to the training sample scene label; extracting features of a cognitive layer of a training sample from the in-vehicle monitoring data and the out-vehicle monitoring data of the training sample through the cognitive layer; and carrying out iterative optimization on the cognitive layer according to the training sample cognitive layer characteristics and the target human driving experience scene cognitive characteristics. The technical problem that the safety of automatic driving is lower in the related art is solved.

Description

Automatic driving optimization methods, control methods, devices, electronic equipment and media

Technical field

The present application relates to the field of vehicle technology, and in particular to an automatic driving optimization method, control method, device, electronic equipment and medium.

Background technique

With the continuous development of technology and continuous breakthroughs in computing power, autonomous driving has developed rapidly. Autonomous driving can provide people with a safer, more convenient and efficient way to travel, and it has also had a profound impact on the entire transportation system and urban planning.

The focus of the autonomous driving architecture in related technologies is on surface behavioral information such as maps and external traffic participant behaviors. The understanding and generalization of complex scenes with multi-layer logic are insufficient. For example, the location of the vehicle is located in other traffic. In the visual blind spot of the participant, since other traffic participants may not be able to spot the car and will not avoid or even accelerate, the car needs to slow down to avoid a collision. For example, when driving on the highway, the road ahead on the right It is a truck. The truck travels slowly and will decelerate the vehicle behind it. The truck may change lanes to the front of the vehicle, so it is necessary to slow down in advance. However, only focusing on surface behavioral information often makes it difficult to flexibly deal with complex scenarios involving multiple traffic participants and multi-layer logical reasoning, which will lead to lower safety of autonomous driving.

Contents of the invention

The main purpose of this application is to provide an automatic driving optimization method, control method, device, electronic equipment and medium, aiming to solve the technical problem of low safety of automatic driving in related technologies.

In order to achieve the above purpose, this application provides an automatic driving optimization method. The automatic driving optimization method is applied to a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer, including the following steps:

Obtain training sample vehicle interior and exterior monitoring data and training sample scene labels, and obtain target human driving experience scene cognitive characteristics corresponding to the training sample scene labels;

Extract training sample cognitive layer features from the training sample vehicle interior and exterior monitoring data through the cognitive layer;

The cognitive layer is iteratively optimized according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene.

This application also provides an automatic driving optimization method. The automatic driving optimization method adopts a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The cognitive layer adopts the automatic driving optimization as described above. The method performs pre-training, and the automatic driving optimization method includes the following steps:

Obtain monitoring data inside and outside the vehicle to be predicted;

The cognitive layer features to be predicted are extracted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted;

The cognitive layer features to be predicted are input into the decision-making layer to determine the automatic driving control parameters.

This application also provides an automatic driving optimization device. A cognitive decision-making model is deployed on the automatic driving optimization device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization device includes:

The first acquisition module is used to acquire training sample vehicle interior and exterior monitoring data and training sample scene labels, and obtain target human driving experience scene cognitive features corresponding to the training sample scene labels;

A first cognitive module, configured to extract training sample cognitive layer features from the training sample vehicle interior and exterior monitoring data through the cognitive layer;

An optimization module, configured to iteratively optimize the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene.

The present application also provides an automatic driving control device. A cognitive decision-making model is deployed on the automatic driving control device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The cognitive layer adopts the automatic driving method as described above. The driving optimization method is pre-trained, and the automatic driving control device includes:

The second acquisition module is used to acquire the vehicle interior and exterior monitoring data to be predicted;

A second cognitive module, configured to extract the cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted;

A decision-making module, used to input the cognitive layer characteristics to be predicted into the decision-making layer and determine automatic driving control parameters.

This application also provides an electronic device. The electronic device is a physical device. The electronic device includes: a memory, a processor, and the automatic driving optimization stored on the memory and capable of running on the processor. The program of the method. When the program of the automatic driving optimization method is executed by the processor, the steps of the automatic driving optimization method as described above can be implemented.

This application also provides a medium. The medium is a computer-readable storage medium. The computer-readable storage medium stores a program for implementing an automatic driving optimization method. When the program for the automatic driving optimization method is executed by a processor, Steps to implement the above-mentioned autonomous driving optimization method.

This application provides an automatic driving optimization method, a control method, a device, an electronic device and a medium. The automatic driving optimization method is applied to a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization method is used to iteratively optimize the cognitive layer of the cognitive decision-making model. First, by obtaining the training sample vehicle interior and exterior monitoring data and the training sample scene labels, and obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene labels, the matching of the training samples and the human driving experience scene cognitive characteristics is achieved. Thus, the target human driving experience scene cognitive features that match the training sample scene label of the training sample can be obtained, and the human driving experience scene cognitive features can reflect human beings' cognition of the current scene; and then through the cognitive layer, the target human driving experience scene cognitive features can be obtained. The cognitive layer features of the training sample are extracted from the inside and outside monitoring data of the training sample, thereby achieving the acquisition of the cognitive layer features of the training sample. The cognitive layer features of the training sample can reflect the self-driving vehicle's cognition of the current scene; further, By iteratively optimizing the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene, the cognitive layer learns human cognition of the scene, and for traffic involving multiple traffic participants and complex scenarios with multi-layer logical reasoning. Compared with autonomous vehicles that only focus on surface phase information, humans can usually more comprehensively combine a large amount of effective information, perform multi-layer logical reasoning, and ultimately make more accurate driving decisions. Therefore, , by making the cognitive layer learn human cognition of the scene, the cognitive layer can have a more comprehensive understanding of complex scenes and have stronger reasoning capabilities, so as to more accurately understand and pay attention to the risk points in the scene, thereby Guide decision-makers to avoid risks more accurately, make safer driving decisions, and improve the safety of autonomous driving.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings needed to describe the embodiments or related technologies. Obviously, for those of ordinary skill in the art, Other drawings can also be obtained based on these drawings without exerting any creative effort.

Figure 1 is a schematic flow chart of the first embodiment of the automatic driving optimization method of the present application;

Figure 2 is a schematic structural diagram of an implementation manner of the cognitive decision-making model in the embodiment of the present application;

Figure 3 is a schematic flow chart of the second embodiment of the automatic driving optimization method of the present application.

Figure 4 is a schematic flow chart of the third embodiment of the automatic driving optimization method of the present application;

Figure 5 is a schematic structural diagram of the automatic driving optimization device in the embodiment of the present application;

Figure 6 is a schematic diagram of the equipment structure of the hardware operating environment involved in the automatic driving optimization method in the embodiment of the present application.

The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings.

Detailed ways

In order to make the above objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without any creative work shall fall within the scope of protection of the present invention.

With the continuous development of technology and continuous breakthroughs in computing power, autonomous driving has developed rapidly. Autonomous driving can provide people with a safer, more convenient and efficient way to travel, and it has also had a profound impact on the entire transportation system and urban planning.

The mainstream technical solutions for autonomous driving systems are mainly divided into two types: rule-based modular architecture and data-driven end-to-end cognitive decision-making architecture. Although the rule-based sub-module architecture has achieved mass production due to its advantages such as low computing power requirements, simple deployment, and explainable processes, it is difficult to achieve full scene domain coverage, and the result of sub-module solution is not the global optimal solution. The end-to-end solution takes raw sensor data as input and generates planning and/or low-level control actions as output. It can perform joint and global optimization, and with the rapid development of deep learning large models, especially artificial intelligence models based on the Transformer structure, With the help of attention mechanism, this type of model has strong multi-scene and multi-objective fusion capabilities, and can understand rich semantic information and handle more complex tasks. End-to-end solutions are becoming increasingly popular.

The end-to-end autonomous driving architecture in related technologies focuses on surface behavioral information such as maps and external traffic participant behaviors. It is easy to overlook some factors that are less accurate in predicting vehicle behavior. However, these factors sometimes cause complications. Key factors in scene safety accidents. For example, when the vehicle is located in the visual blind spot of other traffic participants, the visual blind area is less effective in predicting vehicle behavior. However, other traffic participants may not be able to detect the vehicle because they are unable to detect it. To avoid or even accelerate, the vehicle needs to slow down to avoid a collision. For example, when driving on a highway, there is a truck in front of the road on the right, and the truck is traveling slowly. Before the truck makes a lane change, its prediction Vehicle behavior plays a smaller role, but since a slower truck will slow down the vehicle behind it, the truck may change lanes in front of the vehicle, so it can slow down in advance to avoid a collision with the truck changing lanes. It can be seen from this that only focusing on surface behavioral information has insufficient understanding and generalization of complex scenes with multi-layer logic. It is usually difficult to flexibly deal with these complex scenes involving multiple traffic participants and multi-layer logical reasoning, which will lead to Autonomous driving is less safe.

This application proposes an automatic driving optimization method that pre-trains the cognitive layer based on human driving experience. The automatic driving optimization method is applied to a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization method is used to iteratively optimize the cognitive layer of the cognitive decision-making model. First, by obtaining the training sample vehicle interior and exterior monitoring data and the training sample scene labels, and obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene labels, the matching of the training samples and the human driving experience scene cognitive characteristics is achieved. Thus, the target human driving experience scene cognitive features that match the training sample scene label of the training sample can be obtained, and the human driving experience scene cognitive features can reflect human beings' cognition of the current scene; and then through the cognitive layer, the target human driving experience scene cognitive features can be obtained. The cognitive layer features of the training sample are extracted from the inside and outside monitoring data of the training sample, thereby achieving the acquisition of the cognitive layer features of the training sample. The cognitive layer features of the training sample can reflect the self-driving vehicle's cognition of the current scene; further, By iteratively optimizing the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene, the cognitive layer learns human cognition of the scene, and for traffic involving multiple traffic participants and complex scenarios with multi-layer logical reasoning. Compared with autonomous vehicles that only focus on surface phase information, humans can usually more comprehensively combine a large amount of effective information, perform multi-layer logical reasoning, and ultimately make more accurate driving decisions. Therefore, , by making the cognitive layer learn human cognition of the scene, the cognitive layer can have a more comprehensive understanding of complex scenes and have stronger reasoning capabilities, so as to more accurately understand and pay attention to the risk points in the scene, thereby Guide decision-makers to make more accurate driving decisions and improve the safety of autonomous driving.

Embodiment 1

An embodiment of the present application provides an automatic driving optimization method. Referring to Figure 1, in the first embodiment of the automatic driving optimization method of the present application, the automatic driving optimization method is applied to a cognitive decision-making model, and the cognitive decision-making model includes The cognitive layer and decision-making layer include the following steps:

Step S10, obtain the training sample vehicle interior and exterior monitoring data and the training sample scene labels, and obtain the target human driving experience scene cognitive characteristics corresponding to the training sample scene labels;

The execution subject of the method in this embodiment can be an automatic driving optimization device, or an automatic driving optimization terminal device or server. This embodiment takes an automatic driving optimization device as an example. The automatic driving optimization device can be integrated in a computer with data Processing functions on vehicles, vehicle-mounted terminals, smartphones, computers and other terminal devices.

In this embodiment, it should be noted that the automatic driving optimization method is applied to a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization method is used to analyze the cognitive decision-making model. The cognitive layer of the cognitive decision-making model is iteratively optimized. Before comprehensive training of the cognitive decision-making model, the cognitive layer is pre-trained in advance, so that the cognitive layer can learn human cognition of scenes in complex scenarios. The trained cognitive decision-making model is used to realize end-to-end driving behavior control of autonomous vehicles. The cognitive decision-making model uses the attention mechanism and can be a deep learning model based on the Transformer structure, or it can also use attention for other purposes. The deep learning neural network model of the mechanism can be specifically designed according to the actual situation, and this embodiment is not limited to this.

The cognitive decision-making model at least includes a cognitive layer and a decision-making layer, where the cognitive layer is used to perform feature extraction, fusion, encoding and reasoning on the acquired vehicle interior and exterior monitoring data to obtain the cognitive layer features required by the decision-making layer. ; The decision-making layer is used to plan the driving behavior of the autonomous vehicle in the next time step based on the cognitive layer features transmitted by the cognitive layer, so that the autonomous vehicle can avoid risks and drive safely to the destination.

The training sample vehicle interior and exterior monitoring data refers to the training sample vehicle interior and exterior monitoring data. The vehicle interior and exterior monitoring data refers to the data required for automatic driving control that can be obtained, including exterior vehicle monitoring data and cockpit monitoring data. The vehicle interior and exterior monitoring data can be image data, text data, sound data, signal data, etc. one or more modalities. Exemplarily, the off-vehicle monitoring data may be off-vehicle image data collected through cameras, off-vehicle point cloud data collected through laser radar and millimeter wave radar, vehicle position information data collected through vehicle positioning systems, and vehicle location information data collected through microphones. Sound data outside the car, signal light countdown data obtained through the communication module, etc. For example, the cabin monitoring data may be vehicle driving data such as vehicle speed and acceleration obtained from the vehicle controller or the controller LAN bus, or may be cabin image data collected through a camera, and cabin sound data collected through a microphone. , text data obtained by converting the collected speech, text data extracted from the collected images, user demand signal data collected through the human-computer interaction interface, etc. These monitoring data inside and outside the vehicle can be collected in real time, so they can carry real-time information about the status of other traffic participants, the status of the driver, the status of road conditions and other information that may change at the current moment.

The target human driving experience scene cognitive features refer to the human driving experience scene cognitive features corresponding to the training sample scene labels. The human driving experience scene cognition features are features extracted from human driving experience monitoring data and used to characterize human cognition of the scene. The human driving experience scene cognitive features may be in the form of images, texts, vectors, matrices, etc. In an implementable manner, monitoring data of skilled drivers can be selected to extract scene cognitive features of human driving experience, so that the cognitive layer can learn more effective scene cognitive experience.

In an implementable manner, the extraction method of the cognitive features of the human driving experience scene may be: collecting human driving experience monitoring data during the process of the human driving the vehicle experiencing the first scene, such as taking pictures inside the cockpit and outside the vehicle. images of the environment; and then extract human attention distribution features and driving behavior semantic understanding features from the human driving experience monitoring data as human driving experience scene cognitive features corresponding to the first scene.

In an implementable manner, the method of extracting the cognitive features of the human driving experience scene may also be: collecting multiple human driving experience monitoring data during the process of human driving the vehicle, such as photographing the environment inside the cockpit and outside the vehicle. images; and then extract human attention distribution features and driving behavior semantic understanding features from each of the human driving experience monitoring data as human driving experience scene cognitive features, and then extract the human driving experience scene cognitive features Classify and determine the scene labels corresponding to the cognitive features of each human driving experience scene.

The training sample scene label refers to the scene label of the training sample, such as a ghost probe scene, a pedestrian crossing scene, an intersection scene, etc. The training samples can be annotated and determined manually or by an annotation model.

As an example, the step S10 includes: obtaining a batch of training sample vehicle interior and exterior monitoring data and the training sample scene labels corresponding to each training sample, and based on the preset scene labels and the scene cognitive characteristics of human driving experience. The mapping relationship between the target human driving experience scene cognitive features corresponding to each training sample scene label is obtained. A batch of training samples can be one or more. Since the processing method of each training sample in a batch of training samples is the same, for the convenience of explanation, a single training sample will be used as an example for the following explanation.

Optionally, before the step of obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene label, the step further includes:

Step A10, obtain multiple human driving experience scene monitoring data, and extract human driving experience scene cognitive features from each of the human driving experience scene monitoring data;

Step A20: Perform cluster analysis on the cognitive features of each human driving experience scene to determine the scene labels corresponding to each of the cognitive features of the human driving experience scene.

In this embodiment, it should be noted that since things change in real time, it is almost impossible for human-driven vehicles and autonomous vehicles to experience exactly the same scenes, and many complex scenes themselves rarely occur, so It is almost impossible to find human driving experience vehicle interior and exterior monitoring data that is completely consistent with the vehicle interior and exterior monitoring data of each training sample. Moreover, if autonomous driving performs exactly the same driving behavior as human driving, it will lose the flexibility of autonomous driving. It is difficult to flexibly respond to changing actual situations. Although the actual situation is not exactly the same, there are certain commonalities. For example, although the traffic flow and pedestrian flow at the intersection are different every time you pass an intersection, it is necessary to predict whether the driving trajectories of vehicles or pedestrians' walking trajectories at each intersection will be consistent with the traffic flow. The driving trajectories of the vehicle intersect, determine whether there is a visual blind spot, etc. Therefore, this embodiment classifies according to the scene, and extracts the human driving experience scene cognitive features from the human driving experience scene monitoring data generated by humans driving the vehicle in various scenes. , as the learning object of autonomous vehicles in various scenarios, allowing the cognitive layer to learn the risk points that humans pay attention to and reason about in various scenarios.

Exemplarily, the steps A10-A20 include: collecting a large amount of human driving experience scene monitoring data during the process of human driving the vehicle, and extracting human driving experience scene cognitive features from each of the human driving experience scene monitoring data. ; Then perform cluster analysis on the cognitive characteristics of each of the human driving experience scenes, and then mark the scene labels of each category. Wherein, the method of extracting cognitive features of human driving experience scenes from each of the human driving experience scene monitoring data may be by photographing the driver's facial image and determining the driver's attention distribution by identifying the direction in which the driver's line of sight is pointing, Attention distribution is determined as a feature of human driving experience scene cognition; it can also be used to manually annotate scene semantic understanding information based on the collected human driving experience scene monitoring data, and the manually annotated scene semantic understanding information is determined as human driving experience scene cognition. feature.

Step S20, extract the cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer;

In this embodiment, it should be noted that the training sample cognitive layer feature is the cognitive layer feature of the training sample. The cognitive layer features refer to the features required for automatic driving control. Automatic driving control is to enable the automatic driving vehicle to reach the destination safely, that is, to control the vehicle to drive to the destination while avoiding risks. Therefore, automatic driving The control method can be to predict the high-risk driving area and/or low-risk driving area of the autonomous vehicle in the next time step, thereby planning the driving of the autonomous vehicle to avoid the high-risk driving area or pass through the low-risk driving area in the next time step. Behavior, the driving behavior can be operations that can be directly performed by the vehicle such as acceleration, deceleration, turning, etc., or it can be a driving trajectory to the destination or the next time step, so that the autonomous vehicle drives according to the received driving trajectory.

The cognitive layer features at least include scene cognitive features, and may also include other features. In an implementable manner, the cognitive layer characteristics may include environment cognitive characteristics and user demand characteristics. The environmental cognition features are used to characterize the autonomous vehicle's cognition of the current environment, such as cognition of transportation facilities, cognition of ground obstacles, cognition of the vehicle's driving status, and cognition of other traffic participants. The person's cognition, the cognition of the risks in the scene, the cognition of the risk items in the scene, etc. The environment cognitive characteristics at least include scene cognitive characteristics. The main purpose of autonomous vehicle behavior control is to enable the autonomous vehicle to reach its destination safely, that is, to control the vehicle to drive to the destination while avoiding risks. In an implementable manner, the autonomous driving control method can be Predict the high-risk driving area and/or low-risk driving area of the autonomous vehicle in the next time step, thereby planning the driving behavior of the autonomous vehicle to avoid the high-risk driving area or pass through the low-risk driving area in the next time step. The environmental cognitive features are an objective representation of the current environment of the autonomous driving vehicle. Therefore, the more accurate the extracted environmental cognitive features are, the more accurate the current risk judgment of the autonomous driving vehicle will be, and the safety of the autonomous driving control will be better. The higher. The environment recognition features may include scene recognition features, map features, risk target detection and tracking features, risk target movement features, low-risk driving area features, etc., and may also include other features. The user demand characteristics are used to characterize the needs of users riding in self-driving vehicles, such as the user's demand for the destination, the user's demand for the path to the destination, the user's need to be in a hurry, and the user's desire to drive smoothly to reduce motion sickness. needs, etc. The user demand characteristics can capture user demand information by detecting user operations, collecting user images, etc., and then extract user demand characteristics. For example, user images can be collected through a camera, and the user's current status in the user image can be identified through image recognition technology. When the user is in the sleep state, the user's need for the vehicle to drive smoothly is usually implicit in the sleep state. For example, the user's voice information can also be collected through the microphone. When the user is recognized to say "drive faster", it can be known Users have acceleration needs.

The cognitive layer can be a multi-layer structure, that is, the input of any cognitive layer can be the output of other cognitive layers, and the output of any cognitive layer can also be used as the output of other cognitive layers. Input, it can achieve multi-feature fusion and reasoning, capture the correlation and dependence between various features extracted by the cognitive layer, and achieve a more comprehensive understanding of the current driving actual situation and deeper logical reasoning. Therefore, the scene cognitive loss can be determined by comparing the scene cognitive characteristics of the training sample with the corresponding target human driving experience scene cognitive characteristics, and the entire cognitive layer can be iteratively optimized based on the scene cognitive loss. In an implementable manner, other features of the training sample can also be annotated to obtain other feature annotation data of the training sample, and other features of the other features can be determined by comparing the other feature annotation data of the training sample with other features of the training sample. The extraction loss, together with the scene cognitive loss, is used to iteratively optimize the entire cognitive layer.

The cognitive layer may adopt an attention mechanism. In an implementable manner, the cognitive layer may adopt a transformer structure. The attention mechanism can be used to perform feature extraction, fusion, encoding and reasoning on various modes and types of vehicle monitoring data, and extract real-time information that may change at the current moment, and a period of history before the current moment. The historical information of time composes the information sequence, and captures the correlation and dependence between different types of information, performs feature reasoning, and extracts cognitive layer features with deep logical information for the decision-making layer to automatically drive autonomous vehicles. Control, in this way, the information that may change and the interaction between multiple risk items can be fully taken into account in the process of predicting the driving risks of autonomous vehicles, and the accuracy of prediction of driving risks of autonomous vehicles can be improved, so that autonomous vehicles can More risks can be avoided and the safety of automatic driving control can be improved. Among them, the correlation and dependence between different types of information may have a greater impact on the automatic driving control of autonomous vehicles. For example, there may be interactions between traffic participants, traffic participants and There may be interactions between transportation facilities, and there may also be interactions between transportation participants and scenes. For example, at an intersection, the vehicle is traveling in the east-west direction, and there is a second vehicle traveling in the north-south direction and pedestrians walking in the east-west direction nearby. Assume that the vehicle is traveling in the east-west direction without considering the interaction between traffic participants. There will be no collision between the car and the second vehicle driving at the current speed, but if the second vehicle slows down or stops to avoid pedestrians, it may cause a collision between the own car and the second vehicle. Therefore, when considering the traffic participants and the traffic participants In the case of interaction between traffic participants, the vehicle still needs to slow down or plan other driving routes; and if it is assumed that the second vehicle is in the turning lane, taking into account the interaction between traffic participants and traffic facilities, if the second vehicle turns The driving trajectory of the vehicle does not intersect with the driving trajectory of the vehicle, and the vehicle does not need to slow down; in addition, taking into account the interaction between traffic participants and the scene, in the current scene of passing through the intersection, due to the intersection scene Complexity can increase the risk factor for people and vehicles at intersections, so vehicles may need to be controlled to slow down.

As an example, the step S20 includes: inputting the training sample vehicle interior and exterior monitoring data into the cognitive layer, and performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature inference to obtain Cognitive layer characteristics of training samples. The structure of the cognitive layer and the specific feature processing method can be determined according to the actual needs of the decision-making layer, which is not limited in this embodiment.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the training sample cognitive layer features include training sample scene cognitive features, training sample map features, Risk target detection and tracking features of training samples and risk target movement features of training samples;

The step of extracting the cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer includes:

Step S21: Extract the scene cognitive features of the training sample from the training sample vehicle interior and exterior monitoring data through the scene recognition model, and extract the training sample map from the training sample vehicle interior and exterior monitoring data through the map construction model. Features: extract the training sample risk target detection and tracking features from the training sample vehicle interior and exterior monitoring data through the risk target detection and tracking model;

Step S22, use the motion prediction model to perform risk target motion prediction based on the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features to obtain training sample risk target motion features, Wherein, the motion prediction model adopts an attention mechanism.

In this embodiment, it should be noted that for different modes of vehicle interior and exterior monitoring data, different models can be used for feature extraction. For example, image data can use convolutional neural networks for feature extraction, and time series data can use long-term and short-term features. Memory networks can be used for feature extraction, etc. Different models can also be used for feature processing to extract different cognitive layer features required by the decision-making layer. Therefore, multiple models can be deployed in the cognitive layer for feature processing. The specific model structure and model types can be determined according to the actual situation, and this embodiment does not limit this. The cognitive layer includes a scene cognition model, a map construction model, a risk target detection and tracking model, and a motion prediction model. Risk targets refer to objects that may cause risks to the driving of autonomous vehicles, and can provide information for traffic participants and traffic infrastructure. One or more of the facilities, etc. The scene recognition model, the map construction model, and the training sample scene recognition features, training sample map features, and training sample risk target detection and tracking features output by the risk target detection and tracking model are all useful for predicting the risk target in the next step. The motion of the time step has a certain impact, and there is a certain correlation and dependence between the scene cognitive features, map features and risk target detection and tracking features. Therefore, they can all be used as inputs to the motion prediction model. The attention is adopted through the motion prediction model. The force mechanism captures the correlation and dependence between these features, performs further feature fusion, encoding and reasoning, and determines the risk target motion characteristics of the training sample of at least one risk target.

The training sample cognitive layer features include training sample scene cognitive features, training sample map features, training sample risk target detection and tracking features, and training sample risk target motion features. Scene cognitive features are used to characterize the risk perception of the scene the vehicle is currently in, such as attention distribution, semantic understanding of risk items present in the scene, etc. Map features are used to characterize the ground facilities and roads of the scene the vehicle is in. Map information such as elements, risk target detection and tracking features are used to characterize the identification of risk targets and the location information of risk targets within a period of time before the current moment, and risk target motion features are used to characterize the risk targets within a period of time after the current moment. Movement information may include at least one of position information, trajectory information, movement intention, movement trend, etc.

As an example, the steps S21-S22 include: inputting the training sample vehicle interior and exterior monitoring data into the scene cognitive model, and performing multi-level progressive feature extraction, feature fusion, feature encoding, feature reasoning, etc. Process and extract the cognitive features of the training sample scene; simultaneously, input the training sample vehicle interior and exterior monitoring data into the map to build a model, and perform multi-level progressive features such as feature extraction, feature fusion, feature encoding, and feature reasoning. Process and extract the training sample map features; synchronously, input the training sample vehicle interior and exterior monitoring data into the risk target detection and tracking model to perform multi-level progressive features such as feature extraction, feature fusion, feature encoding, and feature inference. Process and extract training risk target detection and tracking features. Furthermore, the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features are input into the motion prediction model, and the attention mechanism is used to capture the training samples through the motion prediction model. The correlation and dependence between the scene cognitive characteristics, the training sample map characteristics and the training sample risk target detection and tracking characteristics. Based on these correlations and dependencies, risk target operation predictions can be made more accurately, and at least A risk target’s training sample risk target movement characteristics.

Optionally, the training sample risk target detection and tracking features include at least one training sample risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multi-layer perceptron layer and a prediction layer;

The step of using the motion prediction model to perform risk target motion prediction based on the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features to obtain the training sample risk target motion characteristics. include:

Step S221: Input the training sample scene cognitive features, the training sample map features and the training sample risk target detection and tracking features into the attention layer, and use the attention mechanism to determine each of the risk target detection and tracking subdivisions. Training sample risk target interaction features between features, training sample scene interaction features between each of the risk target detection and tracking sub-features and the training sample scene cognitive features, and each of the risk target detection and tracking sub-features and the Training sample map interaction features between training sample map features;

Step S222, input the training sample risk target interaction characteristics, the training sample scene interaction characteristics, and the training sample map interaction characteristics into the multi-layer perceptron layer to obtain the training sample scene cognitive layer characteristics and the training sample motion query characteristics;

Step S223: Input the training sample scene cognitive layer features and the training sample motion query features into the prediction layer, perform risk target motion prediction, and obtain the training sample risk target motion features.

In this embodiment, it should be noted that for complex scenarios, multiple risk targets may be involved. In the case where multiple risk targets are identified, the risk target detection and tracking model can identify multiple risk targets and evaluate multiple risk targets. The targets are tracked synchronously, so that the risk target detection and tracking features obtained will contain relevant information of multiple risk targets. That is, the training sample risk target detection and tracking features include at least one training sample risk target detection and tracking sub-feature. In this case, motion prediction can be performed separately for each risk target. By using the attention mechanism, it is possible to capture the correlation and dependence between each risk target and other risk targets, the correlation and dependence between each risk target and map elements, and the correlation and dependence between each risk target and scene cognition. Based on these correlations and dependencies, risk target operation predictions can be made more accurately for each risk target, and the movement characteristics of the training sample risk target of each risk target can be determined.

Scene cognitive layer features and motion query features represent risks that already exist in the current scene. Based on scene cognitive layer features and motion query features, the motion information of risk targets after the current moment can be further predicted.

As an example, the steps S221-S223 include: inputting the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features into the attention layer, using an attention mechanism. , capture the dependencies between the risk target detection and tracking sub-features of each training sample, to determine the interaction between each risk target and other risk targets, generate the risk target interaction features of the training sample, and capture the risk of each training sample Target detection tracks the dependency between the sub-features and the training sample scene cognitive features to determine the interaction between each risk target and the scene cognitive features, generates the training sample scene interaction features, and captures each of the training samples Risk target detection tracks the dependence between sub-features and the training sample map features to determine the interaction between each risk target and map elements, and generates training sample scene interaction features. Furthermore, after splicing the training sample risk target interaction characteristics, the training sample scene interaction characteristics and the training sample map interaction characteristics, input the multi-layer perceptron layer to obtain the training sample scene cognitive layer characteristics and the training sample motion query feature. Furthermore, the training sample scene cognitive layer characteristics and the training sample motion query characteristics are input into the prediction layer, and risk target motion prediction is performed for each risk target to obtain the training sample risk target motion characteristics of each risk target.

In an implementable manner, the multi-layer perceptron layer can also decode the training sample scene cognitive layer features and the training sample motion query features, and output the decoded scene risk prediction information and risk target motion information as intermediate results, for users to view.

Step S30: Iteratively optimize the cognitive layer according to the training sample cognitive layer characteristics and the target human driving experience scene cognitive characteristics.

As an example, the step S30 includes: determining the scene cognitive loss based on the difference between the training sample scene cognitive characteristics in the training sample cognitive layer characteristics and the target human driving experience scene cognitive characteristics, The scene cognitive loss can be determined as the cognitive layer loss, or the scene cognitive loss and other cognitive layer losses can be aggregated to obtain the cognitive layer loss, and it can be judged whether the cognitive layer loss has converged. If If the cognitive layer loss has not converged, then based on the cognitive layer loss, the gradient descent method can be used to perform a round of updates on the cognitive layer, and return to the acquisition of training sample vehicle interior and exterior monitoring data and training sample scenarios. label, and obtain the cognitive characteristics of the target human driving experience scene corresponding to the training sample scene label, and perform the next round of training; if the cognitive layer loss converges, the cognitive layer completed by iterative optimization can be obtained.

Since the automatic driving optimization method proposed in this embodiment only iteratively optimizes the cognitive layer, it can be pre-trained before formal training of the cognitive decision-making model, so that the cognitive layer can learn from human driving experience. Understand complex scenes more comprehensively and at multiple levels, improve the cognitive layer's ability to understand complex scenes and identify risks, thereby guiding decision-makers to make more accurate driving decisions and improve the safety of autonomous driving.

In an implementable manner, referring to Figure 2, the cognitive layer includes a map construction transformer, a risk target detection and tracking transformer, a scene recognition transformer, and a motion prediction transformer. The map features are extracted through the map construction transformer, and the risk target is extracted through the map construction transformer. The detection and tracking transformer extracts risk target detection and tracking features, and the scene recognition transformer extracts scene recognition features, and further extracts map features, risk target detection and tracking features, and scene recognition features to input interactive features between multi-layer perceptron layers, risk targets The interactive query Q _a is expressed as:

Q _a =MCHA(MHSA(Q),Q _A )

Among them, MHCA represents the multi-head cross attention mechanism, MHSA represents the multi-head self-attention mechanism, Q represents the risk target detection and tracking query, and Q _A represents the detection and tracking query of other traffic participants other than the risk target;

The interactive query _Qm between the risk target and the map is expressed as:

Q _m =MCHA(MHSA(Q),Q _M )

Among them, Q _M represents map query;

The interactive query Q _s between risk targets and scene cognition is expressed as:

Q _s =MCHA(MHSA(Q),Q _s )

Among them, Q _S represents cognitive scene query.

Then Q _a , Q _m and Q _s are passed to the multi-layer perceptron layer, and the obtained scene cognitive layer features and motion query features are passed to the motion prediction transformer to obtain the risk target motion features. The planning transformer is further based on the risk target motion features. Plan the next autonomous driving behavior of the autonomous vehicle. Among them, the scene recognition transformer uses the human scene attention mechanism for pre-training. Human driving experience data of skilled drivers can be collected in advance and stored in the scene experience pool. It can also sample and extract key features for previously unavailable traffic conflict scenarios encountered by autonomous vehicles using cognitive decision-making models, and store them. and scene experience pool; in the process of using the cognitive decision-making model for autonomous vehicles, during the human-computer interaction process with the user, the user is reminded of some key scenes and the scene cognition extracted from the currently collected monitoring data inside and outside the vehicle Features are semantically matched and can also be input into the scene memory pool for storage, which facilitates scene cognitive transformer call training.

In this embodiment, the automatic driving optimization method is applied to a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization method is used to recognize the cognitive decision-making model. The knowledge layer performs iterative optimization. First, by obtaining the training sample vehicle interior and exterior monitoring data and the training sample scene labels, and obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene labels, the matching of the training samples and the human driving experience scene cognitive characteristics is achieved. Thus, the target human driving experience scene cognitive features that match the training sample scene label of the training sample can be obtained, and the human driving experience scene cognitive features can reflect human beings' cognition of the current scene; and then through the cognitive layer, the target human driving experience scene cognitive features can be obtained. The cognitive layer features of the training sample are extracted from the inside and outside monitoring data of the training sample, thereby achieving the acquisition of the cognitive layer features of the training sample. The cognitive layer features of the training sample can reflect the self-driving vehicle's cognition of the current scene; further, By iteratively optimizing the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene, the cognitive layer learns human cognition of the scene, and for traffic involving multiple traffic participants and complex scenarios with multi-layer logical reasoning. Compared with autonomous vehicles that only focus on surface phase information, humans can usually more comprehensively combine a large amount of effective information, perform multi-layer logical reasoning, and ultimately make more accurate driving decisions. Therefore, , by making the cognitive layer learn human cognition of the scene, the cognitive layer can have a more comprehensive understanding of complex scenes and have stronger reasoning capabilities, so as to more accurately understand and pay attention to the risk points in the scene, thereby Guide decision-makers to make more accurate driving decisions and improve the safety of autonomous driving.

Embodiment 2

Furthermore, in the second embodiment of the present application, content that is the same or similar to the above embodiment can be referred to the above introduction, and will not be described again. On this basis, referring to Figure 3, the step of iteratively optimizing the cognitive layer based on the cognitive layer characteristics of the training sample and the cognitive characteristics of the target human driving experience scene includes:

Step S31, obtain training sample map annotation data, training sample risk target detection and tracking annotation data, and training sample motion annotation data;

In this embodiment, it should be noted that when the cognitive layer adopts the attention mechanism, the correlation and dependence between various features extracted in the cognitive layer can be captured. Therefore, each feature in the cognitive layer Models will interact with each other, and each model in the cognitive layer will cause a certain loss. Optimizing one or more other models based on any one of the losses may still cause the entire cognitive layer to pose risks to the current scenario. Cognitive inaccuracies, for example, whether a certain risk target can be identified, may affect the understanding of the behavior of other traffic participants with whom they interact, thereby affecting the risk perception of the entire scenario.

When the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model, the scene cognitive model, the map construction model, the risk target detection and tracking model, and the motion prediction model can be calculated respectively. The losses are aggregated, and the total loss obtained by aggregation is determined as the cognitive layer loss of the entire cognitive layer. Based on the total loss, iterative optimization of each model in the cognitive layer can be simultaneously performed to improve the risk of the entire cognitive layer. Cognitive accuracy.

As an example, the step S31 includes: obtaining the training sample map annotation data, the training sample risk target detection and tracking annotation data and the training sample motion annotation data corresponding to the training sample, wherein the training sample map annotation data, the training sample The sample risk target detection and tracking annotation data and the training sample motion annotation data can be obtained in advance through manual or model annotation based on the training sample vehicle interior and exterior monitoring data.

Step S32: Determine the scene cognitive loss based on the difference between the training sample cognitive layer characteristics and the target human driving experience scene cognitive characteristics, and determine the scene cognitive loss based on the difference between the training sample map characteristics and the training sample map annotation data. The map construction loss is determined based on the difference between the risk target detection and tracking characteristics of the training sample and the risk target detection and tracking annotation data of the training sample, and the risk target detection and tracking loss is determined based on the risk target movement characteristics of the training sample and The difference between the motion annotation data of the training samples determines the motion prediction loss;

In this embodiment, it should be noted that the method of calculating the scene recognition loss, the map construction loss, the risk target detection and tracking loss, and the motion prediction loss can be selected from existing technologies according to actual needs. , for example, the cross entropy loss function, the mean square error loss function, etc. can be used for calculation, or the corresponding loss function can be designed by oneself, which is not limited in this embodiment.

As an example, the step S32 includes: determining the scene cognitive loss according to the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics, and determining the scene cognitive loss according to the training sample map characteristics and the target human driving experience scene cognitive characteristics. The map construction loss is determined based on the difference between the training sample map annotation data, the risk target detection and tracking loss is determined based on the difference between the training sample risk target detection and tracking characteristics and the training sample risk target detection and tracking annotation data, and the risk target detection and tracking loss is determined based on the difference between the training sample risk target detection and tracking annotation data. The difference between the risk target motion characteristics of the training sample and the motion annotation data of the training sample determines the motion prediction loss.

Optionally, the training sample scene cognitive features include a plurality of training sample scene cognitive sub-features, and the target human driving experience scene cognitive features include a plurality of human driving experience scene cognitive sub-features;

The step of determining the scene cognitive loss based on the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics includes:

Step B10, determine the target human driving experience scene cognitive sub-features corresponding to each of the training sample scene cognitive sub-features according to the preset time series correspondence;

Step B20: Calculate the individual difference values between each of the training sample scene cognitive sub-features and the corresponding human driving experience scene cognitive sub-features in sequence, and determine the average difference value of each individual difference value as the scene recognition sub-feature. Know the loss.

In this embodiment, it should be noted that the scene cognitive features contain a lot of information that will change. For example, the behavior and location of traffic participants will change, and the infrastructure may also change. For these changes, Information, data at one moment is often difficult to reflect its intention, so it is difficult to predict its changing trend. Therefore, using information at one moment to perform scene risk recognition is less accurate, which will lead to higher risks in autonomous driving control. Less secure. Therefore, the data generated within a period of time can be extracted, a series of scene cognitive sub-features can be extracted to form a scene cognitive feature sequence, and the change information of the scene within a period of time can be mined, which will help improve the accuracy of risk target movement prediction, thereby Reduce the risk of autonomous driving control and improve the driving safety of autonomous vehicles.

Based on this time series feature, this embodiment proposes a loss determination method. First, match each of the training sample scene cognitive sub-features and the target human driving experience scene cognitive sub-feature according to the time sequence, and calculate each pair of sub-features respectively. The difference between sub-features is then determined based on the average of the differences between sub-features to determine the scene cognitive loss in the current scene.

As an example, the steps B10-B20 include: arranging each of the training sample scene cognitive sub-features into a training sample scene cognitive feature sequence in advance according to the same time sequence arrangement, and assigning each of the target human driving The experience scene cognitive sub-features are arranged into a target human driving experience scene cognitive feature sequence. When calculating the loss, the training sample scene cognitive sub-features and the target human driving experience scene cognitive sub-features corresponding to the same time sequence can be combined into a sub-feature pair. Calculate the individual difference values between the training sample scene cognitive sub-features and the target human driving experience scene cognitive sub-features in each of the sub-feature pairs, respectively, by averaging or weighting the individual difference values The average value is obtained, and the average difference value is determined as the current scene cognitive loss.

Optionally, the single difference value between each of the training sample scene cognitive sub-features and the corresponding human driving experience scene cognitive sub-feature is calculated in sequence, and the average difference value of each single difference value is determined. Steps to cognitive loss for a scenario include:

Input each of the training sample scene cognitive sub-features and each of the target human driving experience scene cognitive sub-features into a preset scene cognitive loss function to calculate the scene cognitive loss, where the scene cognitive loss function is :

in, For scene cognitive loss,/> is the scene cognitive sub-feature of human driving experience at time t,/> is the scene cognitive sub-feature of the training sample at time t, T _f is the preset timestamp, t∈T _f .

In this embodiment, it should be noted that the training sample scene cognitive feature sequence at least includes the training sample scene cognitive sub-features from t ₀ to T _f , and the target human driving experience scene cognitive feature sequence At least include the human driving experience scene cognitive sub-features from t ₀ to T _f . The time information of the training sample vehicle interior and exterior monitoring data may be different from the time information of the human driving experience vehicle interior and exterior monitoring data, and can be recoded. Align and unify the starting time of both to t ₀ .

Optionally, the scene cognitive model includes an attention distribution cognitive model and a semantic understanding model; the training sample scene cognitive features include training sample attention distribution features and training sample semantic understanding features; the target human driving experience Scene cognitive features include target human driving experience attention distribution features and target human driving experience semantic understanding features;

The step of extracting the scene cognitive features of the training sample from the training sample vehicle interior and exterior monitoring data through the scene cognitive model includes:

Step C10: Extract the training sample attention distribution characteristics from the training sample vehicle interior and exterior monitoring data through the attention distribution cognitive model, and extract the training sample attention distribution characteristics from the training sample vehicle interior and exterior monitoring data through the semantic understanding model. Semantic understanding features of training samples;

In this embodiment, it should be noted that the model is trained based on image data, and the trained model lacks semantic understanding of the image, making it difficult to understand the deep semantic information in the scene that requires multi-layer reasoning to obtain, resulting in The model has poor understanding of the scenario, and the trained model cannot meet actual security requirements. By introducing the attention distribution cognitive model and integrating human driving experience to train the attention distribution cognitive model, the cognitive layer can learn the attention distribution of human driving vehicles when encountering complex scenes. However, for more complex scenes, The more intrinsic reasons for human attention distribution are multi-layered and integrated with multiple factors, the model only imitates human attention distribution and is often difficult to flexibly respond to changing actual situations. Therefore, a semantic understanding model can be introduced to extract semantic understanding. Features enable the cognitive layer to not only learn human attention distribution, but also further learn the internal reasons and reasoning processes of human attention distribution, and learn the way humans understand scenes at multiple levels, thus helping to deal with complex scenes.

As an example, the step C10 includes: inputting the training sample vehicle interior and exterior monitoring data into the attention distribution cognitive model, and performing multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning, etc. Obtain the attention distribution characteristics of the training sample; input the training sample vehicle interior and exterior monitoring data into the semantic understanding model, perform feature extraction, feature fusion, feature encoding, feature reasoning and other multi-level progressive feature processing to obtain the semantic understanding of the training sample feature.

The step of determining the scene cognitive loss based on the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics includes:

Step C20, determine the attention distribution cognitive loss based on the difference between the attention distribution characteristics of the training sample and the target human driving experience attention distribution characteristics, and determine the cognitive loss of attention distribution based on the semantic understanding characteristics of the training sample and the target human driving experience. Differences between empirical semantic understanding features determine semantic understanding loss;

Step C30: Aggregate the attention distribution cognitive loss and the semantic understanding loss to obtain the scene cognitive loss.

As an example, the steps C20-C30 include: determining the attention distribution cognitive loss based on the difference between the attention distribution characteristics of the training sample and the target human driving experience attention distribution characteristics, and determining the attention distribution cognitive loss based on the training sample attention distribution characteristics. The difference between the semantic understanding features of the sample and the target human driving experience semantic understanding features determines the semantic understanding loss, and then the attention distribution cognitive loss and The semantic understanding loss is aggregated to obtain the scene recognition loss.

Step S33: Aggregate the scene recognition loss, the map construction loss, the risk target detection and tracking loss, and the motion prediction loss into a cognitive layer loss, and perform the cognitive layer loss based on the cognitive layer loss. Perform iterative optimization.

As an example, step S33 includes: aggregating the scene recognition loss, the map construction loss, the risk target detection and tracking loss, and the motion prediction loss to obtain a cognitive layer loss, where aggregation The method of cognitive layer loss can be summation, weighted sum, weighted average, etc. Furthermore, it is determined whether the cognitive layer loss has converged. If the cognitive layer loss has not converged, then based on the cognitive layer loss, the gradient descent method can be used to perform a round of updates on the cognitive layer and return to execution. The steps of obtaining the training sample vehicle interior and exterior monitoring data and the training sample scene labels, and obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene labels, perform the next round of training; if the cognitive layer loss converges , then the cognitive layer completed by iterative optimization can be obtained.

In this embodiment, by separately calculating the losses of the scene cognitive model, map construction model, risk target detection and tracking model, and motion prediction model, and aggregating them, the total loss obtained by aggregation is determined as the cognitive layer of the entire cognitive layer. Loss, iterative optimization of each model in the cognitive layer based on the total loss can improve the accuracy of the entire cognitive layer's perception of scene risks.

Embodiment 3

Further, embodiments of the present application also provide an automatic driving control method. The automatic driving control method adopts a cognitive decision-making model. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The cognitive layer adopts the above-mentioned method. The above-mentioned automatic driving optimization method is used for pre-training. Contents that are the same or similar to the above-mentioned embodiments can be referred to the above introduction and will not be repeated again.

Referring to Figure 4, the automatic driving control method includes the following steps:

Step D10, obtain the monitoring data inside and outside the vehicle to be predicted;

Step D20: Extract the cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted;

Step D30: Input the cognitive layer features to be predicted into the decision-making layer to determine automatic driving control parameters.

In this embodiment, it should be noted that the cognitive decision-making model has been model trained in advance. During the model training process, the cognitive layer is pre-trained using the automatic driving optimization method as described above, and the pre-training is completed. The cognitive layer and the decision-making layer conduct formal training together. The formal training can be carried out using the same or similar methods of the existing technology, so I will not go into details here. The cognitive decision-making model obtained after formal training can be used in the automatic driving control method to perform automatic driving control on the automatic driving vehicle. Wherein, the pre-training of the cognitive layer can be performed on the first device, the formal training of the cognitive decision-making model can be performed on the second device, and the automatic driving control method can be used on the third device. Any two of the device, the second device, and the third device may be the same or different, and the details may be determined according to actual conditions, which is not limited in this embodiment.

The autonomous driving control parameters may include driving trajectories, vehicle control parameters, etc., where the driving trajectory refers to the trajectory of the autonomous vehicle for a period of time after the current moment, and the autonomous driving vehicle can be controlled to drive based on the driving trajectory, and the vehicle control Parameters include speed, accelerator pedal opening, steering wheel angle, etc.

As an example, the steps D10-D30 include: collecting sensor data at the current moment as the vehicle interior and exterior monitoring data to be predicted through one or more sensors installed on the self-driving vehicle, or through communication on the self-driving vehicle. The module communicates with the external device and obtains the external data provided by the external device as the monitoring data inside and outside the vehicle to be predicted, such as traffic light signals, traffic accident information, roadside monitoring data collected by the roadside monitoring equipment, etc. Furthermore, the vehicle interior and exterior monitoring data to be predicted is input into the cognitive layer, and multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning is performed to obtain the cognitive layer features to be predicted, where, The cognitive layer features to be predicted at least include cognitive features of the scene to be predicted. Furthermore, the cognitive layer features to be predicted are input into the decision-making layer to determine the automatic driving control parameters of the automatic driving vehicle at the current moment or within a period of time after the current moment.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the cognitive layer features to be predicted also include map features to be predicted, risk target detection and tracking to be predicted Characteristics and movement characteristics of risk targets to be predicted;

The step of extracting cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer includes:

Step D21: Use the scene cognitive model to extract the scene cognitive features to be predicted from the vehicle interior and exterior monitoring data to be predicted, and extract the map to be predicted from the vehicle interior and exterior monitoring data to be predicted using the map construction model. Features: extract the risk target detection and tracking features to be predicted from the vehicle interior and exterior monitoring data to be predicted through the risk target detection and tracking model;

Step D22, use the motion prediction model to perform risk target motion prediction based on the to-be-predicted scene cognitive features, the to-be-predicted map features, and the to-be-predicted risk target detection and tracking features to obtain the to-be-predicted risk target motion features, Wherein, the motion prediction model adopts an attention mechanism.

As an example, the steps D21-D22 include: inputting the vehicle interior and exterior monitoring data to be predicted into the scene cognitive model, and performing multi-level progressive features such as feature extraction, feature fusion, feature encoding, and feature reasoning. Processing to extract the cognitive features of the scene to be predicted; simultaneously, input the vehicle interior and exterior monitoring data to be predicted into the map construction model to perform multi-level progressive features such as feature extraction, feature fusion, feature encoding, and feature reasoning. Process to extract the map features to be predicted; simultaneously, input the vehicle interior and exterior monitoring data to be predicted into the risk target detection and tracking model to perform multi-level progressive features such as feature extraction, feature fusion, feature encoding, and feature reasoning. Process and extract the detection and tracking features of the risk target to be predicted. Furthermore, the cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted are input into the motion prediction model, and the attention mechanism is used through the motion prediction model to capture the features to be predicted. The correlation and dependence between the scene cognitive features, the map features to be predicted and the risk target detection and tracking features to be predicted. Based on these correlations and dependencies, risk target operation predictions can be made more accurately, and at least The motion characteristics of a risk target to be predicted.

Optionally, the risk target detection and tracking features to be predicted include at least one risk target detection and tracking sub-feature to be predicted; the motion prediction model includes an attention layer, a multi-layer perceptron layer and a prediction layer;

The step of predicting the movement of risky targets through the motion prediction model based on the cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted, and obtaining the motion characteristics of the risky targets to be predicted include:

Step D221: Input the cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted into the attention layer, and use the attention mechanism to determine the detection of each risk target to be predicted. To-be-predicted risk target interaction features between tracking sub-features, to-be-predicted scene interaction features between each of the to-be-predicted risk target detection tracking sub-features and the to-be-predicted scene cognitive features, and to each of the to-be-predicted risk target detection Track the to-be-predicted map interaction features between the sub-features and the to-be-predicted map features;

Step D222, input the risk target interaction features to be predicted, the scene interaction features to be predicted, and the map interaction features to be predicted into the multi-layer perceptron layer to obtain the cognitive layer features of the scene to be predicted and the motion query features to be predicted;

Step D223: Input the cognitive layer features of the scene to be predicted and the motion query features to be predicted into the prediction layer, perform risk target motion prediction, and obtain the risk target motion features to be predicted.

As an example, the steps D221-D223 include: inputting the cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted into the attention layer, using an attention mechanism. , capture the dependencies between the detection and tracking sub-features of each of the risk targets to be predicted, to determine the interaction between each risk target and other risk targets, generate interaction features of the risk targets to be predicted, and capture each of the risks to be predicted Target detection tracks the dependence between sub-features and the cognitive features of the scene to be predicted to determine the interaction between each risk target and the scene cognition, generate interaction features of the scene to be predicted, and capture each of the scene cognitions to be predicted. Risk target detection tracks the dependence between sub-features and the map features to be predicted to determine the interaction between each risk target and map elements, and generate interaction features of the scene to be predicted. Furthermore, after splicing the risk target interaction features to be predicted, the scene interaction features to be predicted and the map interaction features to be predicted, the multi-layer perceptron layer is input to obtain the cognitive layer features of the scene to be predicted and the motion query to be predicted. feature. Furthermore, the cognitive layer features of the scene to be predicted and the motion query features to be predicted are input into the prediction layer, and risk target motion prediction is performed for each risk target to obtain the risk target motion features to be predicted for each risk target.

In an implementable manner, the multi-layer perceptron layer can also decode the cognitive layer features of the scene to be predicted and the motion query features to be predicted, and output the decoded scene risk prediction information and the motion information of the risk target as intermediate results, for users to view.

In this embodiment, the cognitive layer obtained after training based on the cognitive characteristics of human driving experience scenes has learned human driving experience, so it has a more comprehensive cognition of complex scenes and has stronger reasoning capabilities, so that it can be more accurate Comprehensively understand and pay attention to the risk points in the scene, and apply it to autonomous driving control, which can guide decision-makers to avoid risks more accurately, make safer driving decisions, and improve the safety of autonomous driving.

Embodiment 4

Further, embodiments of the present application also provide an automatic driving optimization device. Referring to Figure 5, a cognitive decision-making model is deployed on the automatic driving optimization device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. Autonomous driving optimization devices include:

The first acquisition module 10 is used to acquire training sample vehicle interior and exterior monitoring data and training sample scene labels, and obtain target human driving experience scene cognitive features corresponding to the training sample scene labels;

The first cognitive module 20 is configured to extract the cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer;

The optimization module 30 is configured to iteratively optimize the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the training sample cognitive layer features include training sample scene cognitive features, training sample map features, Risk target detection and tracking features of training samples and risk target movement features of training samples;

The first cognitive module 20 is also used to:

The scene cognitive model extracts the training sample scene cognitive features from the training sample vehicle interior and exterior monitoring data, and the map construction model extracts the training sample map features from the training sample vehicle interior and exterior monitoring data. The risk target detection and tracking model extracts the training sample risk target detection and tracking characteristics from the training sample vehicle interior and exterior monitoring data;

The motion prediction model performs risk target motion prediction based on the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features to obtain training sample risk target motion features, where The motion prediction model described above uses an attention mechanism.

Optionally, the training sample risk target detection and tracking features include at least one training sample risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multi-layer perceptron layer and a prediction layer;

The first cognitive module 20 is also used to:

The training sample scene cognitive features, the training sample map features and the training sample risk target detection and tracking features are input into the attention layer, and an attention mechanism is used to determine the relationship between each of the risk target detection and tracking sub-features. The training sample risk target interaction features, the training sample scene interaction features between each of the risk target detection and tracking sub-features and the training sample scene cognitive features, and each of the risk target detection and tracking sub-features and the training sample map Training sample map interaction features between features;

Input the training sample risk target interaction characteristics, the training sample scene interaction characteristics and the training sample map interaction characteristics into the multi-layer perceptron layer to obtain the training sample scene cognitive layer characteristics and the training sample motion query characteristics;

The training sample scene cognitive layer features and the training sample motion query features are input into the prediction layer, and risk target motion prediction is performed to obtain training sample risk target motion features.

Optionally, the optimization module 30 is also used to:

Obtain training sample map annotation data, training sample risk target detection and tracking annotation data, and training sample motion annotation data;

The scene cognitive loss is determined based on the difference between the cognitive layer characteristics of the training sample and the cognitive characteristics of the target human driving experience scene, and the scene cognitive loss is determined based on the difference between the training sample map characteristics and the training sample map annotation data. The map construction loss is determined based on the difference between the risk target detection and tracking characteristics of the training sample and the risk target detection and tracking annotation data of the training sample, and the risk target detection and tracking loss is determined based on the risk target motion characteristics of the training sample and the training sample. The difference between the sample motion annotation data determines the motion prediction loss;

The scene recognition loss, the map construction loss, the risk target detection and tracking loss, and the motion prediction loss are aggregated into a cognitive layer loss, and the cognitive layer is iteratively optimized based on the cognitive layer loss. .

Optionally, the training sample scene cognitive features include a plurality of training sample scene cognitive sub-features, and the target human driving experience scene cognitive features include a plurality of human driving experience scene cognitive sub-features;

The optimization module 30 is also used to:

According to the preset time sequence correspondence, determine the target human driving experience scene cognitive sub-feature corresponding to each of the training sample scene cognitive sub-features;

The individual difference values between each of the training sample scene cognitive sub-features and the corresponding human driving experience scene cognitive sub-features are calculated in sequence, and the average difference value of each individual difference value is determined as the scene cognitive loss.

Optionally, the optimization module 30 is also used to:

Input each of the training sample scene cognitive sub-features and each of the target human driving experience scene cognitive sub-features into a preset scene cognitive loss function to calculate the scene cognitive loss, where the scene cognitive loss function is :

in, For scene cognitive loss,/> is the scene cognitive sub-feature of human driving experience at time t,/> is the scene cognitive sub-feature of the training sample at time t, T _f is the preset timestamp, t∈T _f .

Optionally, the scene cognitive model includes an attention distribution cognitive model and a semantic understanding model; the training sample scene cognitive features include training sample attention distribution features and training sample semantic understanding features; the target human driving experience Scene cognitive features include target human driving experience attention distribution features and target human driving experience semantic understanding features;

The first cognitive module is also used to:

The attention distribution characteristics of the training sample are extracted from the training sample vehicle interior and exterior monitoring data through the attention distribution cognitive model, and the training sample semantics are extracted from the training sample vehicle interior and exterior monitoring data through the semantic understanding model. understand characteristics;

The step of determining the scene cognitive loss based on the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics includes:

Determine the attention distribution cognitive loss based on the difference between the attention distribution characteristics of the training sample and the target human driving experience attention distribution characteristics, and understand the semantic understanding based on the semantic understanding characteristics of the training sample and the target human driving experience Differences between features determine semantic understanding loss;

The attention distribution cognitive loss and the semantic understanding loss are aggregated to obtain the scene cognitive loss.

Optionally, before the operation of obtaining the target human driving experience scene cognitive characteristics corresponding to the training sample scene label, the automatic driving optimization device further includes a scene label determination module, and the scene label determination module is used to:

Acquire multiple human driving experience scene monitoring data, and extract human driving experience scene cognitive features from each of the human driving experience scene monitoring data;

Perform cluster analysis on the cognitive features of each of the human driving experience scenes to determine the scene labels corresponding to the cognitive features of each of the human driving experience scenes.

The automatic driving optimization device provided by the present invention adopts the automatic driving optimization method in the above embodiment to solve the technical problem of low safety of automatic driving in related technologies. Compared with related technologies, the benefits of the automatic driving optimization device provided by the embodiments of the present invention are the same as those of the automatic driving optimization method provided by the above-mentioned embodiments, and other technical features in the automatic driving optimization device are the same as those disclosed in the above-mentioned embodiments. The characteristics are the same and will not be repeated here.

Embodiment 5

Further, embodiments of the present application also provide an automatic driving control device. A cognitive decision-making model is deployed on the automatic driving control device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The cognitive layer adopts The automatic driving optimization method as mentioned above is pre-trained, and the automatic driving control device includes:

The second acquisition module is used to acquire the vehicle interior and exterior monitoring data to be predicted;

A second cognitive module, configured to extract the cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted;

A decision-making module, used to input the cognitive layer characteristics to be predicted into the decision-making layer and determine automatic driving control parameters.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the cognitive layer features to be predicted also include map features to be predicted, risk target detection and tracking to be predicted Characteristics and movement characteristics of risk targets to be predicted;

The second cognitive model is also used for:

The scene cognitive features to be predicted are extracted from the vehicle interior and exterior monitoring data to be predicted through the scene cognitive model, and the map features to be predicted are extracted from the vehicle interior and exterior monitoring data to be predicted through the map construction model. The risk target detection and tracking model extracts the risk target detection and tracking features to be predicted from the vehicle interior and exterior monitoring data to be predicted;

The motion prediction model performs motion prediction of risk targets based on the cognitive features of the scene to be predicted, the map features to be predicted, and the risk target detection and tracking features to be predicted, and obtains the motion characteristics of the risk targets to be predicted, wherein, The motion prediction model described above uses an attention mechanism.

Optionally, the risk target detection and tracking features to be predicted include at least one risk target detection and tracking sub-feature to be predicted; the motion prediction model includes an attention layer, a multi-layer perceptron layer and a prediction layer;

The second cognitive model is also used for:

The cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted are input into the attention layer, and an attention mechanism is used to determine each of the risk target detection and tracking sub-features to be predicted. The interaction features of risk targets to be predicted, the scene interaction features to be predicted between each of the risk target detection and tracking sub-features to be predicted and the cognitive features of the scene to be predicted, and the detection and tracking sub-features of each risk target to be predicted To-be-predicted map interaction features with the to-be-predicted map features;

Input the risk target interaction features to be predicted, the scene interaction features to be predicted, and the map interaction features to be predicted into the multi-layer perceptron layer to obtain the cognitive layer features of the scene to be predicted and the motion query features to be predicted;

The cognitive layer features of the scene to be predicted and the motion query features to be predicted are input into the prediction layer, and risk target motion prediction is performed to obtain the risk target motion features to be predicted.

The automatic driving optimization device provided by the present invention adopts the automatic driving optimization method in the above embodiment to solve the technical problem of low safety of automatic driving in related technologies. Compared with related technologies, the benefits of the automatic driving optimization device provided by the embodiments of the present invention are the same as those of the automatic driving optimization method provided by the above-mentioned embodiments, and other technical features in the automatic driving optimization device are the same as those disclosed in the above-mentioned embodiments. The characteristics are the same and will not be repeated here.

Embodiment 6

Further, embodiments of the present invention provide an electronic device. The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions Executed by at least one processor, so that at least one processor can execute the automatic driving optimization method in the above embodiment.

Referring now to FIG. 6 , a schematic structural diagram of an electronic device suitable for implementing embodiments of the present disclosure is shown. Electronic devices in embodiments of the present disclosure may include, but are not limited to, Bluetooth headsets, mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablets), PMPs (Portable Multimedia Players), vehicle Mobile terminals such as car navigation terminals and fixed terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 6 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 6 , the electronic device may include a processing device (such as a central processing unit, a graphics processor, etc.) that may be loaded into a random access memory (RAM) according to a program stored in a read-only memory (ROM) or from a storage device. perform various appropriate actions and processing according to the program in it. In RAM, various programs and arrays required for the operation of electronic equipment are also stored. The processing device, ROM and RAM are connected to each other via a bus. Input/output (I/O) interfaces are also connected to the bus.

Typically, the following systems can be connected to the I/O interface: input devices including, for example, touch screens, touch pads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (LCDs), speakers, vibrators Output devices, etc.; storage devices including tapes, hard disks, etc.; and communication devices. The communication device may allow the electronic device to communicate wirelessly or wiredly with other devices to exchange arrays. Although the figures illustrate electronic devices having various systems, it is to be understood that implementation or availability of all illustrated systems is not required. More or fewer systems may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network through the communication device, or from a storage device, or from a ROM. When the computer program is executed by the processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

The electronic device provided by the present invention adopts the automatic driving optimization method in the above embodiment to solve the technical problem of low safety of automatic driving in related technologies. Compared with related technologies, the benefits of the electronic device provided by the embodiments of the present invention are the same as those of the autonomous driving optimization method provided by the above-mentioned embodiments, and other technical features in the electronic device are the same as those disclosed by the above-mentioned embodiments. This will not be described in detail.

It should be understood that various parts of the present disclosure may be implemented in hardware, software, firmware, or combinations thereof. In the above description of the embodiments, specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention. should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Embodiment 7

Further, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon. The computer-readable program instructions are used to execute the automatic driving optimization method in the above embodiment.

The computer-readable storage medium provided by the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this embodiment, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, system, or device. Program code contained on a computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable storage medium may be included in the electronic device; it may also exist independently without being assembled into the electronic device.

The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by an electronic device, the electronic device: obtains the training sample vehicle interior and exterior monitoring data and the training sample scene tags, and obtains the training sample. Cognitive features of the target human driving experience scene corresponding to the scene label; extract the cognitive layer features of the training sample from the inside and outside monitoring data of the training sample through the cognitive layer; according to the cognitive layer features of the training sample and the Based on the cognitive characteristics of the target human driving experience scene, the cognitive layer is iteratively optimized.

Alternatively, the computer-readable storage medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains the vehicle interior and exterior monitoring data to be predicted; and obtains the vehicle interior and exterior monitoring data from the vehicle through the cognitive layer. The cognitive layer features to be predicted are extracted from the monitoring data inside and outside the vehicle, where the cognitive layer features to be predicted include the scene cognitive features to be predicted; the cognitive layer features to be predicted are input into the decision-making layer to determine the automatic driving Control parameters.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The modules involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of the module does not constitute a limitation on the unit itself under certain circumstances.

The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned automatic driving optimization method, and solves the technical problem of low safety of automatic driving in related technologies. Compared with related technologies, the benefits of the computer-readable storage medium provided by the embodiments of the present invention are the same as the benefits of the automatic driving optimization method provided by the above-mentioned embodiments, and will not be described again here.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of the present application may be directly or indirectly used in other related technical fields. , are all similarly included in the patent processing scope of this application.

Claims (15)

1. An automatic driving optimization method, characterized in that the automatic driving optimization method is applied to a cognitive decision-making model, and the cognitive decision-making model includes a cognitive layer and a decision-making layer, including the following steps: Obtain training sample vehicle interior and exterior monitoring data and training sample scene labels, and obtain target human driving experience scene cognitive characteristics corresponding to the training sample scene labels; Extract training sample cognitive layer features from the training sample vehicle interior and exterior monitoring data through the cognitive layer; The cognitive layer is iteratively optimized according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene. 2. The automatic driving optimization method according to claim 1, characterized in that the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model and a motion prediction model; the training sample cognitive layer Features include training sample scene cognitive features, training sample map features, training sample risk target detection and tracking features, and training sample risk target motion features; The step of extracting the cognitive layer features of the training sample from the training sample vehicle interior and exterior monitoring data through the cognitive layer includes: The scene cognitive model extracts the training sample scene cognitive features from the training sample vehicle interior and exterior monitoring data, and the map construction model extracts the training sample map features from the training sample vehicle interior and exterior monitoring data. The risk target detection and tracking model extracts the training sample risk target detection and tracking characteristics from the training sample vehicle interior and exterior monitoring data; The motion prediction model performs risk target motion prediction based on the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features to obtain training sample risk target motion features, where The motion prediction model described above uses an attention mechanism. 3. The automatic driving optimization method according to claim 2, wherein the training sample risk target detection and tracking features include at least one training sample risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multi-level layer perceptron layer and prediction layer; The step of using the motion prediction model to perform risk target motion prediction based on the training sample scene cognitive features, the training sample map features, and the training sample risk target detection and tracking features to obtain the training sample risk target motion characteristics. include: The training sample scene cognitive features, the training sample map features and the training sample risk target detection and tracking features are input into the attention layer, and an attention mechanism is used to determine the relationship between each of the risk target detection and tracking sub-features. The training sample risk target interaction features, the training sample scene interaction features between each of the risk target detection and tracking sub-features and the training sample scene cognitive features, and each of the risk target detection and tracking sub-features and the training sample map Training sample map interaction features between features; Input the training sample risk target interaction characteristics, the training sample scene interaction characteristics and the training sample map interaction characteristics into the multi-layer perceptron layer to obtain the training sample scene cognitive layer characteristics and the training sample motion query characteristics; The training sample scene cognitive layer features and the training sample motion query features are input into the prediction layer, and risk target motion prediction is performed to obtain training sample risk target motion features. 4. The automatic driving optimization method according to claim 2, wherein the cognitive layer is iteratively optimized according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene. The steps include: Obtain training sample map annotation data, training sample risk target detection and tracking annotation data, and training sample motion annotation data; The scene cognitive loss is determined based on the difference between the cognitive layer characteristics of the training sample and the cognitive characteristics of the target human driving experience scene, and the scene cognitive loss is determined based on the difference between the training sample map characteristics and the training sample map annotation data. The map construction loss is determined based on the difference between the risk target detection and tracking characteristics of the training sample and the risk target detection and tracking annotation data of the training sample, and the risk target detection and tracking loss is determined based on the risk target motion characteristics of the training sample and the training sample. The difference between the sample motion annotation data determines the motion prediction loss; The scene recognition loss, the map construction loss, the risk target detection and tracking loss, and the motion prediction loss are aggregated into a cognitive layer loss, and the cognitive layer is iteratively optimized based on the cognitive layer loss. . 5. The automatic driving optimization method according to claim 4, wherein the training sample scene cognitive characteristics include a plurality of training sample scene cognitive sub-features, and the target human driving experience scene cognitive characteristics include a plurality of human driving experience sub-features. Driving experience-like scene cognition sub-features; The step of determining the scene cognitive loss based on the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics includes: According to the preset time sequence correspondence, determine the target human driving experience scene cognitive sub-feature corresponding to each of the training sample scene cognitive sub-features; The individual difference values between each of the training sample scene cognitive sub-features and the corresponding human driving experience scene cognitive sub-features are calculated in sequence, and the average difference value of each individual difference value is determined as the scene cognitive loss. 6. The automatic driving optimization method according to claim 5, characterized in that said sequentially calculates the single difference between each of the training sample scene cognitive sub-features and the respective corresponding human driving experience scene cognitive sub-features. The steps to determine the average difference value of each individual unit difference value as scene cognitive loss include: Input each of the training sample scene cognitive sub-features and each of the target human driving experience scene cognitive sub-features into a preset scene cognitive loss function to calculate the scene cognitive loss, where the scene cognitive loss function is : in, For scene cognitive loss,/> is the scene cognitive sub-feature of human driving experience at time t,/> is the scene cognitive sub-feature of the training sample at time t, T _f is the preset timestamp, t∈T _f . 7. The automatic driving optimization method according to claim 4, wherein the scene cognitive model includes an attention distribution cognitive model and a semantic understanding model; the training sample scene cognitive characteristics include a training sample attention distribution. Features and training sample semantic understanding features; the target human driving experience scene cognitive features include target human driving experience attention distribution features and target human driving experience semantic understanding features; The step of extracting the scene cognitive features of the training sample from the training sample vehicle interior and exterior monitoring data through the scene cognitive model includes: The attention distribution characteristics of the training sample are extracted from the training sample vehicle interior and exterior monitoring data through the attention distribution cognitive model, and the training sample semantics are extracted from the training sample vehicle interior and exterior monitoring data through the semantic understanding model. understand characteristics; The step of determining the scene cognitive loss based on the difference between the cognitive layer characteristics of the training sample and the target human driving experience scene cognitive characteristics includes: Determine the attention distribution cognitive loss based on the difference between the attention distribution characteristics of the training sample and the target human driving experience attention distribution characteristics, and understand the semantic understanding based on the semantic understanding characteristics of the training sample and the target human driving experience Differences between features determine semantic understanding loss; The attention distribution cognitive loss and the semantic understanding loss are aggregated to obtain the scene cognitive loss. 8. The automatic driving optimization method according to claim 1, characterized in that before the step of obtaining the cognitive characteristics of the target human driving experience scene corresponding to the training sample scene label, it also includes: Acquire multiple human driving experience scene monitoring data, and extract human driving experience scene cognitive features from each of the human driving experience scene monitoring data; Perform cluster analysis on the cognitive features of each of the human driving experience scenes to determine the scene labels corresponding to the cognitive features of each of the human driving experience scenes. 9. An automatic driving control method, characterized in that the automatic driving control method adopts a cognitive decision-making model, and the cognitive decision-making model includes a cognitive layer and a decision-making layer, and the cognitive layer adopts the method as claimed in claim 1- The automatic driving optimization method described in any one of 8 is pre-trained, and the automatic driving control method includes the following steps: Obtain monitoring data inside and outside the vehicle to be predicted; The cognitive layer features to be predicted are extracted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted; The cognitive layer features to be predicted are input into the decision-making layer to determine the automatic driving control parameters. 10. The automatic driving control method according to claim 9, wherein the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model and a motion prediction model; the cognitive layer to be predicted Features also include map features to be predicted, risk target detection and tracking features to be predicted, and risk target motion features to be predicted; The step of extracting cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer includes: The scene cognitive features to be predicted are extracted from the vehicle interior and exterior monitoring data to be predicted through the scene cognitive model, and the map features to be predicted are extracted from the vehicle interior and exterior monitoring data to be predicted through the map construction model. The risk target detection and tracking model extracts the risk target detection and tracking features to be predicted from the vehicle interior and exterior monitoring data to be predicted; The motion prediction model performs motion prediction of risk targets based on the cognitive features of the scene to be predicted, the map features to be predicted, and the risk target detection and tracking features to be predicted, and obtains the motion characteristics of the risk targets to be predicted, wherein, The motion prediction model described above uses an attention mechanism. 11. The automatic driving control method according to claim 10, wherein the risk target detection and tracking features to be predicted include at least one risk target detection and tracking sub-feature to be predicted; the motion prediction model includes an attention layer, a multi-level layer perceptron layer and prediction layer; The step of predicting the movement of risky targets through the motion prediction model based on the cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted, and obtaining the motion characteristics of the risky targets to be predicted include: The cognitive features of the scene to be predicted, the map features to be predicted and the risk target detection and tracking features to be predicted are input into the attention layer, and an attention mechanism is used to determine each of the risk target detection and tracking sub-features to be predicted. The interaction features of risk targets to be predicted, the scene interaction features to be predicted between each of the risk target detection and tracking sub-features to be predicted and the cognitive features of the scene to be predicted, and the detection and tracking sub-features of each risk target to be predicted To-be-predicted map interaction features with the to-be-predicted map features; Input the risk target interaction features to be predicted, the scene interaction features to be predicted, and the map interaction features to be predicted into the multi-layer perceptron layer to obtain the cognitive layer features of the scene to be predicted and the motion query features to be predicted; The cognitive layer features of the scene to be predicted and the motion query features to be predicted are input into the prediction layer, and risk target motion prediction is performed to obtain the risk target motion features to be predicted. 12. An automatic driving optimization device, characterized in that a cognitive decision-making model is deployed on the automatic driving optimization device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The automatic driving optimization device includes: The first acquisition module is used to acquire training sample vehicle interior and exterior monitoring data and training sample scene labels, and obtain target human driving experience scene cognitive features corresponding to the training sample scene labels; A first cognitive module, configured to extract training sample cognitive layer features from the training sample vehicle interior and exterior monitoring data through the cognitive layer; An optimization module, configured to iteratively optimize the cognitive layer according to the cognitive layer characteristics of the training samples and the cognitive characteristics of the target human driving experience scene. 13. An automatic driving control device, characterized in that a cognitive decision-making model is deployed on the automatic driving control device. The cognitive decision-making model includes a cognitive layer and a decision-making layer. The cognitive layer adopts the method described above. The automatic driving optimization method is pre-trained. The automatic driving control device includes: The second acquisition module is used to acquire the vehicle interior and exterior monitoring data to be predicted; A second cognitive module, configured to extract the cognitive layer features to be predicted from the vehicle interior and exterior monitoring data through the cognitive layer, where the cognitive layer features to be predicted include the scene cognitive features to be predicted; A decision-making module, used to input the cognitive layer characteristics to be predicted into the decision-making layer and determine automatic driving control parameters. 14. An electronic device, characterized in that the electronic device includes: at least one processor; and, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions executable by at least one of the processors, and the instructions are executed by at least one of the processors to enable at least one of the processors to perform as claimed in any one of claims 1 to 8 The steps of the automatic driving optimization method described above, or the steps of the automatic driving control method according to any one of claims 9 to 11. 15. A medium, characterized in that the medium is a computer-readable storage medium, and a program for implementing an automatic driving optimization method is stored on the computer-readable storage medium. The program for realizing an automatic driving optimization method is processed by a processor. Executed to implement the steps of the automatic driving optimization method according to any one of claims 1 to 8, or the steps of the automatic driving control method according to any one of claims 9 to 11.