CN117141520B - Real-time track planning method, device and equipment - Google Patents

Abstract

The disclosure belongs to the technical field of automatic driving, and particularly relates to a real-time track planning method, device and equipment. Wherein the method comprises the following steps: basic navigation point information and pose information of a vehicle at the current moment are obtained, a target path reference line is determined based on the basic navigation point information, the pose information is converted from a Cartesian coordinate system to a Flunar coordinate system based on the target path reference line, and a transverse offset-displacement map and a rasterized map are obtained; determining a path planning initial value of a current vehicle through cost searching and reinforcement learning models, smoothing the path planning initial value to obtain an initial path plan, and performing multi-objective optimization on the initial path plan to obtain an optimal path plan; and determining an optimal speed curve corresponding to the optimal path planning, obtaining a target track by combining the optimal path planning and the optimal speed curve, and carrying out cost selection on the multi-lane target track to obtain a final control track.

Description

A real-time trajectory planning method, device and equipment

Technical field

The present disclosure belongs to the field of autonomous driving technology, and specifically relates to a real-time trajectory planning method, device and equipment.

Background technique

At present, the autonomous driving scenarios that real-time trajectory planning of autonomous driving deals with are becoming more and more complex and changeable. When abnormal scenarios occur, traditional decoupled planning or optimization solutions are often difficult to deal with due to issues such as search dimensions and cost parameters, resulting in real-time planning. fail.

Therefore, the existing technology directly generates a rough solution of the trajectory by introducing reinforcement learning and then performs back-end optimization to solve the final trajectory. Although this method can obtain a relatively reliable and safe trajectory, the DQN reinforcement learning method is often difficult to obtain good results due to the limitations of the network itself. The rough solution of the trajectory makes the back-end optimization unstable.

Contents of the invention

The embodiment of the present disclosure proposes a real-time trajectory planning solution to solve the problem of insufficient processing capabilities of abnormal scenes in the existing technology.

A first aspect of the embodiments of the present disclosure provides a real-time trajectory planning method, including:

Obtain the basic navigation point information and pose information of the vehicle at the current moment, determine the target path reference line based on the basic navigation point information, and convert the pose information from the Cartesian coordinate system to Frye based on the target path reference line Under the nano coordinate system, obtain the lateral offset-displacement map and rasterized map, where the pose information includes the vehicle's own status, surrounding traffic participants and local road network information;

Determine the initial path planning value of the current vehicle through cost search and reinforcement learning models, smooth the initial path planning value to obtain the initial path planning, and perform multi-objective optimization on the initial path planning to obtain the optimal path planning;

Determine the optimal speed curve corresponding to the optimal path planning, obtain the target trajectory by combining the optimal path planning and the optimal speed curve, perform cost selection on the multi-lane target trajectory, and obtain the final control trajectory.

In some embodiments, determining the target path reference line based on the basic navigation point information includes:

Obtain basic navigation points within the first preset distance ahead and the second preset distance behind in real time according to the positioning information of the current frame of the vehicle;

Determine the loss function and boundary constraints of the target path reference line, where the loss function includes a smoothing cost, a compact cost and a similarity cost, the smoothing cost is used to measure the smoothness of the target path reference line, the The compactness cost is used to measure the degree of uniformity between the target path reference points corresponding to the basic navigation points, and the similarity cost is used to measure the deviation of the target path reference line from the original navigation path;

The basic navigation point that satisfies the boundary constraint condition is obtained based on the quadratic solver, and the path reference point corresponding to the basic navigation point constitutes the target path reference line.

In some embodiments, determining the initial value of the path planning of the current vehicle through cost search and reinforcement learning model includes:

Sampling is performed along the target path reference line, the cost value of the preset cost of each sampling node is calculated, and the first path planning initial value corresponding to the preset cost is composed of the sampling node with the smallest sum of cost values;

According to the current and past three consecutive frames of the rasterized map based on the target path reference line, the decision-making neural network model is used to make decisions about the actions to be performed by the vehicle at the current moment, and the decision-making actions of the vehicle at the current moment are obtained. value, the decision action value constitutes the initial value of the second path planning;

Calculate the cost value of the preset cost for the second path planning initial value and compare it with the cost value of the first path planning initial value corresponding to the preset cost, and compare the second path planning initial value and the third path planning initial value. Among the initial path planning values, the one with the smaller cost value is used as the initial path planning value of the current vehicle.

In some embodiments, the training method of the decision-making neural network model includes:

The state space is constituted by a three-frame continuous path-based grid map centered on the vehicle, in which the positions of the vehicle and surrounding obstacles are projected onto the grid map;

Constructing an action space consisting of transverse continuous coordinate values of multiple path points sampled within a preset rectangular range in front of the vehicle, wherein a fixed interval is maintained between the longitudinal path points, and the transverse coordinate values are determined by the decision-making neural network model;

Design a reward function based on the reward for reaching the target, the penalty for collision, the curvature of the output path, and the smooth transition between output paths;

Construct a training set with the state space as input and the action space as output;

Design a simulation environment including static scenes and complex scenes. In the simulation environment, based on the reward function, the SAC reinforcement learning method is used to train the neural network model on the training set to obtain the decision-making neural network model.

In some embodiments, smoothing the path planning initial value to obtain the initial path planning includes:

Use the loss function and boundary constraints of the target path reference line as the loss function and boundary constraints of the initial value of the path planning;

The initial path planning value is optimized based on a quadratic solver to obtain the initial path planning.

In some embodiments, performing multi-objective optimization on the initial path plan to obtain the optimal path plan includes:

Determine the boundary constraint conditions of the path based on the lateral offset-displacement map and the initial path planning value, and perform multi-objective optimization on the initial path planning based on the boundary constraint conditions and preset optimization goals to obtain the target space The optimal path planning within one or more of sex;

When the optimal path planning in the target space is solved abnormally, the initial path planning is used as the optimal path planning.

In some embodiments, determining the optimal speed curve corresponding to the optimal path planning includes:

Construct a displacement-time diagram for the static obstacles and dynamic obstacles of the vehicle in the Fleener coordinate system;

The displacement-time graph is sampled, the cost value of the preset cost of each sampling node is calculated, and the initial value of the velocity curve corresponding to the preset cost is composed of the sampling node with the smallest sum of cost values, wherein, The preset costs include speed cost, acceleration cost, jerk cost and collision cost;

A convex space for speed optimization is obtained from the initial value of the speed curve, boundary constraints are extracted in the convex space, and multi-objective optimization is performed on the speed curve based on the boundary constraints and preset optimization goals to obtain the optimal speed. Curve, wherein the preset optimization target includes one or more of cruising speed, acceleration and jerk.

In some embodiments, combining the optimal path planning and the optimal speed curve to obtain the target trajectory includes:

Based on the optimal speed curve, the optimal path planning is interpolated through displacement to obtain the speed and path information at each moment to generate a target trajectory.

A second aspect of the embodiment of the present disclosure provides a real-time trajectory planning device, including:

Determination module, used to obtain the basic navigation point information and pose information of the vehicle at the current moment, determine the target path reference line based on the basic navigation point information, and convert the pose information into Cartesian coordinates based on the target path reference line. The system is transformed into the Fleener coordinate system to obtain the lateral offset-displacement map and the rasterized map, where the pose information includes the vehicle's own status, surrounding traffic participants and local road network information;

The optimization module is used to determine the initial value of the current vehicle's path planning through cost search and reinforcement learning models, smooth the initial value of the path planning to obtain the initial path planning, and perform multi-objective optimization on the initial path planning to obtain the optimal route plan;

An acquisition module is used to determine the optimal speed curve corresponding to the optimal path planning, obtain the target trajectory by combining the optimal path planning and the optimal speed curve, perform cost selection on the multi-lane target trajectory, and determine the final Control the trajectory.

A third aspect of the embodiment of the present disclosure provides a real-time trajectory planning device, including a memory and a processor:

The memory is used to store computer programs;

The processor is configured to implement the method according to the first aspect of the present disclosure when executing the computer program.

To sum up, unlike the existing technology that directly obtains a rough solution of the trajectory through reinforcement learning and then optimizes the optimal trajectory, the real-time trajectory planning method, device and equipment provided by each embodiment of the present disclosure uses reinforcement learning to obtain a rough solution for the path. And fuse it with the rough solution obtained by the cost search to obtain the rough solution of the target path, then optimize it to obtain the target path, and finally merge it with the speed optimized result to form the target trajectory. Because the target path reference line is determined through multi-layer cost and smoothing processing of basic navigation points, the deviation of the model or search space is solved, making the initial path of the reference line of dynamic programming more stable and reliable; through the method of combining cost search and reinforcement learning , which solves the problem of selecting fine-grained spatial solutions and enhances the accuracy and safety of the initial value of path planning; through the multi-objective optimization strategy based on accurate initial value of path planning, it solves the problem of frequent parameter adjustment of the planning module in different driving scenarios. , so that the system has good abnormal scene processing capabilities.

Description of the drawings

The features and advantages of the present disclosure will be more clearly understood by reference to the accompanying drawings, which are schematic and should not be construed as limiting the disclosure in any way, in which:

Figure 1 is a schematic diagram of a real-time trajectory planning system to which the present disclosure is applicable;

Figure 2 is a flow chart of a real-time trajectory planning method according to some embodiments of the present disclosure;

Figure 3 is a schematic diagram of a real-time trajectory planning device according to some embodiments of the present disclosure;

Figure 4 is a schematic diagram of a real-time trajectory planning device according to some embodiments of the present disclosure.

Detailed ways

In the following detailed description, numerous specific details of the present disclosure are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one of ordinary skill in the art that the present disclosure may be practiced without these details. It should be understood that the terms "system", "device", "unit" and/or "module" are used in this disclosure as a means of distinguishing between different components, elements, portions or assemblies at different levels in a sequential arrangement. method. However, these terms may be replaced by other expressions if they serve the same purpose.

It will be understood that when a device, unit or module is referred to as being "on," "connected to" or "coupled to" another device, unit or module, it can be directly on the other device, unit or module on, connected to, coupled to, or in communication with other devices, units or modules, or intervening devices, units or modules may be present, unless the context clearly indicates an exception. For example, as used in this disclosure, the term "and/or" includes any and all combinations of one or more of the associated listed items.

The terminology used in this disclosure is for the purpose of describing specific embodiments only and does not limit the scope of the disclosure. As shown in this disclosure and claims, words such as "a", "an", "an" and/or "the" do not specifically refer to the singular and may include the plural unless the context clearly indicates an exception. Generally speaking, the terms "include" and "include" only imply the inclusion of clearly identified features, integers, steps, operations, elements and/or components, and such expressions do not constitute an exclusive list. Other features, Integers, steps, operations, elements and/or components may also be included.

These and other features and characteristics of the present disclosure, methods of operation, function of associated elements of construction, combination of parts, and economics of manufacture may be better understood by reference to the following description and accompanying drawings, which form part of the instruction manual. It will be expressly understood, however, that the drawings are included for the purpose of illustration and description only and are not intended to limit the scope of the present disclosure. It will be understood that the drawings are not drawn to scale.

Various structural diagrams are used in the present disclosure to illustrate various modifications according to the embodiments of the present disclosure. It should be understood that the preceding or following structures are not intended to limit the present disclosure. The scope of protection of the present disclosure shall be determined by the claims.

Figure 1 is a schematic diagram of a real-time trajectory planning system to which the present disclosure is applicable. As shown in the system shown in Figure 1, the planning module is connected with the data of the perception module and the control module. The planning module performs real-time trajectory planning based on the digital world model perceived by the perception module in the Cartesian coordinate system, and sends the generated control trajectory to the control module. The module controls the vehicle to drive along the control trajectory.

Among them, the sensing module includes various sensors for sensing the vehicle's own status, such as posture, speed, acceleration sensors, etc.; it also includes sensors for sensing the local road network and surrounding traffic participant information, such as cameras and lidar. wait. The sensing module can also obtain vehicle chassis status information such as gear, speed, steering angle, etc. through the CANBUS bus.

The planning module can be any one of a single machine, a cluster, or a distributed server. In particular, the planning module may be an on-board controller of an autonomous vehicle. The planning module can perform dynamic planning of the path based on the digital world model to obtain a rough path solution, perform secondary optimization on the rough path solution to obtain the optimal path, and simultaneously perform dynamic planning on the speed and then perform secondary optimization to obtain the optimal speed. Finally, The optimal path and optimal speed are combined to generate a control trajectory.

The control module is the mechanical and electronic control unit of the autonomous vehicle, such as the body control unit, steering wheel and steering system, braking system, stability system, etc.

Figure 2 is a flow chart of a real-time trajectory planning method according to some embodiments of the present disclosure. In some embodiments, the real-time trajectory planning method is executed by the planning module in the system shown in Figure 1, and the real-time trajectory planning method includes the following steps:

S210: Obtain the basic navigation point information and pose information of the vehicle at the current moment, determine the target path reference line based on the basic navigation point information, and convert the pose information from the Cartesian coordinate system to Under the Fleener coordinate system, a lateral offset-displacement map and a rasterized map are obtained, where the pose information includes the vehicle's own status, surrounding traffic participants and local road network information.

Specifically, the digital world model perceived by the current vehicle in the Cartesian coordinate system is obtained, including the vehicle's own status, local road network, and information about surrounding traffic participants.

Based on the positioning information of the current frame of the vehicle, the basic navigation points 50m in front and 10m behind the vehicle are dynamically intercepted in real time, and the intercepted points are optimized. Here, the basic navigation points are mainly processed in three aspects: smoothness, similarity and compactness to obtain the target path reference line. The loss function corresponding to the optimization solution is constructed as follows:

Smoothing cost: ,

Compact cost: ,

Similar cost: ,

in, They are any three adjacent consecutive points of the target path reference point,/> It is the original navigation point corresponding to the target path reference point. Smoothing cost/> Measures the smoothness of the path reference line, compactness cost/> Measuring the degree of uniformity and similarity cost between path reference points/> Measure the deviation of the path reference line from the original navigation path.

The boundary constraints of the optimization solution are as follows:

,

in, are respectively the i-th path reference point, the original navigation point, and the upper and lower boundary points between the original navigation point and the path reference point.

The solution method uses the quadratic solver OSQP to solve the optimization problem. The optimization problem includes a loss function and boundary constraints to obtain a path reference point that satisfies the constraint conditions. It is smoother and more stable than the rough original navigation point, and provides a good basis for subsequent Planning module provides accurate reference.

This disclosure determines the target path reference line through multi-layer cost and smoothing processing of basic navigation points, solving the deviation of the model or search space, making the initial path of the dynamic planning reference line more stable and reliable.

Finally, based on the target reference line, various pose information (including boundaries) in the digital world model, such as self-vehicle pose Te, static and dynamic obstacles To, lane line point Tl and drivable area boundary point Tb Project from the Cartesian coordinate system to the Fleener coordinate system to obtain the lateral offset-displacement (SL) map, and rasterize the information in the map to obtain the rasterized map of the current frame.

S220: Determine the initial value of the path planning of the current vehicle through cost search and reinforcement learning model, smooth the initial value of the path planning to obtain the initial path planning, and perform multi-objective optimization on the initial path planning to obtain the optimal path planning.

Specifically, first, in the lateral offset-displacement (SL) diagram, the horizontal L is uniformly scattered, and the longitudinal S is non-uniformly scattered (here, the close and far distance method is used), and the adjacent sampling points in the sampling space are The path generation model between is as follows:

,

in, represents the horizontal distance; s represents the vertical distance,/> Fifth degree polynomial coefficients representing lateral distance: .

The cost calculation of each sampling node is constructed as follows:

Smoothing cost: ,

Offset cost: ,

Collision cost: , ,

in, Respectively, the vertical distance/> The corresponding lateral distance and its first, second and third order derivatives below,/> is the coordinate of the obstacle in the SL diagram,/> are the minimum and maximum cost distances from obstacles, /> The cost between distances adopts a linear relationship/> to express,/> is the slope, is the intercept term.

The cost value of each node is calculated recursively according to the above formula, and the initial value I of the path planning with the smallest cost in the discrete space is obtained through backtracking.

Then based on the current and past path-based three consecutive frames of state space information (grid map), the decision-making neural network model is used to make decisions about the actions that the own vehicle will perform at the current moment, and the decision-making action value of the own vehicle at the current moment is obtained (i.e. Path planning initial value II).

Here, the decision-making neural network model is obtained by training the neural network using the SAC reinforcement learning method. Decision action values include: horizontal continuous coordinate values of multiple path points. The training process of the decision neural network model includes the following steps:

1. Construct a state space: The state space is a three-frame continuous path-based laser raster map centered on the vehicle. The positions of the vehicle and surrounding obstacles are projected onto the raster map.

2. Construct an action space composed of horizontal continuous coordinate values of multiple path points. The scope of waypoint screening is a certain rectangular range in front of the vehicle, in which the longitudinal coordinate values of the waypoints are discretized, a fixed interval is maintained between the longitudinal direction points, and the transverse coordinate values of the waypoints are output by the decision-making neural network model.

3. Based on the generated rough path points, use Bezier curves to fit a smoother navigation path with continuous curvature.

4. Design reward function: The entire algorithm reward function considers four designs, the reward for reaching the target, the penalty for collision, the curvature of the output path, and the smooth migration between output paths.

5. Design various and complex simulation environments (including static scenes and complex scenes), based on reward function design, use SAC reinforcement learning method to train neural network models in the simulation environment, and obtain decision-making neural network models.

6. The entire training process of the decision-making neural network model adopts course learning, and the difficulty of the training environment ranges from simple to complex to accelerate the training of the decision-making neural network model and improve the robustness of the decision-making neural network.

The SAC reinforcement learning method is used to train the neural network model in the simulation environment. When the corresponding path action is applied to the own vehicle in the simulation environment, the control module ensures that the vehicle accurately follows the policy path, and at the same time calculates the corresponding reward, and then uses the reward adjustment The policy output corresponding to the corresponding status.

Finally, the same cost calculation is performed on the path planning initial value I obtained by searching in discrete space and the path planning initial value II output by reinforcement learning in continuous space. The cost function is designed as follows:

Fluctuation cost: ,

Curvature cost: ,

Smoothing cost, offset cost, collision cost: consistent with the aforementioned calculation method,

Among them, the fluctuation cost is considered The path of three consecutive frames in the past,/> They are respectively the lateral displacements corresponding to the current frame path and the i-th frame path at the same time, and the fluctuation cost measures the degree of fluctuation of the current path compared with the historical path. The curvature cost calculation considers that the sampling points are small enough and uniform,/> ,/> , They are any three adjacent consecutive points of the path point, and the curvature cost measures the degree of curvature between the path points.

Through the evaluation of the above cost information, the total cost of the two path planning initial values I and II is compared, and the path with a smaller cost is selected as the final path planning initial value.

Then, in order to further optimize the initial value of path planning and reduce the constraint dependence of subsequent continuous space optimization, primary optimization processing is performed on the initial value of path planning. The optimization target problem, loss function construction and boundary constraints are consistent with those in S210, and finally a Smooth and stable initial path planning.

Finally, based on the lateral offset-displacement (SL) diagram and the initial path planning with decision information, the quadratic optimization problem can be constructed as follows:

The target path equality constraint is:

,

in, for/> The corresponding lateral offset of the position and its first and second order derivatives, also/> for/> The lateral offset corresponding to the position and its first and second derivatives,/> is the adjacent longitudinal displacement increment. The constraint of this equation is that the piecewise acceleration is continuous, ensuring that/> The third derivative is a constant/> , thereby meeting vehicle comfort requirements.

The boundary inequality constraint of convex space is:

,

in, respectively/> The upper and lower limits of the convex space corresponding to the position.

The path boundary inequality constraints are:

,

in, as the starting point/> The corresponding lateral offset and its first and second derivatives, They are the minimum speed, minimum acceleration, maximum speed, and maximum acceleration of the vehicle's lateral offset, respectively.

The optimization goals are divided into the following three parts:

Reference line consistency goals:

,

in, is the vertical distance/> The corresponding lateral distance below measures the deviation of the target path from the reference line.

Rough explanation of consistency goals:

,

in, Rough solution for path in longitudinal distance/> The corresponding lateral distance below measures the deviation of the target path from the rough solution.

Smooth continuity target:

,

in, respectively/> The weight coefficients corresponding to the first, second and third derivatives of the lateral offset. This goal measures the continuity of the goal path.

Use a quadratic programming solver to solve this optimization problem, which includes the above-mentioned target path segmented acceleration continuity equation constraints, convex space boundary inequality constraints, and path boundary inequality constraints. Based on the reference line consistency and rough solution consistency And smooth continuity is the optimization goal, and the optimal path in the target space is solved. In particular, when the optimization solution is abnormal, the initial path plan is returned as the target path.

This disclosure solves the problem of selecting fine-grained space solutions through a method that combines cost search and reinforcement learning, and enhances the accuracy and security of the initial value of path planning; through a multi-objective optimization strategy based on accurate initial value of path planning, it solves This solves the problem of frequent parameter adjustment of the planning module in different driving scenarios, giving the system good ability to handle abnormal scenarios.

S230: Determine the optimal speed curve corresponding to the optimal path planning, obtain the target trajectory by combining the optimal path planning and the optimal speed curve, perform cost selection on the multi-lane target trajectory, and obtain the final control trajectory.

Specifically, first, a displacement-time (ST) diagram is constructed for the static and dynamic obstacles in the Fleener coordinate system. In the displacement-time (ST) diagram, the time T is uniformly dotted, and the longitudinal displacement S is non-uniformly Sprinkle points (the method of close and far is used here), taking into account the boundaries and obstacles in the space, and use the node cost (including speed, acceleration, acceleration and collision cost) search method to obtain the initial speed curve.

Similar to S220, the convex space of speed optimization is obtained from the initial speed curve, and the boundary constraint information is extracted in the convex space. The constraints here include the target speed curve segmented acceleration continuity equation constraint, the convex space boundary inequality constraint, and the variable boundary inequality constraint. ; The optimization goals include cruising speed, acceleration and acceleration, and the final optimal speed curve is solved using quadratic programming optimization.

Then based on the optimal speed curve, the optimal path planning is interpolated through the displacement (S) to obtain the speed and path information at each time (T _i ) to generate the target trajectory. Taking into account the state constraints of the self-vehicle, including speed, acceleration, curvature and other limit information, verify whether the target trajectory is within the range of the state limits.

Finally, through design cost calculation, the multi-lane target trajectory is selected to obtain the final control trajectory. in:

The single lane cost calculation is constructed as follows:

Lateral offset cost: ,

Lateral comfort cost: ,

Longitudinal comfort cost: ,

in, are the first and second derivatives of longitudinal displacement with respect to time respectively,/> For the planning cycle,/> It is the maximum longitudinal acceleration limit of the vehicle.

By calculating the cost information of each lane target trajectory in parallel, the target trajectory with the smallest cost is selected as the final output trajectory of the current frame.

Figure 3 is a schematic diagram of a real-time trajectory planning device according to some embodiments of the present disclosure. As shown in FIG. 3 , the real-time trajectory planning device 300 includes a determination module 310 , an optimization module 320 , and an acquisition module 330 . The real-time trajectory planning function can be executed by the planning module in the system shown in Figure 1. in:

The determination module 310 is used to obtain the basic navigation point information and pose information of the vehicle at the current moment, determine a target path reference line based on the basic navigation point information, and convert the pose information into a Cartesian model based on the target path reference line. The coordinate system is converted to the Fleener coordinate system to obtain a lateral offset-displacement map and a rasterized map, where the pose information includes the vehicle's own status, surrounding traffic participants and local road network information;

The optimization module 320 is used to determine the initial value of the current vehicle's path planning through cost search and reinforcement learning models, smooth the initial value of the path planning to obtain the initial path planning, and perform multi-objective optimization on the initial path planning to obtain the optimal path planning. Optimal path planning;

The acquisition module 330 is used to determine the optimal speed curve corresponding to the optimal path planning, obtain the target trajectory by combining the optimal path planning and the optimal speed curve, perform cost selection on the multi-lane target trajectory, and determine the final control trajectory.

One embodiment of the present disclosure provides a real-time trajectory planning device. As shown in Figure 4, the real-time trajectory planning device 400 includes a memory 420 and a processor 410. The memory 420 is used to store a computer program; the processor 410 is used to implement the figure when executing the computer program. Methods described in S210-S230 in 2.

To sum up, unlike the existing technology that directly obtains a rough solution of the trajectory through reinforcement learning and then optimizes the optimal trajectory, the real-time trajectory planning method, device and equipment provided by each embodiment of the present disclosure uses reinforcement learning to obtain a rough solution for the path. And fuse it with the rough solution obtained by the cost search to obtain the rough solution of the target path, then optimize it to obtain the target path, and finally merge it with the speed optimized result to form the target trajectory. Because the target path reference line is determined through multi-layer cost and smoothing processing of basic navigation points, the deviation of the model or search space is solved, making the initial path of the reference line of dynamic programming more stable and reliable; through the method of combining cost search and reinforcement learning , which solves the problem of selecting fine-grained spatial solutions and enhances the accuracy and safety of the initial value of path planning; through the multi-objective optimization strategy based on accurate initial value of path planning, it solves the problem of frequent parameter adjustment of the planning module in different driving scenarios. , so that the system has good abnormal scene processing capabilities.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described equipment and modules can be referred to the corresponding descriptions in the foregoing device embodiments, and will not be described again here.

Although the subject matter described herein is provided in the general context of being performed in conjunction with an operating system and application programs executing on a computer system, those skilled in the art will recognize that other types of program modules may also be performed in conjunction with other types of programs. accomplish. Generally speaking, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will appreciate that the subject matter described herein may be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like , may also be used in distributed computing environments where tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each specific application, but such implementations should not be considered to be beyond the scope of this disclosure.

It should be understood that the above-described specific embodiments of the present disclosure are only used to illustrate or explain the principles of the present disclosure, and do not constitute a limitation of the present disclosure. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present disclosure should be included in the protection scope of the present disclosure. Furthermore, the appended claims of the present disclosure are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or equivalents of such scopes and boundaries.

Claims (9)

1. A method of real-time trajectory planning, comprising:

basic navigation point information and pose information of a vehicle at the current moment are obtained, a target path reference line is determined based on the basic navigation point information, the pose information is converted from a Cartesian coordinate system to a Flunar coordinate system based on the target path reference line, and a transverse offset-displacement map and a rasterization map are obtained, wherein the pose information comprises the state of the vehicle, surrounding traffic participants and local road network information;

determining a path planning initial value of a current vehicle through cost searching and reinforcement learning models, smoothing the path planning initial value to obtain an initial path plan, and performing multi-objective optimization on the initial path plan to obtain an optimal path plan;

determining an optimal speed curve corresponding to the optimal path planning, combining the optimal path planning and the optimal speed curve to obtain a target track, and carrying out cost selection on the multi-lane target track to obtain a final control track;

wherein, the determining the path planning initial value of the current vehicle through the cost searching and reinforcement learning model comprises:

sampling along the target path reference line, calculating the cost value of the preset cost of each sampling node, and forming a first path planning initial value corresponding to the preset cost by the sampling nodes with the minimum sum of cost values;

according to the three continuous frames of the grid map based on the target path reference line in the current and past, utilizing a decision neural network model to make a decision of the action to be executed by the vehicle at the current moment, and obtaining a decision action value of the vehicle at the current moment, wherein the decision action value forms a second path planning initial value;

and calculating the cost value of the preset cost for the second path planning initial value, comparing the cost value with the cost value of the first path planning initial value corresponding to the preset cost, and taking the smaller one of the second path planning initial value and the first path planning initial value as the path planning initial value of the current vehicle.

2. The method of claim 1, wherein the determining a target path reference line based on the base navigation point information comprises:

acquiring basic navigation points in a first preset distance in front and a second preset distance in rear in real time according to the positioning information of the current frame of the vehicle;

determining a loss function and a boundary constraint condition of the target path reference line, wherein the loss function comprises a smooth cost, a compact cost and a similar cost, the smooth cost is used for measuring the smoothness of the target path reference line, the compact cost is used for measuring the uniformity degree between target path reference points corresponding to the basic navigation points, and the similar cost is used for measuring the deviation degree of the target path reference line and an original navigation path;

and obtaining the basic navigation points meeting the boundary constraint condition based on a quadratic solver, wherein the path reference points corresponding to the basic navigation points form the target path reference line.

3. The method of claim 1, wherein the training method of the decision neural network model comprises:

constructing a state space from a path-based three-frame continuous grid map centered on a vehicle, wherein the positions of the vehicle and surrounding obstacles are projected onto the grid map;

constructing an action space formed by transverse continuous coordinate values of a plurality of path points sampled in a preset rectangular range in front of the vehicle, wherein the longitudinal intervals of the path points are kept at fixed intervals, and the transverse coordinate values are determined by a decision neural network model;

designing a reward function based on rewards to reach the target, penalties for collision occurrence, curvature of the output paths, and smooth migration between the output paths;

constructing a training set by taking the state space as input and the action space as output;

designing a simulation environment comprising a static scene and a complex scene, training the neural network model for the training set by adopting a SAC reinforcement learning method based on the reward function in the simulation environment, and obtaining the decision neural network model.

4. The method of claim 1, wherein said smoothing the path plan initial value to obtain an initial path plan comprises:

taking the loss function and the boundary constraint condition of the target path reference line as the loss function and the boundary constraint condition of the path planning initial value;

and optimizing the path planning initial value based on a quadratic solver to obtain the initial path planning.

5. The method of claim 1, wherein said multi-objective optimizing said initial path plan to obtain an optimal path plan comprises:

determining a boundary constraint condition of the path based on the transverse offset-displacement diagram and the path planning initial value, and performing multi-objective optimization on the initial path planning based on the boundary constraint condition and a preset optimization objective to obtain an optimal path planning in a target space, wherein the preset optimization objective comprises one or more of a deviation degree of a target path from a target path reference line, a deviation degree of the target path from the path planning initial value and continuity of the target path;

and when the optimal path planning in the target space solves for abnormality, taking the initial path planning as the optimal path planning.

6. The method of claim 1, wherein the determining an optimal speed profile corresponding to the optimal path plan comprises:

constructing a displacement-time diagram for static obstacles and dynamic obstacles of the vehicle under a flener coordinate system;

sampling the displacement-time diagram, calculating the cost value of preset cost of each sampling node, and forming an initial speed curve corresponding to the preset cost by the sampling nodes with the minimum sum of cost values, wherein the preset cost comprises speed cost, acceleration adding cost and collision cost;

obtaining a convex space of speed optimization from the initial speed curve, extracting boundary constraint conditions in the convex space, and performing multi-objective optimization on the initial speed curve based on the boundary constraint conditions and a preset optimization target to obtain an optimal speed curve, wherein the preset optimization target comprises one or more of cruising speed, acceleration and acceleration.

7. The method of claim 1, wherein the combining the optimal path plan and the optimal speed profile to obtain a target trajectory comprises:

and interpolating the optimal path planning through displacement based on the optimal speed curve to obtain speed and path information at each moment so as to generate a target track.

8. A real-time trajectory planning device, comprising:

the determining module is used for acquiring basic navigation point information and pose information of the vehicle at the current moment, determining a target path reference line based on the basic navigation point information, converting the pose information from a Cartesian coordinate system to a Flunar coordinate system based on the target path reference line, and acquiring a transverse offset-displacement map and a rasterized map, wherein the pose information comprises the state of the vehicle, surrounding traffic participants and local road network information;

the optimizing module is used for determining a path planning initial value of the current vehicle through cost searching and reinforcement learning models, smoothing the path planning initial value to obtain an initial path plan, and carrying out multi-objective optimization on the initial path plan to obtain an optimal path plan;

the acquisition module is used for determining an optimal speed curve corresponding to the optimal path planning, combining the optimal path planning and the optimal speed curve to obtain a target track, and carrying out cost selection on the multi-lane target track to acquire a final control track;

wherein, the determining the path planning initial value of the current vehicle through the cost searching and reinforcement learning model comprises:

sampling along the target path reference line, calculating the cost value of the preset cost of each sampling node, and forming a first path planning initial value corresponding to the preset cost by the sampling nodes with the minimum sum of cost values;

according to the three continuous frames of the grid map based on the target path reference line in the current and past, utilizing a decision neural network model to make a decision of the action to be executed by the vehicle at the current moment, and obtaining a decision action value of the vehicle at the current moment, wherein the decision action value forms a second path planning initial value;

and calculating the cost value of the preset cost for the second path planning initial value, comparing the cost value with the cost value of the first path planning initial value corresponding to the preset cost, and taking the smaller one of the second path planning initial value and the first path planning initial value as the path planning initial value of the current vehicle.

9. A real-time trajectory planning device comprising a memory and a processor:

the memory is used for storing a computer program;

the processor being adapted to implement the method according to any of claims 1-7 when executing the computer program.