CN112590774B - A deep reinforcement learning-based drift storage control method for smart electric vehicles - Google Patents

Abstract

The invention relates to a deep reinforcement learning-based drift storage control method for an intelligent electric vehicle, comprising the following steps: 1) constructing a vehicle dynamics model for deep reinforcement learning and a tire model under the tire force saturation condition; 2) using The TD3 algorithm for drift warehousing control realizes the drift warehousing of intelligent electric vehicles. Compared with the prior art, the present invention has high control precision and good robustness, and can make the vehicle accurately complete the drifting and warehousing action, and during the drifting process, the vehicle can accurately reach the warehouse position by continuously adjusting the steering wheel angle, and when the vehicle During the drifting process, the center position of the storage location can be changed actively, so that the vehicle can drift to the updated storage location.

Description

A deep reinforcement learning-based drift storage control method for smart electric vehicles

technical field

The invention relates to the field of vehicle storage control, in particular to a deep reinforcement learning-based drift storage control method for an intelligent electric vehicle.

Background technique

The vehicle continues to drive in a state where the rear tires are saturated and the rear axle slips, which is called drift. There are two different drift states:

(1) The rear axle is driven and the rear wheel is slipping. At this time, the vehicle mass center slip angle and vehicle speed can be kept at a constant value by controlling the rear axle driving force and the steering angle of the front wheel, so that the vehicle is in a stable state. Most cars are driven by the front axle, so the research value of drift action in this state is relatively small.

(2) Reproducing the drift action according to the open-loop control law may be disturbed by the external environment and the state of the vehicle, so that the vehicle cannot drift into the storage space. When the drift action is triggered, the preset drift trigger pose state is not fully satisfied, and there is a certain deviation. According to the open-loop controller to complete the drift action, the deviation will be retained until the end of the drift. In addition, due to the response limit of the underlying actuator, the open-loop control Under this circumstance, it is impossible to guarantee the consistent response of each actuator. When the response deviates, the vehicle will deviate from the preset drift trajectory; uneven road surface causes sudden changes in tire force during the drift process, which changes the drift path.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a kind of intelligent electric vehicle drift storage control method based on deep reinforcement learning in order to overcome the above-mentioned defects in the prior art. Implement, design a drift controller, adjust the steering wheel angle according to the relative position between the vehicle and the storage space and vehicle state parameters, so that the vehicle drifts into the storage space.

The object of the present invention can be realized through the following technical solutions:

1. a kind of intelligent electric vehicle drift storage control method based on deep reinforcement learning, is characterized in that, comprises the following steps:

1) Build a vehicle dynamics model for deep reinforcement learning and a tire model under tire force saturation conditions;

2) The TD3 algorithm for drift warehousing control is used to realize the drift warehousing of intelligent electric vehicles.

In the described step 1), the vehicle dynamics model is specifically a four-wheel, three-degree-of-freedom vehicle dynamics _model that considers front-rear and left-right load transfer. and the yaw angular velocity ω.

In the four-wheel three-degree-of-freedom vehicle dynamics model, the expression of the four-wheel vertical force considering the longitudinal and lateral acceleration is:

In the formula, h _m is the height of the center of mass, b _f and _br are the front and rear wheelbases, a _x and a _y are the longitudinal and lateral accelerations at the center of mass without considering the influence of body rotation, F _zFL , F _zFR , F _zRL , F _zRR are the vertical forces of the left front, right front, left rear, and right rear wheels, respectively, m is the mass of the electric vehicle, g is the acceleration of gravity, l is the wheelbase, l _f , l _r are the distances from the front and rear axles to the center of mass, F _xFL , F _xFR , F _xRL , and F _xRR are the longitudinal forces of the left front, right front, left rear, and right rear wheels, respectively, and F _yFL , F _yFR , F _yRL , and F _yRR are the left front, right front, left rear, and right rear wheels, respectively The lateral force, δ is the front wheel angle.

During the drifting process, considering that a certain wheel is lifted off the ground due to the excessive load transfer, the vertical load of the wheel is reduced to 0 and the load transfer reaches the upper limit. When the steering wheel is turned to the left and the load is transferred to the right, When the left rear wheel is off the ground, the vertical force of the left rear wheel is 0. At this time, the excessively transferred load is redistributed to the left front wheel and the right rear wheel according to the longitudinal and lateral acceleration, wheelbase and wheel distance, then Have:

ΔF _trans = |F _zRL |

F' _zRL = 0

Among them, ΔF _trans is the excessively transferred load, F′ _zRL is the vertical force of the left rear wheel after distribution, F′ _zRR is the vertical force of the right rear wheel after distribution, and F′ _zFL is the vertical force of the left front wheel after distribution to force.

The force analysis of the four-wheel three-degree-of-freedom vehicle dynamics model considering the load transfer between front and rear and left and right is carried out, and the vehicle dynamic balance equation is obtained as:

φ=β+ψ

According to this calculation, the longitudinal vehicle speed v _mx and the lateral vehicle speed v _my are obtained, then:

v _mx = v _m ·cosβ

v _my = v _m ·sinβ

为车辆质心处速度的变化率，

为质心侧偏角速度，φ为质心处车速全局方位角，

为质心处车速全局方位角速度，

为横摆角速度的变化率，ψ为车头全局方位角，I_z为横摆转动惯量，v_x为车辆纵向车速，v_y为车辆侧向车速。in,

is the rate of change of the velocity at the center of mass of the vehicle,

is the side-slip angular velocity of the centroid, φ is the global azimuth of the vehicle speed at the centroid,

is the global azimuth velocity of the vehicle speed at the center of mass,

is the rate of change of the yaw rate, ψ is the global azimuth angle of the front of the vehicle, I _z is the yaw moment of inertia, v _x is the longitudinal speed of the vehicle, and v _y is the lateral speed of the vehicle.

In the step 1), the tire model used for deep reinforcement learning training includes a front tire force model and a rear tire force model.

For the tire force model of the rear wheel, during the drifting process, the rear wheel brake locks and there is pure friction on the road surface, and the direction of the tire force of the rear wheel is opposite to the direction of the instantaneous speed of the wheel center. The expression of the longitudinal and lateral tire force components of the rear wheels is:

For the left rear wheel:

For the right rear wheel:

F _{r_sat} = μ ₁ F _z

Among them, v _xRL and v _yRL are the longitudinal and lateral speeds at the center of the left rear wheel, respectively, v _xRR and _vyRR are the longitudinal and lateral speeds at the center of the right rear wheel, respectively, and λ _L and λ _R are the left and right velocities, respectively. Rear wheel center slip angle, F _xRL and F _yRL are the longitudinal and lateral forces of the left rear wheel, respectively, F _xRR and F _yRR are the longitudinal and lateral forces of the right rear wheel, respectively, and F _{rRL_sat} and F _{rRR_sat} are the left and right forces, respectively The horizontal saturated tire force of the rear wheel, F _{r_sat} represents the horizontal saturated tire force of the corresponding wheel, μ ₁ is the road surface utilization adhesion coefficient when the wheel is locked, and F _z represents the vertical force of the corresponding wheel.

For the front tire force model, during the drifting process, the front tire force is not saturated, then the improved Burckhardt tire model is used to fit the tire force to express the relationship between the lateral force and the slip angle, as follows:

Among them, θ ₁ to θ ₅ are fitting parameters, and α is the front wheel slip angle;

The left wheel slip angle α _L and the right wheel slip angle α _R can be obtained by the following formulas:

Since the front wheel does not apply braking force and driving force, it is in a free rolling state, with F _xFL = 0, F _xFR = 0, only the lateral force is considered when determining the tire force direction of the front wheel, then the tire force direction of the front wheel is perpendicular to the tire The plane is determined by the steering angle of the front wheels.

Described step 2) specifically comprises the following steps:

21) Design TD3 algorithm for drift warehousing control, build Actor network and Critic network, specifically:

车辆质心处速度v_m、质心侧偏角β以及横摆角速度ω，所述的动作为方向盘转角；Both the Critic network and the Actor network are BP neural networks composed of fully connected layers. The input of the Critic network is the vehicle state and action, and the output is the Q value. The input of the Actor network is the vehicle state and the output is the action. The vehicle state is The parameters that characterize the state of the vehicle during the drift process, including the location coordinates (e _x , e _y ) and the location orientation in the relative coordinate system with the center of mass of the vehicle as the origin and the vehicle's head orientation as the positive direction of the y-axis

the velocity _vm at the center of mass of the vehicle, the side-slip angle β of the center of mass and the yaw angular velocity ω, the actions are the steering wheel angle;

22) Construct the reward function r(k), then there are:

分别为e_x、e_y和

的权重，k为时间；Among them, w _x , w _y ,

are e _x , e _y and

The weight of , k is time;

23) Train the Actor network and the Critic network, and complete the drift storage of smart electric vehicles accordingly.

In step 23), before the Actor network and the Critic network are trained, the boundary of the drift storage controller is determined first, and the target location of each vehicle drift is randomly selected according to the boundary. In the iterative training, the vehicle The vehicle status is calculated with the randomly selected target location and orientation, and the Critic network and the Actor network are trained accordingly. By randomly updating the target location during the training process, the training data set is expanded and the ability to improve.

Compared with the prior art, the present invention has the following advantages:

1. Based on the deep reinforcement learning TD3 algorithm, a control method of intelligent electric vehicle drift storage is designed, which improves the control accuracy, overcomes the problem of drift storage error caused by uneven road surface, and can also change the center point of the storage location. , so that the vehicle moves to the updated warehouse position, which improves the robustness of the control system.

2. During the process of drifting into the warehouse, the vehicle can adjust its posture by continuously adjusting the angle of the steering wheel, so that the vehicle can accurately drift into the warehouse.

Description of drawings

FIG. 1 is a flow chart of the method of the present invention.

Figure 2 is a schematic diagram of the definition of some state parameters in the drift process.

Figure 3 shows the flow of the drift control algorithm based on deep reinforcement learning.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

As shown in FIG. 1 , the present invention provides a deep reinforcement learning-based intelligent electric vehicle drift storage control method, which includes the following steps:

1) Build a vehicle dynamics model and a tire model for deep reinforcement learning training, including the following steps:

11) Building a vehicle dynamics model for deep reinforcement learning

A four-wheel three-degree-of-freedom vehicle dynamics model considering the load transfer between front and rear and left and right, the three degrees of freedom are respectively the velocity of the vehicle's center of mass v _m , the center of mass slip angle β, and the yaw angular velocity ω;

Since the longitudinal and lateral accelerations of the vehicle are large during the drifting process, the influence of the vehicle's front and rear and left and right load transfer on the vertical force of the tire must be considered. The four-wheel vertical force calculation formula considering the longitudinal and lateral acceleration is as formula (1):

In the formula, h _m is the height of the center of mass, b _f and _br are the front and rear wheelbases, and a _x and a _y are the longitudinal and lateral accelerations at the center of mass without considering the influence of the body rotation, which can be obtained from formula (2):

During the drifting process, it is necessary to consider the situation that a certain wheel is lifted off the ground due to the excessive load transfer, so that the vertical load of the corresponding wheel is reduced to 0, and the load transfer reaches the upper limit. Since it is a tail-flick braking process, the load is transferred to the front axle, so only the possibility of the rear wheel getting off the ground is considered. Assuming that the steering wheel drifts to the left and the load shifts to the right, the left rear wheel may leave the ground. When F _zRL <0 calculated according to the formula, let the vertical force of the wheel be 0, and redistribute the excessively transferred load to the left front wheel and the right rear wheel according to the longitudinal and lateral acceleration, wheelbase and wheelbase, the formula Expressed as:

The force analysis of the vehicle model is carried out, and the dynamic balance equation of the vehicle is obtained as:

为速度方向变化率；ψ为车头全局方位角，

为车辆横摆角速度，即车头方向变化率；根据φ＝β+ψ和对上式积分，即可得到各时刻车辆的质心速度v_m、质心侧偏角β和横摆角速度ω，再根据式(8)求得车辆纵侧向车速：In the formula, δ is the front wheel rotation angle; φ is the global azimuth angle of the vehicle speed at the center of mass,

is the rate of change of the speed direction; ψ is the global azimuth of the head of the vehicle,

is the yaw rate of the vehicle, that is, the rate of change of the head direction; according to φ=β+ψ and the integral of the above formula, the center of mass velocity v _m , the side-slip angle of the center of mass β and the yaw rate ω of the vehicle at each moment can be obtained, and then according to the formula (8) Obtain the longitudinal and lateral speed of the vehicle:

12) Build a tire model for deep reinforcement learning under tire force saturation conditions

Different from the normal driving conditions, when drifting, the tire force of the rear wheel is saturated, the lateral speed of the vehicle body and the side-slip angle of the center of mass are large, and the longitudinal and lateral vehicle speeds are in a state of rapid change, so the vehicle system is a strong nonlinear at this time. , the time-varying system of the longitudinal side height coupling, the real-time saturated tire force of the vehicle can be obtained by formula (9):

F _{r_sat} = μ ₁ F _z (9)

为水平轮胎力合力，μ₁为车轮抱死时路面利用附着系数，μ₁＝0.9μ_max，即为0.9倍的峰值附着系数，峰值附着系数为1。In the formula,

is the resultant horizontal tire force, μ ₁ is the road surface adhesion coefficient when the wheel is locked, μ ₁ =0.9 μ _max , which is 0.9 times the peak adhesion coefficient, and the peak adhesion coefficient is 1.

121) Rear tire force model

In the process of rear axle locking and braking, the tire force is saturated. No matter how the sideslip angle changes, the resultant force of the longitudinal and lateral forces remains unchanged, which means that the change of the sideslip angle can be ignored when calculating the horizontal tire force of the rear wheel during the drifting process. Directly find the tire force of the rear axle in the drift state.

Since the rear wheel is locked, the wheels are purely rubbed on the road surface, so the direction of the tire force is determined by the direction of the wheel center speed, that is, the direction of the tire force is opposite to the direction of the instantaneous speed of the wheel center. By analyzing the force of the rear wheel during the drift process, the expression of the longitudinal and lateral tire force components of the rear wheel can be obtained:

Left rear wheel:

Right rear wheel:

In the formula, v _xRL and v _yRL are the longitudinal and lateral speeds at the wheel center of the left rear wheel, respectively, v _xRR , v _yRR are the longitudinal and lateral speeds at the wheel center of the right rear wheel, respectively; λ _L , λ _R are the left, Right rear wheel center slip angle; F _xRL and F _yRL are the longitudinal and lateral forces of the left rear wheel, respectively, F _xRR and F _yRR are the longitudinal and lateral forces of the right rear wheel, respectively; F _{rRL_sat} and F _{rRR_sat} are the left, The right rear wheel level saturated tire force.

122) Front tire force model

The tire force of the front wheel is not saturated, so the longitudinal side is decoupled, and the tire lateral force is obtained by using a tire model suitable for quasi-static conditions. The tire force is fitted by the improved Burckhardt tire model, and the relationship between the lateral force and the slip angle is expressed as follows:

In the formula, θ ₁ to θ ₅ are fitting parameters, α is the front wheel slip angle, and the left and right wheel slip angles can be obtained by formulas (13) and (14).

Since no braking force and driving force are applied, it is considered that the front wheel is in a free rolling state, and the longitudinal force of the wheel is approximately 0, that is, F _xFL =0, F _xFR =0. Only the lateral force is considered when determining the tire force direction of the front wheel, so the force direction of the front wheel tire is perpendicular to the tire plane and is determined by the steering angle of the front wheel.

2) TD3 algorithm design for drift storage control.

In the drifting process, the deep reinforcement learning algorithm is used, based on the built drifting vehicle dynamics model, and according to the end-to-end drift controller, the vehicle can be accurately drifted into the warehouse, specifically:

In the TD3 algorithm, the input of the Critic network is the vehicle state and action, and the output is the Q value; the input of the Actor network is the vehicle state, and the output is the action, that is, the steering wheel angle;

车辆的纵侧向车速的合速度v_m、质心侧偏角β以及横摆角速度ω。e_x、e_y和

反应了漂移过程中车辆当前时刻位置和航向角与期望状态之差，如图2所示，v_m、β和ω表征前三者的变化率。The parameters that characterize the vehicle state during the drift process are selected as the input of the Critic network and the Actor network. This group of parameters should be able to uniquely represent the vehicle state at a certain moment during the drift process, and there is a dynamic correlation with the input value of the steering wheel angle. The 6 state parameters are: the warehouse location coordinates e _x , e _y and the warehouse location orientation in the relative coordinate system with the center of mass of the vehicle as the origin and the vehicle head orientation as the positive direction of the y-axis

The resultant velocity _vm of the vehicle's longitudinal and lateral vehicle speeds, the center of mass slip angle β, and the yaw angular velocity ω. e _x , e _y and

It reflects the difference between the vehicle's current position and heading angle and the expected state during the drift process. As shown in Figure 2, v _m , β and ω represent the rate of change of the first three.

After determining the deep neural network trained by the reinforcement learning algorithm, the reward function is designed to calculate the reward value corresponding to the different states of the vehicle during the drifting process. The reward function is designed as follows:

分别为e_x、e_y和

的权重。由于所关注的是车辆停稳时与库位中心的位移误差和与库位朝向的航向角误差，因此将车速的三次方放在分母项，可以使得当车辆车速越低、越接近停止时，其纵侧向位移误差和航向角误差所计算得到的奖励值绝对值越大。根据算法原理，当车辆最终停在远离库位的位置，会计算得到一个很小的奖励值；而当车辆停在库位中心附近时，计算得到的奖励值接近于0，使前序状态和动作对应的目标Q值较大。Actor网络在根据车辆状态计算方向盘转角时会尽可能使Q值最大，使车辆最终停入库位。在进行奖励函数的设计时，应将被控量放入奖励函数中，但是在漂移入库的过程中，方向盘转角一直在调整，这是一个连续的过程，无法界定其中一次的转向对漂移入库的结果影响，所以权重系数置0。In the formula, w _x , w _y ,

are e _x , e _y and

the weight of. Since the focus is on the displacement error from the center of the storage location and the heading angle error from the orientation of the storage location when the vehicle is stationary, the cube of the vehicle speed is placed in the denominator term, so that when the vehicle speed is lower and closer to stopping, the The greater the absolute value of the reward value calculated by the longitudinal and lateral displacement error and the heading angle error. According to the algorithm principle, when the vehicle is finally parked far from the storage space, a small reward value will be calculated; when the vehicle is parked near the center of the storage space, the calculated reward value will be close to 0, making the pre-order state and the The target Q value corresponding to the action is larger. The Actor network will try to maximize the Q value when calculating the steering wheel angle based on the vehicle state, so that the vehicle will eventually park in the storage space. When designing the reward function, the controlled quantity should be put into the reward function, but in the process of drifting into the library, the steering wheel angle has been adjusted. This is a continuous process, and it is impossible to define one of the steering pairs. The result of the library is affected, so the weight coefficient is set to 0.

Before network training, the "boundary" of the vehicle-mounted drift storage controller is first determined, and it is believed that no matter what steering wheel angle is applied, the vehicle's final position and final heading angle will not exceed this boundary.

According to the boundary of the controller, the target location of each vehicle drift is randomly selected. When a complete drift process is over, set the random target location (X _aim , Y _aim ) and direction ψ _aim , and satisfy the constraints of the controller boundary above.

依此对Critic网络和Actor网络进行训练，通过在训练过程中随机更新目标库位位置，拓展了训练数据集，可以提升网络的泛化能力。In iterative training, the vehicle calculates the vehicle state _{ex, e y and}

Based on this, the Critic network and the Actor network are trained, and the training data set is expanded by randomly updating the target location during the training process, which can improve the generalization ability of the network.

Example

In this embodiment, the control method for drift storage implemented according to the above method is specifically:

Step 1: Build a four-wheel, three-degree-of-freedom vehicle dynamics model based on deep reinforcement learning for drift storage and build a tire model under tire force saturation conditions. A four-wheel, three-degree-of-freedom vehicle dynamics model considering front-to-rear and left-right load transfer. The three degrees of freedom are: the velocity of the vehicle's center of mass v _m , the side-slip angle of the center of mass β, and the yaw rate ω.

Step 2: Design the Critic network and Actor network, and design the reward function based on the vehicle dynamics model of deep reinforcement learning. The input of the Critic network is the vehicle state and action, and the output is the Q value; the input of the Actor network is the vehicle state and the output is the action. The number of inputs and outputs is small, and the corresponding relationship is relatively simple. The BP neural network composed of fully connected layers is used to construct the Critic network and the Actor network. The flow of the drift control algorithm based on deep reinforcement learning is shown in Figure 3.

Claims (1)

1. a kind of intelligent electric vehicle drift storage control method based on deep reinforcement learning, is characterized in that, comprises the following steps: 1) Build a vehicle dynamics model for deep reinforcement learning and a tire model under tire force saturation conditions. The vehicle dynamics model is a four-wheel, three-degree-of-freedom vehicle dynamics model that considers front-rear and left-right load transfer. The described The three degrees of freedom include the vehicle mass center velocity v _m , the mass center slip angle β and the yaw angular velocity ω. In the four-wheel three-degree-of-freedom vehicle dynamics model, the expression of the four-wheel vertical force considering the longitudinal and lateral acceleration is:

In the formula, h _m is the height of the center of mass, b _f and _br are the front and rear wheelbases, a _x and a _y are the longitudinal and lateral accelerations at the center of mass without considering the influence of body rotation, F _zFL , F _zFR , F _zRL , F _zRR are the vertical forces of the left front, right front, left rear, and right rear wheels, respectively, m is the mass of the electric vehicle, g is the acceleration of gravity, l is the wheelbase, l _f , l _r are the distances from the front and rear axles to the center of mass, F _xFL , F _xFR , F _xRL , and F _xRR are the longitudinal forces of the left front, right front, left rear, and right rear wheels, respectively, and F _yFL , F _yFR , F _yRL , and F _yRR are the left front, right front, left rear, and right rear wheels, respectively , δ is the front wheel rotation angle; During the drifting process, considering that a certain wheel is lifted off the ground due to the excessive load transfer, the vertical load of the wheel is reduced to 0 and the load transfer reaches the upper limit. When the steering wheel is turned to the left and the load is transferred to the right, When the left rear wheel is off the ground, the vertical force of the left rear wheel is 0. At this time, the excessively transferred load is redistributed to the left front wheel and the right rear wheel according to the longitudinal and lateral acceleration, wheelbase and wheel distance, then Have: ΔF _trans = |F _zRL | F' _zRL = 0

Among them, ΔF _trans is the excessively transferred load, F′ _zRL is the vertical force of the left rear wheel after distribution, F′ _zRR is the vertical force of the right rear wheel after distribution, and F′ _zFL is the vertical force of the left front wheel after distribution to force; The force analysis of the four-wheel three-degree-of-freedom vehicle dynamics model considering the load transfer between front and rear and left and right is carried out, and the vehicle dynamic balance equation is obtained as:

φ=β+ψ

According to this calculation, the longitudinal vehicle speed v _mx and the lateral vehicle speed v _my are obtained, then: v _mx = v _m ·cosβ v _my = v _m ·sinβ in,

is the rate of change of the velocity at the center of mass of the vehicle,

is the side-slip angular velocity of the centroid, φ is the global azimuth of the vehicle speed at the centroid,

is the global azimuth velocity of the vehicle speed at the center of mass,

is the rate of change of the yaw angular velocity, ψ is the global azimuth angle of the front of the vehicle, I _z is the yaw moment of inertia, v _x is the longitudinal speed of the vehicle, and v _y is the lateral speed of the vehicle; The tire models used for deep reinforcement learning training include the front tire force model and the rear tire force model. For the rear tire force model, during the drifting process, the rear wheel brakes lock and there is pure friction on the road surface, and the rear wheel’s The direction of the tire force is opposite to the direction of the instantaneous speed of the wheel center. Through the force analysis of the rear wheel, the expression of the longitudinal and lateral tire force component of the rear wheel is obtained as: For the left rear wheel:

For the right rear wheel:

F _{r_sat} = μ ₁ F _z Among them, v _xRL and v _yRL are the longitudinal and lateral speeds at the center of the left rear wheel, respectively, v _xRR and _vyRR are the longitudinal and lateral speeds at the center of the right rear wheel, respectively, and λ _L and λ _R are the left and right velocities, respectively. Rear wheel center slip angle, F _xRL and F _yRL are the longitudinal and lateral forces of the left rear wheel, respectively, F _xRR and F _yRR are the longitudinal and lateral forces of the right rear wheel, respectively, and F _{rRL_sat} and F _{rRR_sat} are the left and right forces, respectively The horizontal saturated tire force of the rear wheel, F _{r_sat} represents the horizontal saturated tire force of the corresponding wheel, μ ₁ is the adhesion coefficient of the road surface when the wheel is locked, and F _z represents the vertical force of the corresponding wheel; For the front tire force model, during the drifting process, the front tire force is not saturated, then the improved Burckhardt tire model is used to fit the tire force to express the relationship between the lateral force and the slip angle, as follows:

Among them, θ ₁ to θ ₅ are fitting parameters, and α is the front wheel slip angle; The left wheel slip angle α _L and the right wheel slip angle α _R can be obtained by the following formulas:

Since the front wheel does not apply braking force and driving force, it is in a free rolling state, with F _xFL = 0, F _xFR = 0, only the lateral force is considered when determining the tire force direction of the front wheel, then the tire force direction of the front wheel is perpendicular to the tire The plane is determined by the steering angle of the front wheel; 2) Adopt the TD3 algorithm for drift storage control to realize the drift storage of intelligent electric vehicles, which includes the following steps: 21) Design TD3 algorithm for drift warehousing control, build Actor network and Critic network, specifically: Both the Critic network and the Actor network are BP neural networks composed of fully connected layers. The input of the Critic network is the vehicle state and action, and the output is the Q value. The input of the Actor network is the vehicle state and the output is the action. The vehicle state is The parameters that characterize the state of the vehicle during the drift process, including the location coordinates (e _x , e _y ) and the location orientation in the relative coordinate system with the center of mass of the vehicle as the origin and the vehicle's head orientation as the positive direction of the y-axis

the velocity _vm at the center of mass of the vehicle, the side-slip angle β of the center of mass and the yaw angular velocity ω, the actions are the steering wheel angle; 22) Construct the reward function r(k), then there are:

Among them, w _x , w _y ,

are e _x , e _y and

The weight of , k is time; 23) Train the Actor network and the Critic network, and complete the drift storage of the smart electric vehicle accordingly. Before training the Actor network and the Critic network, determine the boundary of the drift storage controller, and according to the boundary, each vehicle drifts In the iterative training, the vehicle calculates the vehicle state with the randomly selected target location and orientation, and trains the Critic network and the Actor network accordingly, and randomly updates the target during the training process. Warehouse location, expand the training data set, and improve the ability of transformation.

Images