CN117445941A - Track deviation early warning method, terminal and storage medium - Google Patents

Abstract

The embodiment of the application discloses a track deviation early warning method, a terminal and a storage medium, wherein the track deviation early warning method is applied to the terminal, the terminal is arranged in a target vehicle, and when the target vehicle runs, the terminal acquires real-time corner information of the target vehicle and real-time image information in front of a road; carrying out driving track prediction processing based on a wavelet analysis method and real-time corner information to obtain predicted track information; carrying out lane line detection processing based on the target detection model and the real-time image information in front of the road to obtain lane line information; and if the deviation behavior is determined to exist according to the predicted track information and the lane line information, carrying out early warning processing.

Description

A trajectory deviation early warning method, terminal and storage medium

Technical field

This application relates to the field of computer vision technology, and in particular to a trajectory deviation early warning method, terminal and storage medium.

Background technique

In highway driving, the vehicle's unconscious deviation from the lane is an important cause of traffic accidents; this unconscious deviation may be due to the driver's inattention, fatigue driving, etc., causing the vehicle to deviate from the original lane; for this kind of unconscious deviation, One of the functions of the assisted driving system is to provide early warning of unintentional deviations and avoid potential traffic accidents.

The current departure warning technology cannot predict the vehicle position in the future, has poor foresight, and has low accuracy in lane line detection, which affects the effect of departure warning.

Contents of the invention

The embodiments of this application provide a trajectory deviation early warning method, terminal and storage medium, which can effectively improve the deviation early warning effect.

The technical solution of the embodiment of this application is implemented as follows:

In a first aspect, embodiments of the present application provide a trajectory deviation early warning method. The trajectory deviation early warning method is applied to a terminal, and the terminal is installed in a target vehicle. The method includes:

When the target vehicle is driving, obtain the real-time corner information of the target vehicle and the real-time image information ahead of the road;

Perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information;

Perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information;

If it is determined that there is deviation behavior based on the predicted trajectory information and the lane line information, early warning processing is performed.

In the second aspect, embodiments of the present application provide a terminal, which is installed in a target vehicle. The terminal includes an acquisition unit, a prediction unit, a detection unit and an early warning unit,

The acquisition unit is configured to acquire the real-time corner information of the target vehicle and the real-time image information ahead of the road when the target vehicle is driving;

The prediction unit is used to perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information;

The detection unit is used to perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information;

The early warning unit is configured to perform early warning processing if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information.

In a third aspect, embodiments of the present application provide a terminal. The terminal further includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the above-mentioned steps are implemented. Trajectory deviation early warning method.

In the fourth aspect, embodiments of the present application provide a computer-readable storage medium on which a program is stored, which is used in a terminal. When the program is executed by a processor, the above trajectory deviation warning method is implemented.

The embodiment of the present application provides a trajectory deviation early warning method, terminal and storage medium. The terminal is installed in the target vehicle. When the target vehicle is driving, the real-time corner information of the target vehicle and the real-time image information ahead of the road are obtained; based on the wavelet analysis method and Real-time corner information is used for driving trajectory prediction processing to obtain predicted trajectory information; lane line detection processing is performed based on the target detection model and real-time image information ahead of the road to obtain lane line information; if it is determined that there is deviation based on the predicted trajectory information and lane line information, then Perform early warning processing. It can be seen that in this application, the terminal is installed in the target vehicle. During the driving process of the target vehicle, the terminal can perform driving trajectory prediction processing on the real-time corner information based on the wavelet analysis method to obtain the predicted trajectory information for a period of time in the future. This enables the vehicle position to be estimated for a period of time in the future, which is very forward-looking; at the same time, the target detection model is used to perform lane line detection processing on the real-time image information in front of the road, which can improve the accuracy of lane line detection; thus, it can be based on The predicted trajectory information and lane line information predict whether the target vehicle will deviate, and provide early warning processing when it is judged that there is a deviation, effectively improving the effect of deviation warning.

Description of the drawings

Figure 1 is a schematic flow chart of the implementation of the trajectory deviation early warning method proposed in the embodiment of the present application;

Figure 2 is a schematic structural diagram 1 of the terminal proposed by the embodiment of the present application;

Figure 3 is a schematic diagram 1 of the predicted trajectory information proposed by the embodiment of the present application;

Figure 4 is a schematic diagram 2 of the predicted trajectory information proposed by the embodiment of the present application;

Figure 5 is a schematic diagram of the lane line information proposed in the embodiment of the present application;

Figure 6 is a schematic diagram 2 of the implementation flow of the trajectory deviation early warning method proposed in the embodiment of the present application;

Figure 7 is a schematic diagram 1 of the implementation of the wavelet decomposition process proposed in the embodiment of the present application;

Figure 8 is a schematic diagram 2 of the implementation of the wavelet decomposition process proposed by the embodiment of the present application;

Figure 9 is a schematic diagram 3 of the implementation of the wavelet decomposition process proposed in the embodiment of this application;

Figure 10 is a schematic diagram of the target signal proposed by the embodiment of the present application;

Figure 11 is a schematic diagram 3 of the implementation flow of the trajectory deviation early warning method proposed in the embodiment of this application;

Figure 12 is a schematic diagram three of the predicted trajectory information proposed by the embodiment of the present application;

Figure 13 is a schematic flow chart 4 of the implementation of the trajectory deviation early warning method proposed by the embodiment of the present application;

Figure 14 is a schematic diagram 2 of the composition structure of the terminal proposed by the embodiment of the present application;

Figure 15 is a schematic diagram 3 of the composition and structure of the terminal proposed by the embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It can be understood that the specific embodiments described here are only used to explain the relevant application, but not to limit the application. It should also be noted that, for convenience of description, only parts relevant to the relevant application are shown in the drawings.

In highway driving, the vehicle's unconscious deviation from the lane is an important cause of traffic accidents; this unconscious deviation may be due to the driver's inattention, fatigue driving, etc., causing the vehicle to deviate from the original lane; for this kind of unconscious deviation, One of the functions of the assisted driving system is to provide early warning of unintentional deviations and avoid potential traffic accidents.

Common deviation warning methods usually require the installation of additional hardware equipment on the vehicle body or on the road to assist lane line detection, such as radar or infrared equipment. They are not only expensive, but also consume a lot of manpower and material resources to implement. They are difficult to implement and cannot be adapted to various scenarios. More restrictive. Moreover, since it is impossible to accurately distinguish whether the driver actively changes lanes or deviates from the lane unintentionally, warnings may be issued continuously during the vehicle's active lane change process, causing certain interference to the driver; in addition, the current deviation warning Technology is still unable to predict the vehicle's position in the future, and its forward-looking nature is poor.

At the same time, when detecting lane lines, existing methods are only suitable for straight lane lines. In real life, lane lines on the road are diverse, including straight lines, curves, solid lines, dotted lines, white lines, and others. Yellow, so it is difficult for traditional morphology and image processing algorithms to accurately detect all the above lane lines of different shapes. Factors such as weather, lighting, occlusion, etc. also have a great impact on the detection effect.

In addition, for existing target detection networks, in the process of calculating errors for training, Hungarian matching is generally used, and then the parameters between each paired detection target and the real target in the matching are calculated. The principle can be Expressed as the following formula:

Among them, d() is the micro-element symbol; Σ is the accumulation symbol.

In lane line detection, each lane line can be curve-fitted with several parameters. These parameters include: cubic curve equation parameters (k ₁ , k ₂ , k ₃ , b), camera internal and external parameters ((f, R), vertical starting and ending amounts (α, β); then d(l _i ,gt _z(i) ) in the above formula can use these parameters between the detection target and the real target to form the error calculation component, which can be expressed as follows formula:

The above is the common idea of constructing error models in existing target detection; however, combined with the specific method and data characteristics of lane line annotation, it has an obvious flaw: the lane line annotation method is to use software such as labelme to mark points on the image. Labeling. When the labeling is completed, the data we get is a series of discrete points on each lane line. The data of each point is (x, y, n), where x represents the abscissa of the point and y represents the vertical coordinate of the point. Coordinates, n represents which lane line this point belongs to; therefore, the parameters needed in the above formula, including cubic curve equation parameters, etc., need to be calculated using discrete point coordinate data for fitting calculations. Lane lines in the real world do not necessarily conform to the characteristics of cubic curves, and there will be errors to some extent during the fitting process; image distortion, projection errors, etc. during the shooting process will also affect the accuracy of the fitting process; from scatter point fitting Synthesize parameters, participate in error calculation, and then reversely convert the parameters into scatter points. The accumulation of errors in these two conversions will have a certain impact on the accuracy of the error model; that is to say, the existing error model The calculation of the error during the training process is not accurate, which will affect the accuracy of target detection, making it impossible to detect lane lines well.

It can be seen that existing deviation warning methods generally have low accuracy in detecting vehicle deviation and poor early warning effects.

In order to solve the problems existing in the trajectory deviation early warning method in the prior art, embodiments of the present application provide a trajectory deviation early warning method, a terminal and a storage medium. The terminal is installed in the target vehicle. When the target vehicle is driving, the terminal obtains the target vehicle real-time corner information and real-time image information in front of the road; based on wavelet analysis method and real-time corner information, driving trajectory prediction processing is performed to obtain predicted trajectory information; based on the target detection model and real-time image information in front of the road, lane line detection processing is performed to obtain lane line information ; If it is determined that there is a deviation based on the predicted trajectory information and lane line information, early warning processing will be carried out; it is very forward-looking and can also improve the lane line detection accuracy, ultimately effectively improving the deviation warning effect.

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

The embodiment of the present application provides a trajectory deviation early warning method. The trajectory deviation early warning method is applied to a terminal, and the terminal is installed in the target vehicle. Figure 1 is a schematic flow chart of the implementation of the trajectory departure early warning method proposed by the embodiment of the present application. As shown in Figure 1 As shown, the trajectory deviation early warning method can include the following steps:

Step 101: When the target vehicle is driving, obtain the real-time corner information of the target vehicle and the real-time image information of the road ahead.

In the embodiment of the present application, when the target vehicle is driving, the terminal can obtain the real-time corner information of the target vehicle and the real-time image information ahead of the road.

It should be noted that in the embodiment of the present application, the real-time turning angle information can determine the steering angle of the target vehicle, thereby determining the forward trajectory of the target vehicle; for example, the real-time turning angle information can be based on the real-time steering angle of the steering wheel obtained from the steering wheel of the target vehicle. Corner information.

It should be noted that in the embodiment of the present application, during the driving process of the target vehicle, the instantaneous steering angle of the steering wheel can determine the deflection angle of the front wheel of the target vehicle, so that the short-term forward trajectory can be determined by the turning angle of the steering wheel; for example , when the steering wheel angle is 0, the deflection angle of the front wheels is 0, and the target vehicle's driving trajectory is two forward straight lines; when the steering wheel angle is not 0, the target vehicle turns, and the trajectory of the front wheels is a pair of concentric arcs. The radius and arc length of the arc are determined by the wheel deflection angle and body parameters of the target vehicle; therefore, relevant information about the target vehicle's driving trajectory can be obtained by obtaining the steering wheel angle of the target vehicle.

Exemplarily, in the embodiment of the present application, Figure 2 is a schematic structural diagram of the terminal proposed by the embodiment of the present application. As shown in Figure 2, the terminal 10 is installed in the target vehicle 20, and the target vehicle 20 can also be configured There is a camera 30; through the camera, the real-time image information of the road ahead can be collected in real time when the target vehicle is driving; the terminal can obtain the real-time image information of the road front collected by the camera.

Further, in the embodiment of the present application, the camera in the target vehicle is calibrated in advance; the purpose of calibration is to obtain the inner orientation elements, distortion parameters and outer orientation elements of the camera, so that with these parameters, it is possible to establish The conversion relationship between a three-dimensional point in space and its projection point in a two-dimensional image.

For example, in the embodiment of this application, first, a customized spatial rectangular coordinate system is established as the basis for all subsequent spatial transformation relationships and spatial absolute distance measurements; the customized spatial rectangular coordinate system can be based on the target The vertical projection point of the center point of the vehicle's front axle on the ground is the origin. The X-axis faces the right side of the target vehicle's body, the Y-axis faces the opposite direction of the target vehicle's forward direction, and the Z-axis faces directly above; then, in this custom space In the rectangular coordinate system, according to Zhang Zhengyou's calibration method, first use a moving checkerboard to take multiple photos to calculate the internal orientation elements of the camera, then optimize the distortion parameters, and finally take a calibration plate tiled on the ground to calculate the camera's right angle in this space The exterior orientation element in the coordinate system.

Among them, the inner orientation elements are recorded as: focal length f, the pixel dimensions dx and dy of the camera in the x and y directions, and the pixel coordinates of the main image point (u0, v0); the outer orientation elements are recorded as: translation matrix T and rotation matrix R ;Thus, the conversion relationship between a point (x, y, z) in space and its position (u, v) in the image captured by the camera in the target vehicle can be expressed as the following formula:

z _c in the formula represents a constant parameter.

It should be noted that in the embodiment of the present application, for any target vehicle, after the camera is installed, the camera calibration only needs to be performed once, and the parameters obtained after calibration can be used as fixed system parameters in the camera. .

Step 102: Perform driving trajectory prediction processing based on the wavelet analysis method and real-time corner information to obtain predicted trajectory information.

In the embodiment of the present application, after the terminal obtains the real-time corner information of the target vehicle and the real-time image information ahead of the road, it can perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain the predicted trajectory information.

It should be noted that in the embodiment of the present application, the predicted trajectory information refers to the predicted information of the target vehicle's driving trajectory in a future period of time.

Further, in some embodiments of the present application, the terminal can first determine the signal sequence based on real-time corner information; then perform wavelet decomposition processing on the signal sequence to obtain a signal combination; and perform noise filtering on the signal combination according to the preset noise filtering interval. Process to obtain the target signal; then process the target signal based on time information to obtain the prediction signal corresponding to the time information; finally reconstruct the prediction signal to obtain the predicted trajectory information.

It should be noted that in the embodiments of the present application, the wavelet analysis method can be used to predict the driving trajectory of the target vehicle in a period of time in the future, which has good foresight, so that when it is judged that there is a deviation, an early warning can be issued in advance to avoid Accidents happen.

Exemplarily, Figure 3 is a schematic diagram 1 of the predicted trajectory information proposed by the embodiment of the present application. As shown in Figure 3, the predicted trajectory information is converted and displayed into the real-time image information of the road ahead captured by the camera. After projection, you can see It is shown that the predicted trajectory information is a straight trajectory; Figure 4 is a schematic diagram 2 of the predicted trajectory information proposed by the embodiment of the present application. As shown in Figure 4, the predicted trajectory information is a turning trajectory.

Step 103: Perform lane line detection processing based on the target detection model and real-time image information ahead of the road to obtain lane line information.

In embodiments of the present application, after acquiring the real-time corner information of the target vehicle and the real-time image information ahead of the road, the terminal can perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain the lane line information.

It should be noted that in the embodiments of this application, the target detection model is obtained after training the initial detection model based on the preset error calculation model; the preset error calculation model proposed in this application can improve the training of the initial detection model. The effect is to improve the detection accuracy of the target detection model to achieve more accurate lane line detection.

It can be understood that in the embodiment of the present application, the target detection model can perform lane line detection processing based on the real-time image information in front of the road, and the lane line information obtained is the detected lane line; for example, Figure 5 is this The schematic diagram of the lane line information proposed in the application embodiment is shown in Figure 5. In the real-time image information ahead of the road, the target detection model can detect the lane line ahead and mark it.

It should be noted that in the embodiments of the present application, when using the target detection model to detect lane lines on real-time image information ahead of the road, not only can the lane lines of the current lane where the target vehicle itself is located be detected, if the left and right sides of the current lane are If there are same-direction lanes, the lane lines of the same-direction lane can also be detected; that is, the lane line information can include the lane lines of the lane where the target vehicle is currently located, and if there are same-direction lanes on the left and right sides of the current lane, Also includes lane markings in the same direction as the current lane.

Further, after obtaining the lane line information, the terminal can directly determine the current lane in which the target vehicle is currently located and whether there is a lane in the same direction based on the lane line information.

Step 104: If it is determined that there is deviation behavior based on the predicted trajectory information and lane line information, perform early warning processing.

In the embodiment of the present application, the terminal performs driving trajectory prediction processing based on the wavelet analysis method and real-time corner information to obtain predicted trajectory information, and performs lane line detection processing based on the target detection model and real-time image information in front of the road to obtain lane line information. Afterwards, if it is determined that there is deviation based on the predicted trajectory information and lane line information, early warning processing will be performed.

It should be noted that in some embodiments of the present application, the terminal can first calculate the distance information between the predicted trajectory information and the lane line information; if the distance information is less than the preset threshold, the predicted driving direction is determined based on the predicted trajectory information. The lane line information determines the current lane of the target vehicle; if it is determined based on the lane line information that the current lane does not have a lane in the same direction in the predicted driving direction, it is determined that there is a deviation behavior.

Further, in the embodiment of the present application, if the distance information is less than the preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the current lane of the target vehicle is determined based on the lane line information; further, if the current lane exists in the predicted driving direction If the lane is in the same direction, the first historical corner information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval are obtained; determined based on the first historical corner information and the first historical lane line information. difference curve, and determine the turning point information based on the difference curve; where the turning point information represents the starting information of the driving angle deviating from the lane line; the first vector, the second vector and the third vector are determined based on the turning point information and the predicted trajectory information; if If the vector product of a first vector and a second vector is greater than or equal to the vector product of a second vector and a third vector, it is determined that there is a deviation; and if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector vector product, the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained; the second historical turning angle information and the second historical turning angle information are calculated based on the preset time interval. At least one difference integral information between lane line information; determine the difference integral change trend corresponding to the second historical time interval based on at least one difference integral information; if the difference integral change trend is an increasing trend, it is determined that it does not exist Deviating behavior.

It should be noted that in the embodiment of this application, the deviation behavior refers to the target vehicle's unconscious deviation, rather than the active deviation; that is to say, the target vehicle can actively deviate, such as actively changing lanes. In the case of such active deviation, Under this condition, no pre-warning processing is performed, and pre-warning processing will only be performed when the target vehicle deviates unintentionally.

It can be understood that in the embodiment of the present application, if it is determined that there is no deviation behavior based on the predicted trajectory information and lane line information, no early warning processing will be performed.

Further, in the embodiment of the present application, early warning processing can be performed by means of sound warning, warning information display, etc.

Further, in the embodiment of the present application, the terminal performs lane line detection processing based on the target detection model and the real-time image information ahead of the road. Before obtaining the lane line information, that is, before step 103, the following steps may also be included:

Step 105: Obtain a training data set; the training data set includes lane marking point information.

In the embodiment of the present application, the terminal performs lane line detection processing based on the target detection model and real-time image information ahead of the road. Before obtaining the lane line information, a training data set may be obtained first; the training data set includes lane line mark point information.

It should be noted that in the embodiment of the present application, the lane marking point information is the relevant data of the marking points obtained by marking based on the real lane lines.

Step 106: Use the initial detection model to perform target detection processing on the training data set, and obtain lane marking point detection results corresponding to the lane marking point information.

In the embodiment of the present application, after acquiring the training data set, the terminal can use the initial detection model to perform target detection processing on the training data set, and obtain the lane marking point detection results corresponding to the lane marking point information.

It can be understood that in the embodiment of the present application, since the training data set contains lane line marking point information, the initial detection model can perform target detection processing on these lane line marking point information, thereby obtaining information about these lane line marking point information. Lane marking point detection results; there are errors between the lane marking point detection results and the lane marking point information, so these errors are calculated based on the preset error calculation model proposed in this application to update and optimize the initial detection model.

It should be noted that in the embodiment of the present application, the initial detection model may be a transformer-based deep learning network; its structure may include a convolutional layer, an encoder (Encoder), a decoder (Decoder), and a parameter feedforward layer. wait.

Step 107: Calculate the error information between the lane marking point detection result and the lane marking point information, and update the initial detection model based on the error information to obtain the target detection model.

In the embodiment of the present application, after the terminal uses the initial detection model to perform target detection processing on the training data set and obtains the lane marking point detection results corresponding to the lane marking point information, the terminal can calculate the lane marking point detection results and the lane markings. Mark the error information between point information, and update the initial detection model based on the error information to obtain the target detection model.

It should be noted that in the embodiment of the present application, a preset error calculation model can be used to calculate the error between the lane marking point information (marking point coordinates) and the lane marking point detection result (detected point coordinates), where , the preset error calculation model can be expressed as the following formula:

Among them, cost represents the error information; p _l,x represents the x-axis coordinate in the lane mark point detection result, p _gt,x represents the x-axis coordinate in the lane mark point information; p _l,y represents the lane mark point detection The y-axis coordinate in the result, p _gt,y, represents the y-axis coordinate in the lane mark point information.

Therefore, the preset error calculation model proposed in this application uses the point error sequence on the lane line as a component of error calculation, which is a discretized error calculation method. This can avoid the existing error calculation method in the simulation. The accumulation of errors caused during the process of combination and conversion also avoids the impact of errors caused by image distortion or projection, which can improve the training effect and improve the detection accuracy of the target detection model.

Figure 6 is a schematic diagram 2 of the implementation process of the trajectory deviation early warning method proposed by the embodiment of the present application. As shown in Figure 6, the terminal performs driving trajectory prediction processing based on the wavelet analysis method and real-time corner information, and obtains the method of predicting trajectory information, that is, step 102 The proposed method can include the following steps:

Step 102a: Determine a signal sequence based on real-time corner information.

In the embodiments of the present application, the terminal performs driving trajectory prediction processing based on the wavelet analysis method and real-time corner information to obtain the predicted trajectory information; in some embodiments of the present application, the terminal may first determine the signal sequence based on the real-time corner information.

For example, in the embodiment of the present application, the present application can collect real-time corner information within a period of time, and organize the real-time corner information within this period of time into a one-dimensional signal sequence that changes with time, and the signal sequence can Expressed as: A=f(t); where, t=t ₀ , t ₁ , t ₂ ,...t _k .

That is to say, the terminal can obtain and save the real-time corner information within a period of time, so that after converting the real-time corner information within this period of time into a signal sequence, the driving trajectory prediction is converted into a signal trend prediction problem.

Step 102b: Perform wavelet decomposition processing on the signal sequence based on the wavelet analysis method to obtain a signal combination.

In the embodiment of the present application, after the terminal determines the signal sequence based on real-time corner information, the terminal can perform wavelet decomposition processing on the signal sequence based on the wavelet analysis method to obtain a signal combination.

It should be noted that in the embodiment of the present application, due to the collection error, transmission error of the sensor in the target vehicle, and the occasional steering fluctuation when the vehicle driver operates the steering wheel, the real-time corner information obtained by the terminal has some noise. , therefore, it is necessary to remove these noises in the signal sequence; through experiments, it is found that the real information in the signal sequence is concentrated in the lower frequency components, and the disturbance caused by factors such as acquisition, transmission, and operation errors to the signal is noise. It shows high-frequency and sporadic characteristics; therefore, the method adopted by this application is to perform wavelet decomposition processing on the signal sequence, which can decompose the signal sequence into an approximate part and a detailed part, thereby obtaining a signal combination; where the approximate part is The real signal is reflected, and the details are the reflection of noise.

Illustratively, in the embodiment of the present application, a quasi-trigonometric function wavelet is constructed as the wavelet base for wavelet decomposition processing, so that the signal sequence is decomposed by two-layer decomposition; the wavelet base can be expressed as the following formula:

Among them, C is a constant parameter, e is a natural constant, t represents time, and cos(t) represents the cosine function.

Figure 7 is a schematic diagram 1 of the implementation of the wavelet decomposition process proposed by the embodiment of the present application. Figure 7 shows the signal sequence before the wavelet decomposition process is performed; furthermore, Figure 8 is a schematic diagram of the implementation of the wavelet decomposition process proposed by the embodiment of the present application. Second, as shown in Figure 8, the approximate part of the signal combination (Approximation A1) obtained by performing two-layer wavelet decomposition processing on the signal sequence in Figure 7 through the above wavelet basis; Figure 9 shows the wavelet decomposition proposed by the embodiment of the present application. Schematic diagram 3 of the processing implementation. Figure 9 shows the details of the signal combination (Detail D1) obtained by performing two-layer wavelet decomposition processing on the signal sequence in Figure 3 using the above-mentioned wavelet basis.

Further, after wavelet decomposition processing, the signal sequence is converted into a linear combination of a set of multi-resolution signals, which is equivalent to wavelet function fitting at different scales and times, thus obtaining the signal combination; the signal combination can be expressed as the following formula:

f(t)＝∑ _k ∑ _j α _j,k ψ _j,k (t) (6)

Among them, α _j,k are harmonic parameters, ψ _j,k (t) is the wavelet function family, and j and k are wavelet coefficients.

Step 102c: Perform noise filtering processing on the signal combination according to the preset noise filtering interval to obtain the target signal.

In the embodiment of the present application, after the terminal performs wavelet decomposition processing on the signal sequence and obtains the signal combination, it can perform noise filtering processing on the signal combination according to the preset noise filtering interval to obtain the target signal.

It should be noted that in the embodiments of the present application, since in the wavelet domain, real information is concentrated on low-frequency components, their wavelet coefficients are generally larger, while noise is generally concentrated on higher-frequency components, and their wavelet coefficients are Generally, it is relatively small. Therefore, this application collects some typical noise components, performs statistics on their wavelet coefficients, records their average wavelet coefficients as σ, and obtains the preset noise filtering interval based on the average wavelet coefficients.

Illustratively, in the embodiment of the present application, the preset noise filtering interval is [-3×σ, 3×σ]; thus, in the signal combination, the wavelet coefficient falls within the signal of this preset noise filtering interval. Filter out to complete the noise filtering process. The obtained target signal can be expressed as the following formula:

f′(t)＝∑ _k ∑ _j α _j,k ψ _j,k (t) (7)

Among them, α>3σ or α<3σ.

Exemplarily, in the embodiment of the present application, Figure 10 is a schematic diagram of the target signal proposed in the embodiment of the present application. Figure 10 shows the target signal obtained after noise filtering processing.

Step 102d: Determine predicted trajectory information based on the target signal.

In the embodiment of the present application, after the terminal performs noise filtering processing on the signal combination according to the preset noise filtering interval and obtains the target signal, the terminal can determine the predicted trajectory information based on the target signal.

In some embodiments of the present application, the terminal can process the target signal based on time information to obtain a prediction signal corresponding to the time information; and then reconstruct the prediction signal to obtain predicted trajectory information.

Figure 11 is a schematic diagram 3 of the implementation process of the trajectory deviation early warning method proposed by the embodiment of the present application. As shown in Figure 11, the method for the terminal to determine the predicted trajectory information based on the target signal, that is, the method proposed in step 102d may include the following steps:

Step 102d1: Process the target signal based on the time information to obtain a prediction signal corresponding to the time information.

In the embodiments of this application, the terminal determines the predicted trajectory information based on the target signal; in some embodiments of this application, the terminal may first process the target signal based on the time information to obtain the prediction signal corresponding to the time information.

It should be noted that in the embodiment of the present application, the time information refers to a period of time in the future, and the prediction signal to be obtained is a signal corresponding to a period of time in the future.

Exemplarily, in the embodiment of the present application, the target signal is processed based on the time information t+Δt, and the predicted signal corresponding to t+Δt is obtained, which can be expressed as the following formula:

f′(t+△t)＝∑ _k ∑ _j α _j,k ψ _j,k (t+△t) (8)

In other words, by substituting the time information t+Δt into the target signal, the predicted signal corresponding to the time information can be obtained.

Step 102d2: Perform reconstruction processing on the predicted signal to obtain predicted trajectory information.

In the embodiment of the present application, after the terminal processes the target signal based on the time information and obtains the prediction signal corresponding to the time information, the terminal can reconstruct the prediction signal to obtain the predicted trajectory information.

It should be noted that in the embodiment of the present application, the reconstruction process refers to the inverse transformation of decomposing the prediction signal.

Further, Figure 12 is a schematic diagram three of the predicted trajectory information proposed by the embodiment of the present application. Figure 12 shows a schematic diagram of the predicted trajectory information obtained after processing by the above series of wavelet analysis methods.

Further, in the embodiment of the present application, if the terminal determines that there is a deviation behavior based on the predicted trajectory information and lane line information, the following steps may be included before performing the early warning process, that is, before step 104:

Step 108: Calculate the distance information between the predicted trajectory information and the lane line information.

In the embodiment of the present application, if the terminal determines that there is a deviation behavior based on the predicted trajectory information and the lane line information, the distance information between the predicted trajectory information and the lane line information is calculated before performing the early warning process.

It should be noted that, in the embodiment of the present application, distance information may be calculated by calculating the distance on the image and then performing projection conversion.

Step 109: If the distance information is less than the preset threshold, determine the predicted driving direction based on the predicted trajectory information, and determine the current lane of the target vehicle based on the lane line information.

In the embodiment of the present application, after the terminal calculates the distance information between the predicted trajectory information and the lane line information, if the distance information is less than the preset threshold, the terminal determines the predicted driving direction based on the predicted trajectory information, and determines the target vehicle based on the lane line information. of the current lane.

It should be noted that in the embodiment of the present application, if only the distance information between the predicted trajectory information and the lane line information is used as the basis for judging deviation, then when the vehicle actively changes lanes, it will also be judged as deviation, and continuously It is obviously inappropriate to issue an early warning. Therefore, this application adopts a series of behavioral analysis methods to distinguish the situation where the target vehicle actively changes lanes and the situation where it deviates. When the target vehicle deviates, a deviation warning is issued. When it actively changes lanes, Can issue lane change prompts.

Therefore, when this application determines that the distance information is less than the preset threshold, it first determines the predicted driving direction based on the predicted trajectory information, and determines the current lane of the target vehicle based on the lane line information; thus, it is judged that the target vehicle is driving in the predicted direction. Whether there is a lane in the same direction on the side where the line is about to be pressed.

For example, in the embodiment of the present application, while the target vehicle is driving, the distance information obtained at a certain moment is 19cm, and the preset threshold is 20cm. At this time, the predicted driving direction needs to be determined based on the predicted trajectory information, Determine the current lane of the target vehicle based on the lane line information.

It can be understood that, in the embodiment of the present application, the terminal can directly determine the current lane of the target vehicle based on the lane line information.

Step 110: If it is determined based on the lane line information that the current lane does not have a lane in the same direction in the predicted driving direction, it is determined that there is a deviation behavior.

In the embodiment of the present application, after calculating the distance information between the predicted trajectory information and the lane line information, if the terminal determines based on the lane line information that there is no same direction lane in the current lane in the predicted driving direction, it determines that there is a deviation behavior.

It can be understood that in the embodiment of the present application, if it is determined based on the lane line information that the current lane does not have a lane in the same direction in the predicted driving direction, the target vehicle must not actively change lanes, and it is determined that there is a deviation behavior.

Further, in the embodiment of the present application, if the distance information is less than the preset threshold, the predicted driving direction is determined based on the predicted trajectory information. After determining the current lane of the target vehicle based on the lane line information, that is, after step 109, the following steps may be included: :

Step 110: If the current lane has a same-direction lane in the predicted driving direction, obtain the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval.

In the embodiment of the present application, if the distance information is less than the preset threshold, the terminal determines the predicted driving direction based on the predicted trajectory information. After determining the current lane of the target vehicle based on the lane line information, if the current lane has a same-direction lane in the predicted driving direction , then the first historical turning angle information of the steering wheel corresponding to the first historical time interval and the first historical lane line information corresponding to the first historical time interval are obtained.

It can be understood that in the embodiment of the present application, if the current lane has a lane in the same direction in the predicted driving direction, lane change is possible, and other information needs to be combined to further determine whether there is a deviation.

It should be noted that in the embodiment of the present application, the first historical time interval refers to a period of time before the current time of the target vehicle, and the first historical turning angle information is the steering wheel angle information corresponding to the first historical time interval. A piece of historical lane marking information is the lane marking information corresponding to the first historical time interval.

The first historical lane line information is obtained based on the real-time image information of the road ahead corresponding to the first historical time interval.

Step 111: Determine the difference curve according to the first historical turning angle information and the first historical lane line information, and determine the turning point information according to the difference curve; wherein the turning point information represents the starting information of the driving angle deviating from the lane line.

In the embodiment of the present application, if the current lane has a lane in the same direction in the predicted driving direction, the terminal obtains the first historical turning angle information of the steering wheel corresponding to the first historical time interval, and the first historical lane corresponding to the first historical time interval. After the line information is obtained, the difference curve can be determined based on the first historical corner information and the first historical lane line information, and the turning point information can be determined based on the difference curve; wherein the turning point information represents the starting information of the driving angle deviating from the lane line.

It should be noted that, in the embodiment of the present application, the turning point information represents the starting information of the driving angle deviating from the lane line.

It can be understood that when the target vehicle is driving normally, the steering wheel angle information and the angle of the lane line should be "consistent", but from the moment the target vehicle starts to deviate, the steering angle information and the angle of the lane line are "inconsistent" "; This application determines the difference curve based on the first historical corner information and the first historical lane line information, and then performs extreme value analysis on the difference curve, and can obtain this difference from "consistent" to "inconsistent" in the difference curve Turning point, that is, turning point information.

Step 112: Determine whether there is deviation based on the turning point information.

In the embodiment of the present application, the terminal determines the difference curve based on the first historical corner information and the first historical lane line information, and after determining the turning point information based on the difference curve, it can determine whether there is a deviation behavior based on the turning point information.

In some embodiments of the present application, the terminal can determine the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information; further, if the vector product of the first vector and the second vector is greater than or equal to the sum of the second vector The vector product of the third vector determines the existence of deviation behavior.

It should be noted that in the embodiment of the present application, the turning point information includes the time information of the turning point and the position information of the turning point; it can be understood that the time information of the turning point is the moment when the turning point occurs, and the position information of the turning point is the turning point. location of occurrence.

Further, in the embodiment of the present application, the method for the terminal to determine whether there is a deviation behavior based on the turning point information, that is, the method proposed in step 112 may include the following steps:

Step 112a: Determine the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information.

In the embodiment of the present application, the terminal determines whether there is a deviation behavior based on the turning point information; in some embodiments of the present application, the terminal may first determine the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information.

It should be noted that in the embodiment of the present application, the first vector takes the position information of the turning point as the starting point and the position information corresponding to the next moment of the time information of the turning point in the predicted trajectory information as the end point, forming vector.

It should be noted that in the embodiment of the present application, the position information passing through the turning point is drawn as a line parallel to the lane line, and a vector of unit length on this line is taken to form the second vector.

It should be noted that in the embodiment of the present application, the third vector takes the position information of the turning point as the starting point and the position information corresponding to the last moment of the time information of the turning point in the predicted trajectory information as the end point. vector.

Step 112b: If the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, it is determined that there is a deviation behavior.

In the embodiment of the present application, after the terminal determines the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information, if the vector product of the first vector and the second vector is greater than or equal to the second vector and the third vector, The vector product of three vectors indicates the existence of deviation behavior.

Illustratively, in the embodiment of this application, the first vector is expressed as The second vector is expressed as/> The third vector is expressed as/> Compute the vector product:

If L1>L2 or L1=L2, it is determined that there is deviation behavior and early warning processing is required.

Further, in the embodiment of the present application, after the terminal determines the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information, that is, after step 112a, the following steps may also be included:

Step 112c. If the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, obtain the second historical angle information of the steering wheel corresponding to the second historical time interval, corresponding to the second historical time interval. The second historical lane line information.

In the embodiment of the present application, after the terminal determines the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector By vector product, the second historical turning angle information of the steering wheel corresponding to the second historical time interval and the second historical lane line information corresponding to the second historical time interval are obtained.

It should be noted that in the embodiment of the present application, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, it can be determined that the target vehicle is likely to be steered by the driver steering the steering wheel. The steering wheel deviated from the lane and turned; further obtain the second historical turning angle information of the steering wheel corresponding to the second historical time interval, and the second historical lane line information corresponding to the second historical time interval to determine whether it is an active lane change behavior. If not, If the vehicle actively changes lanes, an early warning will be issued.

Further, in the embodiment of the present application, the second historical time interval refers to the period from the time information of the turning point to the current moment; the second historical corner information is the steering wheel angle information corresponding to the second historical time interval. The second historical lane marking information is the lane marking information corresponding to the second historical time interval.

It can be understood that, in the embodiment of the present application, the second historical lane line information is obtained based on the real-time image information of the road ahead corresponding to the second historical time interval.

Step 112d: Calculate at least one difference integral information between the second historical corner information and the second historical lane line information based on the preset time interval.

In the embodiment of the present application, if the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, the terminal obtains the second historical angle information of the steering wheel corresponding to the second historical time interval, and After the second historical lane marking information corresponding to the second historical time interval, at least one difference integral information between the second historical corner information and the second historical lane marking information may be calculated based on the preset time interval.

It should be noted that in the embodiment of the present application, the corresponding driving trajectory information can be determined based on the second historical corner information, and then the driving trajectory information and the second historical lane line corresponding to the second historical corner information can be calculated based on the preset time interval. At least one difference between the messages integrates the messages.

That is to say, the difference integral between the driving trajectory information corresponding to the second historical corner information and the second historical lane line information is calculated every preset time interval, so that at least one difference integral information can be obtained.

For example, in the embodiment of the present application, the calculation method of the difference integral information can be expressed as the following formula:

Among them, f _car (t) represents the second historical corner information, and f _lane (t) represents the second historical lane line information.

Step 112e: Determine the difference integral change trend corresponding to the second historical time interval based on at least one difference integral information.

In an embodiment of the present application, after the terminal calculates at least one difference integral information between the second historical corner information and the second historical lane line information based on the preset time interval, the terminal can determine the second historical value based on the at least one difference integral information. The change trend of the difference integral corresponding to the time interval.

It should be noted that, in the embodiment of the present application, the difference integral change trend refers to the change trend presented by at least one difference integral information obtained from the time information of the turning point in the second historical time interval to the current time.

Step 112f: If the change trend of the difference integral is an increasing trend, it is determined that there is no deviation behavior.

In the embodiment of the present application, after the terminal determines the difference integral change trend corresponding to the second historical time interval based on at least one difference integral information, if the difference integral change trend is an increasing trend, it is determined that there is no deviation behavior.

It should be noted that in the embodiment of the present application, if the change trend of the difference integral is an increasing trend, it means that the target vehicle is an active lane change behavior caused by the driver actively operating the steering wheel, and is not a deviation behavior. At this time, Lane change prompt information can be generated.

It should be noted that, in the embodiment of the present application, if the change trend of the difference integral is not an increasing trend, for example, decreases or remains unchanged, it is determined that there is a deviation behavior, and early warning processing needs to be performed.

To sum up, Figure 13 is a schematic diagram 4 of the implementation process of the trajectory deviation warning method proposed by the embodiment of the present application. As shown in Figure 13, it is judged whether the distance information between the predicted trajectory information and the lane line information is less than the preset threshold (step 201), if so, continue to determine whether there is a same-direction lane on the side about to cross the line (step 202). If there is a same-direction lane, first determine the turning point information (step 203), and then perform the vector product L1 and Calculate L2 (step 204), and then determine whether L1 is less than L2 (step 205); and if there is no same-direction lane, perform a departure warning (step 206); then, if it is determined based on step 205 that L1 < L2, calculate at least A difference integral information (step 207), and then determine whether the change trend of at least one difference integral information is an increasing trend (step 208). If the change trend is an increasing trend, it is determined that there is no deviation behavior, and then Generate lane change prompt information (step 209), and if the change trend is not an increasing trend, it is determined that there is a deviation behavior, and a deviation warning is performed (step 206); and if it is judged based on step 201 that it is not less than the preset threshold, then it is determined that it is not smaller than the preset threshold. There is a deviation (step 210), and no deviation warning is required.

It can be seen that in the trajectory deviation early warning method of this application, the driving trajectory prediction is based on the wavelet analysis method; the traditional target detection deep learning network is improved on the error model, making the training effect better and more suitable for lane lines. detection; at the same time, it can also distinguish active lane changes or unconscious deviations based on predicted trajectory information and lane line information; the trajectory deviation early warning method in this application can be applied to automatic intelligent vehicle-road collaboration projects, and self-driving trucks equipped with this technology During field testing, lane lines of various shapes and road scenes can be well detected, and the predicted driving trajectory is also consistent with the actual driving trajectory, showing a high overall accuracy. When the vehicle is likely to deviate from the lane due to unintentional deviation or abnormal driving, it can detect the danger in time and issue an early warning, which helps to maintain the vehicle's lane and ensure driving safety; at the same time, when actively manipulating the steering wheel to change lanes, it can Correctly identifying the lane change intention will not trigger a deviation warning, but will display a lane change prompt, making the autonomous driving function more intelligent, solving the problem of false lane change warnings that has existed before, and improving the automatic intelligent vehicle-road collaboration project. The working efficiency of the remote driving console realizes intelligent trajectory deviation warning.

The embodiment of the present application provides a trajectory deviation early warning method, terminal and storage medium. The terminal is installed in the target vehicle. When the target vehicle is driving, the real-time corner information of the target vehicle and the real-time image information ahead of the road are obtained; based on the wavelet analysis method and Real-time corner information is used for driving trajectory prediction processing to obtain predicted trajectory information; lane line detection processing is performed based on the target detection model and real-time image information ahead of the road to obtain lane line information; if it is determined that there is deviation based on the predicted trajectory information and lane line information, then Perform early warning processing. It can be seen that in this application, the terminal is installed in the target vehicle. During the driving process of the target vehicle, the terminal can perform driving trajectory prediction processing on the real-time corner information based on the wavelet analysis method to obtain the predicted trajectory information for a period of time in the future. This enables the vehicle position to be estimated for a period of time in the future, which is very forward-looking; at the same time, the target detection model is used to perform lane line detection processing on the real-time image information in front of the road, which can improve the accuracy of lane line detection; thus, it can be based on The predicted trajectory information and lane line information predict whether the target vehicle will deviate, and provide early warning processing when it is judged that there is a deviation, effectively improving the effect of deviation warning.

In another embodiment of the present application, Figure 14 is a schematic diagram 2 of the composition structure of the terminal proposed by the embodiment of the present application. As shown in Figure 14, the terminal 10 proposed by the embodiment of the present application may include an acquisition unit 11, a prediction unit 12, Detection unit 13, early warning unit 14 and calculation unit 15.

The acquisition unit 11 is configured to acquire the real-time corner information of the target vehicle and the real-time image information ahead of the road when the target vehicle is traveling.

The prediction unit 12 is configured to perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information.

The detection unit 13 is configured to perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information;

The early warning unit 14 is configured to perform early warning processing if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information.

Further, the acquisition unit 11 is also configured to obtain a training data set before performing lane line detection processing based on the target detection model and the real-time image information of the road ahead to obtain the lane line information; wherein the training data set includes Lane marking point information.

Further, the detection unit 13 is also used to perform target detection processing on the training data set using an initial detection model to obtain the lane marking point detection results corresponding to the lane marking point information; and calculate the lane marking point detection result. The error information between the marked point detection result and the lane marking point information is updated, and the initial detection model is updated according to the error information to obtain the target detection model.

Further, the acquisition unit 11 is also used to determine a signal sequence based on the real-time corner information; and perform wavelet decomposition processing on the signal sequence to obtain a signal combination; and combine the signals according to a preset noise filtering interval. Perform noise filtering processing to obtain a target signal; and determine the predicted trajectory information based on the target signal.

Further, the acquisition unit 11 is further configured to process the target signal based on time information to obtain a prediction signal corresponding to the time information; and perform reconstruction processing on the prediction signal to obtain the prediction trajectory information. .

The calculation unit 15 is configured to calculate the predicted trajectory information and the lane line information before performing early warning processing if the warning unit 14 determines that there is a deviation behavior based on the predicted trajectory information and the lane line information. distance information between them; if the distance information is less than the preset threshold, determine the predicted driving direction based on the predicted trajectory information, and determine the current lane of the target vehicle based on the lane line information; if based on the lane line information If it is determined that the current lane does not have a same-directional lane in the predicted driving direction, it is determined that the deviation behavior exists.

Further, the acquisition unit 11 is also configured to determine the predicted driving direction according to the predicted trajectory information and determine the target according to the lane line information if the distance information is less than a preset threshold. After the current lane of the vehicle, if there is a same-direction lane in the predicted driving direction of the current lane, the first historical turning angle information of the steering wheel corresponding to the first historical time interval is obtained, corresponding to the first historical time interval. The first historical lane line information.

Further, the calculation unit 15 is further configured to determine a difference curve based on the first historical corner information and the first historical lane line information, and determine turning point information based on the difference curve; wherein, the turning point The information represents the starting information of the driving angle deviating from the lane line; and based on the turning point information, it is determined whether the deviation behavior exists.

Further, the calculation unit 15 is further configured to determine the first vector, the second vector and the third vector based on the turning point information and the predicted trajectory information; and if the first vector and the second vector If the vector product is greater than or equal to the vector product of the second vector and the third vector, it is determined that the deviation behavior exists.

Further, the calculation unit 15 is further configured to determine the first vector, the second vector and the third vector based on the turning point and the predicted trajectory information, if the vector of the first vector and the second vector If the product is less than the vector product of the second vector and the third vector, then the second historical angle information of the steering wheel corresponding to the second historical time interval and the second historical lane corresponding to the second historical time interval are obtained line information; and calculating at least one difference integral information between the second historical corner information and the second historical lane line information based on a preset time interval; and determining the first difference integral information based on the at least one difference integral information. The difference integral change trend corresponding to the two historical time intervals; if the difference integral change trend is an increasing trend, it is determined that the deviation behavior does not exist.

Figure 15 is a schematic diagram 3 of the composition structure of the terminal proposed by the embodiment of the present application. As shown in Figure 15, the terminal 10 proposed by the embodiment of the present application may also include a processor 16 and a memory 17 that stores instructions executable by the processor 16. Further The terminal 10 may also include a communication interface 18, and a bus 19 for connecting the processor 16, the memory 17 and the communication interface 18.

In the embodiment of the present application, the above-mentioned processor 16 can be an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a digital signal processor (Digital Signal Processor, DSP), a digital terminal (Digital Signal Processing Device, DSPD), or At least one of Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor . It can be understood that for different devices, the electronic device used to implement the above processor function may also be other, which is not specifically limited in the embodiment of the present application. The processor 16 may also include a memory 17, which may be connected to the processor 16, where the memory 17 is used to store executable program codes, which include computer operating instructions. The memory 17 may include high-speed RAM memory, or may also Includes non-volatile memory, such as at least two disk memories.

In the embodiment of the present application, the bus 19 is used to connect the communication interface 18, the processor 16 and the memory 17 as well as the mutual communication between these devices.

In the embodiment of the present application, the memory 17 is used to store instructions and data.

Further, in the embodiment of the present application, the above-mentioned processor 16 is used to obtain the real-time corner information of the target vehicle and the real-time image information ahead of the road when the target vehicle is driving; based on the wavelet analysis method and the real-time Perform driving trajectory prediction processing on the corner information to obtain predicted trajectory information; perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information; if determined based on the predicted trajectory information and the lane line information If there is deviation, early warning processing will be carried out.

In practical applications, the above-mentioned memory 17 can be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory. (Read-Only Memory, ROM), flash memory (flash memory), hard disk (Hard DiskDrive, HDD) or solid-state drive (Solid-State Drive, SSD); or a combination of the above types of memories, and is provided to the processor 16 instructions and data.

In addition, each functional module in this embodiment can be integrated into one analysis unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or software function modules.

If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially The contribution made to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which can be a personal computer). A computer, server, or network device, etc.) or processor executes all or part of the steps of the method in this embodiment. The aforementioned storage media include: U disk, mobile hard disk, read only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various media that can store program codes.

The embodiment of the present application provides a trajectory deviation early warning method, terminal and storage medium. The terminal is installed in the target vehicle. When the target vehicle is driving, the real-time corner information of the target vehicle and the real-time image information ahead of the road are obtained; based on the wavelet analysis method and Real-time corner information is used for driving trajectory prediction processing to obtain predicted trajectory information; lane line detection processing is performed based on the target detection model and real-time image information ahead of the road to obtain lane line information; if it is determined that there is deviation based on the predicted trajectory information and lane line information, then Perform early warning processing. It can be seen that in this application, the terminal is installed in the target vehicle. During the driving process of the target vehicle, the terminal can perform driving trajectory prediction processing on the real-time corner information based on the wavelet analysis method to obtain the predicted trajectory information for a period of time in the future. This enables the vehicle position to be estimated for a period of time in the future, which is very forward-looking; at the same time, the target detection model is used to perform lane line detection processing on the real-time image information in front of the road, which can improve the accuracy of lane line detection; thus, it can be based on The predicted trajectory information and lane line information predict whether the target vehicle will deviate, and provide early warning processing when it is judged that there is a deviation, effectively improving the effect of deviation warning.

Specifically, the program instructions corresponding to a trajectory deviation early warning method in this embodiment can be stored on storage media such as optical disks, hard disks, and U disks; when the program instructions corresponding to a trajectory deviation early warning method in the storage medium When read or executed by an electronic device, the following steps are included:

When the target vehicle is driving, obtain the real-time corner information of the target vehicle and the real-time image information ahead of the road;

Perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information;

Perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information;

If it is determined that there is deviation behavior based on the predicted trajectory information and the lane line information, early warning processing is performed.

Those skilled in the art will understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage and optical storage, etc.) embodying computer-usable program code therein.

The present application is described with reference to schematic flowcharts and/or block diagrams of implementations of methods, devices (systems), and computer program products according to embodiments of the present application. It will be understood that each process and/or block in the flowchart illustrations and/or block diagrams, and combinations of processes and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions executed by the processor of the computer or other programmable data processing device produce a use A device for realizing the functions specified in one process or multiple processes of the flow diagram and/or one block or multiple blocks of the block diagram.

These computer program instructions may also be stored in a computer-readable memory that causes a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction means, the instructions The device implements the functions specified in implementing one process or multiple processes in the flow diagram and/or one block or multiple blocks in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device, causing a series of operating steps to be performed on the computer or other programmable device to produce computer-implemented processing, thereby executing on the computer or other programmable device. The instructions provide steps for implementing the functions specified in implementing a process or processes of the flowchart diagram and/or a block or blocks of the block diagram.

The above descriptions are only preferred embodiments of the present application and are not intended to limit the scope of protection of the present application.

Claims (11)

1. A trajectory deviation early warning method, characterized in that the trajectory deviation early warning method is applied to a terminal, and the terminal is installed in a target vehicle, and the method includes: When the target vehicle is driving, obtain the real-time corner information of the target vehicle and the real-time image information ahead of the road; Perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information; Perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information; If it is determined that there is deviation behavior based on the predicted trajectory information and the lane line information, early warning processing is performed. 2. The method according to claim 1, characterized in that, before performing lane line detection processing based on the target detection model and the real-time image information of the road ahead, and obtaining the lane line information, the method includes: Obtain a training data set; wherein the training data set includes lane line marking point information; Using an initial detection model to perform target detection processing on the training data set, obtain lane line marking point detection results corresponding to the lane marking point information; Calculate error information between the lane line marking point detection result and the lane line marking point information, and update the initial detection model according to the error information to obtain the target detection model. 3. The method according to claim 1, characterized in that the driving trajectory prediction processing is performed based on the wavelet analysis method and the real-time corner information to obtain the predicted trajectory information, including: Determine a signal sequence based on the real-time corner information; Perform wavelet decomposition processing on the signal sequence based on the wavelet analysis method to obtain a signal combination; Perform noise filtering processing on the signal combination according to the preset noise filtering interval to obtain the target signal; The predicted trajectory information is determined based on the target signal. 4. The method of claim 3, wherein determining the predicted trajectory information based on the target signal includes: Process the target signal based on time information to obtain a prediction signal corresponding to the time information; The predicted signal is reconstructed to obtain the predicted trajectory information. 5. The method according to claim 1, characterized in that, if it is determined that there is a deviation behavior according to the predicted trajectory information and the lane line information, before performing early warning processing, the method includes: Calculate distance information between the predicted trajectory information and the lane line information; If the distance information is less than a preset threshold, determine the predicted driving direction based on the predicted trajectory information, and determine the current lane of the target vehicle based on the lane line information; If it is determined based on the lane line information that the current lane does not have a same-direction lane in the predicted driving direction, it is determined that the deviation behavior exists. 6. The method of claim 5, wherein if the distance information is less than a preset threshold, the predicted driving direction is determined based on the predicted trajectory information, and the target vehicle is determined based on the lane line information. After the current lane, the method includes: If the current lane has a same-direction lane in the predicted driving direction, obtain the first historical turning angle information of the steering wheel corresponding to the first historical time interval, and the first historical lane line corresponding to the first historical time interval. information; Determine a difference curve based on the first historical corner information and the first historical lane line information, and determine turning point information based on the difference curve; wherein the turning point information represents the starting information of the driving angle deviating from the lane line; Determine whether the deviation behavior exists based on the turning point information. 7. The method according to claim 6, wherein the determining whether there is the deviation behavior based on the turning point information includes: Determine a first vector, a second vector and a third vector based on the turning point information and the predicted trajectory information; If the vector product of the first vector and the second vector is greater than or equal to the vector product of the second vector and the third vector, it is determined that the deviation behavior exists. 8. The method according to claim 7, wherein after determining the first vector, the second vector and the third vector based on the turning point and the predicted trajectory information, the method includes: If the vector product of the first vector and the second vector is less than the vector product of the second vector and the third vector, obtain the second historical angle information of the steering wheel corresponding to the second historical time interval, Second historical lane line information corresponding to the second historical time interval; Calculate at least one difference integral information between the second historical corner information and the second historical lane line information based on a preset time interval; Determine the difference integral change trend corresponding to the second historical time interval based on the at least one difference integral information; If the change trend of the difference integral is an increasing trend, it is determined that the deviation behavior does not exist. 9. A terminal, characterized in that the terminal is installed in a target vehicle, and the terminal includes an acquisition unit, a prediction unit, a detection unit and an early warning unit, The acquisition unit is configured to acquire the real-time corner information of the target vehicle and the real-time image information ahead of the road when the target vehicle is driving; The prediction unit is used to perform driving trajectory prediction processing based on the wavelet analysis method and the real-time corner information to obtain predicted trajectory information; The detection unit is used to perform lane line detection processing based on the target detection model and the real-time image information ahead of the road to obtain lane line information; The early warning unit is configured to perform early warning processing if it is determined that there is a deviation behavior based on the predicted trajectory information and the lane line information. 10. A terminal, characterized in that the terminal includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, any one of claims 1-8 is implemented. method described in the item. 11. A computer-readable storage medium, characterized in that a program is stored thereon and is applied to a terminal. When the program is executed by a processor, the method according to any one of claims 1-8 is implemented.