CN114559960A - A Collision Warning System Based on Fusion of Forward-looking Camera and Rear Millimeter-Wave Radar - Google Patents

Abstract

The invention relates to the technical field of automatic driving of intelligent vehicles, and discloses a collision early warning system based on fusion of a front-view camera and a rear millimeter wave radar, which comprises a visual detection module, a front-view radar module and a rear-view radar module, wherein the visual detection module is used for shooting and processing road condition images in front of an intelligent vehicle to obtain front road condition information; the rear millimeter wave radar is used for enabling a user to obtain rear obstacle information of the intelligent vehicle, wherein the rear obstacle information comprises obstacle speed; the data processing module is used for processing the road condition information and the rear obstacle information and comprises a rear early warning processing unit, wherein the rear early warning processing unit is used for obtaining collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction and giving an alarm when the collision time reaches a rear collision threshold value. The early warning function has guaranteed intelligent vehicle's safe traveling around this scheme, greatly reduced the probability that the vehicle bumps, improve intelligent vehicle's security of traveling.

Description

A Collision Warning System Based on Fusion of Forward-looking Camera and Rear Millimeter-Wave Radar

technical field

The invention relates to the technical field of automatic driving of intelligent vehicles, in particular to a collision warning system based on a fusion front-view camera and a rear millimeter-wave radar.

Background technique

In recent years, artificial intelligence technology has been widely used in the automobile industry, and automobiles are constantly developing in the direction of intelligence. Many vehicles are equipped with certain driving assistance devices and intelligent devices to assist drivers to perceive and understand the vehicle. control, so as to further improve the safety of driving. When driving in front and rear, the smart car can use the automatic emergency braking technology in the on-board driving assistance device to avoid rear-end collision with the vehicle in front, but when the smart car is rear-ended by the rear vehicle, it cannot actively avoid danger. Or when the self-driving vehicle is waiting for the traffic light at the intersection, the self-driving vehicle stops in an incorrect posture, or the parking position is in the lane waiting to turn, and the body is parked at a certain angle. Parking at an angle, resulting in a false alarm for rear collision warning.

SUMMARY OF THE INVENTION

In view of the deficiencies in the prior art, the purpose of the present invention is to provide a collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar.

In order to achieve the above object, the present invention provides the following technical solutions:

A collision warning system based on a fusion front-view camera and rear millimeter-wave radar,

The visual detection module is used to capture and process images of road conditions in front of the intelligent vehicle to obtain information on the road conditions ahead;

Rear millimeter-wave radar, the user obtains the obstacle information behind the intelligent vehicle, and the rear obstacle information includes the obstacle speed;

The data processing module is used to process the above-mentioned road condition information and rear obstacle information, including a rear early warning processing unit, which is used to obtain the collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction. The alarm will be issued when the collision time reaches the rear collision threshold. The calculation formula of the trigger time is:

Among them, t is the trigger time, and V _r is the relative speed of the rear obstacle and the intelligent vehicle in the y direction.

In the present invention, further, the method for obtaining the relative speed of the rear obstacle and the intelligent vehicle in the y direction is v _r =(v ₁ -v ₂ )tanθ, where V1 is the vehicle speed of the obstacle, and V2 is the intelligent vehicle Speed, which is the heading angle of the intelligent vehicle.

In the present invention, further, the method for obtaining the relative heading angle is as follows: determining the lane reference line of the intelligent vehicle, and constructing a right triangle to obtain the relative heading angle by previewing two points in front of the intelligent vehicle.

In the present invention, further, the visual detection module includes at least a front-view camera, and the front-view camera is used for capturing images of road conditions ahead.

In the present invention, further, the data processing module further includes a forward warning unit, which is used for processing the image obtained by the forward-looking camera to obtain a lane reference line, and collecting forward obstacle information.

In the present invention, further, the processing of the image obtained by the front-view camera includes reading the image, preprocessing the image, extracting the image features to obtain the lane reference line, and obtaining the lateral position of the intelligent vehicle through the lane line fitting method, When the lateral position of the intelligent vehicle is less than the preset departure threshold, the lane departure warning signal is triggered.

In the present invention, further, the collection of front obstacle information includes the speed of the front obstacle and the distance of the front obstacle. When the speed difference between the intelligent vehicle and the front obstacle is greater than the set speed threshold and the distance difference between the two is greater than the distance threshold, the start is triggered. Warning sign.

In the present invention, further, the method for obtaining the lane reference line is: selecting a distance along the direction of the heading angle of the vehicle, sampling uniformly at a certain distance within the distance, and solving the first derivative and the second derivative of the polynomial function fitted by the reference line The order derivative calculates the curvature value of the uniform sampling point, performs statistical analysis on the curvature value of many sampling points, calculates the maximum value, minimum value, mean value and variance, builds an algorithm model to solve the weighted curvature value, and characterizes the lane reference line through the weighted curvature.

In the present invention, further, the obtaining of the lateral position of the intelligent vehicle by means of lane line fitting includes indicating the change rate of the relative position of the intelligent vehicle by the relative change of the vehicle offset and the heading angle in the two frames of images:

表示航向角的变化率，

是航向角的变化率，Δc是相邻两帧图像智能车辆偏移量的变化量，Δt是该两帧图像的间隔时间，Δθ是该两帧图像中航向角的变化量。in,

represents the rate of change of the heading angle,

is the change rate of the heading angle, Δc is the change of the offset of the intelligent vehicle in the adjacent two frames of images, Δt is the interval time between the two frames of images, and Δθ is the change of the heading angle in the two frames of images.

In the present invention, further, the movement of the intelligent vehicle adopts a lateral feedback control strategy.

Compared with the prior art, the beneficial effects of the present invention are:

The device of the invention detects front and rear obstacles by integrating the front-view camera and the rear millimeter-wave radar, obtains the collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction, and gives an alarm when the collision time reaches the rear collision threshold. Solve the problem that when the intelligent vehicle is waiting for the traffic light at the intersection, the self-vehicle stops in an incorrect posture, or the parking position is in the lane waiting to turn, and the vehicle is parked at a certain angle, etc., when there is a target passing by in the side lane, because the self-vehicle is parked at an angle. cause a collision safety hazard. At the same time, this solution also integrates the front warning unit to realize the lane line departure warning in front of the intelligent vehicle, and when waiting for a red light, the vehicle ahead has already driven out, but the intelligent vehicle does not start within the preset time, it will trigger the start warning Signal. In this way, the safe driving of the intelligent vehicle is ensured, the probability of vehicle collision is greatly reduced, and the driving safety of the intelligent vehicle is improved.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the attached image:

1 is an overall structural diagram of a collision warning system based on a fusion front-view camera and a rear millimeter-wave radar according to the present invention;

2 is an overall structural diagram of a collision warning system based on a fusion front-view camera and a rear millimeter-wave radar according to the present invention;

3 is a diagram of a collision warning system based on a fusion front-view camera and a rear millimeter-wave radar according to the present invention;

FIG. 4 is a diagram of a collision warning system based on a fusion front-view camera and a rear millimeter-wave radar according to the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that when a component is referred to as being "fixed to" another component, it can be directly on the other component or there may also be a centered component. When a component is considered to be "connected" to another component, it can be directly connected to the other component or there may be an intervening component at the same time. When a component is considered to be "set on" another component, it may be directly set on the other component or there may be a co-existing centered component. The terms "vertical," "horizontal," "left," "right," and similar expressions are used herein for illustrative purposes only.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.

Referring to FIGS. 1 and 2 , a preferred embodiment of the present invention provides a collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar, including:

The visual detection module is used to capture and process images of road conditions in front of the intelligent vehicle to obtain information on the road conditions ahead;

Rear millimeter-wave radar, the user obtains the obstacle information behind the intelligent vehicle, and the rear obstacle information includes the obstacle speed;

The data processing module is used to process the above-mentioned road condition information and rear obstacle information, including a rear early warning processing unit, which is used to obtain the collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction. The alarm will be issued when the collision time reaches the rear collision threshold. The calculation formula of the trigger time is:

Among them, t is the trigger time, and V _r is the relative speed of the rear obstacle and the intelligent vehicle in the y direction.

Specifically, multiple rear millimeter-wave radars can be set, which are mainly used to detect the motion status of rear obstacles, including vehicle speed, vehicle distance, etc. Using the characteristics of rear millimeter-wave radars, this system is equipped with a rear early warning processing unit to solve the problem when When waiting for a traffic light at an intersection, the parking position of the self-vehicle is not correct, or the parking position is in the lane waiting to turn, and the vehicle is parked at a certain angle. Since the self-vehicle is stationary, when there is a target passing by in the side lane, because the self-vehicle is parked at an angle, the target The relative speed of the vehicle and the vehicle will have a certain component in the y direction of the vehicle, causing a potential safety hazard in collision.

To this end, this scheme obtains the collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction, and alarms when the collision time reaches the rear collision threshold, where the rear collision threshold is set to 1.4s. The method for obtaining the relative speed of the rear obstacle and the smart vehicle in the y direction is v _r =(v ₁ -v ₂ )tanθ, where V1 is the speed of the obstacle, V2 is the speed of the smart vehicle, and θ is the relative speed of the smart vehicle Heading.

In the present invention, further, the method for obtaining the relative heading angle is as follows: determining the lane reference line of the intelligent vehicle, and constructing a right-angled triangle to obtain the relative heading angle by previewing two points in front of the intelligent vehicle.

Specifically, as shown in Figure 4, the vertical distances between X0 and X2 from the distance calculation reference line at the distances X0 and X2 in the direction of the heading angle of the vehicle are D0 and D2 respectively, and a right triangle can be constructed. Solve for the relative heading angle θ.

Based on the above embodiments, in this embodiment, as shown in FIG. 3 , the movement of the intelligent vehicle adopts a lateral feedback control strategy. Specifically, in view of the highly nonlinear dynamic characteristics and parameter uncertainty of intelligent vehicles, a lateral controller composed of a feedforward controller and a feedback controller that can simulate human driving behavior is designed. In order to improve the control accuracy and eliminate the steady-state error, a lateral feedback control strategy is proposed. The visual detection module determines the relative position and attitude of the intelligent vehicle and the reference line, and compensates the actual angle output through the feedback controller. Through the feedforward and feedback controller of the lateral controller, the actual steering wheel angle of the vehicle is compensated for quickly, smoothly and stably following the expected steering wheel angle, and the vehicle is controlled to travel along the planned path, and the driving safety, stability and ride comfort of the vehicle are ensured. Among them, the feedforward controller is the feedforward information such as the relative heading angle, the lateral deviation, the preview curvature and the longitudinal speed of the vehicle, which are calculated at the forward preview point when the intelligent vehicle based on visual navigation is driving. Strategies and methods, such as regression analysis, neural network and principal component analysis methods, build an algorithmic model to calculate the expected steering wheel angle and control the vehicle to travel along a specified path.

In the present invention, further, the visual detection module includes at least a front-view camera, and the front-view camera is used for capturing images of road conditions ahead. The data processing module further includes a forward warning unit, which is used for processing the image obtained by the forward-looking camera to obtain a lane reference line, and collecting forward obstacle information.

Specifically, the processing of the image obtained by the forward-looking camera includes reading the image, preprocessing the image, and extracting the image features to obtain the lane reference line.

Based on the above embodiment, preprocessing the image includes:

Perform grayscale conversion on the collected image, specifically, grayscale calculation can be performed by a weighted average calculation method;

Image enhancement is performed within a spatial area to improve image clarity. Specifically, methods such as mean filtering or frequency filtering can be used to enhance the image.

In this way, after the above-mentioned image processing, it is possible to prevent noise in the image due to the influence of light, environment and other factors when taking pictures, so as to obtain a clearer image, which is beneficial to subsequent lane recognition.

In the present invention, further, the method for obtaining the lane reference line is: selecting a distance along the direction of the heading angle of the vehicle, sampling uniformly at a certain distance within the distance, and solving the first derivative and the second derivative of the polynomial function fitted by the reference line The order derivative calculates the curvature value of the uniform sampling point, performs statistical analysis on the curvature value of many sampling points, calculates the maximum value, minimum value, mean value and variance, builds an algorithm model to solve the weighted curvature value, and characterizes the lane reference line through the weighted curvature.

Based on the above embodiment, as shown in FIG. 4 , the distance X3 in the direction of the heading angle of the vehicle is selected, and the samples are uniformly sampled at a certain interval from the distance 0 to the distance X3, and the first-order derivative and the second-order derivative are calculated for the polynomial function fitted by the reference line. Uniform the curvature values of the sampling points, and then perform statistical analysis on the curvature values of many sampling points, calculate the maximum value, minimum value, mean value and variance, build an algorithm model to solve the weighted curvature value, and use the weighted curvature to characterize whether the reference line is a left curve or a right curve Corner or straight, and the size of the corner radius.

In the present invention, further, in this solution, the lateral position of the intelligent vehicle is obtained by means of lane line fitting, and when the lateral position of the intelligent vehicle is less than a preset deviation threshold, a lane departure alarm signal is triggered. The method of lane fitting is mainly used to determine the attitude and position of the vehicle.

Specifically, the obtaining of the lateral position of the intelligent vehicle by means of lane line fitting includes indicating the change rate of the relative position of the intelligent vehicle by the relative change of the vehicle offset and the heading angle in the two frames of images:

表示航向角的变化率，

是航向角的变化率，Δc是相邻两帧图像智能车辆偏移量的变化量，Δt是该两帧图像的间隔时间，Δθ是该两帧图像中航向角的变化量。in,

represents the rate of change of the heading angle,

is the change rate of the heading angle, Δc is the change of the offset of the intelligent vehicle in the adjacent two frames of images, Δt is the interval time between the two frames of images, and Δθ is the change of the heading angle in the two frames of images.

可以获得只能车辆的运动偏差以及变化率，以确定智能车辆的位置，如此有利于对只能车辆的动态分析。Real-time acquisition by continuously processing each frame of image

The motion deviation and change rate of the intelligent vehicle can be obtained to determine the position of the intelligent vehicle, which is beneficial to the dynamic analysis of the intelligent vehicle.

After determining the position of the vehicle in the lane, the distance between the intelligent vehicle and the left and right reference lines can be determined. When the distance between the intelligent vehicle and the reference line is less than the preset preset deviation threshold, a lane departure warning is performed, where the deviation threshold is When it is set to 0, it means that the intelligent vehicle presses the line.

In the present invention, further, when waiting for a red light, the vehicle in front has already driven out, and the intelligent vehicle may not be started due to reasons such as the driver himself, which may cause congestion of vehicles behind and insufficient time to pass the light. In order to solve the above solution, the information of the obstacles ahead is collected including the speed of the obstacles in front and the distance of the obstacles in front. When the speed difference between the intelligent vehicle and the obstacle in front is greater than the set speed threshold and the distance difference between the two is greater than the distance threshold, the start warning signal is triggered. That is to say, when the vehicle is waiting for the traffic light, most of the vehicle is in a stationary state. Since the speed of the vehicle will not be too large when it is just started or set off, after the vehicle ahead has driven a certain distance, if the intelligent vehicle It is still in a relatively static state, and the speed difference between the two will also increase. At this time, a start warning will be issued. Through the two judgments of speed and distance, the probability of misjudgment can be effectively reduced.

In this way, this solution also integrates the front warning unit to realize the lane line departure warning in front of the intelligent vehicle, and. In this way, the safe driving of the intelligent vehicle is ensured, the probability of vehicle collision is greatly reduced, and the driving safety of the intelligent vehicle is improved.

It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can also be made to the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may implement the described functionality using different methods for each particular application, but such implementations should not be considered beyond the scope of the present invention.

In the several embodiments provided in this application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit.

It should be understood that, in the embodiments of the present application, the rights, various embodiments, and features can be combined with each other, and the aforementioned technical problems can be solved.

The functions, if implemented in the form of software functional units and sold or used as independent products, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications completed under the technical spirit suggested by the present invention shall belong to This invention covers the scope of the patent.

Claims (10)

1. a collision warning system based on fusion forward-looking camera and rear millimeter wave radar, is characterized in that, comprises The visual detection module is used to capture and process images of road conditions in front of the intelligent vehicle to obtain information on the road conditions ahead; Rear millimeter-wave radar, the user obtains the obstacle information behind the intelligent vehicle, and the rear obstacle information includes the obstacle speed; The data processing module is used to process the above-mentioned road condition information and rear obstacle information, including a rear early warning processing unit, which is used to obtain the collision time according to the relative speed of the rear obstacle and the intelligent vehicle in the y direction. The alarm will be issued when the collision time reaches the rear collision threshold. The calculation formula of the trigger time is:

Among them, t is the trigger time, and V _r is the relative speed of the rear obstacle and the intelligent vehicle in the y direction. 2. a kind of collision warning system based on fusion front-view camera and rear millimeter wave radar according to claim 1, is characterized in that, the acquisition method of the relative velocity V _r of described rear obstacle and intelligent vehicle in y direction is: v _r =(v ₁ -v ₂ )tanθ, where V1 is the vehicle speed of the obstacle, V2 is the speed of the intelligent vehicle, and θ is the heading angle of the intelligent vehicle. 3. a kind of collision warning system based on fusion front-view camera and rear millimeter wave radar according to claim 1, is characterized in that, described relative heading angle acquisition method is: determine the lane reference line of intelligent vehicle, Preview two points in front of the vehicle, and construct a right-angled triangle to obtain the relative heading angle. 4. The collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar according to claim 1, wherein the visual detection module at least comprises a front-view camera, and the front-view camera is used for photographing the front Traffic image. 5 . The collision warning system based on the fusion front-view camera and the rear millimeter wave radar according to claim 4 , wherein the data processing module further comprises a forward warning unit, and the forward warning unit is used to The image obtained by the camera is processed to obtain the lane reference line, and the information of the obstacles ahead is collected. 6 . The collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar according to claim 5 , wherein the processing of the image obtained by the front-view camera comprises reading the image, and pre-processing the image. 7 . Processing, extracting image features to obtain a lane reference line, obtaining the lateral position of the intelligent vehicle by means of lane line fitting, and triggering a lane departure alarm signal when the lateral position of the intelligent vehicle is less than a preset deviation threshold. 7. A collision warning system based on the fusion of front-view camera and rear millimeter-wave radar according to claim 5, characterized in that, collecting front obstacle information includes front obstacle speed and front obstacle distance. When the speed difference of the obstacles in front is greater than the set speed threshold and the distance difference between the two sides is greater than the distance threshold, an early warning signal is triggered. 8. a kind of collision warning system based on fusion front-view camera and rear millimeter wave radar according to claim 6, is characterized in that, the described method of obtaining lane reference line is: select distance along the heading angle direction of vehicle, in this Sampling uniformly at a certain interval within the distance, solve the first-order derivative and second-order derivative of the polynomial function fitted by the reference line, calculate the curvature value of the uniform sampling point, perform statistical analysis on the curvature value of many sampling points, calculate the maximum value, minimum value, The mean and variance are used to construct an algorithm model to solve the weighted curvature value, and the lane reference line is represented by the weighted curvature. 9 . The collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar according to claim 5 , wherein obtaining the lateral position of the intelligent vehicle by means of lane line fitting comprises passing the vehicle in two frames of images. 10 . The offset and the relative change of the heading angle indicate the rate of change of the relative position of the smart vehicle:

in,

represents the rate of change of the heading angle,

is the change rate of the heading angle, Δc is the change of the offset of the intelligent vehicle in the adjacent two frames of images, Δt is the interval time between the two frames of images, and Δθ is the change of the heading angle in the two frames of images. 10 . The collision warning system based on the fusion of a front-view camera and a rear millimeter-wave radar according to claim 1 , wherein the movement of the intelligent vehicle adopts a lateral feedback control strategy. 11 .

Images