CN107886043A - The vehicle front-viewing vehicle and pedestrian anti-collision early warning system and method for visually-perceptible - Google Patents

Abstract

The invention discloses a kind of vehicle front-viewing vehicle of visually-perceptible and pedestrian anti-collision early warning system and method, the video obtained to vehicle front-viewing camera is read frame by frame, and with training cascade classifier, vehicle in frame of video, pedestrian are identified for realization, and in combination with straight-line detection, to be filtered to recognition result, non-vehicle, non-pedestrian part are removed, is completed using depth DBN networks to this front side vehicle, the judgement of pedestrian information.After confirming front truck, pedestrian position, image coordinate is converted into car body coordinate, so as to obtain Ben Che and front truck, Ben Che and above pedestrian relative distance and relative angle, when calculating safe away from, again with Kalman filter prediction vehicle, pedestrian position, if less than it is safe when away from then sending alarm.The present invention can realize more efficient and accurate traffic identification, so as to more accurately carry out anticollision early warning.

Description

Vehicle and pedestrian anti-collision warning system and method based on visual perception

technical field

The invention relates to the technical field of image processing, and in particular to a visual perception vehicle and pedestrian anti-collision warning system and method.

Background technique

It is particularly important for a car to recognize cars and pedestrians in its front-sight area in a complex traffic environment. Through the recognition of forward-looking vehicles and pedestrians, it can effectively make up for the lack of the driver's sensory ability, thereby reducing traffic accidents. Therefore, the recognition of forward-looking vehicles and pedestrians when the car is driving has important theoretical and practical significance.

With the advancement of science and technology, vehicle and pedestrian recognition technology has also been developed, but in the actual implementation process, as the scene and shape change, the recognition effect is not satisfactory, and various recognition algorithms only judge and recognize based on image features , thus limiting the use of various algorithms.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention aims to provide a visual perception vehicle and pedestrian anti-collision warning system and method, using a deep learning algorithm to obtain images of vehicles and pedestrians by performing feature training and extraction on a large number of samples The highest-level representation of the image of vehicles and pedestrians is understood from another angle, so that the computer has a deep understanding of the image, and at the same time completes the recognition of vehicles and pedestrians.

In order to achieve the above object, the present invention adopts the following technical solutions:

Vision-aware forward-looking vehicle and pedestrian anti-collision warning systems, including:

Vehicle front-view area acquisition module: used to collect the video information of the vehicle front-view area and transmit it to the frame acquisition module;

Frame acquisition module: used to continuously acquire the video information collected by the vehicle front view area acquisition module frame by frame, for detection by the Haar detection module;

Haar detection module: for adopting the Haar classifier to identify the area where the front vehicle or pedestrian is located in the frame information obtained by the frame acquisition module;

Horizontal line matching module: used to detect the area obtained by the Haar detection module and obtain the area where the horizontal line is detected, compare the area obtained by the horizontal line detection with the area obtained by the Haar detection module, and eliminate the Haar detection module Regions detected but not detected in the horizontal line matching module;

DBN network detection module: used to perform DBN network detection on the area obtained after processing by the horizontal line matching module, and obtain the final area where the vehicle or pedestrian in front is located;

Collision warning module: It is used to predict whether the detected distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is located, and if so, issue a collision warning.

Further, the horizontal line matching module includes a contour detection module, a horizontal line detection module, and a comparison module; wherein:

The contour detection module is used to perform contour detection on the area detected by the Haar detection module;

The horizontal straight line detection module is used to detect the horizontal straight line on the detection result obtained by the contour detection module, and only keep the horizontal straight line to obtain the area with the horizontal straight line;

The comparison module is used to compare the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and remove the area not detected by the horizontal straight line detection module in the area of the preceding vehicle or pedestrian detected by the Haar detection module.

Further, the DBN network detection module includes an RBM layer training module and a Softmax regression module; wherein:

RBM layer training module: used to perform RBM unsupervised training with the area of the front vehicle or pedestrian obtained after processing by the horizontal line matching module as input;

Softmax regression module: used to classify the RBM unsupervised training results obtained by the RBM module, judge whether the detected area is the area where the vehicle in front or the pedestrian is located, and if so, mark it.

Furthermore, the DBN network detection module includes at least three layers of RBM layer training modules, and the output of the previous layer of RBM layer training modules is used as the input of the subsequent layer of RBM layer training modules.

Further, the DBN network detection module also includes a BP feedback adjustment module, which is used for fine-tuning the feedback of the entire DBN network.

Further, the collision warning module includes a coordinate conversion module, a Kalman filter module, a safety distance calculation module, and a comparison module; wherein:

Coordinate transformation module: used to lock the vehicle or pedestrian in front and perform coordinate transformation according to the detection result finally obtained by the DBN network detection module, and determine the position of the vehicle or pedestrian in front in the vehicle body coordinate system;

Kalman filter module: used to use Kalman filter to predict the distance between the car and the vehicle or pedestrian in front at the next time;

Safety distance calculation module: used to calculate the safety distance between the car and the vehicle in front or the car and pedestrians;

Comparison module: compare the predicted distance between the car and the vehicle in front or the pedestrian with the safety distance between the car and the vehicle in front or the pedestrian, if the safety distance between the car and the vehicle in front or the pedestrian is greater than the predicted distance between the car and the pedestrian in the next time A collision warning will be issued if the distance between the vehicle or pedestrian in front is exceeded.

The anti-collision warning method for vehicles and pedestrians using the above-mentioned system includes the following steps:

S1 installs the car front-view area acquisition module under the car logo in front of the car. After the system is started, it first collects video information through the car front-view area acquisition module, and transmits the collected video to the frame acquisition module;

The S2 frame acquisition module continuously acquires the frame information of the video information, and the acquired frame information exists in the form of pictures;

The S3Haar detection module uses a Haar classifier to detect and identify the front vehicle and pedestrian in the frame information, and obtain the area where the front vehicle or pedestrian is located;

The S4 horizontal straight line matching module detects the horizontal straight line on the area obtained by the Haar detection module and obtains the area where the horizontal straight line is detected, compares the area obtained by the horizontal straight line detection with the area obtained by the Haar detection module, and eliminates the area detected by the Haar detection module But there is no detected area in the horizontal line matching module;

The S5DBN network detection module performs DBN network detection on the areas obtained after the processing of the horizontal line matching module, and further identifies whether these areas are vehicles or pedestrians in front, and if so, mark them to obtain the final area where the vehicles or pedestrians in front are located;

The S6 collision warning module predicts whether the distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is locked, and if so, issues a collision warning.

It should be noted that step S4 is specifically:

4.1) The contour detection module carries out contour detection to the area detected by the Haar detection module;

4.2) The horizontal straight line detection module carries out horizontal straight line detection to the detected contour, and after the detection is completed, only the horizontal straight line is retained to obtain the area with the horizontal straight line;

4.3) the comparison module compares the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and removes the area that is not detected by the horizontal straight line detection module in the area of the vehicle in front or pedestrians that the Haar detection module detects. Obtain the detection result of the horizontal line matching module.

It should be noted that step S5 is specifically:

5.1) Utilize the RBM layer training module to complete the RBM unsupervised training on the area where the vehicle in front and the pedestrian are located after being processed by the horizontal line matching module, and complete the feature extraction;

5.2) Use the Softmax regression module to classify the extracted features, and identify the actual front vehicle and pedestrian areas in the area where the front vehicles and pedestrians are located after being processed by the horizontal line matching module.

It should be further explained that 5.3) is also included: use the BP feedback adjustment module to adjust the entire DBN network, improve the learning effect, and make the parameter state of the DBN network reach the optimal state.

It should be further noted that at least three layers of RBM layer training modules are included, the output of the previous layer of RBM layer training modules is used as the input of the next layer of RBM layer training modules, and the output of the last layer of RBM layer training modules is used as the input of the Softmax regression module .

It should be noted that, in step 5.1), the area of the vehicle in front and pedestrians obtained after processing by the horizontal line matching module is converted into an image form, scaled to a set size, and adaptive binarization is performed, and then the raster order Scan into an array form, and the formed array is used as the input of the RBM layer training module.

The beneficial effect of the present invention is that: vehicles and pedestrians are non-rigid objects, and different vehicles and pedestrians have different forms, but there are always some invariable characteristics between the form changes. The present invention processes the pictures of vehicles and pedestrians through adaptive binarization, which can better retain the characteristics of vehicles and pedestrians, and is sufficient to distinguish them from other objects, and uses the processing results to train the DBN network. Feature extraction for better identification.

Description of drawings

Fig. 1 is a schematic flow chart of an embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that the following examples are based on the technical solution and provide detailed implementation and specific operation process, but the scope of the present invention is not limited to this implementation example.

As shown in Figure 1, the vision-aware car forward-looking vehicle and pedestrian anti-collision warning system includes:

Vehicle front-view area acquisition module: used to collect the video information of the vehicle front-view area and transmit it to the frame acquisition module;

Frame acquisition module: used to continuously acquire the video information collected by the vehicle front view area acquisition module frame by frame, for detection by the Haar detection module;

Haar detection module: for adopting the Haar classifier to identify the area where the front vehicle or pedestrian is located in the frame information obtained by the frame acquisition module;

Horizontal line matching module: used to detect the area obtained by the Haar detection module and obtain the area where the horizontal line is detected, compare the area obtained by the horizontal line detection with the area obtained by the Haar detection module, and eliminate the Haar detection module Regions detected but not detected in the horizontal line matching module;

DBN network detection module: used to perform DBN network detection on the area obtained after processing by the horizontal line matching module, and obtain the final area where the vehicle or pedestrian in front is located;

Collision warning module: It is used to predict whether the detected distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is located, and if so, issue a collision warning.

Further, the horizontal line matching module includes a contour detection module, a horizontal line detection module, and a comparison module; wherein:

The contour detection module is used to perform contour detection on the area detected by the Haar detection module;

The horizontal straight line detection module is used to detect the horizontal straight line on the detection result obtained by the contour detection module, and only keep the horizontal straight line to obtain the area with the horizontal straight line;

The comparison module is used to compare the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and remove the area not detected by the horizontal straight line detection module in the area of the preceding vehicle or pedestrian detected by the Haar detection module.

Further, the DBN network detection module includes an RBM layer training module and a Softmax regression module; wherein:

RBM layer training module: used to perform RBM unsupervised training with the area of the front vehicle or pedestrian obtained after processing by the horizontal line matching module as input;

Softmax regression module: used to classify the RBM unsupervised training results obtained by the RBM module, judge whether the detected area is the area where the vehicle in front or the pedestrian is located, and if so, mark it.

Furthermore, the DBN network detection module includes at least three layers of RBM layer training modules, and the output of the previous layer of RBM layer training modules is used as the input of the subsequent layer of RBM layer training modules.

Further, the DBN network detection module also includes a BP feedback adjustment module, which is used for fine-tuning the feedback of the entire DBN network.

Further, the collision warning module includes a coordinate conversion module, a Kalman filter module, a safety distance calculation module, and a comparison module; wherein:

Coordinate transformation module: used to lock the vehicle or pedestrian in front and perform coordinate transformation according to the detection result finally obtained by the DBN network detection module, and determine the position of the vehicle or pedestrian in front in the vehicle body coordinate system;

Kalman filter module: used to use Kalman filter to predict the distance between the car and the vehicle or pedestrian in front at the next time;

Safety distance calculation module: used to calculate the safety distance between the car and the vehicle in front or the car and pedestrians;

Comparison module: compare the predicted distance between the car and the vehicle in front or the pedestrian with the safety distance between the car and the vehicle in front or the pedestrian, if the safety distance between the car and the vehicle in front or the pedestrian is greater than the predicted distance between the car and the pedestrian in the next time A collision warning will be issued if the distance between the vehicle or pedestrian in front is exceeded.

The anti-collision warning method for vehicles and pedestrians using the above-mentioned system includes the following steps:

S1 installs the car front-view area acquisition module under the car logo in front of the car. After the system is started, it first collects video information through the car front-view area acquisition module, and transmits the collected video to the frame acquisition module;

The S2 frame acquisition module continuously acquires the frame information of the video information, and the acquired frame information exists in the form of pictures;

The S3Haar detection module uses a Haar classifier to detect and identify the front vehicle and pedestrian in the frame information, and obtain the area where the front vehicle or pedestrian is located;

The S4 horizontal straight line matching module detects the horizontal straight line on the area obtained by the Haar detection module and obtains the area where the horizontal straight line is detected, compares the area obtained by the horizontal straight line detection with the area obtained by the Haar detection module, and eliminates the area detected by the Haar detection module But there is no detected area in the horizontal line matching module;

The S5DBN network detection module performs DBN network detection on the areas obtained after the processing of the horizontal line matching module, and further identifies whether these areas are vehicles or pedestrians in front, and if so, mark them to obtain the final area where the vehicles or pedestrians in front are located;

The S6 collision warning module predicts whether the distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is locked, and if so, issues a collision warning.

It should be noted that step S4 is specifically:

4.1) The contour detection module carries out contour detection to the area detected by the Haar detection module;

4.2) The horizontal straight line detection module carries out horizontal straight line detection to the detected contour, and after the detection is completed, only the horizontal straight line is retained to obtain the area with the horizontal straight line;

4.3) the comparison module compares the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and removes the area that is not detected by the horizontal straight line detection module in the area of the vehicle in front or pedestrians that the Haar detection module detects. Obtain the detection result of the horizontal line matching module.

It should be noted that step S5 is specifically:

5.1) Utilize the RBM layer training module to complete the RBM unsupervised training on the area where the vehicle in front and the pedestrian are located after being processed by the horizontal line matching module, and complete the feature extraction;

5.2) Use the Softmax regression module to classify the extracted features, and identify the actual front vehicle and pedestrian areas in the area where the front vehicles and pedestrians are located after being processed by the horizontal line matching module.

It should be further explained that 5.3) is also included: use the BP feedback adjustment module to adjust the entire DBN network, improve the learning effect, and make the parameter state of the DBN network reach the optimal state.

It should be further noted that at least three layers of RBM layer training modules are included, the output of the previous layer of RBM layer training modules is used as the input of the next layer of RBM layer training modules, and the output of the last layer of RBM layer training modules is used as the input of the Softmax regression module .

It should be noted that, in step 5.1), the area of the vehicle in front and pedestrians obtained after processing by the horizontal line matching module is converted into an image form, scaled to a set size, and adaptive binarization is performed, and then the raster order Scan into an array form, and the formed array is used as the input of the RBM layer training module.

The present invention will be further illustrated and described below.

After the vehicle is started, the visual perception vehicle and pedestrian anti-collision warning system is activated through the on-board equipment in the vehicle, and the software and hardware of each part of the system start to work.

At this time, you can enter the system control panel, and choose whether to enter the learning mode on the control panel. If you choose to enter the learning mode, the system enters the DBN network learning module, and continuously updates the parameters after learning and training until it is optimal. After adjusting the parameters, the system can start to identify the vehicle in front and pedestrians. Specifically

1. Vehicle front view area acquisition module (camera) acquisition

A camera is installed under the front logo of the car. After the system is started, the video information is first collected through the camera, and the collected video is transmitted to the frame acquisition module.

2. The frame acquisition module realizes the continuous acquisition of the frame information of the video file. The obtained frame exists in the form of a picture, and the next step is to complete the processing of the picture.

3. Use the Haar classifier to detect the front car and pedestrian in the video. This link is the focus of the preliminary work, and the goal of this link is to detect the front vehicle and pedestrians in the frame information. The method adopted is: through the training based on the cascade classifier, the cascade classifier is used to realize the recognition of vehicles and pedestrians in the frame. The training method of the cascade classifier is described below.

1) Produce positive sample description files

2) Generate a classifier.

The Haar classifier cascades the strong classifiers trained by the AdaBoost algorithm, and adopts rectangular features and integral maps. This classifier is a "Haar feature checker" generated by thresholding the sum or difference of pixels in a rectangular area, which consists of the sum or difference of a rectangular area of the image. As a result of cascade classifier training, the generated xml file can be used for vehicle and pedestrian detection, but the detection results are not good. When vehicles and pedestrians are detected, there will also be false detection areas. In order to improve the detection of error regions, a horizontal line matching module needs to be introduced.

Cascade classifier training requires a large number of samples to be trained, and the number of negative samples is controlled to be 3 times that of positive samples.

4. Use the horizontal line matching module to process the detection results of the Haar classifier. During the running of vehicles and pedestrians, at least one horizontal straight line is often formed, and the length of the horizontal straight line is smaller than the width of the detection area of vehicles and pedestrians. For the misdetection part of the detection result of the Haar classifier, such straight lines are often lacking, so the horizontal straight line detection can be introduced into the detection of the preceding vehicle and pedestrians to improve the accuracy rate of the preceding vehicles and pedestrians.

Its specific implementation steps are as follows:

1) Contour detection;

2) Line detection;

3) Keep only horizontal straight lines;

4) Comparing the sample collection area with the horizontal straight line area;

5) Eliminate the regions where the comparison fails.

Vehicles and pedestrians will form a certain degree of horizontal straight line, and match the horizontal straight line with the detection area to eliminate a certain degree of false detection results.

5. DBN network processing

The present invention utilizes the deep learning method to complete the judgment of the vehicle in front and the pedestrian. At the top of the model, softmax regression is used for processing, and then backpropagation is used for parameter tuning, that is to say, it is a deep learning model composed of multi-layer unsupervised learning and one-layer supervised learning.

The DBN network is mainly divided into the following parts: first, use RBM to complete the feature extraction, then use the Softmax regression algorithm to classify the extracted features, and finally use BP feedback to adjust the entire DBN network, so that the DBN network has a better recognition effect.

In DBN, each layer will undergo RBM unsupervised training. And the output of the first layer will be used as the input of the second layer to train the second layer of RBM, so until the last layer of RBM, in this process, each layer of RBM uses the comparison and divergence algorithm to update the parameters, and finally completes Extract feature information. The present invention uses a three-layer RBN for feature extraction.

At the top of DBN, the softmax algorithm is used to classify the data obtained by unsupervised training, which is the so-called supervised learning.

BP performs feedback fine-tuning on the entire deep learning network to improve the learning effect and make the parameter state of the DBN network optimal.

The core content of deep learning-based recognition and detection of vehicles and pedestrians is deep learning, and the recognition of vehicles and pedestrians is completed by using deep learning. The present invention adopts a hidden layer composed of three RBM layers to extract the features of the original data, so that the data can be classified using the classification algorithm more efficiently, that is, the classification work performed by the final Softmax regression layer.

The specific implementation method is: convert the area detected by the horizontal line matching module into an image form, scale it to a specified size, and perform adaptive binarization processing, and then scan it into an array form in raster order, and then perform the formation of the array Carry out DBN network detection, judge whether the area is a vehicle or pedestrian in front according to the detection result, and mark it if it is a vehicle or pedestrian in front.

By introducing the DBN network, it has a better effect on judging the front vehicle and pedestrians detected in the video.

Due to real-time problems, the training parameters of the previous DBN network are directly introduced into the program, and the assignment operation is completed for the DBN network. In this way, a DBN network can be quickly constructed.

5.1 Data collection

Before deep learning training, samples required for training need to be collected, including positive samples (vehicles or pedestrians) and negative samples (non-vehicles or non-pedestrians). Positive samples only contain vehicles or pedestrians, and negative samples contain everything that may appear on the road, but do not contain any vehicles or pedestrians.

5.2 Sample handling

Due to the input relationship of deep learning, all the collected sample pictures must be scaled to a uniform size of 32*32, and then processed in grayscale, so that the image is converted into a matrix composed of grayscale pixel values of 32*32. Since a large amount of data needs to be input once to train the DBN network, this embodiment converts the 32*32 matrix into a 1*1024 one-dimensional array in raster order, and integrates all samples into a large two-dimensional matrix.

Similarly, the processing of non-vehicle and non-pedestrian image information is completed in this way. And store the processed information in a file for next use.

5.3 Training set and test set generation module

In order to ensure the effectiveness of DBN network training, the present invention allows all vehicles and non-vehicles, pedestrians and non-pedestrians to be cross-distributed, which not only enables the DBN network to be trained effectively, but also simplifies label assignment.

5.4 RBM layer training module

One layer of the restricted Boltzmann machine is the visible layer, and the other layer is the hidden layer, but there is no connection between the layers, the visible layer and the hidden layer are fully connected, and the internal unit states of each layer are independent of each other, which is overcome by Gibbs sampling. The RBM distribution instability problem caused by the independence of the internal unit states of each layer.

Use c to represent the data of the display layer, which is converted from image information; h represents the data of the hidden layer, which is the feature extraction of the image. W represents the weight between the data, and a and b represent the bias of c and h, respectively.

Introducing the contrastive divergence algorithm into the RBM model greatly improves the performance of the RBM, both in terms of training effect and training speed. Specific steps are as follows:

a. Calculate the probability of hidden layer nodes with the existing visible layer training samples, and obtain the hidden layer data h;

b. Use h to construct the visible layer c', and then use c' to construct the hidden layer h';

c. Update weights.

Use the comparison and divergence algorithm to complete the training of the RBM layer. The settings for other parameters are as follows: Weight initialization: each number comes from a random number in the interval (-1/N, 1/N), and N represents the number of neural units in the input layer. number; learning rate: 0.1/N; weight decay: reduce 0.0002 on the basis of the source; RBM training process is layer-by-layer training, first use the input data to train the first layer of RBM, after training, use the output of the first layer of RBM as the second The input of the first layer trains the second layer RBM, which is trained layer by layer.

5.5 Softmax regression module

The softmax regression module targets multi-classification problems. x _i is a type of data, k corresponds to the number of classification results, A is a parameter, each data corresponds to k different classification results, and among the k types of results, each data represents the probability of each type of x _i results. As long as the parameters are processed, the desired classification result can be obtained. Train parameters A such that A minimizes the cost function. Next, the parameters are updated in an iterative manner, and the weight attenuation term is introduced, so that the cost function becomes a convex function, ensuring the uniqueness of the solution, and thus obtaining the global optimal solution. Implement the Softmax regression algorithm to solve the classification problem, classify the data, and perform Softmax regression on the features extracted by the RBM layer by layer.

5.6 BP feedback fine-tuning module

The weight parameters are updated using the backpropagation method. BP feedback adjustment is performed in the DBN network. The DBN network has completed the calculation of the activation values of all RBM nodes, including the output of the final softmax layer. For each layer of the network, its output cannot fully reach the state required by the target. From the DBN network The last layer is calculated forward layer by layer, and the residual of this layer network and the input data of this layer network are used to update the parameters of this layer. a _l is the output of L layer, δ _l is the residual of L layer, and the residual of L-1 layer can be calculated by using the residual of L layer δ _l-1 , a _l-1 is the output data of L-1 layer, L Layer input data, using a _l-1 and δ _l can calculate the update amount of parameter A.

Using the features extracted by RBM to convert to the source data layer by layer, the backpropagation algorithm fine-tunes the entire neural network through feedback to reduce the degree of data loss and improve the accuracy of feature extraction.

6. Coordinate transformation

After solving the constraint equation of the external parameters of the camera, the relative relationship between the camera, the image and the vehicle body coordinate system can be completely determined, so that the radar scanning point can be projected onto the image pixel coordinate system through the camera model. Establish the conversion relationship between the object point P(x _v ,y _v ,z _v ) in the vehicle body coordinate system and the image point p(u,v) in the image pixel coordinate system. Its pixel-level data fusion model is as follows:

(u,v,l) ^T ＝K(R _c (x _v ,y _v ,z _v ) ^T +T _c ) (1)

The internal parameter matrix of the camera is

In the above model, l is the coefficient of (u, v) written in homogeneous coordinates, T _c is the translation matrix, R _c is the rotation matrix of the external parameters of the camera, which is the translation vector of the external parameters of the camera, and f _x and f _y are x and the equivalent focal length in the y direction, u ₀ and v ₀ are the coordinates of the image pixel center. The spatial data fusion is completed through the formula, and the first purpose of the data fusion is to combine the camera coordinate system, the image pixel coordinate system and the vehicle body coordinate system together. The unification and establishment of the coordinate system is conducive to the measurement of the specific distance and orientation of the environment and obstacles by the environmental perception sensor. The calibration of the external parameters of the camera can realize the inverse perspective projection transformation and provide important parameters for vehicle visual navigation.

7. Kalman filter

The present invention uses the Kalman filter to predict the change of the vehicle speed in the next time, and applies it to the subsequent calculation of the safety distance.

First, use the third-order Kalman filter to predict the effective target information obtained in the current period, and select the state x _n =[d _n,e ,v _n,e ,a _n,e ] ^T , where d _n,e ,v _{n , e} , a _{n, and e} are the Y-direction relative distances, speeds, and accelerations between the “forward-looking vehicle and pedestrian” and the vehicle obtained through processing in the nth detection cycle, and the predicted value of the target state in the next cycle is as follows: Shown:

In the formula, t is the period, and the value is 0.05s; d _(n+1)|n , v _(n+1)|n , a _(n+1)|n are the predicted (n+1)th The Y-direction relative distance, velocity and acceleration between the valid target and the vehicle within the detection period. The predicted value of effective target information in the (n+1)th detection period is obtained through formula (2), and the consistency check with the primary target information in this period is carried out, and the following judgment criteria are adopted, as shown in formula (3) :

In formula (3), d _n+l , v _n+1 , and a _n+1 are the y-direction relative distances and speeds between the primary selected "foresight vehicles and pedestrians" and the workshop in the (n+1)th detection cycle, respectively. and acceleration; d ₀ , v ₀ , a ₀ are the maximum allowable Y-direction relative distance, velocity and acceleration errors respectively. d ₀ , v ₀ , a ₀ are mainly determined by the measurement error and the error generated by using formula (2) for prediction, and in the present invention:

[d ₀ v ₀ a ₀ ] ^T ＝[3 2 0.25] ^T (4)

For the primary selection "forward-looking vehicles and pedestrians" obtained in the (n+1)th cycle, if its information value satisfies formula (4), it is considered that the primary selection target is consistent with the effective target obtained in the nth cycle, and then proceed Update of target information.

8. In the present invention, data such as speed and acceleration of "the vehicle in front and pedestrians" can be calculated, so when designing the safety distance algorithm, it should conform to people's driving habits, for example, the distance between the vehicle in front and pedestrians relative to the vehicle. If the speed is very high, the safety distance should be appropriately reduced. If the "vehicle in front and pedestrian" are driving with a relatively large positive acceleration, the safety distance should also be appropriately reduced. The "time distance between people" is inversely proportional to the relative speed of "two vehicles or vehicle-pedestrian" and inversely proportional to the acceleration of "the vehicle in front and pedestrian". The formula for calculating the safe inter-vehicle interval t' is as follows:

t'=T-a'*v_re-b'*ar_1 (5)

In the formula, T is a fixed value of 1.5s, and v_re is the relative speed v_re=vr_1-v_1 of "vehicle-vehicle or vehicle-pedestrian". vr_1 is the speed of the "vehicle in front and pedestrian" after 1 second of predicted time, and v_1 is the speed of the vehicle after 1 second of predicted time. ar_1 is the acceleration of the "vehicle in front and pedestrian" after 1 second of the predicted time. a', b' are positive constants.

Since the "vehicle-to-vehicle time distance and vehicle-to-human time distance" cannot be negative and cannot be too large, the saturation function is specifically used to make the safe "vehicle-to-vehicle, vehicle-to-human" time distance more reasonable, and the obtained safe "vehicle-to-vehicle, The calculation formula of the time interval t' between vehicles and people is shown in formula (6)

t'_min is the minimum value of the set safe "vehicle-to-vehicle, vehicle-to-person" time interval, and t'_max is the set maximum value of the safe "vehicle-to-vehicle to vehicle-to-human" time interval. The formula for the final safe "vehicle-to-vehicle, vehicle-to-human" time distance is as follows:

safdis=v_1*t'+d' (7)

The specific implementation of the safe "vehicle-to-vehicle, vehicle-to-person" time distance is as follows:

Step1: Calculate the "two vehicles or vehicles - pedestrians" time distance, if the "two vehicles or vehicles - pedestrians" time distance is greater than 2.2s, set the "two vehicles or vehicles - pedestrians" time distance to 2.2s, if "two vehicles or vehicles - pedestrians" time distance is greater than 2.2s If the time distance between vehicles and pedestrians is less than 0.2s, set the time distance between two vehicles or vehicles and pedestrians to 0.2s.

t'=1.5-0.2*v_re-1.2*ar_1;

ift'>2.2

t'=2.2;

end

ift'<0.2

t'=0.2;

end

Step2: Calculate the safe distance.

safdis=v_1*t'+2.5;

Formula 6 and Formula 7 can dynamically adjust the safety distance between "two vehicles or vehicle-pedestrian" according to the real-time data of the operation of the vehicle and the "vehicle in front and pedestrian", so that the calculation of the safety distance has a stronger predictive ability , and the ability to resist interference. Finally, the stability and dynamic performance of the "vehicle-to-vehicle, vehicle-to-person" safety time distance when the external environment changes are improved.

Vehicles and pedestrians are non-rigid objects, and different vehicles and pedestrians have different forms, but there are always some invariable characteristics between these form changes. The present invention processes the pictures of vehicles and pedestrians through self-adaptive binarization, which can better retain the characteristics of the vehicles and pedestrians. And enough to distinguish it from other objects. And the processing results are used to train the DBN network and perform feature extraction for better recognition.

The present invention uses deep learning and superimposes multiple RBM DBN networks to extract features layer by layer, uses the Softmax regression layer in the final stage, and uses labels for supervision and training, and finally uses BP feedback adjustment to adjust the entire network. After efficiently and accurately identifying the vehicle and pedestrian in front, the relative position of the vehicle in front and pedestrian is determined according to the coordinate transformation, and the distance value relationship between vehicle-vehicle and vehicle-pedestrian in the next time is predicted by Kalman filter, and the "vehicle-to-vehicle, vehicle-to-vehicle The "human world" safety time distance adjusts the safety distance, which improves the safety of the vehicle.

For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims (10)

1. A car forward-looking vehicle and pedestrian anti-collision warning system with visual perception, characterized in that it includes: Vehicle front-view area acquisition module: used to collect the video information of the vehicle front-view area and transmit it to the frame acquisition module; Frame acquisition module: used to continuously acquire the video information collected by the vehicle front view area acquisition module frame by frame, for detection by the Haar detection module; Haar detection module: for adopting the Haar classifier to identify the area where the front vehicle or pedestrian is located in the frame information obtained by the frame acquisition module; Horizontal line matching module: used to detect the area obtained by the Haar detection module and obtain the area where the horizontal line is detected, compare the area obtained by the horizontal line detection with the area obtained by the Haar detection module, and eliminate the Haar detection module Regions detected but not detected in the horizontal line matching module; DBN network detection module: used to perform DBN network detection on the area obtained after processing by the horizontal line matching module, and obtain the final area where the vehicle or pedestrian in front is located; Collision warning module: It is used to predict whether the detected distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is located, and if so, issue a collision warning. 2. the car front-view vehicle and pedestrian anti-collision warning system of visual perception according to claim 1, is characterized in that, described horizontal straight line matching module comprises contour detection module, horizontal straight line detection module, comparison module; Wherein: The contour detection module is used to perform contour detection on the area detected by the Haar detection module; The horizontal straight line detection module is used to detect the horizontal straight line on the detection result obtained by the contour detection module, and only keep the horizontal straight line to obtain the area with the horizontal straight line; The comparison module is used to compare the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and remove the area not detected by the horizontal straight line detection module in the area of the preceding vehicle or pedestrian detected by the Haar detection module. 3. the automobile front view vehicle of visual perception according to claim 1 and pedestrian anti-collision warning system, it is characterized in that, DBN network detection module comprises RBM layer training module and Softmax regression module; Wherein: RBM layer training module: used to perform RBM unsupervised training with the area of the front vehicle or pedestrian obtained after processing by the horizontal line matching module as input; Softmax regression module: used to classify the RBM unsupervised training results obtained by the RBM module, judge whether the detected area is the area where the vehicle in front or the pedestrian is located, and if so, mark it. 4. the automobile front view vehicle of visual perception according to claim 3 and pedestrian anti-collision warning system, it is characterized in that, described DBN network detection module comprises at least three layers of RBM layer training modules, the previous layer of RBM layer training modules The output is used as the input of the RBM layer training module of the next layer; The DBN network detection module also includes a BP feedback adjustment module for fine-tuning the entire DBN network. 5. the automobile front-view vehicle and pedestrian anti-collision warning system of visual perception according to claim 1, is characterized in that, described collision warning module comprises coordinate conversion module, Kalman filter module, safe distance calculation module, comparison module; in: Coordinate transformation module: used to lock the vehicle or pedestrian in front and perform coordinate transformation according to the detection result finally obtained by the DBN network detection module, and determine the position of the vehicle or pedestrian in front in the vehicle body coordinate system; Kalman filter module: used to use Kalman filter to predict the distance between the car and the vehicle or pedestrian in front at the next time; Safety distance calculation module: used to calculate the safety distance between the car and the vehicle in front or the car and pedestrians; Comparison module: compare the predicted distance between the car and the vehicle in front or the pedestrian with the safety distance between the car and the vehicle in front or the pedestrian, if the safety distance between the car and the vehicle in front or the pedestrian is greater than the predicted distance between the car and the pedestrian in the next time A collision warning will be issued if the distance between the vehicle or pedestrian in front is exceeded. 6. Utilize the automobile forward-looking vehicle of system described in claim 1 and pedestrian anti-collision early warning method, it is characterized in that: comprise the steps: S1 installs the car front-view area acquisition module under the car logo in front of the car. After the system is started, it first collects video information through the car front-view area acquisition module, and transmits the collected video to the frame acquisition module; The S2 frame acquisition module continuously acquires the frame information of the video information, and the acquired frame information exists in the form of pictures; The S3Haar detection module uses a Haar classifier to detect and identify the front vehicle and pedestrian in the frame information, and obtain the area where the front vehicle or pedestrian is located; The S4 horizontal straight line matching module detects the horizontal straight line on the area obtained by the Haar detection module and obtains the area where the horizontal straight line is detected, compares the area obtained by the horizontal straight line detection with the area obtained by the Haar detection module, and eliminates the area detected by the Haar detection module But there is no detected area in the horizontal line matching module; The S5DBN network detection module performs DBN network detection on the areas obtained after the processing of the horizontal line matching module, and further identifies whether these areas are vehicles or pedestrians in front, and if so, mark them to obtain the final area where the vehicles or pedestrians in front are located; The S6 collision warning module predicts whether the distance between the vehicle in front and the pedestrian and the car is less than the safe distance according to the final area where the vehicle in front or the pedestrian is locked, and if so, issues a collision warning. 7. The method according to claim 6, wherein step S4 is specifically: 4.1) The contour detection module carries out contour detection to the area detected by the Haar detection module; 4.2) The horizontal straight line detection module carries out horizontal straight line detection to the detected contour, and after the detection is completed, only the horizontal straight line is retained to obtain the area with the horizontal straight line; 4.3) the comparison module compares the area obtained by the horizontal straight line detection module with the area detected by the Haar detection module, and removes the area that is not detected by the horizontal straight line detection module in the area of the vehicle in front or pedestrians that the Haar detection module detects. Obtain the detection result of the horizontal line matching module. 8. The method according to claim 6, characterized in that step S5 is specifically: 5.1) Utilize the RBM layer training module to complete the RBM unsupervised training in the area obtained after the horizontal line matching module is processed, and complete the feature extraction; include at least three layers of RBM layer training modules, and the output of the previous layer of RBM layer training modules is used as the latter The input of the RBM layer training module of the first layer, and the output of the RBM layer training module of the last layer is used as the input of the Softmax regression module. 5.2) Use the Softmax regression module to classify the extracted features, and identify the actual area where the vehicle in front or the pedestrian is located in the area obtained after processing by the horizontal line matching module. 9. The method according to claim 8, further comprising 5.3): utilizing the BP feedback adjustment module to adjust the entire DBN network to improve the learning effect so that the parameter state of the DBN network reaches optimum. 10. The method according to claim 8, characterized in that, in step 5.1), the region obtained after processing by the horizontal line matching module is converted into a picture form, scaled to a set size, and adaptive binarization is performed , and then scanned into an array form in raster order, and the formed array is used as the input of the RBM layer training module.