CN110069993B

CN110069993B - A target vehicle detection method based on deep learning

Info

Publication number: CN110069993B
Application number: CN201910206458.5A
Authority: CN
Inventors: 瞿三清; 许仲聪; 卢凡; 陈广; 董金虎; 陈凯
Original assignee: Tongji University
Current assignee: Tongji University
Priority date: 2019-03-19
Filing date: 2019-03-19
Publication date: 2021-10-08
Anticipated expiration: 2039-03-19
Also published as: CN110069993A

Abstract

The present invention relates to a target vehicle detection method based on deep learning, comprising the following steps: 1) collecting the tail feature point cloud data of the target vehicle through two single-line laser radars arranged at the tail of the parking robot, and performing preprocessing to obtain a binary image 2) Annotate the binary image to obtain the position of the tail of the target vehicle to generate a training data set; 3) Construct a deep convolutional neural network and its loss function suitable for target vehicle detection; 4) Transfer the training data set After the augmentation, it is input into the deep convolutional neural network, and the parameters in the convolutional neural network are trained and updated according to the difference between the output value and the training true value, and the optimal network parameters are obtained. network is detected. Compared with the prior art, the present invention has the advantages of high robustness, no dependence on manual features, and low detection cost.

Description

Target vehicle detection method based on deep learning

Technical Field

The invention relates to the technical field of intelligent parking, in particular to a target vehicle detection method based on deep learning.

Background

In the field of intelligent driving, detection of a target vehicle is one of key tasks for guaranteeing safe driving of an unmanned vehicle. In the technical field of intelligent parking, the detection of the position of a target vehicle is a key step for realizing the accurate alignment of a parking robot on the target vehicle. Because the laser radar is slightly influenced by the environment and can acquire accurate point cloud data of a target vehicle, the laser radar becomes the most important sensor for detecting and positioning the vehicle in the field of intelligent parking.

At present, in the technical field of intelligent parking, a traditional detection algorithm based on manual characteristics of target vehicles is mainly adopted for a target vehicle detection method. Although the traditional detection algorithm is small in calculated amount and high in speed, the characteristics of the target vehicle cannot be well matched with the manual characteristics in many scenes, so that the traditional detection algorithm for the target vehicle is low in recall rate and poor in robustness.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provide a target vehicle detection method based on deep learning.

The purpose of the invention can be realized by the following technical scheme:

a target vehicle detection method based on deep learning is used for realizing intelligent parking, and comprises the following steps:

1) acquiring tail characteristic point cloud data of a target vehicle through two single-line laser radars arranged at the tail of the parking robot, and preprocessing the data to obtain a binary image;

2) labeling the binary image to obtain the position of the tail of the target vehicle, so as to generate a training data set;

3) constructing a deep convolutional neural network suitable for target vehicle detection and a loss function thereof;

4) and the training data set is input into the deep convolutional neural network after being augmented, parameters in the convolutional neural network are trained and updated according to the difference between the output value and the training true value to obtain the optimal network parameters, and detection is performed according to the trained deep convolutional neural network.

The step 1) specifically comprises the following steps:

11) converting the collected point cloud data into a globally uniform Cartesian coordinate system according to a polar coordinate system taking a single-line laser radar as a coordinate origin;

12) and meshing the point cloud data after coordinate conversion and converting the point cloud data into a binary image.

In the step 11), the conversion expression is:

(x_j0,y_j0)＝(x_j1,y_j1)R+t

wherein (r)_j,φ_j) Is the position coordinate of the point j in the original point cloud data, (x)_j1,y_j1) For point j to a position coordinate in a Cartesian coordinate system with the lidar as the origin of coordinates, (x)_j0,y_j0) And the coordinates in the global unified Cartesian coordinate system, R is a conversion rotation matrix, and t is a conversion translation vector.

In the step 2), the labeling content includes pixel level labeling of the image and boundary frame labeling of the target vehicle.

The deep convolutional neural network is a Faster R-CNN convolutional neural network, takes a binary image with a set size as input, and takes the position and the confidence degree of a corresponding target vehicle on the input binary image as output.

The loss function of the deep convolutional neural network is expressed as:

Loss＝L_cls(p,u)+λ[u＝1]L_loc(t^u,v)

L_cls(p,u)＝-log(p)

x＝(t^u-v)

wherein L is_cls(p, u) detection of loss subfunction for target classification, L_loc(t^uV) is a distance loss subfunction, p is a predictor for the target class, u is an actual factor for the corresponding class, λ is a weighted weight of the loss function, when u is 1, it means that the target vehicle is in the region of interest, when u is 0, it means that the region of interest is the background, t is^uV is in the training sample for the predicted location factorThe true position factor, x is the deviation of the predicted value from the true value.

The specific steps for augmenting the training data set are:

and performing random horizontal turning, cutting and unified zooming on the image to a fixed size, and performing corresponding turning, cutting and zooming on the marking data.

The training deep convolutional neural network specifically comprises the following steps:

and (4) according to the loss function, carrying out iterative updating on the parameters of the deep convolutional neural network by using a gradient descent back propagation method, and taking the network parameters obtained after iteration to the maximum set times as the optimal network parameters to finish training.

Compared with the prior art, the invention has the following advantages:

firstly, the robustness is high: the laser radar can acquire accurate target vehicle point cloud data under various complex working conditions, and a target vehicle detection algorithm with high robustness is used as an auxiliary tool, so that the vehicle detection method has high robustness, and the relative accuracy of a detection result can be ensured under the complex working conditions.

Secondly, do not rely on manual characteristics: compared with the traditional target vehicle detection algorithm, the method provided by the invention has the advantages that the target characteristics are learned through the deep neural network, the manual characteristics are not relied on, and the recall rate of the detection result is high.

Thirdly, the detection cost is low: the invention adopts two single line laser radars as sensors, and has low detection cost compared with a multi-line laser radar.

Drawings

FIG. 1 is a flow chart of the detection method of the present invention.

Fig. 2 is a schematic structural diagram of an intelligent parking robot in an embodiment of the invention.

Fig. 3 is a schematic diagram of a deep convolutional network structure of a target vehicle in an embodiment of the present invention.

Detailed Description

The invention is described in detail below with reference to the figures and specific embodiments.

The invention provides a method for detecting a target vehicle by utilizing a single-line laser radar based on deep learning. As shown in fig. 1, the method comprises the steps of:

(1) acquiring tail characteristic point cloud data of a target vehicle by using 2 single-line laser radars, and preprocessing the acquired point cloud data;

(2) manually marking the position of the tail of the target vehicle in the acquired data to construct a data set for training

(3) Construction of deep convolutional neural network and loss function suitable for target vehicle detection

(4) And (3) amplifying the data used for training in the step (2), and inputting the data into the deep convolutional neural network constructed in the step (3). And training and updating parameters in the convolutional neural network according to the difference between the output value and the training true value, and finally obtaining more ideal network parameters.

In this embodiment, the preprocessing of the point cloud data in step (1) includes two steps of coordinate transformation and image transformation of the point cloud data, as follows:

(1-1) the 2 single-line laser radars in the embodiment are located on two sides of the rear of the intelligent parking robot, the structure of the intelligent parking robot is shown in fig. 2, the two single-line laser radars collect surrounding point cloud data according to a certain ground frame rate, and the collection results take the laser radars as coordinate origin and are stored in a polar coordinate mode. And converting the collected point cloud data into a global unified Cartesian coordinate system of the parking robot.

The conversion expression is as follows:

(x_j0,y_j0)＝(x_j1,y_j1)R+t

in the above formula, (r)_j,φ_j) Representing acquired raw point cloud data(x)_j1,y_j1) Is a representation of a point in the collected point cloud data converted into a cartesian coordinate system with the laser radar as the origin of coordinates. (x)_j0,y_j0) The corresponding point cloud data points are converted to a representation in the global uniform cartesian coordinate system of the parking robot. R is the translation rotation matrix and t is the translation vector.

And (1-2) carrying out imaging processing on the point cloud data. In this embodiment, the upper limit of the distance acquired by the laser radar is set to 10m, the point cloud data after coordinate conversion is gridded, and then the binary image with the size of 250 × 250 is adjusted, if there is a data point in the corresponding grid, the binary image is set to 1, otherwise the binary image is set to 0.

In the present embodiment, step (2) is to construct a data set required for deep learning training. After the collected laser radar point cloud data is processed, training data needs to be manually marked to form a data set required by training. The labeling mode includes, but is not limited to, pixel level labeling of the image, and bounding box labeling of the target vehicle. The position of the target vehicle at least needs to be included during marking, but the attitude information of the target vehicle can be expanded and increased.

In this embodiment, step (3) is to construct a deep convolutional neural network and a loss function suitable for target vehicle detection. The construction of the deep convolutional neural network is directly related to the training data set prepared in the step (2), in the embodiment, the step (2) is marked by using a boundary box of the target vehicle, so that the structure of the deep convolutional neural network is similar to that of the Faster R-CNN in the embodiment, the construction of the main structure refers to the Faster R-CNN, and the structure of the convolutional neural network is shown in FIG. 3.

In this embodiment, in step (3), the loss function of the deep convolutional neural network is constructed with two-part weighting:

Loss＝L_cls(p,u)+λ[u＝1]L_loc(t^u,v)

(3-1) construction of an object Classification loss function L_cls(p, u), where p is the predictor for the target class and u represents the actual factor for the corresponding class. Usually constructed using a log-loss function, where p represents the prediction probability of a certain class,the closer p is to 1, the higher the confidence, the smaller the loss.

L_cls(p,u)＝-log(p)

(3-2) construction of the target detection distance L_loc(t^uV). Where λ represents the weighting weight of the loss function, and may be generally taken to be λ ═ 1. [ u-1 ]]The expression takes 1 when the region of interest is the target vehicle and 0 when the region of interest is the background, i.e., if the current region of interest is an environment-independent thing, its distance loss is not considered. T in the formula^uThe representation represents the predicted location factor and v represents the true location factor in the training sample. The distance-loss sub-function is usually expressed in the form of a smoothed Manhattan distance equation

Constructing the expression: wherein x is (t)^u-v) representing the deviation of the predicted value from the true value.

In this embodiment, the step (4) of augmenting the training data mainly includes performing random horizontal flipping, cropping, and unified scaling on the image to a fixed size, and performing corresponding flipping, cropping, and scaling on the labeled data, and performing normalization processing on the obtained image according to a channel on the basis, where the fixed size adopted in this embodiment is 250 × 250.

In this embodiment, when initializing the training network model in step (4), pre-training the object feature extraction network by using a SoftMax loss function on the ImageNet or other image classification data sets, and using the obtained parameter values as initial parameters of the network.

In this embodiment, when the network is trained in step (4), a weighted loss function is used to calculate a comprehensive loss value, then a back propagation calculation gradient is performed, and an optimizer such as Adam is used to update network parameters, and a final result is obtained by iterating for a certain number of times. And setting the final parameter result as the network model parameter of the target vehicle detector for the detection of the target vehicle.

The invention provides a method for detecting a target vehicle by utilizing a single-line laser radar based on deep learning. The intelligent parking robot has the advantages of excellent detection performance, higher robustness and low implementation cost, and is easy to deploy to the existing intelligent parking robot for detecting the target vehicle.

It will be readily apparent to those skilled in the art that various modifications to these embodiments may be made, and the generic principles described herein may be applied to other embodiments without the use of the inventive faculty. Therefore, the present invention is not limited to the embodiments described herein, and those skilled in the art should make improvements and modifications within the scope of the present invention based on the disclosure of the present invention.

Claims

1. a target vehicle detection method based on deep learning, in order to realize intelligent parking, is characterized in that, comprises the following steps:

1) Collect the tail feature point cloud data of the target vehicle through two single-line lidars set at the tail of the parking robot, and perform preprocessing to obtain a binary image, which specifically includes the following steps:

11) Convert the collected point cloud data from the polar coordinate system with the single-line lidar as the coordinate origin to a globally unified Cartesian coordinate system. The conversion expression is:

(x _j0 ,y _j0 )＝(x _j1 ,y _j1 )R+t

Among them, (r _j , φ _j ) is the position coordinate of point j in the original point cloud data, (x _j1 , y _j1 ) is the position coordinate of point j converted to the Cartesian coordinate system with lidar as the coordinate origin, ( x _j0 , y _j0 ) are the coordinates in the global unified Cartesian coordinate system, R is the transformation rotation matrix, and t is the transformation translation vector;

12) Gridding the point cloud data after coordinate transformation and converting it into a binary image;

2) Annotate the binary image, and obtain the position of the tail of the target vehicle, thereby generating a training data set, and the annotation content includes the pixel-level annotation of the image and the bounding box annotation of the target vehicle;

3) Construct a deep convolutional neural network and its loss function suitable for target vehicle detection. The deep convolutional neural network is a Faster R-CNN convolutional neural network, which takes a binary image of a set size as an input to match The position and confidence of the corresponding target vehicle on the input binary image are output, and the expression of the loss function of the deep convolutional neural network is:

Loss=L _cls (p,u)+λ[u=1]L _loc (t ^u ,v)

L _cls (p,u)=-log(p)

x=(t ^u -v)

Among them, L _cls (p, u) is the target classification detection loss sub-function, L _loc (t ^u , v) is the distance loss sub-function, p is the predictor for the target category, u is the actual factor of the corresponding category, and λ is The weighted weight of the loss function, when u=1, it means that the region of interest is the target vehicle, when u=0, it means that the region of interest is the background, t ^u is the predicted location factor, v is the actual location factor in the training sample, x is the deviation of the predicted value from the actual value;

4) The training data set is augmented and input into the deep convolutional neural network, and the parameters in the convolutional neural network are trained and updated according to the difference between the output value and the training true value to obtain the optimal network parameters, and according to the training A good deep convolutional neural network is used for detection, and the training data set is augmented as follows:

The image is randomly flipped horizontally, cropped and scaled uniformly to a fixed size, and the annotation data is also flipped, cropped and scaled accordingly;

The training of deep convolutional neural network is as follows:

According to the loss function, the gradient descent backpropagation method is used to iteratively update the parameters of the deep convolutional neural network, and the network parameters obtained after iterating to the maximum set number of times are used as the optimal network parameters to complete the training.