Specific implementation method
Further illustrate the present invention program in the following with reference to the drawings and specific embodiments.
The present invention is based on the vehicle speed measuring methods of video encoder server and Radar Signal Fusion, include the following steps:
Step 1, vehicle identification tracking: vehicle video information is acquired using camera, carries out the identification and tracking of vehicle.Make
For a kind of specific embodiment, vehicle identification is carried out using YOLOv3 algorithm, carries out vehicle tracking using Kalman filter tracking.
The process of vehicle identification is carried out as shown in figure 4, the specific method is as follows using YOLOv3 algorithm:
The frame picture read to camera is normalized, and is divided into S × S-grid, selects object centre bit
The detection that the grid fallen in is responsible for object is set, since each bounding box possesses 5 parameters, the respectively centre coordinate (x of target0,
y0), wide w, high h and bounding box confidence level, by centre coordinate (x0, y0) normalized to relative to target's center's cell at place
Between 0-1, the wide and high of w, h image itself is normalized between 0 and 1;
Bounding box confidence level is determined according to the bounding-box perimeter after normalizationWherein Pr (object)
Indicate bounding box a possibility that containing target, when there is target in bounding box, Pr (object)=1, when in bounding box without target
When, Pr (object)=0,It is expressed as the accuracy of bounding box, calculation formula isIts
Middle BoxtruthRepresent true object boundary frame, BoxpredThe object boundary frame for representing prediction, according toThreshold value removal
The relatively low window of possibility removes redundancy window using non-maxima suppression, exports final target detection frame.
Kalman filter tracking specifically choose vehicle center-of-mass coordinate be vehicle trace point, by tracking be divided into initialization,
Prediction, matching and corrigendum four-stage, steps are as follows:
Vehicle is appeared in into the pixel coordinate in first frame image as the initial value of Kalman filter tracking;
Kalman prediction is carried out to current all moving targets, since the movement of object in a short time can be seen
Work is linear uniform motion, thus define its state-transition matrix A be A=[1,0, △ t, 0;0,1,0,△t;0,0,1,0;0,
0,0,1], wherein △ t indicates that video reads the time of a frame, and the prediction result of state matrix and present frame is substituted into state transfer
Equation obtains the status predication value of next frame;
Multiple tracking targets are matched, since the time that video handles a frame is short, so the operating range of vehicle is not
Can be far, set threshold value a T, xk+1,xkIt is the abscissa of video next frame and present frame vehicle centroid, y respectivelyk+1,ykRespectively
It is the ordinate of video next frame and present frame vehicle centroid, when matching appears in the pixel in video before and after frames using vehicle and sits
Mark calculates Euclidean distanceWhen minimum Eustachian distance be less than threshold value T and corresponding tracking vehicle with
When the marker of detection vehicle is all displayed without successful match, it will test target and tracking target matched, otherwise, it is determined that vehicle
It fails to match;
The estimated value of Kalman filtering is modified, the estimated value of obtained vehicle centroid coordinate and measured value are utilized
Kalman's coefficient can calculate the optimal estimation value of current vehicle center-of-mass coordinate.The present invention may be implemented using Kalman filter
To the real-time estimation of target, without hysteresis effect.
Step 2, measurement of testing the speed: the echo-signal obtained using radar determines speed, the distance and bearing information of vehicle.
Radar emits signal by transmitter and transmit-receive switch, and receiver receives the small-signal being reflected back when encountering object.By
It is high-frequency signal in the signal received, needs to carry out it data processing, the flow chart of Radar Signal Processing is as shown in Figure 5.
As a kind of specific embodiment, measurement of testing the speed method particularly includes:
Echo-signal that receiver receives is subjected to A/D and Frequency mixing processing is converted to Beat Signal, and using window function into
Row FFT transform does non-coherent accumulation processing to obtained spectrogram to inhibit the interference of other target clutters;
It carries out Ordered Statistic CFAR detection and if it is greater than thresholding, then judges target by processing result compared with threshold value
In the presence of, the decoupling operation of speed and distance is carried out, since the frequency of Beat Signal is made of speed and range information, according to
The peak value spectral line of spectrogram calculates the phase difference between consecutive frame, using not fuzzy distance and beat signal frequency solve away from
From range-to-go is obscured to obtain, formula is substituted intoThe radial velocity of vehicle can be solved, wherein f is beat letter
Number frequency, μ is chirp rate, and c is the light velocity, and R is the distance of target, and V is the radial velocity of target, f0It is radar emission signal
Frequency;
When the azimuth information of radar is based on radar antenna radiated electromagnetic wave, antenna beam axis and echo when target alignment
The characteristics of signal is most strong, and echo-signal dies down when antenna beam axis and target deviation determines the direction of target.The present invention according to
Wave path-difference determines the azimuth of vehicleCalculation formula isWherein, λ is the wavelength of radar, and Δ R is to measure its wave
Path difference.
As a kind of more specific embodiment, the transmitted waveform of radar uses step-by-step movement multi order linear frequency keying (MS-
LFSK) waveform, the thinking of MS-LFSK Waveform Design is that waveform is divided into two parts of upper sweep frequency band and lower sweep frequency band, in such as Fig. 1
Shown in 2, each frequency range generates the waveform of a variety of different frequencies respectively, generates a variety of not fuzzy distances with this.Upper sweep frequency band
It is made of M linear frequency keying signal, the frequency keying time width of each LFSK signal is Tstep, swept bandwidth B, kth+1
A signal and k-th of signal have fixed difference on the frequency fshfit k=fk+1-fk, the identical upper sweep frequency band of lower sweep frequency band.
Step 3, result fusion: according to the vehicle distances and azimuth information of detections of radar, vehicle centroid coordinate and row are determined
Direction is sailed, the vehicle centroid coordinate and driving direction that it is obtained with video detection match, and vehicle is shown on video
Velocity information.As a kind of specific embodiment, as a result merge method particularly includes:
First matching: the vehicle heading of detections of radar is compared with the vehicle heading of video tracking, if
Unanimously, then first successful match, into Secondary Match, otherwise it fails to match;
Secondary Match: the three-dimensional vehicle space coordinate of detections of radar is projected into image coordinate system, calculates the matter of vehicle
Heart coordinate (u, v), if the height of radar installation is H, the angle with horizontal plane is θ, and the distance that radar measures vehicle is R, orientation
Angle is ψ, X=Rsin ψ, Z=Rcos ψ of the vehicle under radar fix system,Y
=H-h1, the Y-axis information that the Y calculated is the absence of, according to camera monocular calibration principle [u, v, 1]=K × [R × [X, Y,
Z, 1]+T] three dimensional space coordinate of vehicle is projected in image coordinate system, the center-of-mass coordinate (u of vehicle is calculatedcal,
vcal), wherein K is the internal reference of camera, is obtained according to Zhang Zhengyou chessboard calibration method, and R and T are that radar is put relative to camera respectively
The spin matrix and translation matrix put;The vehicle centroid of the vehicle centroid coordinate (u, v) and video detection that calculate detections of radar is sat
The Euclidean distance D of mark (u ', v ');When Euclidean distance D is less than threshold value T, then the vehicle and video detection of detections of radar are judged
Vehicle is same vehicle, and shows the speed of radar measuring car above this vehicle in video;When Euclidean distance D is greater than
When threshold value T, then it is assumed that it fails to match.
Embodiment 1
In order to verify the validity of the method for the present invention, road carries out vehicle speed measuring at optional one, specific as follows:
One frame image of video acquisition, is normalized into 416 × 416 for picture, 13 × 13 grids is then split into, if vehicle
Center fall in the grid, then this grid just be responsible for detect this vehicle.
Grid can generate 3 bounding boxes, and each bounding box is using sigmoid function to the center of vehicle, target on boundary
Probability and detection classification confidence level in frame are predicted.It calculatesSelectionIt is maximum
Bounding box predicted, the detection block of final output target.
Using the center of the detection block of output as the initial value of Kalman filter tracking, Kalman is carried out to current vehicle
Filter tracking.When tracking 2 automobiles simultaneously, the Euclidean distance of pixel coordinate in vehicle before and after frames is calculated, if Euclidean distance
Minimum value be less than threshold value 50, be determined as that the vehicle of before and after frames tracking is same, and find sub-minimum and continue to track
Matching;If it is greater than threshold value 50, it is determined as that vehicle tracking is lost, re-establishes new tracking.
At the same time, radar emission MS-LFSK signal, and AD sampling and data rearrangement are carried out to echo-signal, obtain difference
Clap signal.FFT processing is done to every group of LFSK Beat Signal, obtains spectrogram Wk[N] does cyclic graph accumulation to it, obtains amplitude-frequency
ResponseOrdered Statistic CFAR detection be greater than thresholding and in the form of peak value existing for detect it is single
Member.If one of detection unit is p, for frequency expression corresponding to the detection unit where spectral line are as follows:Calculate the corresponding phase of peak value spectral line p in spectrogram
Adjacent phase is obtained as differenceAccording to not fuzzy distancePhase differenceWith Beat Signal fMS-LFSKCalculate corresponding fuzzy distance Then basis
Range ambiguity resolving goes out distance and speed apart from velocity solution couple solution.
The azimuth of vehicleIt can be determined by wave path-difference,Wherein, λ is the wavelength of radar, measures its wave path-difference
Δ R, so that it may determine target direction
The distance, speed and azimuth information of detections of radar to vehicle are transferred to video by host computer and handle port.Depending on
The center-of-mass coordinate for the vehicle that frequency processor record detects when starting, continues the mass center of calculating vehicle during the tracking of vehicle
Coordinate can finally judge the traveling of vehicle according to the placement position of camera from video according to the center-of-mass coordinate difference of vehicle
Direction.Then radar, which is transmitted through the speed come, also may determine that the driving direction of vehicle from positive and negative dividing.If video detection
Vehicle Speed is consistent with the radar biography direction of car speed, then enters second step and match;Otherwise it is assumed that it fails to match.
In second step matching, if the height of radar installation is H, the angle with horizontal plane is θ, and installation is as shown in Figure 3.Thunder
It is R up to the distance for measuring vehicle, azimuth is ψ, X=Rsin ψ, Z=Rcos ψ of the vehicle under radar fix system,Y=H-h1, Y-axis information that the Y calculated is the absence of.According to camera
The three dimensional space coordinate of vehicle can be projected to image seat by monocular calibration principle [u, v, 1]=K × [R × [X, Y, Z, 1]+T]
In mark system, the center-of-mass coordinate (u of vehicle is calculatedcal, vcal).Wherein K is the internal reference of camera, can be according to Zhang Zhengyou chessboard mark
The method of determining obtains, the spin matrix and translation matrix that R and T, which are radar respectively, to be put relative to camera, can put according to 2
Position determines that [u, v] is the pixel coordinate that the three dimensional space coordinate of object projects in image coordinate system.Video to vehicle into
Also center-of-mass coordinate (the u of available one group of vehicle during row detectionreal, vreal).If threshold value T is 30, if metThen judge that the vehicle of video detection and the vehicle that radar is surveyed are same
Vehicle shows the speed of corresponding vehicle in video;IfThen it fails to match.
From the present embodiment as can be seen that the present invention can test the speed simultaneously to more automobiles, and it can use video
The travel situations of intuitive display road vehicle.Not only rate accuracy with higher in this way, and it is visual strong, it is cheap.