CN107886523A - Vehicle target movement velocity detection method based on unmanned plane multi-source image - Google Patents

Abstract

The present invention discloses a kind of vehicle target movement velocity detection method based on unmanned plane multi-source image, comprises the following steps：Build unmanned plane multi-source image acquisition platform；Image acquisitions are carried out using image collection platform, the multi-source image of Visible Light Camera and thermal infrared camera is obtained by the computer main board of unmanned aerial vehicle platform, multi-source image data are transmitted to ground monitoring client by 4G, complete the real-time acquisition of image；The image data of acquisition is corrected, to reduce the geometric distortion of image data；Registration is carried out to the multi-source image data after correction；Fusion is weighted to the polynary image data after registration；Based on the multi-source image data after Weighted Fusion, vehicle target is detected；Vehicle target is tracked；Calculate vehicle target movement velocity.The present invention provides a kind of easy to operate, high efficiency, the speed monitor mode of maneuverability for vehicle monitoring management department.

Description

Velocity detection method of vehicle target based on multi-source image of UAV

technical field

The invention relates to the technical field of video image processing, in particular to a method for detecting the moving speed of a vehicle target based on multi-source images of an unmanned aerial vehicle.

Background technique

With the continuous development of science and technology, modern transportation is becoming more and more important in economic activities, and with the modernization of transportation, more and more traffic problems have emerged. Under such a large environment, Intelligent Traffic System (Intelligent Traffic System, ITS) emerged as the times require, and researchers from all over the world are actively researching and developing to improve traffic monitoring methods, among which computer vision technology provides important technical support for intelligent transportation systems. The detection of vehicle speed is an important part of the intelligent transportation system, and it is also a hot and difficult point in the research. On the one hand, the monitoring of vehicle speed can monitor speeding violations and judge road congestion. On the other hand, it can monitor interested vehicles. And then quickly take relevant measures to maintain road traffic, so as to realize the intelligent traffic system.

Vehicle speed detection is the basis of intelligent traffic monitoring system. At present, most vehicle speed detection relies on electronic camera system for traffic control, so as to realize the purpose of unmanned monitoring. The commonly used methods at home and abroad are: magnetic speed measurement, radar speed measurement, infrared speed measurement, Laser speed measurement and video speed measurement, etc. Among them, the detection technology based on the principle of electromagnetic induction is relatively complicated in layout, and the monitoring location is relatively fixed; the relatively high cost of laser and radar detection technology has certain errors, making it difficult to promote in practical applications. This means that the current vehicle speed monitoring has the problem of poor flexibility of monitoring locations, certain dependence on lighting conditions, and difficulty in detecting a single vehicle of interest.

With the rapid development of UAV (Unmanned Aerial Vehicle), it is easy to operate, can carry multi-task equipment, and complete various tasks. With the development of high-altitude power technology, precision take-off and landing technology, and communication technology, the performance of drones has gradually improved, their functions have been expanded, and their applications have become more extensive. They occupy an indispensable position in civil resource surveys. This means that drones provide a good data collection platform for vehicle object detection. And with the continuous development of thermal infrared cameras, it has good imaging capabilities. The wavelength of thermal infrared exceeds that of visible light, so it can provide more diverse data. In addition, the cost of thermal infrared cameras is also falling.

Contents of the invention

The object of the present invention is to provide a vehicle target motion speed detection method based on UAV multi-source images, which overcomes the defects in the prior art and increases vehicle speed detection The flexibility of the detection method expands the application range of the detection method and reduces the dependence on lighting conditions.

In order to achieve the above object, the technical solution adopted in the present invention is: a method for detecting the moving speed of a vehicle target based on multi-source images of an unmanned aerial vehicle, characterized in that: comprising the following steps:

Step 1. Build a UAV multi-source image acquisition platform;

Step 2, using the image acquisition platform to collect image data;

Step 3, correcting the acquired image data to reduce the geometric distortion of the image data;

Step 4, registering the corrected multi-source image data;

Step 5, performing weighted fusion on the registered multivariate image data;

Step 6. Based on the weighted and fused multi-source image data, the vehicle target is detected;

Step 7, tracking the vehicle target;

Step 8: Calculating the target moving speed of the vehicle.

The scheme of further improvement in above-mentioned technical scheme is as follows:

1. In the above scheme, the UAV multi-source image acquisition platform is a UAV platform equipped with a visible light camera and a thermal infrared camera for multi-source image acquisition.

2. In the above scheme, the third step is to use the traditional checkerboard to perform geometric correction on the acquired visible light image.

3. In the above scheme, the fourth step is to use the homography matrix to register the geometrically corrected optical image and thermal infrared image.

4. In the above solution, the sixth step is to use the weighted and fused multi-source image information to improve the accuracy of vehicle target detection and optimize the position of the target frame.

5. In the above scheme, YOLO (You Only Look Once) deep learning is used to detect vehicle targets.

6. In the above scheme, the detection of the vehicle target in the step 6 specifically includes the following sub-steps:

Sub-step S61, cropping the weighted and fused multi-source images;

Sub-step S62, mark the vehicle target in the multi-source image, and divide the marked vehicle target into a training data set and a test data set;

Sub-step S63, setting training parameters, which mainly include: batch size (batch size), weight decay and learning rate;

Sub-step S64, train the divided training data set to obtain a converged training model;

Sub-step S65 , using the training model to detect vehicle targets one by one, and saving the detection results.

7. In the above scheme, the vehicle target is tracked in the step 7, that is, according to the vehicle target detection result in the step 6, the vehicle is matched and tracked between adjacent video frames, specifically including the following sub-steps:

Sub-step S71, for each pair of consecutive images, extract a set of corresponding SURF (Speeded Up Robust Features) feature points;

Sub-step S72, calculate the relative transformation matrix according to the feature points, and estimate the relative pose of the current view, that is, the position relative to the previous view;

In sub-step S73 , the discrete Kalman filter is used to track and predict the identified vehicle, and the vehicle position is corrected according to the relative transformation matrix in sub-step S72 , so as to prepare for the subsequent calculation of vehicle movement distance.

8. In the above scheme, the eighth step is to calculate the moving speed of the vehicle target, that is, calculate the image resolution based on the known target length, and calculate the pixel movement of the target vehicle according to the vehicle tracking results, and then calculate the vehicle driving distance. The frequency calculates the time interval of image acquisition, and finally calculates the vehicle speed according to the speed calculation formula.

Due to the use of the above-mentioned technical solutions, the present invention has the following advantages compared with the prior art:

The present invention is based on the UAV multi-source image vehicle target motion speed detection method, which is based on a flexible UAV platform to realize the speed detection and monitoring of road vehicles or interested vehicle targets; it can not only flexibly obtain multi-source information on road traffic conditions image, and the addition of thermal infrared images can make this method also applicable to situations without lighting conditions, providing an easy-to-operate, high-efficiency, and flexible vehicle speed monitoring method for vehicle monitoring and management departments.

Description of drawings

Accompanying drawing 1 is the flow chart of the method for detecting the moving speed of a vehicle target in the present invention.

Detailed ways

The present invention will be further described below in conjunction with accompanying drawing and embodiment:

Embodiment: a kind of vehicle target motion speed detection method based on unmanned aerial vehicle multi-source image, it is characterized in that: comprise the following steps:

Step 1. Build a UAV multi-source image acquisition platform;

Step 2: Use the image acquisition platform to collect image data, obtain the multi-source images of the visible light camera and thermal infrared camera through the computer motherboard of the UAV platform, and transmit the multi-source image data to the ground monitoring client through 4G to complete the real-time monitoring of the images Obtain;

Step 3, correcting the acquired image data to reduce the geometric distortion of the image data;

Step 4, registering the corrected multi-source image data;

Step 5, performing weighted fusion on the registered multivariate image data;

Step 6. Based on the weighted and fused multi-source image data, the vehicle target is detected;

Step 7, tracking the vehicle target;

Step 8: Calculating the target moving speed of the vehicle.

The above-mentioned UAV multi-source image acquisition platform is a UAV platform equipped with visible light camera and thermal infrared camera for multi-source image acquisition.

In the third step above, geometric correction is performed on the acquired visible light image using a traditional checkerboard.

Step 4 above, using the homography matrix to register the geometrically corrected optical image and the thermal infrared image.

The sixth step above is to use the weighted and fused multi-source image information to improve the accuracy of vehicle target detection and optimize the position of the target frame.

Vehicle target detection using YOLO (You Only Look Once) deep learning.

The detection of the vehicle target in the above step six specifically includes the following sub-steps:

Sub-step S61, cropping the weighted and fused multi-source images, so that the area ranges of the images are consistent;

Sub-step S62, mark the vehicle target in the multi-source image, and divide the marked vehicle target into a training data set and a test data set;

Sub-step S63, setting training parameters for deep learning, which mainly include: batch size, weight decay and learning rate;

Sub-step S64, train the divided training data set to obtain a converged training model;

Sub-step S65 , using the training model to detect vehicle targets one by one, and saving the detection results.

In the above step seven, the vehicle target is tracked, that is, according to the vehicle target detection result in step six, the matching and tracking prediction of the vehicle between adjacent video frames are carried out, which specifically includes the following sub-steps:

Sub-step S71, for each pair of consecutive images, extract a set of corresponding SURF (Speeded Up Robust Features) feature points;

Sub-step S72, calculate the relative transformation matrix according to the feature points, and estimate the relative pose of the current view, that is, the position relative to the previous view;

In sub-step S73 , the discrete Kalman filter is used to track and predict the identified vehicle, and the vehicle position is corrected according to the relative transformation matrix in sub-step S72 , so as to prepare for the subsequent calculation of vehicle movement distance.

The eighth step above is to calculate the moving speed of the vehicle target, that is, calculate the image resolution based on the known target length, and calculate the pixel movement of the target vehicle according to the vehicle tracking result, and then calculate the vehicle driving distance, and calculate the image acquisition time interval by the frequency of image acquisition , and finally calculate the vehicle speed according to the speed calculation formula.

This implementation is further explained as follows:

The UAV multi-source data acquisition system proposed by the present invention includes: UAV platform, power supply, computer motherboard, ground monitoring client, visible light camera, thermal infrared camera, camera fixing frame, image acquisition card, 4G module and base station, The UAV platform is equipped with a flight controller and has a power system, GPS and battery, etc., and supports module expansion. The computer motherboard, visible light camera and thermal infrared camera are all fixed on the UAV platform. The image acquisition The card is used to ensure that the computer mainboard obtains the image data of the thermal infrared camera. The computer mainboard is equipped with an image acquisition card driver, and adopts the supporting SDK development structure of the image acquisition card. Programming synchronously acquires the acquisition data of the optical camera and the thermal infrared camera. The 4G module is mounted on the computer motherboard and connected to the base station through automatic dialing, and the ground monitoring client is connected to the base station to ensure that the computer motherboard mounted on the drone is connected to the ground monitoring client.

In specific implementation, the output voltage of the power supply is 12V, and the output current is 1A; the visible light camera is an industrial camera connected to USB, with a pixel size of 5.0μm×5.2μm, a rated voltage of 12V, and a rated current of 80mA; the thermal infrared The wavelength range of the camera is 8-14 μm, the resolution is 640×480 (pixels), and the rated voltage is 12V; the camera holder is printed by a 3D printer.

In the specific implementation, the main board of the computer uses JPEG-turbo encoding, a fast jpeg compression encoding library, to compress and encode the acquired multi-source images, and uses boost asio TCP for image transmission, and transmits the compressed images to the ground monitoring client. The client uses the corresponding boost asio TCP to receive the compressed image, and uses JPEG-turbo to decode and display it.

When the speed detection method of vehicle targets based on multi-source images of UAVs is used, the speed detection and monitoring of road vehicles or vehicle targets of interest can be realized, and multi-source images of road traffic conditions can be flexibly obtained, and it is suitable for applications that do not have lighting conditions It provides an easy-to-operate, high-efficiency, and flexible vehicle speed monitoring method for the vehicle monitoring and management department.

The above-mentioned embodiments are only to illustrate the technical concept and characteristics of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and not to limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention shall fall within the protection scope of the present invention.

Claims (9)

1. A kind of 1. vehicle target movement velocity detection method based on unmanned plane multi-source image, it is characterised in that：Including following step Suddenly：

   Step 1: build unmanned plane multi-source image acquisition platform；

   Step 2: carry out image acquisitions using image collection platform；

   Step 3: the image data of acquisition is corrected, to reduce the geometric distortion of image data；

   Step 4: registration is carried out to the multi-source image data after correction；

   Step 5: fusion is weighted to the polynary image data after registration；

   Step 6: based on the multi-source image data after Weighted Fusion, vehicle target is detected；

   Step 7: vehicle target is tracked；

   Step 8: calculate vehicle target movement velocity.
2. 2. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：The unmanned plane multi-source image acquisition platform is carrying Visible Light Camera and thermal infrared camera and carries out multi-source image collection Unmanned aerial vehicle platform.
3. 3. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：It is described Step 3: visible image to acquisition, uses traditional gridiron pattern to carry out geometric correction.
4. 4. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：It is described Step 4: using homography matrix will be after geometric correction optical image and thermal infrared imagery can be subjected to registration.
5. 5. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：It is described Step 6: using the multi-source image information after Weighted Fusion, to improve the precision of vehicle target detection and optimize mesh Mark the position of frame.
6. 6. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：Use YOLO（You Only Look Once）Deep learning detects to vehicle target.
7. 7. the vehicle target movement velocity detection method according to claim 6 based on unmanned plane multi-source image, its feature It is：Detection is carried out to vehicle target in the step 6 and specifically includes following sub-step：

   Sub-step S61, the multi-source image after Weighted Fusion cut；

   Sub-step S62, mark multi-source image in vehicle target, and by the vehicle target marked be divided into training dataset and Test data set；

   Sub-step S63, training parameter is set, wherein mainly including：Criticize size（batch size）, weights decay and learning rate；

   Sub-step S64, the training dataset marked off is trained, obtains convergent training pattern；

   Sub-step S65, application training model detect to vehicle target one by one, and testing result preserves.
8. 8. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：Vehicle target is tracked in the step 7, i.e., according to vehicle target testing result in step 6, vehicle carried out The matching of adjacent video interframe and tracking prediction, specifically include following sub-step：

   Sub-step S71, to each pair consecutive image, SURF corresponding to one group of extraction（Speeded Up Robust Features）It is special Sign point；

   Sub-step S72, relative transform matrix calculated according to characteristic point, estimate the relative attitude of active view, i.e., relative to upper one The position of view；

   Sub-step S73, using Kalman Filtering for Discrete device the vehicle identified is tracked and predicted, according to sub-step S72 In relative transform matrix correct vehicle location, calculate and prepare for follow-up vehicle movement distance.
9. 9. the vehicle target movement velocity detection method according to claim 1 based on unmanned plane multi-source image, its feature It is：It is described Step 8: calculate vehicle target movement velocity, i.e., foundation known target length computation image resolution ratio, and according to Vehicle tracking result calculates the pixel movement of target vehicle, and then calculates vehicle operating range, by the frequency meter for gathering image The time interval of IMAQ is calculated, speed is finally calculated according to speed calculation formula.