CN109934161B - Vehicle identification and detection method and system based on convolutional neural network - Google Patents

Abstract

The invention relates to a vehicle identification and detection method and system based on a convolutional neural network, which are characterized by comprising the following steps: extracting a vehicle picture sample and marking; carrying out region segmentation and category analysis training on the marked picture sample; extracting a group of random continuous images to be identified in the video to be identified; predicting the positions and types of vehicles in all images to be recognized; outputting a motion state of the vehicle; the invention adopts the vehicle identification system to carry out intelligent and automatic management on related vehicles, controls and manages urban traffic by adopting modern technical means, controls the running state of each vehicle in real time, can conveniently realize vehicle dispatching, can guide the dangerous vehicle to track and treat when danger occurs, and lays a solid foundation for timely dredging and efficient monitoring by management departments.

Description

Vehicle recognition and detection method and system based on convolutional neural network

technical field

The invention relates to the technical field of vehicle identification and management, in particular to a method and system for vehicle identification and detection based on a convolutional neural network.

Background technique

With the rapid development of social economy, the number of cars in major cities in various countries is increasing day by day. Illegal parking of vehicles is one of the major causes of traffic congestion. Parking. At present, the traffic department mainly adopts manual patrol to supervise illegal parking. Therefore, the supervision of illegal parking by manual patrol requires a lot of manpower and material resources. Few devices can satisfy real-time performance, accuracy and Validity requirements.

In addition, in recent years, the number of special vehicles such as oil tank trucks, hazardous chemical transport vehicles, dump trucks, and military and police vehicles has also increased. In order to determine the position of a moving vehicle, widely used A method of positioning by using radio signals from the GPS (Global Positioning System: Global Positioning System). However, the accuracy of using GPS to locate the vehicle contains an error of about tens of meters, and it is difficult to determine the detailed position with higher accuracy. Hidden danger.

As an important detection technology, traffic information collection based on video image processing has been widely valued at home and abroad. With the development of society and economy and the progress of science and technology, video detection technology has also achieved rapid development. Video detection products in After experiencing the two important development stages of analog and digital, it is now in the high-definition development stage. At present, high-definition video detection products have appeared in the market. The traffic video detection sensor obtains traffic scene images through the video acquisition equipment located above the road, and uses computer image processing , artificial intelligence, pattern recognition and other technologies to automatically analyze and process scene image information to obtain traffic information. Because it is a non-contact traffic information collection device, it can be installed, debugged and maintained without affecting the operation of the vehicle without closing the road section. At the same time, the video detection sensor can detect multiple lanes at the same time. Traffic monitoring in the scene has the characteristics of low cost, comprehensive and intuitive information, easy maintenance and installation, etc., so it has a high application prospect in intelligent transportation systems.

Contents of the invention

The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a method and system for vehicle identification and detection based on convolutional neural network.

The present invention is achieved through the following technical solutions:

A kind of vehicle identification and detection method based on convolution neural network, it is characterized in that, comprises the following steps:

a. Extract a group of image samples of specific forms of different types of vehicles, and mark all types of vehicles in the image samples;

b. Use mask-rcnn to perform region segmentation and category analysis training on marked image samples;

c. Establish a communication connection with the video monitoring terminal to extract a group of random and continuous images to be recognized in the video to be recognized;

d. Predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation;

e. Output the motion state of the vehicle through video motion tracking;

f. Bind the motion state to the vehicle monitoring business logic, and output monitoring results and instructions.

According to the above technical solution, preferably, the images to be recognized are a group of images selected continuously within a time period randomly extracted from the video to be recognized.

According to the above technical solution, preferably, step e includes: using an object tracking algorithm to track the positions of all vehicles in the image to be recognized in adjacent frames; and outputting the motion state of the vehicle according to the vehicle tracking situation.

According to the above technical solution, preferably, step f includes: when the vehicle is stationary in a specific area, sending a reminder instruction to the monitoring terminal.

According to the above technical solution, preferably, step f includes: sending a reminder instruction to the monitoring terminal when the stationary time of the vehicle exceeds a preset duration.

A vehicle recognition and detection system based on a convolutional neural network, characterized in that it includes: a marking unit for extracting a group of picture samples of specific forms of different types of vehicles, and marking all types of vehicles in the picture samples; the training unit, It is used to use mask-rcnn to perform region segmentation and category analysis training on marked image samples; the extraction unit is used to establish a communication connection with the video monitoring terminal and extract a group of random and continuous images to be recognized in the video to be recognized; vehicle information recognition The unit is used to predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation; the vehicle displacement recognition unit is used to output the motion state of the vehicle through video motion tracking; the monitoring unit is used to bind the motion state Vehicle monitoring business logic, output monitoring results and instructions.

According to the above technical solution, preferably, the vehicle displacement recognition unit includes: a tracking module, which is used to track the positions of all vehicles in the image to be recognized in adjacent frames through an object tracking algorithm; an output module, which is used to output the position of the vehicle according to the vehicle tracking situation state of motion.

According to the above technical solution, preferably, the monitoring unit includes: a first judging module, configured to send a reminder instruction to the monitoring terminal when the vehicle is stationary in a specific area.

According to the above technical solution, preferably, the monitoring unit includes: a second judging module, configured to send a reminder instruction to the monitoring terminal when the stationary time of the vehicle exceeds a preset duration.

The beneficial effects of the present invention are:

Through the images recorded by the video monitoring terminal, the video clips are extracted for identification and analysis, and the motion state of the vehicle is identified through the motion tracking of the video. For the vehicle state that meets specific conditions, the information instruction is transmitted to the vehicle monitoring management center through the Internet, and the vehicle identification system is used to identify the vehicle. Relevant vehicles are intelligently and automatically managed. In order to control and manage urban traffic with modern technical means, and to control the running status of each vehicle in real time, it is convenient to realize vehicle scheduling. When danger occurs, it can guide emergency vehicles to track and deal with it. Lay a solid foundation for timely guidance and efficient monitoring of the management department.

Description of drawings

Fig. 1 is a schematic diagram of the working process of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and preferred embodiments.

As shown in the figure, the present invention discloses a vehicle recognition and detection method based on a convolutional neural network, which is characterized in that it includes the following steps: a. extracting a group of picture samples of specific forms of different types of vehicles, and taking the picture samples All types of vehicles in the vehicle are marked, and the contours of all vehicles in the intercepted picture samples containing various vehicle specific forms are finely marked using the VIA image marking algorithm framework, forming a contour surrounded by multiple closed polygons, and marking the name of the vehicle type at the same time, and Export the marked information into a json file; b. Use mask-rcnn to perform region segmentation and type analysis training on the marked image samples. In this example, use the mask-rcnn algorithm under the tensorflow framework to train the json file samples; c. Combine with video monitoring The terminal establishes a communication connection and extracts a group of random and continuous images to be recognized in the video to be recognized. The video monitoring terminal in this example can be a surveillance camera for monitoring whether ordinary vehicles have invaded a specific area, or it can be a video shot by a satellite. It is used to monitor the motion state of special vehicles in different environments; d. Predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation; e. Output the motion state of vehicles through video motion tracking; f. Bind vehicle monitoring business logic, output monitoring results and instructions. Through the images recorded by the video monitoring terminal, the video clips are extracted for identification and analysis, and the motion state of the vehicle is identified through the motion tracking of the video. For the vehicle state that meets specific conditions, the information instruction is transmitted to the vehicle monitoring management center through the Internet, and the vehicle identification system is used to identify the vehicle. Relevant vehicles are intelligently and automatically managed. In order to control and manage urban traffic with modern technical means, and to control the running status of each vehicle in real time, it is convenient to realize vehicle scheduling. When danger occurs, it can guide emergency vehicles to track and deal with it. Lay a solid foundation for timely guidance and efficient monitoring of the management department.

According to the above-mentioned embodiment, preferably, the image to be identified is a group of images continuously selected in the time period randomly extracted from the video to be identified. In this example, the image algorithm framework opencv is used to import the video to be identified, and a certain amount of images is extracted from the video. Time period, the extracted time period is related to the calculation speed and observation time, it can be infinitely long but not too short, a group of images is randomly selected continuously as the image to be recognized.

According to the above-mentioned embodiment, preferably, step e includes: using an object tracking algorithm to track the positions of all vehicles in the images to be recognized in adjacent frames, starting from the first frame in the image to be recognized, observing the positions of all vehicles in the images of adjacent frames, using The matrix algorithm calculates the center of mass of each vehicle outline, takes the center of mass of each vehicle in the first image as the center, and searches in turn with the preset pixel as the radius, checks the displacement of each vehicle in the next frame, and judges the distance between the object and the object where the center of mass is located Whether the shape difference is within a certain range, if it is, the current object is considered to be the object after the displacement of the object where the center of mass of the previous image is located, otherwise it is considered to be a different type of vehicle; output the motion state of the vehicle according to the vehicle tracking situation, if within a certain period of time If the center of mass of the same vehicle does not move, the vehicle is considered to be at rest.

According to the above-mentioned embodiment, preferably, step f includes: when the vehicle is stationary in a specific area, sending a reminder instruction to the monitoring terminal; when the vehicle is in a stationary state at a prohibited parking position, the system sends a reminder instruction to the traffic management center, It enables traffic managers to quickly judge the movement status of vehicles, improves the intelligence level of road surveillance video, and provides traffic managers with timely and effective means and basis for handling accidents.

According to the above-mentioned embodiment, preferably, step f includes: when the stationary time of the vehicle exceeds the preset duration, sending a reminder instruction to the monitoring terminal, setting a reasonable alarm threshold for the stationary duration of the vehicle, When the alarm threshold is exceeded, the system sends a reminder command to the monitoring terminal, which is convenient for real-time control of the running status of each vehicle. When danger occurs, it can guide the rescue vehicle to track and deal with it, laying a solid foundation for the management department to guide and monitor efficiently.

The present invention also discloses a vehicle identification and detection system based on convolutional neural network, which is characterized in that it includes: a marking unit, used to extract a group of picture samples of different types of vehicles with specific forms, and perform all types of vehicles in the picture samples Marking; training unit, used to use mask-rcnn to perform region segmentation and type analysis training on marked image samples; extraction unit, used to establish a communication connection with the video monitoring terminal, and extract a group of random and continuous images to be identified in the video to be identified Image; vehicle information recognition unit, used to predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation; vehicle displacement recognition unit, used to output the motion state of the vehicle through video motion tracking; monitoring unit, used for Bind the motion state to the vehicle monitoring business logic, and output monitoring results and instructions.

According to the above-mentioned embodiment, preferably, the vehicle displacement identification unit includes: a tracking module, used to track the positions of all vehicles in the images to be identified in adjacent frames through an object tracking algorithm; an output module, used to output the position of the vehicle according to the vehicle tracking situation state of motion.

According to the above embodiment, preferably, the monitoring unit includes: a first judging module, configured to send a reminder instruction to the monitoring terminal when the vehicle is stationary in a specific area.

According to the above embodiment, preferably, the monitoring unit includes: a second judging module, configured to send a reminder instruction to the monitoring terminal when the stationary time of the vehicle exceeds a preset duration.

Through the images recorded by the video monitoring terminal, the video clips are extracted for identification and analysis, and the motion state of the vehicle is identified through the motion tracking of the video. For the vehicle state that meets specific conditions, the information instruction is transmitted to the vehicle monitoring management center through the Internet, and the vehicle identification system is used to identify the vehicle. Relevant vehicles are intelligently and automatically managed. In order to control and manage urban traffic with modern technical means, and to control the running status of each vehicle in real time, it is convenient to realize vehicle scheduling. When danger occurs, it can guide emergency vehicles to track and deal with it. Lay a solid foundation for timely guidance and efficient monitoring of the management department.

The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.

Claims (3)

1. A vehicle identification and detection method based on convolutional neural network, is characterized in that, comprises the following steps: a. Extract a group of image samples of specific forms of different types of vehicles, and mark all types of vehicles in the image samples; b. Use mask-rcnn to perform region segmentation and category analysis training on marked image samples; c. Establish a communication connection with the video monitoring terminal to extract a group of random and continuous images to be recognized in the video to be recognized; d. Predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation; e. Use the object tracking algorithm to track the position of all vehicles in the image to be recognized in the adjacent frame, start from the first frame in the image to be recognized, observe the positions of all the vehicles in the image of the adjacent frame, and use the matrix algorithm to calculate the centroid of each vehicle outline, Take the center of mass of each vehicle in the first image as the center of the circle, search in turn with the preset pixel as the radius, check the displacement of each vehicle in the next frame, and judge whether the shape difference between the object and the object where the center of mass is located is within the preset range, if so It is considered that the current object is the object after the displacement of the object where the center of mass of the previous image is located, otherwise it is considered to be a different type of vehicle; the motion state of the vehicle is output according to the vehicle tracking situation, if the center of mass of the same vehicle does not move within the preset time, the vehicle is considered to be at rest; f. Bind the motion state to the vehicle monitoring business logic, and output the monitoring results and instructions. Among them, when the vehicle’s stationary time exceeds the preset duration, a reminder instruction is sent to the monitoring terminal, and a reasonable alarm threshold for the vehicle’s stationary duration is set. When the vehicle is stationary When the residence time in some specific environments exceeds the alarm threshold, the system will send a reminder command to the monitoring terminal. 2. A method for vehicle identification and detection based on convolutional neural networks according to claim 1, wherein the images to be identified are a group of images selected continuously within a time period randomly extracted from the video to be identified. 3. A vehicle identification and detection system based on convolutional neural network, characterized in that, comprising: The marking unit is used to extract a group of picture samples of specific forms of different types of vehicles, and mark all types of vehicles in the picture samples; The training unit is used to perform region segmentation and category analysis training on marked image samples using mask-rcnn; The extraction unit is used to establish a communication connection with the video monitoring terminal to extract a group of random and continuous images to be identified in the video to be identified; The vehicle information recognition unit is used to predict the position and type of vehicles in all images to be recognized through mask-rcnn semantic segmentation; The vehicle displacement recognition unit is used to track the position of all vehicles in the image to be recognized in the adjacent frame through the object tracking algorithm, start from the first frame in the image to be recognized, observe the positions of all the vehicles in the image of the adjacent frame, and use the matrix algorithm to calculate the position of each vehicle The center of mass of the outline is located in the first image, with the center of mass of each vehicle in the first image as the center, and the preset pixel is used as the radius to search in turn, check the displacement of each vehicle in the next frame, and judge whether the shape difference between the object and the object where the center of mass is located is within the preset Set the range, if it is, the current object is considered to be the object after the displacement of the object where the center of mass of the previous image is located, otherwise it is considered to be a different type of vehicle; output the motion state of the vehicle according to the vehicle tracking situation, if the same vehicle within the preset time If the center of mass does not move, the vehicle is considered to be at rest; The monitoring unit is used to bind the motion state to the vehicle monitoring business logic, output monitoring results and instructions, wherein, when the vehicle's stationary time exceeds the preset duration, send a reminder instruction to the monitoring terminal, and set a reasonable vehicle stationary duration alarm threshold, When the vehicle stays in some specific environment for more than the alarm threshold, the system will send a reminder command to the monitoring terminal.

Images