Disclosure of Invention
The invention aims to provide a method, a device and electronic equipment for detecting vehicle violation events, which can accurately detect the violation events of vehicles in various scenes and reduce the occurrence frequency of the vehicle violation events.
In order to solve the technical problem, an embodiment of the present invention provides a method for detecting a vehicle violation event, including: acquiring image data of all vehicles in a preset area in real time, wherein the image data comprises at least 2 frames of images in the area; determining the motion track of each vehicle in a preset area according to the image data; determining a traffic incident to which each vehicle belongs currently according to the motion trail of each vehicle and a traffic incident determination model, wherein the traffic incident determination model is a corresponding relation between the motion trail of the vehicle and the traffic incident to which the vehicle belongs; and outputting alarm information indicating that the vehicles belong to the illegal traffic events according to the traffic events to which each vehicle belongs.
The embodiment of the invention also provides a device for detecting the vehicle violation event, which comprises the following steps: the device comprises an acquisition module, a first determination module, a second determination module and an information output module; the acquisition module is used for acquiring image data of all vehicles in a preset area in real time, wherein the image data comprises images in at least 2 frame areas; the first determining module is used for determining the motion track of each vehicle in a preset area according to the image data; the second determining module is used for determining a traffic incident to which each vehicle belongs at present according to the motion trail of each vehicle and a traffic incident determining model, wherein the traffic incident determining model is a corresponding relation between the motion trail of the vehicle and the traffic incident to which the vehicle belongs; the information output module is used for outputting alarm information indicating that the vehicles belong to the illegal traffic incident according to the traffic incident to which each vehicle belongs.
An embodiment of the present invention also provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method for vehicle violation event detection described above.
Compared with the prior art, the method and the device have the advantages that the motion trail of each vehicle in the preset area is determined by acquiring the image data of all vehicles in the preset area and according to the frame images in the image data; the traffic incident to which the current vehicle belongs can be determined according to the movement track of each vehicle and the traffic incident determination model, different detection methods are not required to be set for the vehicles according to different detection requirements, the range of applicable scenes for detecting violation incidents of the vehicles is widened, the traffic incident to which the current vehicle belongs can be accurately determined only by detecting once, and the detection speed of the vehicles is accelerated; because the traffic events of the vehicles are related to the motion states of the vehicles, the acquired motion tracks of each vehicle and the traffic event determination model jointly determine the traffic events of the vehicles, so that the accuracy of detecting the traffic events can be improved, and the occurrence of road violation events can be effectively reduced.
In addition, prior to determining the traffic events to which each vehicle currently belongs, the method of vehicle violation event detection further comprises: acquiring historical image data of vehicles in a first preset time period in a preset area, and acquiring a traffic incident to which each vehicle belongs in the first preset time period; determining the historical motion track of each vehicle in a preset area within a first preset time period according to historical image data; and constructing a traffic incident determination model according to the historical motion trail and the traffic incident to which each vehicle belongs within the first preset time period. A large amount of historical data is used as a basis for constructing a traffic incident determination model, so that the determined traffic incident determination model is more accurate.
In addition, the traffic incident determination model is constructed in a deep neural network mode, wherein an input layer of the traffic incident determination model comprises n nodes, each node corresponds to one sampling point of the motion trail of each vehicle, an output layer comprises m output nodes, each output node corresponds to one traffic incident, and n and m are integers larger than 1. A traffic incident determination model is constructed in a deep neural network mode, so that the traffic incident determination model is more practical and has high accuracy; the number of sampling points of the motion trail corresponds to the number of nodes, so that the deep neural network model can be accurately constructed.
In addition, determining the current traffic incident of each vehicle according to the motion trail of each vehicle and the traffic incident determination model specifically comprises: acquiring the number of nodes of a traffic incident determination model input layer; respectively preprocessing the motion trail of each vehicle according to the number of nodes of the input layer; and taking the preprocessed motion trail of each vehicle as the input of a traffic incident determination model to determine the current traffic incident of each vehicle. And preprocessing the motion trail of each vehicle according to the number of nodes of the input layer of the traffic determination model so that the motion trail of each vehicle conforms to the input of the traffic incident determination model.
In addition, according to the number of nodes of the input layer, the motion trail of each vehicle is preprocessed, and the method specifically comprises the following steps: the following preprocessing is carried out on the motion track of each vehicle: judging whether the number of sampling points contained in the motion trail is equal to the number of nodes or not, if the number of the sampling points contained in the motion trail is determined to be not equal to the number of the nodes, adjusting the number of the sampling points in the motion trail so that the number of the sampling points in the adjusted motion trail is equal to the number of the nodes. Through adjusting the number of sampling points to ensure that no matter how many actual sampling points are few, can all determine corresponding traffic incident according to the sampling point in the motion trail.
In addition, after obtaining the number of nodes of the traffic event determination model input layer, the method for vehicle violation event detection further comprises: and carrying out normalization processing on the motion track of each vehicle. Through normalization operation, the subsequent rapid processing of the motion trail is facilitated, and meanwhile, due to the fact that the motion trail is mapped between 0 and 1, large errors caused by large distance difference of two sampling points in the motion trail are reduced.
In addition, according to the image data, determining a motion track of each vehicle in a preset area specifically comprises: detecting the position of each vehicle in each frame of image in a preset area; the following processing is performed for each vehicle in the image: identifying vehicles in the images of the continuous frames, and taking the positions of the vehicles in each frame of image as sampling points in the motion trail of the vehicles; and determining the motion trail of the vehicle according to the sampling points of the motion trail of the vehicle in the area. The method can accurately identify the vehicles in each frame of image, and can accurately track the position of each vehicle in the image due to the continuity of the image, so that the corresponding motion track of each vehicle in the preset area image can be quickly and accurately determined, and the time for determining the motion track is shortened.
In addition, the method for determining the traffic incident of each vehicle by taking the preprocessed motion trail of each vehicle as the input of a traffic incident determination model specifically comprises the following steps: each vehicle is processed as follows: obtaining the confidence coefficient of each traffic event corresponding to the current vehicle; and determining the current traffic event of the vehicle according to the confidence coefficient. And the traffic incident to which the vehicle belongs can be objectively determined according to the confidence coefficient.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, it will be appreciated by those of ordinary skill in the art that numerous technical details are set forth in order to provide a better understanding of the present application in various embodiments of the present invention. However, the technical solution claimed in the present application can be implemented without these technical details and various changes and modifications based on the following embodiments.
A first embodiment of the invention relates to a method of vehicle violation event detection. The method for detecting the vehicle violation event is applied to a terminal or electronic equipment, and the terminal or the electronic equipment is installed at the positions of a road, a parking lot and the like needing to detect the vehicle violation event so as to detect vehicles in a preset area in real time. The specific flow of the vehicle violation event detection method is shown in fig. 1.
Step 101: and acquiring image data of all vehicles in a preset area in real time, wherein the image data comprises images in at least 2 frame areas.
Specifically, the image data of all vehicles in the preset area can be collected in real time by using equipment for collecting image data, such as a camera and infrared monitoring equipment. The preset area can be determined according to a scene of vehicle violation event detection, and the detected scene can be a parking lot, a highway intersection or a road and the like. The preset area may be set as a maximum acquisition range of the acquisition device for acquiring the current scene, for example, if the acquisition device is a camera and the detection scene is an intersection position, a range of an image shot by the camera at the current intersection may be used as the preset area. Since the acquisition device is usually fixedly arranged in the detection scene, the preset area is fixed.
The image data includes at least 2 frames of images in the preset area, for example, if a camera is used to collect images in the preset area in real time, the collected images are consecutive frames of images.
It should be noted that, before determining the current traffic event of each vehicle, a traffic event determination model of the current preset area needs to be constructed in advance. The manner in which the traffic event determination model is determined may be as follows.
In a specific implementation, historical image data of vehicles in a first preset time period in a preset area is obtained, and a traffic incident to each vehicle in the first preset time period is obtained; determining the historical motion track of each vehicle in a preset area within a first preset time period according to historical image data; and constructing a traffic incident determination model according to the historical motion trail and the traffic incident to which each vehicle belongs within the first preset time period.
Specifically, there are various scenes in which it is necessary to detect whether a vehicle violates a rule, and the preset areas are different, so that the traffic event determination model corresponding to each preset area can be determined. The traffic event determination model may be constructed in a neural network learning manner. The traffic event determination model may be constructed by learning from a large amount of historical data. The first preset time period may be selected from historical time periods, and a time period adjacent to the current time may be selected for learning accuracy. For example, if it is currently 1/2018, the first preset time period may be a time period between 1/2017 and 31/2017/12/2017. The method comprises the steps of obtaining historical image data of vehicles in a preset area within a first preset time period, identifying the vehicles in each frame of image, determining the motion track of each vehicle in the preset area image according to continuous frame images, taking the historical motion track of each vehicle as input data of a traffic event model, taking the acquired traffic event to which each vehicle belongs as an output result of the traffic event model, and constructing the traffic event model in the preset area by utilizing a machine learning algorithm or a neural network algorithm. It will be appreciated that the present embodiment is not limited to the algorithm for constructing the traffic event determination model.
Step 102: and determining the motion track of each vehicle in the preset area according to the image data.
In one specific implementation, the position of each vehicle in a preset area in each frame of image is detected; the following processing is performed for each vehicle in the image: identifying vehicles in the images of the continuous frames, and taking the positions of the vehicles in each frame of image as sampling points in the motion trail of the vehicles; and determining the motion trail of the vehicle according to the sampling points of the motion trail of the vehicle in the area.
Specifically, the vehicles in each frame of image included in the image data are identified, the position of each vehicle in the image is detected, the preset area is fixed, and the acquisition device is also fixed, so that the background (a fixed building) in each frame of image is the same, and the movement track of each vehicle in the image of the preset area can be determined through the position information of each vehicle in different frame of images, wherein the position of each vehicle in each frame of image can be used as a sampling point in the movement track.
For example, as shown in fig. 2, a frame a, a frame B and a frame C respectively represent the areas where the vehicle y is located in 3 continuous images, a center point s1 of the frame a, a center point s2 of the frame B and a center point s3 of the frame C are obtained, and a connection line of three points s1, s2 and s3 can be determined as the motion trajectory of the vehicle y.
The recognition of the vehicles can also be realized through a yolo (young Only Look one, abbreviated as "yolo") real-time target detection method or an object detection algorithm ssd (single shot multi-box detector, abbreviated as "ssd"), and then the positions of the vehicles belonging to the same area in the preset area image can be tracked through the hungarian algorithm, so that the movement locus of each vehicle can be obtained, wherein the specific calculation processes of yolo or ssd and the hungarian algorithm are not repeated in the embodiment.
Step 103: and determining the traffic incident to which each vehicle belongs currently according to the motion trail of each vehicle and a traffic incident determination model, wherein the traffic incident determination model is the corresponding relation between the motion trail of the vehicle and the traffic incident to which the vehicle belongs.
In a specific implementation, the following processing is performed on the motion track of each vehicle: taking the motion trail of the vehicle as the input of a traffic incident determination model to obtain the confidence coefficient of each traffic incident corresponding to the vehicle; and determining the current traffic event of the vehicle according to the confidence coefficient.
Specifically, according to the input parameter requirements of the traffic incident determination model, the motion trail of each vehicle is input into the traffic incident determination model, the traffic incident model calculates the confidence degree of each traffic incident corresponding to the vehicle according to the input motion trail, and the traffic incident to which the current vehicle belongs can be determined according to the confidence degree of each traffic incident. For example: if the corresponding output results in the traffic event determination model include three traffic events, namely, illegal parking, vehicle reverse running and normal running of the vehicle, after the running track of the vehicle A is input into the traffic event determination model, the confidence degrees corresponding to the three traffic events are obtained, namely, the parking confidence degree is 0.1, the confidence degree of the vehicle reverse running is 0.8, and the confidence degree of the normal running of the vehicle is 0.1, so that the traffic event to which the vehicle A belongs can be determined to be the reverse running according to the confidence degree of the vehicle. Of course, it is understood that the type of the corresponding belonging traffic event in the traffic event determination model may be set according to actual needs, and is not limited to the three traffic events listed in the embodiment.
Step 104: and outputting alarm information indicating that the vehicles belong to the illegal traffic events according to the traffic events to which each vehicle belongs.
Specifically, when it is determined that the traffic event to which the vehicle belongs is a traffic event in violation, the warning information indicating that the vehicle belongs to the traffic event in violation can be output, and the output mode can be a voice report or uploading to a related network platform (such as a traffic network platform).
Compared with the prior art, the method and the device have the advantages that the motion trail of each vehicle in the preset area is determined by acquiring the image data of all vehicles in the preset area and according to the frame images in the image data; the traffic incident to which the current vehicle belongs can be determined according to the movement track of each vehicle and the traffic incident determination model, different detection methods are not required to be set for the vehicles according to different detection requirements, the range of applicable scenes for detecting violation incidents of the vehicles is widened, the traffic incident to which the current vehicle belongs can be accurately determined only by detecting once, and the detection speed of the vehicles is accelerated; because the traffic events of the vehicles are related to the motion states of the vehicles, the acquired motion tracks of each vehicle and the traffic event determination model jointly determine the traffic events of the vehicles, so that the accuracy of detecting the traffic events can be improved, and the occurrence of road violation events can be effectively reduced.
A second embodiment of the invention relates to a method of vehicle violation event detection. The second embodiment is the step 103 of the first embodiment: and determining the concrete refinement of the current traffic incident of each vehicle according to the motion trail of each vehicle and the traffic incident determination model. The specific process of determining the traffic event to which each vehicle belongs currently is shown in fig. 3.
Step 2031: and acquiring the number of nodes of the input layer of the traffic event determination model.
In a specific implementation, the traffic incident determination model is constructed in a deep neural network manner, wherein an input layer of the traffic incident determination model comprises n nodes, each node corresponds to one sampling point of a motion track of each vehicle, an output layer comprises m output nodes, each output node corresponds to one traffic incident, and n and m are integers greater than 1.
Specifically, a Deep Neural Network (DNN) model includes three parts, namely an input layer, a hidden layer and an output layer. Each node of the input layer corresponds to one sampling point of a vehicle motion trajectory, for example, if the input layer includes 4 nodes, the 4 nodes correspond to 4 sampling points of the motion trajectory of the vehicle a. The hidden layer can be a single layer or multiple layers, each layer can also be provided with a plurality of nodes, the more the hidden layers are, the more the calculation is complex, and the number of the hidden layers and the number of the nodes in each layer can be set according to actual needs. The output layer comprises m output nodes, and each output node corresponds to a traffic event, such as normal driving, roadside parking, reverse driving and the like.
Because the traffic incident determination model is constructed in advance, the number of nodes of the input layer of the traffic incident determination model can be directly acquired.
Step 2032: and respectively preprocessing the motion trail of each vehicle according to the number of the nodes of the input layer.
In one specific implementation, the following preprocessing is performed on the motion trajectory of each vehicle: judging whether the number of sampling points contained in the motion trail is equal to the number of nodes or not, if the number of the sampling points contained in the motion trail is determined to be not equal to the number of the nodes, adjusting the number of the sampling points in the motion trail so that the number of the sampling points in the adjusted motion trail is equal to the number of the nodes.
The following description will take the motion trajectory of one of the vehicles as an example.
Specifically, the number of sampling points included in the movement track of the vehicle is firstly obtained, whether the number of the sampling points is equal to the number of nodes in an input layer or not is judged, if yes, each sampling point in the movement track of the vehicle is directly used as the input of a traffic event determination model, and if not, the number of the sampling points in the movement track of the vehicle is adjusted, so that the number of the sampling points is equal to the number of the nodes in the input layer. And when the number of the sampling points in the motion trail is greater than the number of the nodes of the input layer, randomly deleting any coordinate data between the maximum value and the minimum value of the coordinate data of the sampling points of the motion trail until the number of the coordinate data of the motion trail of the vehicle is equal to n. And when the number of the sampling points in the motion trail is less than the number of the nodes of the input layer, randomly simulating and generating new sampling points between the maximum value and the minimum value of the coordinate data of the sampling points of the motion trail of the vehicle until the number of the coordinate data of the motion trail of the vehicle is equal to n.
It should be noted that after the number of nodes of the input layer of the traffic event determination model is obtained and before the motion trajectory of each vehicle is preprocessed, the motion trajectory of each vehicle may be normalized. Through normalization operation, the subsequent rapid processing of the motion trail is facilitated, and meanwhile, due to the fact that the motion trail is mapped between 0 and 1, large errors caused by large distance difference of two sampling points in the motion trail are reduced.
Step 2033: and taking the preprocessed motion trail of each vehicle as the input of a traffic incident determination model to determine the current traffic incident of each vehicle.
In a specific embodiment, the confidence of each traffic event corresponding to the current vehicle is obtained; and determining the current traffic event of the vehicle according to the confidence coefficient. This process is substantially the same as the process of determining a traffic event according to the confidence in the first embodiment, and will not be described here.
According to the method for detecting the vehicle violation event, the traffic event determination model is constructed in a deep neural network mode, so that the traffic event determination model is more practical and has high accuracy; the number of sampling points of the motion trail corresponds to the number of nodes, so that the deep neural network model can be constructed; and through adjusting the number of sampling points to ensure that no matter how many actual sampling points are few, can all determine corresponding traffic incident according to the sampling point in the motion trail.
The steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, so long as the same logical relationship is included, which are all within the protection scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.
A third embodiment of the present invention relates to a vehicle violation event detection device 30, including: the specific structure of the acquisition module 301, the first determination module 302, the second determination module 303 and the information output module 304 is shown in fig. 4.
The acquisition module 301 is configured to acquire image data of all vehicles in a preset area in real time, where the image data includes images in at least 2 frame areas; the first determining module 302 is configured to determine a motion trajectory of each vehicle in a preset area according to the image data; the second determining module 303 is configured to determine a traffic event to which each vehicle belongs currently according to the motion trajectory of each vehicle and a traffic event determination model, where the traffic event determination model is a correspondence between the motion trajectory of the vehicle and the traffic event to which the vehicle belongs; the information output module 304 is configured to output warning information indicating that the vehicle belongs to an illegal traffic event according to the traffic event to which each vehicle belongs.
It should be understood that this embodiment is an example of the apparatus corresponding to the first embodiment, and may be implemented in cooperation with the first embodiment. The related technical details mentioned in the first embodiment are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the first embodiment.
It should be noted that each module referred to in this embodiment is a logical module, and in practical applications, one logical unit may be one physical unit, may be a part of one physical unit, and may be implemented by a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, elements that are not so closely related to solving the technical problems proposed by the present invention are not introduced in the present embodiment, but this does not indicate that other elements are not present in the present embodiment.
A fourth embodiment of the present invention relates to an electronic apparatus 40 including: at least one processor 401; and a memory 402 communicatively coupled to the at least one processor 401; the memory 402 stores instructions executable by the at least one processor 401, and the instructions are executed by the at least one processor 401 to enable the at least one processor 401 to perform the method for detecting vehicle violation events in the first embodiment or the second embodiment. The specific structure of the electronic device is shown in fig. 5.
The memory 402 and the processor 401 are connected by a bus, which may include any number of interconnected buses and bridges that link one or more of the various circuits of the processor 401 and the memory 402. The bus may also link various other circuits such as peripherals, voltage regulators, power management circuits, and the like, which are well known in the art, and therefore, will not be described any further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be one element or a plurality of elements, such as a plurality of receivers and transmitters, providing a means for communicating with various other apparatus over a transmission medium. The data processed by the processor is transmitted over a wireless medium via an antenna, which further receives the data and transmits the data to the processor.
The processor 401 is responsible for managing the bus and general processing and may provide various functions including timing, peripheral interfaces, voltage regulation, power management, and other control functions. And the memory may be used to store data used by the processor in performing operations.
Those skilled in the art can understand that all or part of the steps in the method of the foregoing embodiments may be implemented by a program to instruct related hardware, where the program is stored in a storage medium and includes several instructions to enable a device (which may be a single chip, a chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It will be understood by those of ordinary skill in the art that the foregoing embodiments are specific examples for carrying out the invention, and that various changes in form and details may be made therein without departing from the spirit and scope of the invention in practice.