CN115187886A - Vehicle violation detection method and device and electronic equipment - Google Patents

Abstract

The embodiment of the present invention relates to a method for detecting illegal behavior of vehicles, the method is executed by a vehicle violation detection system, and the method includes: acquiring video frame data of a target scene; Trajectory information of the motor vehicle; based on the trajectory information, determine whether the non-motor vehicle has an illegal act. This method is not limited to frontal non-motor vehicle data, it can detect non-motor vehicles in multi-angle, multi-pose, and multi-scenario situations, and the detection of non-motor vehicle illegal events is clear and clear, without manual secondary classification; The motor vehicle has been classified, which can determine the category of non-motor vehicles, and determine the specific categories of non-motor vehicles with illegal acts, which is conducive to the supervision of specific types of non-motor vehicles, and effectively solves the problems in the existing technology.

Description

A vehicle violation detection method, device and electronic device

technical field

Embodiments of the present invention relate to the field of computer technologies, and in particular, to a method, device, and electronic device for detecting vehicle violations.

Background technique

With the further implementation of the construction of a civilized city and a safe city, efforts to create a safe, orderly and smooth traffic environment have become the primary goal of the traffic management departments of major cities, and a series of traffic violations against motor vehicles and non-motor vehicles are concentrated on Remediation work has become the top priority of traffic police work at this stage. However, in the work of rectifying illegal activities, there are the following problems: the video resources built by the traffic police department are facing waste of content, the built equipment is not intelligent, the manual capture manpower is insufficient, and there is a significant mismatch between the supply and demand of urban roads; With the development of urbanization information technology, the scale of urban intelligent transportation industry is also expanding year by year. Artificial intelligence empowers the transportation industry, which is of great significance to accelerating the construction of digital transportation and a strong transportation country.

SUMMARY OF THE INVENTION

The present invention provides a method, device and electronic device for detecting illegal behavior of vehicles, which are used to solve technical problems such as traffic disorder and personal safety threats caused by non-motor vehicle violations in the prior art, and provide assistance for the supervision of non-motor vehicles.

In a first aspect, the present invention provides a vehicle violation detection method, the method is executed by a vehicle violation detection system, and the method includes:

Get the video frame data of the target scene;

Based on the video frame data, obtain the trajectory information of the non-motor vehicle in the target scene;

Based on the trajectory information, determine the illegal behavior of non-motor vehicles.

In a possible implementation manner, the trajectory information includes thumbnail data of non-motor vehicles; based on the trajectory information, the illegal behavior of non-motor vehicles is determined, specifically including:

Based on the thumbnail data, perform helmet detection;

If no helmet is detected, obtain the head portrait of the non-motor vehicle driver;

Perform gray value analysis on the head avatar

If the analysis result satisfies the preset threshold, it is determined that the driver of the non-motor vehicle has an illegal act of not wearing a helmet.

In a possible implementation manner, the trajectory information includes tracking frame information of the non-motor vehicle; based on the trajectory information, the illegal behavior of the non-motor vehicle is determined, specifically including:

Obtain the track points of non-motor vehicles according to the tracking frame information;

Obtain the lane stop line and traffic light status in the target scene based on the video frame data;

In the red light state, if the trajectory points appear in the lane stop line in turn, cross the lane stop line and appear at the opposite intersection, it is judged that the non-motor vehicle has an illegal act of running a red light.

In a possible implementation manner, the track information includes tracking frame information and thumbnail data; based on the track information, the illegal behavior of non-motor vehicles is determined, specifically including:

Obtain the lane line information in the target scene based on the video frame data;

According to the lane line information, preset the lane direction line;

Obtain the track line of the non-motor vehicle according to the tracking frame information;

Calculate the angle between the track line and the preset lane direction line;

Identify the head direction of the non-motor vehicle according to the thumbnail data;

If the front direction of the non-motor vehicle is opposite to the direction of the lane direction line, and the included angle is greater than the preset threshold, it is determined that the non-motor vehicle has a wrong-way violation.

In a possible implementation manner, the trajectory information includes tracking frame information; based on the trajectory information, the illegal behavior of non-motor vehicles is determined, specifically including:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, it is judged that the non-motor vehicle has the illegal act of occupying the motor vehicle lane.

In a possible implementation manner, the trajectory information includes tracking frame information; based on the trajectory information, the illegal behavior of non-motor vehicles is determined, specifically including:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, calculate the number of the motor vehicle lane occupied by the trajectory point;

If the number is greater than the preset threshold, it is determined that the non-motor vehicle has an illegal act of crossing the road.

In a possible implementation manner, the track information includes classification information of non-motor vehicles; after judging the illegal behavior of non-motor vehicles based on the track information, it also includes:

According to the classification information of non-motor vehicles, determine the platform to which the non-motor vehicles that have violated the law belong.

In a second aspect, the present invention provides a vehicle violation detection device, the device comprising:

Video frame data module: used to obtain video frame data of the target scene;

The trajectory information module is used to obtain the trajectory information of the non-motor vehicle in the target scene based on the video frame data;

The illegal behavior judgment module is used to judge the illegal behavior of non-motor vehicles based on the trajectory information.

In a third aspect, the present invention provides an electronic device, the electronic device carries a vehicle illegal behavior detection system, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface, and the memory complete the communication between each other through the communication bus. communication;

memory for storing computer programs;

The processor is configured to implement the steps of any one of the vehicle violation detection methods of the first aspect when executing the program stored in the memory.

In a fourth aspect, the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the vehicle violation detection method according to any one of the first aspect.

Compared with the prior art, the above-mentioned technical solutions provided by the embodiments of the present invention have the following advantages:

The technical solutions provided by the embodiments of the present invention are not limited to frontal non-motor vehicle data, and can detect non-motor vehicles in multi-angle, multi-pose, and multi-scenario situations, and the detection of non-motor vehicle illegal incidents is clear and definite, without the need for manual secondary The classification of non-motor vehicles is carried out, the category of non-motor vehicles can be judged, and the specific categories of non-motor vehicles with illegal acts can be determined, which is conducive to the supervision of vehicles and effectively solves the problems in the prior art.

Description of drawings

1 is a schematic flowchart of a method for detecting illegal behavior of vehicles according to an embodiment of the present invention;

2 is a schematic flowchart of a method for detecting illegal behavior without a helmet provided by an embodiment of the present invention;

3 is a schematic flowchart of a method for detecting illegal acts of running a red light provided by an embodiment of the present invention;

4 is a schematic flowchart of a method for detecting a retrograde illegal act provided by an embodiment of the present invention;

5 is a schematic flowchart of a method for detecting illegal acts of occupying motor vehicle lanes provided by an embodiment of the present invention;

6 is a schematic flowchart of a method for detecting violations of road crossings provided by an embodiment of the present invention;

7 is a schematic structural diagram of a vehicle illegal behavior detection device provided by an embodiment of the present invention;

8 is a schematic structural diagram of another vehicle illegal behavior detection device provided by an embodiment of the present invention;

FIG. 9 provides a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In order to facilitate the understanding of the embodiments of the present invention, further explanation will be given below with specific embodiments in conjunction with the accompanying drawings, and the embodiments do not constitute limitations to the embodiments of the present invention.

In the process of using vehicles to go out, it is easy to cause illegal incidents, especially with the development of economy and traffic, the illegal incidents of non-motor vehicles are also increasing day by day, such as not wearing a helmet, running a red light, running the wrong way, occupying a motor vehicle lane and crossing the road, etc. , causing adverse effects on both traffic and personal safety. In view of the above problems, an embodiment of the present invention provides a method for detecting illegal behavior of vehicles, as shown in FIG. 1 for details.

In the deep learning-based vehicle illegal behavior detection method provided by the present invention, a corresponding algorithm model is constructed before detection. Specifically, first, non-motor vehicle images in actual scenes are collected as a basic data set, and a large-scale training data set is established to make it It not only has samples from various angles and scenes of non-motor vehicles, but also has its detailed characteristics. Then, deep learning is used to train a non-motor vehicle detection model suitable for practical scenarios on the training dataset. At the same time, according to the non-motor vehicle data set, the non-motor vehicle classification model is trained, and the non-motor vehicle type classification model is established. In actual use, the video stream is framed, and then non-motorized vehicles are detected, the detected non-motorized vehicles are classified, and the non-motorized vehicles of the type to be detected are subsequently processed according to the actual needs, and the non-motorized vehicles are obtained through the tracking algorithm. Using the track information, lane line information, and traffic light status information of non-motor vehicles to monitor the driving process of non-motor vehicles, and determine whether there are no helmets, running red lights, running the wrong way, occupying the motor vehicle lane, crossing the road or not during the driving process. Illegal acts such as crossing the road.

In addition to the model training described above, the system also builds a target tracking model. The following is a brief introduction to individual models and the knowledge points involved:

(1) Establish a non-motor vehicle detection data set

Collect non-motor vehicle pictures and images in actual traffic scenes, and collect multi-angle samples from multiple scenes and multiple time periods. Each image requires the target to be clear and easy to distinguish, and the pixel value of the target is greater than 100*100 to construct large-scale non-motor vehicle detection data. set. The non-motor vehicle data set is processed, such as using a variety of data enhancement methods, such as mosaic enhancement, random erasure, adding noise, etc., to further expand the data set. For the obtained processed data set, the non-motor vehicle rectangular frame is manually calibrated as a training data set.

(2) Non-motor vehicle detection model training optimization

Based on the training data set obtained in (1) above, the yolov5 structured model is used to train the model for non-motor vehicles in practical application scenarios to improve the generalization of the model in practical application scenarios.

Specifically, the basic principle of yolov5 structured model training is to perform fine-tuning based on large-scale pre-training data sets. First, collect a large amount of non-motor vehicle data from various scenes, and then train a version of the model based on these data to obtain pre-training weights. When the model is to be deployed to a new scene, a small amount of data including non-motor vehicles is collected from the scene, and then fine-tuned on the existing pre-training weights. Since the pre-training weights are trained based on different scene data, it is guaranteed that The diversity of data can greatly improve the generalization ability of the model, and the fine-tuning based on a specific scene also further improves the performance of the model in this scene.

(3) Constructing a non-motor vehicle classification dataset

Based on the training data set in (1) above, crop the non-motor vehicle area to obtain a large number of thumbnails, and then divide them according to different types of non-motor vehicles, and finally obtain different types of non-motor vehicle classification data sets.

(4) Non-motor vehicle type classification model training optimization

In order to obtain different types of non-motor vehicle classification data sets as training data sets, the EfficientNet-b5 classification model is used for training. At the same time, in order to solve the imbalance problem between data set categories, focal loss is used as the training loss function. In order to ensure that The model learns the correct features, and grad-cam is used to visualize the last layer of the convolution to understand the feature regions that the model pays most attention to when classifying.

A spatial attention module is also added to the classification model. The attention module can capture features that are more favorable to the classification results. For example, there are some differences in details in various parts of the body in different types of non-motor vehicles. The force module makes the model pay more attention to the features of these regions and is more conducive to the final classification effect.

In order to solve the problem of class imbalance in the dataset, focal loss is used here. Since there are many types of non-motor vehicles to be classified, the focal loss of two classifications is modified to the focal loss of multiclassification, and then the corresponding focal loss is used. The relevant weight of the number of class samples is used to further optimize the loss, thereby increasing the proportion of classes with a small number of samples and difficult samples in the training model, and finally maintaining the balance of the accuracy of each class.

At the same time, label smoothing, Stochastic Weight Averaging and cosine learning rate decay are added in the training process of the classification model to improve the generalization ability of the model.

(5) Target tracking model

The tracking model is the core module in the entire illegal event detection system. The DeepSort multi-target tracking model is used here. The model is mainly divided into two parts (1) Reid appearance re-identification model part (2) Sort trajectory allocation part

1.reid re-identification model

The Reid re-identification model is often used in cross-camera pedestrian re-identification. This model can identify the appearance features of different objects and represent them with specific vectors, and then judge the similarity of different objects by comparing the distances between the feature vectors. In the detection of non-motor vehicle violations, it is only necessary to identify non-motor vehicles under the same camera device. Therefore, first build a non-motor vehicle Reid data set under the same camera device, and then retrain the Reid model based on these data. Resnet50 is used as the backbone network, and 512-dimensional feature vectors are used to characterize objects. The final effect is that the distance between the feature vectors of the same non-motor vehicle is much smaller than the distance between the feature vectors of different non-motor vehicles.

2. Sort track assignment part

In the Sort part, the Kalman filter and the Hungarian matching algorithm are used. The Kalman filter updates the position and other related information of the object trajectory according to the linear motion model, while the Hungarian matching is responsible for the matching of the trajectory frame and the detection frame. The scene adjusts the state equation of the motion model, uses the center point, aspect ratio and height of the moving object as the state space parameters of the motion model, and adjusts the position and speed parameters, so as to adjust the motion of the non-motor vehicle in the current scene. More accurate modeling.

Feature similarity and intersection between objects are used in Hungarian matching to match tracking and detection boxes more accurately than IOU and Mahalanobis distance, thereby improving tracking accuracy.

In addition, in addition to the above-mentioned models, the present invention also relates to a multi-attribute classification model for identifying attributes of non-motor vehicles such as the front and rear of a vehicle, a traffic light classification model for classifying traffic lights, and non-motor vehicle drivers Wearing a helmet detection model will not be repeated here.

Before introducing the model reasoning described above in detail, first introduce the models and their uses used by the vehicle violation detection system, as shown in Table 1:

Table 1 Summary of models and uses used in the vehicle violation detection system

model name use Yolov5s structured model Detection of non-motor vehicles Efficientnet-b5 classification model Identify different types of non-motorized vehicles DeepSort model multi-target tracking Yolov5s helmet detection model No helmet detection Mobilenet classification model Classification of traffic lights resnet50 multi-attribute classification model Identify attributes such as front and rear of non-motor vehicles

Use tensorrt as the inference backend of the model to optimize the inference speed of the model and reduce the consumption of video memory resources, and use multi-process and multi-thread technology to optimize the judgment logic of illegal event detection, so as to further optimize the execution efficiency of the entire system and ultimately improve the system. Concurrent performance, the average detection speed of the current non-motor vehicle violation detection system to be detected on Tesla P4 is 52.35ms.

A method for detecting illegal behavior of vehicles provided by the present invention is described in detail below. Specifically, FIG. 1 is a schematic flowchart of a method for detecting illegal behaviors of vehicles provided by an embodiment of the present invention. As shown in FIG. The system executes, and the method steps specifically include:

Step 110, acquiring video frame data of the target scene.

Step 120 , based on the video frame data, obtain the track information of the non-motor vehicle in the target scene.

Specifically, the Yolov5s structured model is used to detect non-motor vehicles in the current scene. If there are non-motor vehicles, the Efficientnet-b5 classification model is used to identify the classification information of non-motor vehicles and determine the specific category of non-motor vehicles. When a non-motor vehicle of the category to be detected appears in the scene, the system enters the tracking state, and uses the DeepSort algorithm to track the non-motor vehicle target, and finally obtains the complete non-motor vehicle trajectory information, where the trajectory information includes the current non-motor vehicle track number. , the tracking frame information of non-motor vehicles, the thumbnail data of non-motor vehicles and the classification information of the current non-motor vehicles. If there is no non-motor vehicle in the current scene, the system enters the detection state. Only when the system is in the tracking state, will the Non-motorized vehicles are tracked.

The system obtains the track information of each non-motor vehicle in the tracking state, and each non-motor vehicle has a unique track number, that is, the track number. Therefore, it can reduce the reuse of vehicle information in time series video frames to achieve the purpose of deduplication. According to the trajectory information combined with logical judgment, the illegal behavior of non-motor vehicles can be judged.

Step 130, based on the trajectory information, determine the illegal behavior of the non-motor vehicle.

Before introducing the detection of illegal behaviors, the track information of non-motor vehicles is briefly explained. As described above, the track information includes: track numbers of non-motor vehicles, tracking frame data, thumbnail data and classification of non-motor vehicles. information, wherein the tracking frame data includes the center coordinate point information of the non-motor vehicle in each frame, so the track point of each frame of the non-motor vehicle and the trajectory line of the non-motor vehicle can be obtained through the tracking frame data.

The violations of non-motor vehicles include: non-motor vehicles drivers do not wear helmets, non-motor vehicles run red lights, non-motor vehicles run the wrong way, non-motor vehicles occupy motor vehicle lanes, non-motor vehicles cross the road, and the following are the penalties for each illegal act. Introduce the detection:

(1) Detection of not wearing a helmet

Fig. 2 is a schematic flowchart of a method for detecting illegal behavior without a helmet provided by an embodiment of the present invention. As shown in Fig. 2, the steps for detecting an illegal behavior without wearing a helmet are:

Step 210, based on the thumbnail data, perform helmet detection.

Step 220, if the helmet is not detected, obtain the head image of the non-motor vehicle driver;

Step 230, performing gray value analysis on the head portrait;

Step 240, if the analysis result satisfies the preset threshold, it is determined that the driver of the non-motor vehicle has an illegal act of not wearing a helmet.

Specifically, based on the thumbnail data of the non-motor vehicle corresponding to the track number, the yolov5 model is used for helmet detection. If the helmet is detected, it will return directly. If the helmet is not detected, in order to prevent detection errors, a second identification will be performed to obtain For the head image of a non-motor vehicle driver, use the image grayscale histogram to analyze the gray value of the driver's head image. If the histogram distribution satisfies the preset threshold, it means the distribution and the characteristic distribution of the non-helmeted Consistent, the event of not wearing a helmet is directly output, that is, it is judged that the driver of a non-motor vehicle has an illegal act of not wearing a helmet.

(2) Detection of running a red light

3 is a schematic flowchart of a method for detecting the illegal behavior of running a red light provided by an embodiment of the present invention. As shown in FIG. 3 , the detection steps for the illegal behavior of running a red light are:

Step 310: Obtain the track points of the non-motor vehicle according to the tracking frame information.

Step 320, based on the video frame data, obtain the lane stop line and the traffic light status in the target scene.

Step 330: In the red light state, if the trajectory points appear in the lane stop line, cross the lane stop line, and appear at the opposite intersection in sequence, it is determined that the non-motor vehicle has an illegal act of running a red light.

Specifically, the information of the red street light area in the target scene is obtained in advance from the video frame data, and the red street light image is obtained by cropping, and the Mobilenet classification model is used to identify the traffic light category. If the current state is a red light, the subsequent red light running judgment is performed, otherwise, it returns directly. .

According to the coordinate point information of the corresponding track number, it is judged whether the track point appears in the lane stop line of non-motor vehicles, crosses the lane stop line, and appears at the opposite intersection. If all three conditions are satisfied and the whole process is red If the light is on, it is judged to be running a red light, and an event is output. That is, it is judged that there is an illegal act of running a red light by a non-motor vehicle.

(3) Retrograde detection

FIG. 4 is a schematic flowchart of a method for detecting retrograde illegal acts provided by an embodiment of the present invention. As shown in FIG. 4 , the steps for detecting retrograde illegal acts are:

Step 410 , obtain lane line information in the target scene based on the video frame data.

Step 420, preset lane direction lines according to lane line information.

Step 430: Obtain the trajectory line of the non-motor vehicle according to the tracking frame information.

Step 440: Calculate the angle between the trajectory line and the preset lane direction line.

Step 450 , according to the thumbnail data, identify the head direction of the non-motor vehicle.

Step 460: If the head of the non-motor vehicle is facing in the opposite direction to the direction line of the lane, and the included angle is greater than the preset threshold, it is determined that the non-motor vehicle has a wrong-way violation.

Specifically, the trajectory line and thumbnail data of the non-motor vehicle are obtained, the lane direction line is preset according to the actual direction of the lane in the target scene, that is, the lane line direction is preset according to the lane line information in the actual scene, and then the trajectory is calculated. The angle between the line and the lane direction line, if the angle is greater than the preset threshold, continue to judge, otherwise return directly. Use the resnet50 multi-attribute classification model to identify whether the non-motor vehicle is the front or the rear of the vehicle. If the direction of the front of the vehicle is opposite to the direction of the lane direction line, and the preset threshold calculated earlier, the retrograde event will be output, which is to judge that the non-motor vehicle has retrograde Illegal behavior, otherwise return directly.

(4) Occupied motor vehicle lane detection

5 is a schematic flow chart of a method for detecting illegal acts of occupying motor vehicle lanes provided by an embodiment of the present invention. As shown in 5, the steps for detecting illegal acts in the wrong direction are:

Step 510, based on the video frame data, determine a motor vehicle lane in the target scene.

Step 520: Obtain the track points of the non-motor vehicle according to the tracking frame information.

Step 530, analyze the position of the track point and the vehicle lane.

Step 540, if the trajectory point appears in the motor vehicle lane, it is determined that the non-motor vehicle has an illegal act of occupying the motor vehicle lane.

Specifically, the polygon frame of the motor vehicle lane area is preset according to the actual position of the motor vehicle lane, and then it is judged whether the track point of the non-motor vehicle appears in the motor vehicle lane frame. There is an illegal act of occupying the motor vehicle lane, otherwise, return directly.

(5) Cross-road detection

FIG. 6 is a schematic flowchart of a method for detecting illegal behaviors crossing roads provided by an embodiment of the present invention. As shown in FIG. 6 , the steps for detecting illegal behaviors crossing roads are:

Step 610, based on the video frame data, determine a motor vehicle lane in the target scene.

Step 620: Obtain the track points of the non-motor vehicle according to the tracking frame information.

Step 630, analyze the position of the track point and the vehicle lane.

Step 640, if the trajectory point appears in the motor vehicle lane, calculate the number of the motor vehicle lane occupied by the trajectory point;

Step 650, if the number is greater than the preset threshold, it is determined that the non-motor vehicle has an illegal act of crossing the road.

Specifically, a separate polygon frame is set for each lane according to the actual position of the motor vehicle lane, and then the occupancy of the track points of non-motor vehicles in each motor vehicle lane is determined. If the number of occupied lanes is greater than the set threshold, it means that the vehicle lateral After crossing the road, the illegal events are directly output, that is, it is judged that the non-motor vehicle has the illegal behavior of crossing the road, otherwise it is directly output.

In an example, after judging the illegal behavior of the non-motor vehicle according to the track information of the non-motor vehicle, the method further includes: determining the category of the non-motor vehicle with the illegal behavior according to the classification information of the non-motor vehicle. That is, according to the classification information of non-motor vehicles, the specific categories of non-motor vehicles with illegal acts can be determined.

The detection method provided by the embodiment of the present invention is not limited to frontal non-motor vehicle data, but can detect non-motor vehicles in multi-angle, multi-pose, and multi-scenario situations, and the detection of non-motor vehicle illegal events is clear and clear, without the need for Manual secondary classification; and this method classifies non-motor vehicles, can determine the category of non-motor vehicles, and determine the specific category of non-motor vehicles that have illegal acts, which is conducive to the supervision of specific types of non-motor vehicles and effectively solves the problem. problems in the prior art.

The above are the embodiments of the vehicle illegal behavior detection method provided by the present invention, and the following describes other embodiments of the vehicle illegal behavior detection provided by the present invention. For details, refer to the following.

FIG. 7 is a schematic structural diagram of a vehicle illegal behavior detection device according to an embodiment of the present invention. The device includes: a video frame data module 701 , a trajectory information module 702 , and an illegal behavior judgment module 703 .

Video frame data module 701: used to obtain video frame data of the target scene;

A trajectory information module 702, configured to obtain the trajectory information of the non-motor vehicle in the target scene based on the video frame data;

The illegal behavior judgment module 703 is used for judging the illegal behavior of the non-motor vehicle based on the trajectory information.

The illegal behavior judgment module 703 is specifically used for:

Based on the thumbnail data, perform helmet detection;

If the helmet is not detected, obtain the head image of the non-motor vehicle driver;

Perform gray value analysis on the head image;

If the analysis result satisfies the preset threshold, it is determined that the driver of the non-motor vehicle has an illegal act of not wearing a helmet.

The illegal behavior judgment module 703 is specifically used for:

Obtain the track points of non-motor vehicles according to the tracking frame information;

Obtain the lane stop line and traffic light status in the target scene based on the video frame data;

In the red light state, if the trajectory points appear in the lane stop line in turn, cross the lane stop line and appear at the opposite intersection, it is judged that the non-motor vehicle has an illegal act of running a red light.

The illegal behavior judgment module 703 is specifically used for:

Obtain the lane line information in the target scene based on the video frame data;

According to the lane line information, preset the lane direction line;

Obtain the track line of the non-motor vehicle according to the tracking frame information;

Calculate the angle between the track line and the preset lane direction line;

Identify the head direction of the non-motor vehicle according to the thumbnail data;

If the front direction of the non-motor vehicle is opposite to the direction of the lane direction line, and the included angle is greater than the preset threshold, it is determined that the non-motor vehicle has a wrong-way violation.

The illegal behavior judgment module 703 is specifically used for:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, it is judged that the non-motor vehicle has the illegal act of occupying the motor vehicle lane.

The illegal behavior judgment module 703 is specifically used for:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, calculate the number of the motor vehicle lane occupied by the trajectory point;

If the number is greater than the preset threshold, it is determined that the non-motor vehicle has an illegal act of crossing the road.

In an example, FIG. 8 is a schematic structural diagram of another vehicle illegal behavior detection device provided by an embodiment of the present invention. As shown in FIG. 8 , the device further includes a determination module 704 for determining, according to the classification information of non-motor vehicles, The category of non-motor vehicles with violations.

The functions performed by each component in the device for detecting illegal behavior of vehicles provided in the embodiments of the present invention have been described in detail in any of the above method embodiments, and therefore are not repeated here.

The vehicle illegal behavior detection device provided by the embodiment of the present invention is not limited to frontal non-motor vehicle data, but can detect non-motor vehicles in multi-angle, multi-pose, and multi-scenario situations, and can detect non-motor vehicle illegal events clearly and accurately. It is clear that there is no need for manual secondary classification; and the classification of non-motor vehicles can determine the category of non-motor vehicles and determine the specific categories of non-motor vehicles with illegal acts, which is conducive to the supervision of specific types of non-motor vehicles and effectively solve the problem. problems in the prior art.

FIG. 9 provides a schematic structural diagram of an electronic device according to an embodiment of the present invention. As shown in FIG. 9 , an embodiment of the present invention provides an electronic device. The electronic device includes a processor 111 , a communication interface 112 , a memory 113 and a communication bus 114 , wherein , the processor 111 , the communication interface 112 , and the memory 113 communicate with each other through the communication bus 114 .

a memory 113 for storing computer programs;

In an embodiment of the present invention, the processor 111 is configured to implement the vehicle illegal behavior detection method provided by any one of the foregoing method embodiments when executing the program stored in the memory 113, including:

Get the video frame data of the target scene;

Based on the video frame data, obtain the trajectory information of the non-motor vehicle in the target scene;

Based on the trajectory information, determine the illegal behavior of non-motor vehicles.

Optionally, the trajectory information includes thumbnail data of non-motor vehicles; based on the trajectory information, the illegal behavior of non-motor vehicles is determined, specifically including:

Based on the thumbnail data, perform helmet detection;

If the helmet is not detected, obtain the head image of the non-motor vehicle driver;

Perform gray value analysis on the head image;

If the analysis result satisfies the preset threshold, it is determined that the driver of the non-motor vehicle has an illegal act of not wearing a helmet.

Optionally, the trajectory information includes the tracking frame information of the non-motor vehicle; based on the trajectory information, the illegal behavior of the non-motor vehicle is determined, specifically including:

Obtain the track points of non-motor vehicles according to the tracking frame information;

Obtain the lane stop line and traffic light status in the target scene based on the video frame data;

In the red light state, if the trajectory points appear in the lane stop line in turn, cross the lane stop line and appear at the opposite intersection, it is judged that the non-motor vehicle has an illegal act of running a red light.

Optionally, the track information includes tracking frame information and thumbnail data; based on the track information, the illegal behavior of non-motor vehicles is determined, specifically including:

Obtain the lane line information in the target scene based on the video frame data;

According to the lane line information, preset the lane direction line;

Obtain the track line of the non-motor vehicle according to the tracking frame information;

Calculate the angle between the track line and the preset lane direction line;

Identify the head direction of the non-motor vehicle according to the thumbnail data;

If the front direction of the non-motor vehicle is opposite to the direction of the lane direction line, and the included angle is greater than the preset threshold, it is determined that the non-motor vehicle has a wrong-way violation.

Optionally, the track information includes tracking frame information; based on the track information, the illegal behavior of the non-motor vehicle is determined, specifically including:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, it is judged that the non-motor vehicle has the illegal act of occupying the motor vehicle lane.

Optionally, the track information includes tracking frame information; based on the track information, the illegal behavior of the non-motor vehicle is determined, specifically including:

Determine the motor vehicle lane in the target scene based on the video frame data;

Obtain the track points of non-motor vehicles according to the tracking frame information;

Analyze the location of trajectory points and motor vehicle lanes;

If the trajectory point appears in the motor vehicle lane, calculate the number of the motor vehicle lane occupied by the trajectory point;

If the number is greater than the preset threshold, it is determined that the non-motor vehicle has an illegal act of crossing the road.

Optionally, the track information includes classification information of non-motor vehicles; after judging the illegal behavior of non-motor vehicles based on the track information, it also includes:

According to the classification information of non-motor vehicles, determine the platform to which the non-motor vehicles that have violated the law belong. Embodiments of the present invention further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the vehicle violation detection method provided by any one of the foregoing method embodiments.

It should be noted that, in this document, relational terms such as "first" and "second" etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these There is no such actual relationship or sequence between entities or operations. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion such that a process, method, article or device comprising a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device that includes the element.

The above are only specific embodiments of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims (10)

1. A vehicle violation detection method performed by a vehicle violation detection system, the method comprising:

acquiring video frame data of a target scene;

obtaining track information of the non-motor vehicle in the target scene based on the video frame data;

and judging the illegal behaviors of the non-motor vehicle based on the track information.

2. The method of claim 1, wherein the trajectory information includes thumbnail data of the non-motor vehicle; the judging of the illegal action of the non-motor vehicle based on the track information specifically comprises the following steps:

performing helmet detection based on the thumbnail data;

if the helmet is not detected, acquiring a head image of the non-motor vehicle driver;

performing gray value analysis on the head image;

and if the analysis result meets a preset threshold value, judging that the driver of the non-motor vehicle has an illegal behavior of not wearing a helmet.

3. The method of claim 1, wherein the trajectory information comprises tracking frame information of the non-motor vehicle; the judging the illegal behavior of the non-motor vehicle based on the track information specifically comprises the following steps:

obtaining track points of the non-motor vehicle according to the tracking frame information;

obtaining lane stop lines and traffic light states in the target scene based on the video frame data;

and under the red light state, if the track points sequentially appear in the lane stop line, cross over the lane stop line and appear at the opposite intersection, judging that the non-motor vehicle has illegal behavior of running the red light.

4. The method of claim 1, wherein the trajectory information includes tracking frame information and thumbnail data; the judging of the illegal action of the non-motor vehicle based on the track information specifically comprises the following steps:

acquiring lane line information in the target scene based on the video frame data;

presetting a lane direction line according to the lane line information;

obtaining a track line of the non-motor vehicle according to the tracking frame information;

calculating an included angle between the trajectory line and the preset lane direction line;

identifying the head orientation of the non-motor vehicle according to the thumbnail data;

and if the direction of the head of the non-motor vehicle is opposite to the direction of the lane direction line and the included angle is larger than a preset threshold value, judging that the non-motor vehicle has a reverse illegal behavior.

5. The method of claim 1, wherein the trajectory information comprises tracking box information; the judging the illegal behavior of the non-motor vehicle based on the track information specifically comprises the following steps:

determining a lane of vehicles in the target scene based on the video frame data;

obtaining track points of the non-motor vehicle according to the tracking frame information;

analyzing the positions of the track points and the motor vehicle lane;

and if the track point appears in the motor vehicle lane, judging that the non-motor vehicle has illegal behaviors of occupying the motor vehicle lane.

6. The method of claim 1, wherein the trajectory information comprises tracking box information; the judging of the illegal action of the non-motor vehicle based on the track information specifically comprises the following steps:

determining a lane of vehicles in the target scene based on the video frame data;

obtaining track points of the non-motor vehicle according to the tracking frame information;

analyzing the positions of the track points and the motor vehicle lane;

if the track points appear in the motor vehicle lane, calculating the number of the track points occupying the motor vehicle lane;

and if the number is larger than a preset threshold value, judging that the non-motor vehicle has illegal behaviors crossing the road.

7. The method according to any one of claims 1-6, wherein the track information includes classification information of the non-motor vehicle; after the determining the unlawful act of the non-motor vehicle based on the track information, the method further comprises the following steps:

and determining the non-motor vehicle category with illegal behaviors according to the classification information of the non-motor vehicles.

8. A vehicle law violation detection device, comprising:

a video frame data module: the method comprises the steps of obtaining video frame data of a target scene;

the track information module is used for obtaining track information of the non-motor vehicle in the target scene based on the video frame data;

and the illegal behavior judging module is used for judging the illegal behavior of the non-motor vehicle based on the track information.

9. An electronic device is characterized in that the electronic device bears the vehicle illegal behavior detection system and comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;

a memory for storing a computer program;

a processor for implementing the steps of the method for detecting vehicle unlawful behaviour according to any of claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method for detecting vehicle unlawful behaviour according to any one of claims 1-7.

Images