CN114758319A - A near-field vehicle jamming behavior prediction method based on image input - Google Patents

Abstract

The invention relates to a near-field vehicle jamming behavior prediction method based on image input. The method includes: (1) collecting image sequence information based on forward panoramic images in a real structured road scene, and manually labeling vehicles in the image sequence The location and behavior information of the target; (2) Build a near-field vehicle detection and tracking model suitable for near-field vehicle detection and tracking in structured roads; (3) Build a lane line detection network suitable for lane line detection in structured roads and the corresponding loss function; (4) the vehicle ID and the bounding box position data of the corresponding target obtained based on the near-field vehicle detection and tracking model established in step (2), and the lane obtained by the lane line detection network established in step (3) line to obtain the relative position deviation of the target and the lane line, and according to the formulation of the prior rules, the prediction result of the jamming behavior of the near-field vehicle is obtained. Compared with the prior art, the present invention has high prediction accuracy and high efficiency.

Description

A near-field vehicle jamming behavior prediction method based on image input

technical field

The invention relates to the technical field of intelligent driving, in particular to a near-field vehicle jamming behavior prediction method based on image input.

Background technique

Behavior prediction is a further development based on behavior recognition. As one of the basic tasks in the field of computer vision, with the rapid development of deep learning technology in recent years, behavior prediction algorithms also include algorithms for formulating prior rules and algorithms based on deep neural networks. End-to-end prediction technology. The method of behavior recognition and prediction has developed from the original method based on physical motion features to the SlowFast network based on visual video input, the action recognition network TSN based on bimodal input, and the 3D convolutional neural network based on dilated three-dimensional convolution (I3D). Many good algorithm techniques have emerged. These algorithms have excellent detection effects and performance on open human behavior recognition datasets. However, for the near-field vehicle jamming behavior prediction task, there are the following shortcomings in practical applications:

First, in the existing public datasets, there is a lack of EgoVehicle perspective datasets for near-field vehicle jamming behavior prediction. the further development of technology;

Second, in the dual-modal input technology, the optical flow is a hand-made feature, and it is trained separately from the RGB input, which cannot achieve end-to-end training, and the accuracy of the system needs to be improved. At the same time, the complex algorithm reduces the real-time performance of the system;

Third, the hardware cost and maintenance cost of the LiDAR-based method are high, and there is currently no method for predicting the behavior of the Ego Vehicle from the perspective of the forward panoramic image and video.

Fourth, the end-to-end method based on deep learning has problems such as poor generalization of the training model, unclear specific principles, and high hardware requirements for mobile terminals, making it difficult to quickly implement in actual scenarios.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a near-field vehicle jamming behavior prediction method based on image input in order to overcome the above-mentioned defects in the prior art. The target detection and tracking algorithm based on image input is used to obtain the perception and tracking of the area of interest of the forward target vehicle from the perspective of the ego vehicle. Under the premise of speed, the cost of software and hardware for actual deployment is greatly reduced, and finally a more accurate prediction of the jamming behavior of adjacent vehicles is obtained, which provides sufficient time for the intelligent driving system to avoid risks and improves the overall safety of the intelligent driving system.

The object of the present invention can be realized through the following technical solutions:

A near-field vehicle jamming behavior prediction method based on image input, the method comprising:

(1) Collect the image sequence information based on the forward panoramic image in the real structured road scene, and manually mark the position and behavior information of the vehicle target in the image sequence;

(2) Build a near-field vehicle detection and tracking model suitable for near-field vehicle detection and tracking in structured roads;

(3) Construct a lane line detection network and corresponding loss function suitable for lane line detection in structured roads;

(4) Based on the vehicle ID obtained by the near-field vehicle detection and tracking model established in step (2) and the bounding box position data of the corresponding target, and the lane line obtained by the lane line detection network established in step (3), the target and lane are obtained. According to the relative position deviation of the line, according to the formulation of the prior rules, the prediction result of the jamming behavior of the near-field vehicle is obtained.

Preferably, step (1) specifically includes:

(11) Calibrate the internal and external parameters of the camera, wherein the external parameters include a rotation matrix R and a translation vector T, and the internal parameters include an internal parameter matrix K, and the camera distortion coefficient;

(12) Use a data collection vehicle equipped with a camera to collect video data in a real road scene, and record the type of vehicle target in the image at the time of collection;

(13) Annotate the collected video data with an annotation tool. The annotation methods include vehicle target tracking ID annotation, vehicle target category annotation, target object bounding box annotation, vehicle jamming start, vehicle crossing the midpoint of the lane line, and vehicle completion jamming behavior The key frame annotations and vehicle jamming behavior category annotations of the , and the annotation content needs to include at least the location, keyframes, and jamming behavior category information of the near-field vehicle.

Preferably, step (2) specifically includes:

(21) Construct a near-field vehicle target detection network based on the improved Yolov5, input the input video slices as image time series to the near-field vehicle target detection network, and perform multi-layer convolution and downsampling operations to characterize the input image information Extraction and feature encoding to obtain a multi-dimensional feature tensor that divides the image;

(22) Build a classification network, adopt non-maximum suppression operation, and finally obtain the location information and classification confidence information of each target, including the classification probability and positioning probability of the object;

(23) Build a near-field vehicle target tracking network based on improved Deep-SORT, take the target object bounding box information and classification information obtained by target detection as input, locate and track multiple objects in the video simultaneously, and record ID and trajectory information , especially under the condition of occlusion, reduce the transformation of object ID, and output the tracking ID of the target vehicle, the target category and the bounding box information of the target object.

Preferably, step (3) specifically includes:

(31) Construct a backbone network for lane feature extraction based on convolutional neural network, output features based on shallow residual connection network, and improve the inference speed of the model while ensuring the detection effect by using a larger receptive field;

(32) Construct a lane line semantic segmentation network, up-sample multi-scale features to the same scale during network training, and through transposed convolution, calculate the semantic segmentation loss, enhance the visual feature extraction ability of the backbone network, and finally obtain an enhanced based on Residual connected lane detection backbone network;

(33) According to the features extracted by the backbone network, according to the longitudinal candidate anchor frame of the picture specified by the prior, the candidate points are calculated by the classifier in the global scope, and finally the lane line position node of the lane where the vehicle is located is obtained;

(34) Construct the loss function of lane line detection network, including multi-classification loss, segmentation loss and lane structure loss.

Preferably, the loss function of the lane line detection network is expressed as L _total :

L _total =L _cls +L _seg +ηL _lane

L _cls is the multi-classification loss, L _seg is the segmentation loss, L _lane is the lane structuring loss, and η is the hyperparameter.

Preferably, the multi-class loss L _cls is expressed as:

Among them, L _CE ( ) represents the cross-entropy loss function, P _i,j,: represents the prediction results of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, T _{i, j,:} indicates the true distribution of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, c _i,j,: indicates P _i,j,: and T _{i,j, :} similarity, C and h represent the number of lane line classes and the number of longitudinal anchor points of the lane, respectively, and γ and α are hyperparameters.

Preferably, the lane structure loss L _lane is expressed as:

L _lane =L _sim +λ _Lshp

Among them, L _sim is the similarity loss, L _shp is the shape loss, λ is the hyperparameter representing the loss weight, P _{i, j, k} is the predicted probability of the i-th lane line at the position (j, k), w The number of division cells for each row.

Preferably, the steps of step (4) training the network and formulating a priori rules are as follows:

(41) Perform data preprocessing on the collected image sequence, including: randomly flipping, cropping and uniformly scaling the image to a fixed size horizontally, and flipping, cropping and scaling the labeled data accordingly. The obtained image is normalized by channel;

(42) Using a highly robust regression model to obtain the lane line fitting model of the lane where the ego vehicle is located by using the lane line position node of the lane where the ego vehicle is located, obtained from the lane line detection;

(43) Establish an area of interest for jamming behavior according to the lane line model of the lane where the ego vehicle is located, calculate the position deviation between the bounding box information of the target vehicle and the area of interest for jamming behavior, and establish the jamming behavior expectation of each target according to the target vehicle tracking ID Dictionary of times and vehicle status symbols;

(44) Determine the specific behavior of the target vehicle by setting the threshold of the expected number of jamming behaviors combined with the vehicle status symbol, update the parameters of the established jamming behavior area of interest, and obtain ideal network parameters after iteration.

Preferably, the lane line fitting model of the lane where the ego vehicle is located is a linear model, and the left and right lanes are respectively obtained by fitting according to the predicted position nodes of the lane lines.

Preferably, in the established dictionary of expected times of jamming behavior and vehicle status symbols for each target, the key value is the target tracking ID, and the value is the expected number of times and the vehicle status symbol.

Compared with the prior art, the present invention has the following advantages:

(1) The present invention proposes a new network structure in view of the problem that the current neural network cannot predict the vehicle jamming behavior, and greatly improves the performance of the near-field vehicle real-time behavior prediction under the premise of limited computing power of the mobile terminal, which can be easily deployed In the existing intelligent driving system, the intelligent driving car can respond to the jamming behavior of the near-field vehicle in time, and improve the safety during driving;

(2) The present invention first extracts the video clips of the lane-changing behavior and the bounding box (Bounding Boxes) information of the target, filters out the data set that conforms to the above-defined car jamming behavior, and finally establishes a “proximity” containing the annotation and video data. Object Typical Jamming Behavior Database" dataset;

(3) In the present invention, a forward-view image is obtained with a large field of view and high resolution, including the appearance features of the target and the dependencies between the targets, and a near-vehicle detection and tracking model and a jamming behavior prediction algorithm are developed. The target detection module is deeply improved based on the latest One-Stage target detection algorithm Yolov5, and has a high detection speed on the basis of maintaining a certain detection accuracy;

(4) In the present invention, after outputting the bounding box and category information of the target, the Deep-SORT multi-target tracking algorithm is used to obtain the frame sequence of interest corresponding to each target ID. In order to ensure the real-time performance of the algorithm, this paper does not use optical flow as the extraction of time-domain features, but uses the target sequence as the input of time-space features;

(5) In the present invention, in the jamming behavior prediction module, starting from the characteristic information of jamming behavior, based on the algorithm for formulating a priori rules and the target sequence output obtained in the previous step, an interpretable real-time high robustness in the vehicle driving scene is proposed A method for predicting the jamming behavior of sexual neighbors.

Description of drawings

1 is a schematic flowchart of a method for predicting a near-field vehicle jamming behavior based on image input according to the present invention;

2 is a schematic structural diagram of a near-field vehicle target detection network according to the present invention;

Fig. 3 is the first part of the algorithm flow for predicting the jamming behavior according to the a priori rule;

Fig. 4 is the second part of the algorithm flow for predicting the jamming behavior according to the a priori rule;

FIG. 5 is the third part of the algorithm flow for predicting the jamming behavior according to the prior rules.

Detailed ways

The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Note that the description of the following embodiments is merely an illustration in essence, and the present invention is not intended to limit its application or use, and the present invention is not limited to the following embodiments.

Example

The main steps of the behavior prediction method of the present invention include collecting the image sequence information based on the forward panoramic image in the real structured road scene, marking the position and behavior information of the vehicle target in the image sequence by artificial methods; A deep convolutional neural network for vehicle detection and tracking; construct a deep neural network suitable for lane line detection in structured roads and the corresponding loss function; the vehicle ID obtained from the established near-field vehicle detection and tracking model is related to the corresponding target. The position data of the bounding box and the lane line model obtained by the established lane line detection network are used to obtain the relative position deviation between the target and the lane line. Aiming at the problem that the current neural network cannot predict the vehicle jamming behavior poorly, the present invention proposes a new jamming behavior prediction algorithm, which greatly improves the performance of near-field vehicle real-time behavior prediction under the premise of limited computing capability of the mobile terminal, and can be easily deployed in In the existing intelligent driving system, the intelligent driving car can respond to the jamming behavior of the near-field vehicle in time, so as to improve the safety during driving.

As shown in FIG. 1 , this embodiment provides a near-field vehicle jamming behavior prediction method based on image input, and the method includes:

(1) Collect the image sequence information based on the forward panoramic image in the real structured road scene, and manually mark the position and behavior information of the vehicle target in the image sequence;

(2) Build a near-field vehicle detection and tracking model suitable for near-field vehicle detection and tracking in structured roads;

(3) Construct a lane line detection network and corresponding loss function suitable for lane line detection in structured roads;

(4) Based on the vehicle ID obtained by the near-field vehicle detection and tracking model established in step (2) and the bounding box position data of the corresponding target, and the lane line obtained by the lane line detection network established in step (3), the target and lane are obtained. According to the relative position deviation of the line, according to the formulation of the prior rules, the prediction result of the jamming behavior of the near-field vehicle is obtained.

Specifically, step (1) specifically includes:

(11) Calibrate the internal and external parameters of the camera, wherein the external parameters include a rotation matrix R and a translation vector T, and the internal parameters include an internal parameter matrix K, and the camera distortion coefficient;

(12) Use a data collection vehicle equipped with a camera to collect video data in a real road scene, and record the type of vehicle target in the image at the time of collection;

(13) Annotate the collected video data with an annotation tool. The annotation methods include vehicle target tracking ID annotation, vehicle target category annotation, target object bounding box annotation, vehicle jamming start, vehicle crossing the midpoint of the lane line, and vehicle completion jamming behavior The key frame annotation and the vehicle jamming behavior category annotation of the vehicle should contain at least the position, key frame and jamming behavior category information of the near-field vehicle. The vehicle target ID required to be recorded in this process is unique.

Step (2) specifically includes:

(21) Construct a near-field vehicle target detection network based on the improved Yolov5, input the input video slices as image time series to the near-field vehicle target detection network, and perform multi-layer convolution and downsampling operations to characterize the input image information Extraction and feature encoding to obtain a multi-dimensional feature tensor that divides the picture. The network structure of the near-field vehicle target detection network in this part is shown in Figure 2, which is composed of Backbone, FPN, PAN and other structures;

(22) Build a classification network, adopt non-maximum suppression operation, and finally obtain the location information and classification confidence information of each target, including the classification probability and positioning probability of the object;

(23) Build a near-field vehicle target tracking network based on improved Deep-SORT, take the target object bounding box information and classification information obtained by target detection as input, locate and track multiple objects in the video simultaneously, and record ID and trajectory information , especially under the condition of occlusion to reduce the transformation of object ID, output the tracking ID of the target vehicle, the target category and the bounding box information of the target object, the ReID module in the Deep-SORT network has undergone the reclassification process of the new vehicle weight. Recognition dataset Compcars training.

Step (3) specifically includes:

(31) Construct a backbone network for lane feature extraction based on convolutional neural network, connect network output features based on shallow residuals, and improve the inference speed of the model while ensuring the detection effect by using a larger receptive field. The output of the differentially connected convolutional neural network has four scales, which are 112×112×3, 56×56×64, 28×28×128, 14×14×256:

(32) Construct a lane line semantic segmentation network, up-sample multi-scale features to the same scale during network training, and through transposed convolution, calculate the semantic segmentation loss, enhance the visual feature extraction ability of the backbone network, and finally obtain an enhanced based on Residual connected lane line detection backbone network, lane line semantic segmentation network, multi-scale features are upsampled to the same scale during network training, and are not used in the actual inference process;

(33) According to the features extracted by the backbone network, according to the a priori specified vertical candidate anchor frame of the picture, the candidate points are calculated by the classifier in the global scope, and finally the lane line position node of the lane where the vehicle is located is obtained. In this embodiment, the lane line The value of the number of classes C is 4, and the value of the number of vertical candidate anchor boxes h is 18;

(34) Construct the loss function of the lane line detection network, including multi-classification loss, segmentation loss and lane structure loss. The loss function of the lane line detection network is expressed as L _total :

L _total =L _cls +L _seg +ηL _lane

L _cls is the multi-classification loss, L _seg is the segmentation loss, L _lane is the lane structuring loss, and η is the hyperparameter.

Among them, the multi-class loss L _cls and the segmentation loss L _seg both use the cross entropy loss function, and the segmentation loss L _seg uses the multi-class loss with 2 classifications.

Specifically, the multi-class loss L _cls is expressed as:

Among them, L _CE ( ) represents the cross-entropy loss function, P _i,j,: represents the prediction results of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, T _{i, j,:} indicates the true distribution of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, c _i,j,: indicates P _i,j,: and T _{i,j, :} similarity, C and h represent the number of lane line classes and the number of longitudinal anchor points of the lane, respectively, and γ and α are hyperparameters.

The lane structuring loss L _lane is expressed as:

L _lane =L _sim +λL _shp

Among them, L _sim is the similarity loss, L _shp is the shape loss, λ is the hyperparameter representing the loss weight, P _{i, j, k} is the predicted probability of the i-th lane line at the position (j, k), w is the number of lane line units in each row.

In the embodiment, the value of η is 1, the value of γ is 2, the value of α is 0.25, and the value of λ is 1.25.

Step (4) The steps of training the network and formulating a priori rules are as follows:

(41) Perform data preprocessing on the collected image sequence, including: randomly flipping, cropping and uniformly scaling the image to a fixed size horizontally, and flipping, cropping and scaling the labeled data accordingly. The obtained image is normalized by channel;

(42) Use the highly robust regression model to obtain the lane line fitting model of the own vehicle's lane by using the lane line position node of the own vehicle's lane detected by the lane line detection, and the lane line fitting model of the own vehicle's lane is linear. model, and the left and right lanes are respectively fitted according to the predicted position nodes of the lane line;

(43) Establish an area of interest for jamming behavior according to the lane line model of the lane where the ego vehicle is located, calculate the position deviation between the bounding box information of the target vehicle and the area of interest for jamming behavior, and establish the jamming behavior expectation of each target according to the target vehicle tracking ID The dictionary of times and vehicle status symbols, the key value in the established expected times of jamming behavior of each target and the vehicle status symbol dictionary is the target tracking ID, and the value is the expected number of times and the vehicle status symbol;

(44) Determine the specific behavior of the target vehicle by setting the threshold of the expected number of jamming behaviors combined with the vehicle status symbol, update the parameters of the established jamming behavior area of interest, and obtain ideal network parameters after iteration.

According to the formulation of the prior rules, the specific flow of the algorithm to obtain the prediction result of the jamming behavior of the near-field vehicle is shown in Figure 3, Figure 4, and Figure 5. First, after each adjacent vehicle completes detection and tracking, construct a target state descriptor with the target tracking ID as the key and the initial state [0, 'keep'] as the value, where 0 is the count value and 'keep' is the historical state descriptor. According to the difference of the area where the adjacent vehicle detection bounding box (Bounding Box) is located, it is judged that the initial lane of the adjacent vehicle is the left lane or the right lane; at the same time, according to the lane line model obtained by the lane line detection module, the width on both sides of the lane line is appropriately increased. Set the vehicle jamming prediction region of interest (Cut-In RoI). According to the relative positional relationship between the adjacent vehicle target and the vehicle jamming prediction region of interest (Cut-InRoI), it can be divided into two position states: (1) when the adjacent vehicle target is crossing the edge of the area; (2) when the adjacent vehicle target is in the area. The latter is relatively simple, and the target state descriptor is set to [2, 'follow'] at this time. The former state mostly occurs at the cut-in and cut-out moments of the adjacent vehicle. Under this condition, it can be distinguished according to the historical state descriptor {'keep', 'cut_in', 'follow'}.

If the target state descriptor is [count<Threshold, 'keep'], it is considered that the target adjacent vehicle was in the lane keeping state at the previous moment and there is a potential trend from lane keeping to jamming behavior. At the same time, in order to reduce the warning logic of the jamming behavior of adjacent vehicles If the false detection rate is higher, the prediction time of the jamming warning can be appropriately delayed, and only the targets that have been tracked for a long enough time to be jammed are considered. Specifically, only the points of the adjacent vehicle detection bounding box (Bounding Box) that are far from the cut-in prediction region of interest (Cut-In RoI) are located outside the region, while the points within the cut-in prediction region of interest (Cut-In RoI) are located outside the region. When the number of times the bounding box key point (the distance from the other end point is a certain proportion of the width α) is detected exceeds the threshold, the target state descriptor is [count=count+1, 'keep'] and keeps updating until count=Threshold, The target state descriptor is updated to [count=Threshold, 'cut_in']. In this project, α=0.4 is set, and the threshold of the detected jamming times Threshold=3. At this time, the jamming behavior of adjacent vehicles in the current frame changes from action='keep' to action='cut_in', and the adjacent vehicle detection bounding box (Bounding Box) and jamming prediction region of interest (Cut-In RoI) become red warnings.

If the target state descriptor of the current frame is [count=Threshold, 'cut_in'], it can be considered that the target is continuing the jamming behavior of the previous frame, so keep the target state descriptor as [count=Threshold, 'cut_in'] unchanged, The jamming behavior of the adjacent vehicles in the current frame is changed from action='cut_in', the adjacent vehicle detection bounding box (BoundingBox) and the jamming prediction region of interest (Cut-In RoI) continue to remain red, indicating that the jamming behavior of the neighboring vehicles is taking place.

If the target state descriptor of the current frame is [count=Threshold, 'cut_in'], but at this time, the adjacent vehicle detection bounding box (Bounding Box) completely enters the area of interest (Cut-In RoI) for plugging prediction, it can be considered that the target has been The jamming behavior is completed, so the target state descriptor is updated to [count=Threshold,'follow'], the jamming behavior of the adjacent vehicle in the current frame is from action'keep', the adjacent vehicle detection bounding box (Bounding Box) and the jamming prediction region of interest ( Cut-In RoI) turns green, indicating that the approaching vehicle jamming behavior has been completed and the current frame is in a safe state.

If the target state descriptor of the current frame is pcount=Threshold, 'folow'], and the adjacent vehicle detection bounding box (Bounding Box) is at the edge of the crossed prediction region of interest (Cut-In RoI), it can be considered that the target has been completed. The jamming behavior is leaving the current lane, so the target state descriptor is updated to [count=Threshold,'keep'], the jamming behavior of the adjacent vehicles in the current frame is from action='keep', the adjacent vehicle detection bounding box (BoundingBox) and jamming prediction The region of interest (Cut-In RoI) continues to remain green and the current frame is in a safe state.

In any case, if the adjacent vehicle detection bounding box (Bounding Box) completely leaves the area of interest (Cut-In RoI) for plugging prediction, immediately update the target state descriptor to [count=0, 'keep'], and in the next Continue to repeat the above steps for one frame. Using the above algorithm flow, there is an additional 3 frame delay for the jamming situation. Since the FPS of the input stream is 30, this delay is equivalent to 0.1s; under real conditions, the human driver usually needs at least 0.7s when encountering an unforeseen jamming cut-in situation. to brake. Therefore, even if there is a time delay from the detection of a jamming warning sent, it is not expected to cause serious problems, in line with the actual technical requirements.

After testing, it can be seen that the network proposed by the present invention successfully predicts the jamming behavior of adjacent vehicles and gives an early warning, and at the same time can realize the early warning cancellation after the jamming is completed, and can be successfully distinguished in the above two similar scenarios.

In a word, the present invention proposes a near-field vehicle jamming behavior prediction method based on image input, which greatly improves the performance of near-field vehicle real-time behavior prediction under the premise of limited computing capability of the mobile terminal, and can be easily deployed in the existing In the intelligent driving system, the intelligent driving car can respond in time to the jamming behavior of vehicles in the near field, and improve the safety during driving.

The above-described embodiments are merely examples, and do not limit the scope of the present invention. These embodiments can be implemented in other various forms, and various omissions, substitutions, and changes can be made without departing from the technical idea of the present invention.

Claims (10)

1. a near-field vehicle jamming behavior prediction method based on image input, is characterized in that, the method comprises: (1) Collect the image sequence information based on the forward panoramic image in the real structured road scene, and manually mark the position and behavior information of the vehicle target in the image sequence; (2) Build a near-field vehicle detection and tracking model suitable for near-field vehicle detection and tracking in structured roads; (3) Construct a lane line detection network and corresponding loss function suitable for lane line detection in structured roads; (4) Based on the vehicle ID obtained by the near-field vehicle detection and tracking model established in step (2) and the bounding box position data of the corresponding target, and the lane line obtained by the lane line detection network established in step (3), the target and lane are obtained. According to the relative position deviation of the line, according to the formulation of the prior rules, the prediction result of the jamming behavior of the near-field vehicle is obtained. 2. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 1, is characterized in that, step (1) specifically comprises: (11) Calibrate the internal and external parameters of the camera, wherein the external parameters include a rotation matrix R and a translation vector T, and the internal parameters include an internal parameter matrix K, and the camera distortion coefficient; (12) Use a data collection vehicle equipped with a camera to collect video data in a real road scene, and record the type of vehicle target in the image at the time of collection; (13) Annotate the collected video data with an annotation tool. The annotation methods include vehicle target tracking ID annotation, vehicle target category annotation, target object bounding box annotation, vehicle jamming start, vehicle crossing the midpoint of the lane line, and vehicle completion jamming behavior The keyframe annotations and vehicle jamming behavior category annotations for the vehicle are required to contain at least the location, keyframes, and jamming behavior category information of the near-field vehicle. 3. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 1, is characterized in that, step (2) specifically comprises: (21) Construct a near-field vehicle target detection network based on the improved Yolov5, input the input video slices as image time series to the near-field vehicle target detection network, and perform multi-layer convolution and downsampling operations to characterize the input image information Extraction and feature encoding to obtain a multi-dimensional feature tensor that divides the image; (22) Build a classification network, adopt non-maximum suppression operation, and finally obtain the location information and classification confidence information of each target, including the classification probability and positioning probability of the object; (23) Build a near-field vehicle target tracking network based on improved Deep-SORT, take the target object bounding box information and classification information obtained by target detection as input, locate and track multiple objects in the video simultaneously, and record ID and trajectory information , especially under the condition of occlusion, reduce the transformation of object ID, and output the tracking ID of the target vehicle, the target category and the bounding box information of the target object. 4. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 1, is characterized in that, step (3) specifically comprises: (31) Construct a backbone network for lane feature extraction based on convolutional neural network, output features based on shallow residual connection network, and improve the inference speed of the model while ensuring the detection effect by using a larger receptive field; (32) Construct a lane line semantic segmentation network, up-sample multi-scale features to the same scale during network training, and through transposed convolution, calculate the semantic segmentation loss, enhance the visual feature extraction ability of the backbone network, and finally obtain an enhanced based on Residual connected lane detection backbone network; (33) According to the features extracted by the backbone network, according to the longitudinal candidate anchor frame of the picture specified by the prior, the candidate points are calculated by the classifier in the global scope, and finally the lane line position node of the lane where the vehicle is located is obtained; (34) Construct the loss function of lane line detection network, including multi-classification loss, segmentation loss and lane structure loss. 5. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 4, is characterized in that, the loss function of described lane line detection network is expressed as L _total : L _total =L _cls +L _seg +ηL _lane L _cls is the multi-classification loss, L _seg is the segmentation loss, L _lane is the lane structuring loss, and η is the hyperparameter. 6. The near-field vehicle jamming behavior prediction method based on image input according to claim 5, wherein the multi-class loss L _cls is expressed as:

Among them, L _CE ( ) represents the cross-entropy loss function, P _i,j,: represents the prediction results of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, T _{i, j,:} indicates the true distribution of all (w+1) lane line units for the i-th lane line and the j-th lateral anchor box, c _i,j,: indicates P _i,j,: and T _{i,j, :} similarity, C and h represent the number of lane line classes and the number of longitudinal anchor points of the lane, respectively, and γ and α are hyperparameters. 7. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 5, is characterized in that, described lane structure loss L _lane is expressed as: L _lane =L _sim +λL _shp

Among them, L _sim is the similarity loss, L _shp is the shape loss, λ is the hyperparameter representing the loss weight, P _{i, j, k} is the predicted probability of the i-th lane line at the position (j, k), w The number of division cells for each row. 8. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 1, is characterized in that, step (4) training network and formulating a priori rule step are as follows: (41) Perform data preprocessing on the collected image sequence, including: randomly flipping, cropping and uniformly scaling the image to a fixed size horizontally, and flipping, cropping and scaling the labeled data accordingly. The obtained image is normalized by channel; (42) Using a highly robust regression model to obtain the lane line fitting model of the lane where the ego vehicle is located by using the lane line position node of the lane where the ego vehicle is located, obtained from the lane line detection; (43) Establish an area of interest for jamming behavior according to the lane line model of the lane where the ego vehicle is located, calculate the position deviation between the bounding box information of the target vehicle and the area of interest for jamming behavior, and establish the jamming behavior expectation of each target according to the target vehicle tracking ID Dictionary of times and vehicle status symbols; (44) Determine the specific behavior of the target vehicle by setting the threshold of the expected number of jamming behaviors combined with the vehicle status symbols, update the parameters of the established region of interest for jamming behavior, and obtain ideal network parameters after iteration. 9. A near-field vehicle jamming behavior prediction method based on image input according to claim 8, wherein the lane line fitting model of the lane where the vehicle is located is a linear model, and the left and right lanes are respectively predicted according to the lane lines. Node fitting is obtained. 10. a kind of near-field vehicle jamming behavior prediction method based on image input according to claim 8, is characterized in that, the key value in the expected number of jamming behavior of each target established and the vehicle status symbol dictionary is the target tracking ID, The values are expected times and vehicle status symbols.

Images