CN117437808A - A method for over-the-horizon sensing and early warning in blind spots in underground parking lots - Google Patents

Abstract

The invention provides a method for sensing and early warning beyond visual range of blind areas of an underground parking lot, which comprises the steps of collecting video images of the whole areas of the parking lot; obtaining vehicle pose information taking monitoring as an origin; building a virtual scene simulation platform of the whole domain of the parking lot; a virtual scene display screen device is arranged, a virtual picture of a first driver view angle which is removed from the shielding of the wall column is provided on the display screen, and blind area perspective is realized; predicting pose data change of the vehicle in a future period of time, measuring the relation between the current visual angle and the predicted pose data, and designing the optimal visual angle change; and evaluating the collision risk of the vehicle and carrying out early warning prompt. The invention realizes detection and early warning of the dead zone vehicles and other targets, has the characteristics of low time delay and low cost, and can effectively solve the dead zone safety problem; the method realizes the beyond-visual-distance sensing and early warning of the blind area of the underground parking lot by depending on the environment sensing and the virtual space, and provides important tools and means for solving the frequent occurrence of the accidents of the conflict points of the blind area.

Description

A method for over-the-horizon sensing and early warning in blind spots in underground parking lots

Technical field

The invention relates to the field of transportation technology, and in particular to a method for over-the-horizon sensing and early warning in blind spots in underground parking lots.

Background technique

In order to maintain the space required for urban functions and transportation, underground parking lots continue to emerge in cities. However, its internal wall columns are densely packed, there are a large number of sharp bends and steep slopes, and the lighting is dim, resulting in a large number of potential operational conflicts and safety risks, and a high incidence of accidents in blind spots. In order to expand the driver's field of vision and reduce the impact of blind spots, the traditional method is to install road wide-angle mirrors on key wall pillars. However, it is easily affected by factors such as light, viewing angle range, placement angle, etc., and its reliability is low; the wide-angle lens causes picture distortion, which is not intuitive enough. The driver usually focuses on finding a parking space and is not easy to detect; the wide-angle lens expands the field of view is limited, giving the driver poor reaction Less time. Therefore, the actual reminder effect of road wide-angle lenses is often greatly reduced, and its role in avoiding accidents is extremely limited.

At present, the digital see-through A-pillar system has been implemented in relevant companies. It uses the A-pillar screen to display the transformed blind spot image, achieving a perspective effect that eliminates A-pillar occlusion, and better reduces the occurrence of accidents caused by A-pillar occlusion. . If this idea of eliminating blind spots is directly applied to the parking lot blind spot environment, because the camera is fixed and the driver's position changes all the time, it is impossible to display the image from the first perspective of the service target driver in real time. There is still a problem that the road wide-angle lens picture is not intuitive enough. Therefore, there is an urgent need to use environmental awareness and virtual platforms to run visualization synchronously. By transmitting the picture in the virtual platform with the wall pillars removed to the LED screen installed on the wall pillars, and at the same time providing dangerous vehicle warning signs, we can finally achieve over-the-horizon perception of blind spots in the parking lot. and early warning functions, providing new ideas and new means to solve the "stubborn problem" of frequent accidents at conflict points in underground parking lots.

Contents of the invention

The purpose of the present invention is to propose a method with over-the-horizon sensing and early warning functions in a parking lot, which solves the above technical problems.

In order to achieve the above objectives, the present invention proposes a method for over-the-horizon sensing and early warning in blind spots in underground parking lots, which includes the following steps:

S1: Collect video images of the entire parking lot;

S2: Use the image data for depth estimation based on the DORN algorithm to obtain the vehicle pose information with the monitoring as the origin;

S3: Build a virtual scene simulation platform for the entire parking lot;

S4: Arrange a virtual scene display screen device in the underground parking lot. Based on the vehicle posture information and the virtual scene simulation platform, the display screen device provides a virtual picture from the perspective of the first driver without the obstruction of the wall pillar on the display screen. , to achieve blind spot perspective;

S5: Based on the vehicle's posture data, predict the vehicle's posture data changes in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change;

S51. Perceive the user's needs through the trajectory analysis and prediction algorithm, and extract the position and attitude data of the user's vehicle. Among them, x(t), y(t), z(t) are the position coordinates of the vehicle in the three-dimensional space,/> is the attitude angle of the vehicle (the rotation angle around the x, y, and z axes), and t is the time. Based on the pose data in the past period of time/> Prediction functions via trajectory analysis/> Predict the pose data P'(t _n+1 ) in the future;

S52. Measure the relationship between the current perspective and the predicted pose data through the objective function G(V(t),P'(t _n+1 )). The current perspective V(t)=(v _x (t), v _y (t), v _z (t)), where v _x (t), v _y (t), v _z (t) respectively represent the direction of the current perspective in the x, y, and z axes. The larger the value of the objective function, the greater the The more the current perspective meets the driver’s needs;

S53. Set the optimal viewing angle change ΔV(t) to maximize the value of the objective function G(V(t),P'(t _n+1 )), that is And update the position and posture of the camera in the virtual platform through V(t _n+1 )=V(t)+ΔV(t), thereby switching the display perspective according to user needs;

S6: Based on the time series information of the vehicle position data, evaluate the vehicle collision risk, generate annotation data, and implement collision warning prompts between vehicles.

Further, in S1, the video images are collected through monitoring arranged in the parking lot; in S1, frame cutting and preprocessing of the collected video images are also included.

Further, the preprocessing includes enhancing the contrast, brightness and sharpness of the video image.

Further, the S2 includes the following steps:

S21. Use the DORN algorithm to estimate the depth of the image sequence and obtain the depth image; the algorithm uses a deep regression neural network to estimate a single image, and uses inter-frame information to obtain the depth information of the continuous image sequence;

S22. Input the depth image and original RGB image into the D4LCN model, and use depth-guided convolution to extract image features; the model uses a multi-branch architecture to perform multi-scale and multi-channel convolution on the input depth image and RGB image to extract image features. , thereby realizing the positioning and identification of vehicle location information;

S23. Based on the image feature extraction, use the non-maximum suppression target detection module of D4LCN to identify and locate the vehicle; this algorithm obtains the position, size and location of the target by classifying and regressing the targets in the image. Direction information; at the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and positioned in consecutive frames.

Further, the S3 includes the following steps:

S31. Obtain the basic spatial information of the corresponding underground parking lot, including the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout. Use the Vue framework and three.is engine to perform scene rendering, provide a user interaction interface, and display parking. Basic information about the field;

S32. Use the Flask framework to build a program to extract the vehicle pose data sensed by the camera, retrieve the vehicle pose information with the same ID, sort the trajectory by time, and organize the trajectory data set of the vehicle in the virtual panoramic perspective through perspective transformation P = {P ₁ ,P ₂ ,···,P _j ,···,P _m }, where is the trajectory of vehicle j, x _j , y _j , z _j are the three-dimensional coordinates of vehicle j at time t,/> is the attitude angle of vehicle j at time t;

S33, front-end and back-end interact with data through RESTfulAPI to realize panoramic operation of the virtual platform.

Further, the S4 includes the following steps:

S41. According to the parking space layout and vehicle direction, arrange a virtual scene display device on the wall column. Its height, size, direction and brightness should be in order to facilitate the driver's observation;

S42. Based on the virtual space platform, track the driver's perspective and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

Further, the S6 includes the following steps:

S61. Extract the corresponding motion trajectory and speed timing information based on the vehicle's pose data, through v _f = _vi +at, s=(v _i +v _f )t/2 obtains the speed and acceleration of the vehicle, where v _i and v _f represent the initial speed and final speed of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents The vehicle traveling distance can be obtained from the position data of adjacent frames of the vehicle;

S62. Evaluate the risk of vehicle collision and calculate the time required for a collision between two vehicles traveling at the current speed TTC = L/(v ₁ - v ₂ ), where L is the distance between vehicle 1 and vehicle 2, v ₁ and v ₂ represent the speed of vehicle 1 and vehicle 2 respectively;

S63. Set the distance threshold between the two vehicles according to the vehicle operation conditions in the parking lot. When L is lower than a certain threshold, different risk levels are marked according to the TTC value to complete the collision warning prompt.

Compared with the existing technology, the advantages of the present invention are:

1. The present invention realizes the detection and early warning of vehicles and other targets in blind spots, has the characteristics of low delay and low cost, and can effectively solve the safety problem of blind spots; it relies on environmental perception and virtual space to realize over-the-horizon perception of blind spots in underground parking lots. and early warning, providing important tools and means to solve the frequent accidents at blind spots and conflict points.

2. The present invention accurately positions indoor vehicles, and the display screen device provides accurate viewing angles based on vehicle information, thereby providing an effective guarantee for the safety of vehicles driving in the basement.

Description of the drawings

Figure 1 is a flow chart of a blind spot over-the-horizon sensing and early warning method in an underground parking lot in an embodiment of the present invention.

Figure 2 is a schematic diagram of image preprocessing in the over-the-horizon sensing and early warning method for blind spots in underground parking lots in an embodiment of the present invention;

Figure 3 is a schematic diagram of target detection in the over-the-horizon sensing and early warning method for blind spots in underground parking lots in an embodiment of the present invention;

Figure 4 is a schematic diagram of the practical application of the over-the-horizon sensing and early warning method for blind spots in underground parking lots in an embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be further described below.

As shown in Figure 1, the present invention is an over-the-horizon sensing and early warning method for blind spots in underground parking lots, which includes the following steps:

S1. Read the surveillance video data deployed throughout the parking lot and preprocess the video images.

S11. The parking lot monitoring video stream data is transmitted to the environment sensing module in real time through the interface;

S12. The video is divided into images frame by frame. Use the video processing library (OpenCV) to segment frame by frame to obtain video sequence images;

S13. Image enhancement preprocessing. As shown in Figure 2, the video image is processed using the image enhancement tool in the Pythonpillow library. The enhancement algorithm includes contrast, brightness and sharpness. Image enhancement preprocessing can improve image quality and reduce the loss of feature details during depth extraction.

Further, the specific algorithm of step S13 is:

Contrast enhancement:

Among them, Oi,j represents the enhanced pixel value, Ii,j represents the original pixel value, μ represents the average gray value of the image, σ represents the standard deviation of the image, and k represents the contrast gain.

Brightness enhancement:

Oi,j＝Ii,j×k+b

Where b represents the brightness offset, and the other variables are the same as above.

Sharpness enhancement:

Oi,j＝Ii,j+Ii,j×k-Li,j×k

Among them, Li,j represents the pixel value after Laplacian filtering, k represents the sharpness gain, and the other variables are the same as above.

S2. Use the DORN algorithm to perform depth estimation on the image data, perform three-dimensional target detection on the obtained depth image and the original RGB image based on the D4LCN model, and output the vehicle pose information with the monitoring as the origin of the coordinates.

S21. Use the DORN algorithm to perform depth estimation on the image sequence to obtain a depth image. Assume that the input RGB image is where h and w represent the height and width of the image respectively, and 3 represents the number of channels of the image. The output of DORN is a depth image/> The value of each pixel represents the depth of that point. The depth calculation method is:

where p _k (x, y) represents the probability that the pixel point (x, y) belongs to the k-th depth interval. The resulting depth is image normalized and converted into a depth image in grayscale form.

This algorithm uses a deep regression neural network to estimate a single image, and uses inter-frame information to obtain depth information of continuous image sequences to improve the accuracy and robustness of depth estimation.

S22. Input the depth image and original RGB image into the D4LCN model, and use depth-guided convolution to extract image features. The depth information in the depth map is calculated as the information to guide the convolution module:

Among them, I' is the output image feature information, k is the convolution kernel scale, g _i and g _j are the image unit translation scales respectively, I is the input image matrix, and D is the depth image matrix.

The model uses a multi-branch architecture to perform multi-scale, multi-channel convolution on the input depth image and RGB image to extract image features, thereby achieving positioning and identification of vehicle position information.

S23. Based on the image feature extraction, use the non-maximum suppression (NMS) target detection module of D4LCN to identify and locate the vehicle. The obtained vehicle position information is [x _f,i y _f,i z _f,i ] _3D , which represents the three-dimensional spatial coordinates of the vehicle under the coordinate system of the surveillance probe's perspective, where f is the sequence number of the preprocessed video segmentation image sequence, i is the target signal contained in a single image. The set information of the same vehicle bounding box is [h _f,i w _f,i l _f,i ] _3D , where h, w, l respectively represent the height of the bounding box (vertical straight direction), length (parallel to the direction of target movement) and width. This algorithm obtains the position, size and direction information of the target by classifying and regressing the targets in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and positioned in consecutive frames.

S3. As shown in Figure 3, obtain the basic information of the parking space layout, filter and process the pose data matching the vehicle from the camera sensing data set, build a simulation platform, and realize panoramic operation of the virtual platform.

S31. Obtain the basic spatial information of the corresponding underground parking lot, including but not limited to the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout, etc. Use the Vue framework and three.is engine to perform scene rendering and provide user interaction Interface to display basic parking lot information;

S32. Use the Flask framework to build a program to extract the vehicle pose data sensed by the camera, retrieve the vehicle pose information with the same ID, sort the trajectory by time, and organize the trajectory data set of the vehicle in the virtual panoramic perspective through perspective transformation P = {P ₁ ,P ₂ ,···,P _j ,···,P _m }, where is the trajectory of vehicle j, x _j , y _j , z _j are the three-dimensional coordinates of vehicle j at time t,/> is the attitude angle of vehicle j at time t;

S33, front-end and back-end interact with data through RESTfulAPI to realize panoramic operation of the virtual platform.

S4. Set up a virtual scene display device to track the perspective of the user in the car in the virtual platform. By removing the wall column blocking model in the virtual platform, it provides the driver with a wide field of view and eliminates the hidden danger of blind spots.

S41. According to the parking space layout and vehicle direction, arrange a virtual scene display device on the wall column. Its height, size, direction and brightness should be in order to facilitate the driver's observation;

S42. Based on the virtual space platform, track the driver's perspective and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective.

S5. Based on the vehicle's posture data, predict the vehicle's posture data changes in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change.

S51. Perceive the user's needs through the trajectory analysis and prediction algorithm, and extract the position and attitude data of the user's vehicle. Among them, x(t), y(t), z(t) are the position coordinates of the vehicle in the three-dimensional space,/> is the attitude angle of the vehicle (the rotation angle around the x, y, and z axes), and t is the time. Based on the pose data in the past period of time/> Prediction functions via trajectory analysis/> Predict the pose data P'(t _n+1 ) in the future;

S52. Measure the relationship between the current perspective and the predicted pose data through the objective function G(V(t),P'(t _n+1 )). The current perspective V(t)=(v _x (t), v _y (t), v _z (t)), where v _x (t), v _y (t), v _z (t) respectively represent the direction of the current perspective in the x, y, and z axes. The larger the value of the objective function, the greater the The more the current perspective meets the driver’s needs;

S53. Set the optimal viewing angle change ΔV(t) to maximize the value of the objective function G(V(t),P'(t _n+1 )), that is And the position and posture of the camera in the virtual platform are updated through V(t _n+1 )=V(t)+ΔV(t), thereby switching the display perspective according to user needs.

S6. Based on the time series information of the vehicle position data, evaluate the vehicle collision risk, generate annotation data, and implement collision warning prompts between vehicles.

S61. Extract the corresponding motion trajectory and speed timing information based on the vehicle's pose data, through v _f = _vi +at, s=(v _i +v _f )t/2 obtains the speed and acceleration of the vehicle, where v _i and v _f represent the initial speed and final speed of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents The vehicle traveling distance can be obtained from the position data of adjacent frames of the vehicle;

S62. Evaluate the risk of vehicle collision and calculate the time required for a collision between two vehicles traveling at the current speed TTC = L/(v ₁ - v ₂ ), where L is the distance between vehicle 1 and vehicle 2, v ₁ and v ₂ represent the speed of vehicle 1 and vehicle 2 respectively;

S63. Set the distance threshold between the two vehicles according to the vehicle operation conditions in the parking lot. When L is lower than a certain threshold, different risk levels are marked according to the TTC value to complete the collision warning prompt.

This example is based on an underground parking lot in Pudong as a case study scenario. There are ramps, sharp turns, and dense wall columns in the parking lot. There are many blind spots in the field of vision and many safety hazards. Multiple cameras are deployed to achieve full-area monitoring and environmental perception detection, process the acquired vehicle pose data, render the vehicle's motion status in the virtual space, and further perceive user needs to achieve personalized perspective switching and collision warning annotation.

This embodiment applies the above technical solution, as shown in Figure 4, and its main process is:

Step 1: Read the surveillance video data deployed throughout the parking lot and preprocess the video images.

Step 1.1. The parking lot monitoring video stream data is transmitted to the information processing module in real time through the interface;;

Step 1.2. The video is divided into images frame by frame. Use the video processing library (OpenCV) to segment frame by frame to obtain video sequence images;

Step 1.3, image enhancement preprocessing. Use the image enhancement tools in the Pythonpillow library to process video images. The enhancement algorithms include contrast, brightness and sharpness. Image enhancement preprocessing can improve image quality and reduce the loss of feature details during depth extraction.

Further, the specific algorithm of step 1.3 is:

Contrast enhancement:

Among them, Oi,j represents the enhanced pixel value, Ii,j represents the original pixel value, μ represents the average gray value of the image, σ represents the standard deviation of the image, and k represents the contrast gain.

Brightness enhancement:

Oi,j＝Ii,j×k+b

Where b represents the brightness offset, and the other variables are the same as above.

Sharpness enhancement:

Oi,j＝Ii,j+Ii,j×k-Li,j×k

Among them, Li,j represents the pixel value after Laplacian filtering, k represents the sharpness gain, and the other variables are the same as above.

Step 2: Use the DORN algorithm to perform depth estimation on the image data, perform three-dimensional target detection on the obtained depth image and the original RGB image based on the D4LCN model, and output the vehicle pose information with the monitoring as the origin of the coordinates.

Step 2.1. Use the DORN algorithm to perform depth estimation on the image sequence to obtain a depth image. Assume that the input RGB image is where h and w represent the height and width of the image respectively, and 3 represents the number of channels of the image. The output of DORN is a depth image/> The value of each pixel represents the depth of that point. The depth calculation method is:

where p _k (x, y) represents the probability that the pixel point (x, y) belongs to the k-th depth interval. The resulting depth is image normalized and converted into a depth image in grayscale form.

This algorithm uses a deep regression neural network to estimate a single image, and uses inter-frame information to obtain depth information of continuous image sequences to improve the accuracy and robustness of depth estimation.

Step 2.2: Input the depth image and original RGB image into the D4LCN model, and use depth-guided convolution to extract image features. The depth information in the depth map is calculated as the information to guide the convolution module:

Among them, I' is the output image feature information, k is the convolution kernel scale, g _i and g _j are the image unit translation scales respectively, I is the input image matrix, and D is the depth image matrix.

The model uses a multi-branch architecture to perform multi-scale, multi-channel convolution on the input depth image and RGB image to extract image features, thereby achieving positioning and identification of vehicle position information.

Step 2.3. Based on the image feature extraction, use the non-maximum suppression (NMS) target detection module of D4LCN to identify and locate the vehicle. The obtained vehicle position information is [x _f,i y _f,i z _f,i ] _3D , which represents the three-dimensional spatial coordinates of the vehicle under the coordinate system of the surveillance probe's perspective, where f is the sequence number of the preprocessed video segmentation image sequence, i is the target signal contained in a single image. The set information of the same vehicle bounding box is [h _f,i w _f,i l _f,i ] _3d , where h, w, l respectively represent the height of the bounding box (vertical straight direction), length (parallel to the direction of target movement) and width. This algorithm obtains the position, size and direction information of the target by classifying and regressing the targets in the image. At the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and positioned in consecutive frames.

Step 3: Obtain the basic information of the parking space layout, filter and process the pose data matching the vehicle from the camera sensing data set, combine the two, build a simulation platform, and realize the panoramic operation of the virtual platform.

Step 3.1. Obtain the basic spatial information of the underground parking lot, including but not limited to the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout. Use the Vue framework and three.is engine to render the scene and provide user interaction. Interface to display basic parking lot information;

Step 3.2: Use the Flask framework to build a program to extract the vehicle pose data perceived by the camera, retrieve the vehicle pose information with the same ID, sort the trajectory by time, and organize the vehicle trajectory data set P in the virtual panoramic perspective through perspective transformation. ={P ₁ ,P ₂ ,···,P _j ,···,P _m }, where is the trajectory of vehicle j, x _j , y _j , z _j are the three-dimensional coordinates of vehicle j at time t,/> is the attitude angle of vehicle j at time t;

Step 3.3. The front and back ends interact with data through RESTfulAPI to achieve panoramic operation of the virtual platform.

Step 4: Set up a virtual scene display device, track the perspective of the user in the car in the virtual platform, and remove the wall column blocking model in the virtual platform to provide the driver with a wide field of view and eliminate the hidden danger of blind spots.

Step 4.1. According to the parking lot space layout and vehicle direction, arrange the virtual scene display device on the wall pillar. Its height, size, direction and brightness should be in order to facilitate the driver's observation;

Step 4.2: Track the driver's perspective based on the virtual space platform, and provide a virtual picture of the first driver's perspective with the wall pillars removed on the display screen to achieve blind spot perspective.

Step 5: Based on the vehicle's posture data, predict the vehicle's posture data changes in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change.

Step 5.1: Perceive user needs through trajectory analysis and prediction algorithms, and extract the position and attitude data of the user's vehicle Among them, x(t), y(t), z(t) are the position coordinates of the vehicle in the three-dimensional space,/> is the attitude angle of the vehicle (the rotation angle around the x, y, and z axes), and t is the time. Based on the pose data in the past period of time/> Prediction functions via trajectory analysis/> Predict the pose data P'(t _n+1 ) in the future;

Step 5.2. Measure the relationship between the current perspective and the predicted pose data through the objective function G(V(t),P'(t _n+1 )). The current perspective V(t)=(v _x (t),v _y (t), v _z (t)), where v _x (t), v _y (t), v _z (t) respectively represent the direction of the current viewing angle in the x, y, and z axes. The larger the objective function value, Indicates that the current perspective can better meet the driver's needs;

Step 5.3. Set the optimal viewing angle change ΔV(t) to maximize the value of the objective function G(V(t),P'(t _n+1 )), that is And the position and posture of the camera in the virtual platform are updated through V(t _n+1 )=V(t)+ΔV(t), thereby switching the display perspective according to user needs.

Step 6: Based on the time series information of the vehicle position data, evaluate the vehicle collision risk, generate annotation data, and implement collision warning prompts between vehicles.

Step 6.1. Extract the corresponding motion trajectory and speed timing information based on the vehicle's pose data, through v _f = _vi +at, s=(v _i +v _f )t/2 obtains the speed and acceleration of the vehicle, where v _i and v _f represent the initial speed and final speed of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents The vehicle traveling distance can be obtained from the position data of adjacent frames of the vehicle;

Step 6.2. Evaluate the risk of vehicle collision and calculate the time required for a collision between two vehicles traveling at the current speed TTC = L/(v ₁ - v ₂ ), where L is the distance between vehicle 1 and vehicle 2, v ₁ and v ₂ represents the speed of vehicle 1 and vehicle 2 respectively;

Step 6.3. Based on the vehicle operation conditions in the parking lot, set the distance threshold between the two workshops to 10m. When L is lower than this threshold, different risk levels are marked according to the TTC value, where TTC≥3s is marked as low risk; 1.5s<TTC<3s is marked as medium risk; TTC≤1.5s is marked as high risk. Based on this judgment mechanism, annotation data is generated to complete collision warning prompts.

To sum up, the present invention proposes an over-the-horizon sensing and early warning device for underground parking lots in view of the frequent accidents in blind spots in underground parking lots, which combines the vehicle position and attitude data collected by the camera with virtual space simulation to realize virtualization of real scenes. By removing the obstruction of wall columns in the virtual space, it provides a wide field of view and eliminates the hidden danger of blind spots; it analyzes the vehicle trajectory operation status and uses the pose change closest to the prediction as the constraint to achieve automatic switching of individual perspectives; based on the timing of vehicle position changes Based on the information, corresponding thresholds are set for distance and collision time, the collision risk is evaluated, risk annotations are generated, and early warning prompts are completed. This invention builds a virtual space platform for the blind spots of underground parking lots, combines cameras, display screens and other hardware equipment to detect the movement traces of vehicles and pedestrians in the blind spots, displays virtual pictures that run synchronously with the real scene, and provides users with "intelligent, "Accurate and personalized" prompts and warnings. This invention realizes the detection and early warning of vehicles and other targets in blind spots, has the characteristics of low delay and low cost, and can effectively solve the safety problem of blind spots; it relies on environmental perception and virtual space to realize over-the-horizon sensing and early warning of blind spots in underground parking lots. , providing important tools and means to solve the frequent accidents at blind spots and conflict points.

The above are only preferred embodiments of the present invention and do not limit the present invention in any way. Any person skilled in the technical field who makes any form of equivalent substitution or modification to the technical solutions and technical contents disclosed in the present invention shall not deviate from the technical solutions of the present invention. The contents still fall within the protection scope of the present invention.

Claims (7)

1. A method for over-the-horizon sensing and early warning in blind spots in underground parking lots, which is characterized by including the following steps: S1: Collect video images of the entire parking lot; S2: Use the image data for depth estimation based on the DORN algorithm to obtain the vehicle pose information with the monitoring as the origin; S3: Build a virtual scene simulation platform for the entire parking lot; S4: Arrange a virtual scene display screen device in the underground parking lot. Based on the vehicle posture information and the virtual scene simulation platform, the display screen device provides a virtual picture from the perspective of the first driver without the obstruction of the wall pillar on the display screen. , to achieve blind spot perspective; S5: Based on the vehicle's posture data, predict the vehicle's posture data changes in the future, measure the relationship between the current perspective and the predicted posture data, and design the optimal perspective change; S51. Perceive the user's needs through the trajectory analysis and prediction algorithm, and extract the position and attitude data of the user's vehicle. Among them, x(t), y(t), z(t) are the position coordinates of the vehicle in the three-dimensional space,/> is the attitude angle of the vehicle (the rotation angle around the x, y, and z axes), and t is the time. Based on the pose data in the past period of time/> Prediction functions via trajectory analysis/> Predict the pose data P'(t _n+1 ) in the future; S52. Measure the relationship between the current perspective and the predicted pose data through the objective function G(V(t),P'(t _n+1 )). The current perspective V(t)=(v _x (t), v _y (t), v _z (t)), where v _x (t), v _y (t), v _z (t) respectively represent the direction of the current perspective in the x, y, and z axes. The larger the value of the objective function, the greater the The more the current perspective meets the driver’s needs; S53. Set the optimal viewing angle change ΔV(t) to maximize the value of the objective function G(V(t),P'(t _n+1 )), that is And update the position and posture of the camera in the virtual platform through V(t _n+1 )=V(t)+ΔV(t), thereby switching the display perspective according to user needs; S6: Based on the time series information of the vehicle position data, evaluate the vehicle collision risk, generate annotation data, and implement collision warning prompts between vehicles. 2. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 1, characterized in that in S1, the video images are collected through monitoring of the parking lot layout; in S1, it also includes Perform frame cutting and preprocessing on the collected video images. 3. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 2, wherein the preprocessing includes enhancing the contrast, brightness and sharpness of the video image. 4. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 1, characterized in that the S2 includes the following steps: S21. Use the DORN algorithm to estimate the depth of the image sequence and obtain the depth image; the algorithm uses a deep regression neural network to estimate a single image, and uses inter-frame information to obtain the depth information of the continuous image sequence; S22. Input the depth image and original RGB image into the D4LCN model, and use depth-guided convolution to extract image features; the model uses a multi-branch architecture to perform multi-scale and multi-channel convolution on the input depth image and RGB image to extract image features. , thereby realizing the positioning and identification of vehicle location information; S23. Based on the image feature extraction, use the non-maximum suppression target detection module of D4LCN to identify and locate the vehicle; this algorithm obtains the position, size and location of the target by classifying and regressing the targets in the image. Direction information; at the same time, the Deepsort tracking algorithm is used to track the vehicle to ensure that the vehicle can be accurately identified and positioned in consecutive frames. 5. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 1, characterized in that the S3 includes the following steps: S31. Obtain the basic spatial information of the corresponding underground parking lot, including the number and location of wall columns, the number and location of parking spaces, and the location and angle of camera layout. Use the Vue framework and three.is engine to perform scene rendering, provide a user interaction interface, and display parking. Basic information about the field; S32. Use the Flask framework to build a program to extract the vehicle pose data sensed by the camera, retrieve the vehicle pose information with the same ID, sort the trajectory by time, and organize the trajectory data set of the vehicle in the virtual panoramic perspective through perspective transformation P = {P ₁ ,P ₂ ,···,P _j ,···,P _m }, where is the trajectory of vehicle j, x _j , y _j , z _j are the three-dimensional coordinates of vehicle j at time t,/> is the attitude angle of vehicle j at time t; S33, front-end and back-end interact with data through RESTfulAPI to realize panoramic operation of the virtual platform. 6. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 1, characterized in that the S4 includes the following steps: S41. According to the parking space layout and vehicle direction, arrange a virtual scene display device on the wall column. Its height, size, direction and brightness should be in order to facilitate the driver's observation; S42. Based on the virtual space platform, track the driver's perspective and provide a virtual picture of the first driver's perspective without the obstruction of the wall pillar on the display screen to achieve blind spot perspective. 7. The over-the-horizon sensing and early warning method for blind spots in underground parking lots according to claim 1, characterized in that the S6 includes the following steps: S61. Extract the corresponding motion trajectory and speed timing information based on the vehicle's pose data, through v _f = _vi +at, s=(v _i +v _f )t/2 obtains the speed and acceleration of the vehicle, where v _i and v _f represent the initial speed and final speed of the vehicle, a represents the acceleration of the vehicle, t represents the time interval between adjacent frames, and s represents The vehicle traveling distance can be obtained from the position data of adjacent frames of the vehicle; S62. Evaluate the risk of vehicle collision and calculate the time required for a collision between two vehicles traveling at the current speed TTC = L/(v ₁ - v ₂ ), where L is the distance between vehicle 1 and vehicle 2, v ₁ and v ₂ represent the speed of vehicle 1 and vehicle 2 respectively; S63. Set the distance threshold between the two vehicles according to the vehicle operation conditions in the parking lot. When L is lower than a certain threshold, different risk levels are marked according to the TTC value to complete the collision warning prompt.