CN108447305B - Pre-warning method and pre-warning system for courier pedestrian - Google Patents

Abstract

The invention discloses a polite pedestrian early warning method and early warning system. The method includes: collecting vehicle travel information, pedestrian information and traffic sign line information; using a pedestrian detection algorithm based on deep learning to detect the pedestrian information to complete pedestrian position detection and detection. Category distinction; use the traffic sign line detection algorithm to detect the traffic sign line information, and extract the traffic sign line; according to the above results, determine whether the pedestrian has the intention to cross the road, if so, analyze the relative position of the pedestrian and the vehicle, when When the vehicle meets the regulations of parking courtesy pedestrians, an early warning signal is issued to remind the driver of the vehicle to stop and wait for pedestrians to pass. The system includes: sensors, an embedded platform, and an early warning device for early warning when it is necessary to yield to pedestrians. The early warning method and early warning system for polite pedestrians of the present invention can assist drivers to clarify the rules for polite pedestrians, and ensure the safety of pedestrians.

Description

A comity pedestrian early warning method and early warning system

technical field

The invention relates to an early warning technology in the field of assisted driving, in particular to an early warning method and an early warning system for yielding pedestrians under new traffic regulations.

Background technique

In 2017, courteous pedestrians officially entered China's new traffic regulations, and many cities have also stepped up inspections of vehicles for courteous pedestrians. This regulation is of great significance to ensure the safety of pedestrians and the smooth flow of vehicles.

The rules for giving way to pedestrians are:

1. Pedestrians are walking on the crosswalk on the other side of the road. If they do not reach the center line, they can drive past slowly, otherwise they will yield to pedestrians.

2. When pedestrians wait outside the separation line between the motor vehicle lane and the non-motor vehicle lane and do not enter the motor vehicle lane, slow down and drive slowly, but if the pedestrian crosses the motor vehicle lane line, give an early warning and comity.

3. Pedestrians can drive at a slow speed when they walk through the corresponding lanes, but when pedestrians are waiting on the center line of the green belt, the warning will give way to pedestrians.

But most drivers do not have very clear rules for yielding to pedestrians. The current assisted driving systems are also unable to solve this problem well, such as PCW, etc., cannot monitor the dynamics of pedestrians on both sides of the road. All assisted driving systems have not yet incorporated the current new traffic rules for yielding pedestrians.

Therefore, it is urgent to study a kind of early warning technology that can be applied to pedestrians under the new traffic regulations.

SUMMARY OF THE INVENTION

Aiming at the problems existing in the above-mentioned prior art, the present invention proposes an early warning method and an early warning system for yielding pedestrians under new traffic regulations, which can assist drivers to clarify the rules for yielding pedestrians and ensure the safety of pedestrians.

In order to solve the above-mentioned technical problems, the present invention is achieved through the following technical solutions:

The present invention provides an early warning method for polite pedestrians, which includes:

S11: Collect vehicle travel information, pedestrian information and traffic sign line information;

S12: Use a pedestrian detection algorithm based on deep learning to detect the pedestrian information, and complete pedestrian position detection and category distinction;

S13: use a traffic sign line detection algorithm to detect the traffic sign line information, and extract the traffic sign line;

S14: According to the results obtained in S12 and S13, determine whether the pedestrian has an intention to cross the road, and if there is an intention to cross the road, enter S15;

S15: Analyze the relative positions of the pedestrian and the vehicle, and when the vehicle meets the regulations of parking courtesy pedestrians, issue an early warning signal to prompt the vehicle driver to stop and wait for the pedestrian to pass.

The S11, S12, and S13 may be performed in any order, and may be performed simultaneously in no particular order.

Preferably, after the S15, it also includes:

S16: During the process of pedestrian crossing the road, the position of the pedestrian is always tracked, and the warning signal is stopped once the pedestrian leaves the safe area in front of the vehicle.

Preferably, between S11 and S12 also includes:

S17: Perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information.

Preferably, the S17 is specifically: using a straight line model to perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information.

Preferably, the early warning signal in S15 includes: a light warning signal and a sound warning signal.

The present invention also provides a polite pedestrian warning system, which is used for early warning when a polite pedestrian is required, comprising: a sensor, an embedded platform and an early warning device, wherein,

The sensor is used to collect vehicle travel information, pedestrian information and traffic sign line information;

The embedded platform is used to detect the pedestrian information by using a pedestrian detection algorithm based on deep learning to complete pedestrian position detection and category distinction; it is also used to detect the traffic sign line information by using a traffic sign line detection algorithm, The traffic sign line is extracted; it is also used to judge whether the pedestrian has the intention to cross the road according to the above results. If there is an intention to cross the road, the relative position of the pedestrian and the vehicle is analyzed. When the vehicle meets the regulations of parking courtesy pedestrians, controlling the pre-warning device to issue an early-warning signal;

The pre-warning device is used to send out an early-warning signal.

Preferably, the embedded platform is also used to keep track of the pedestrian's position during the process of pedestrian crossing the road, and once the pedestrian leaves the safe area in front of the vehicle, the pre-warning device is controlled to stop sending out the pre-warning signal.

Preferably, the sensor is a wide-angle camera, and the wide-angle camera can overcome the narrow field of view of ordinary cameras and monitor a large area in front of the vehicle.

Preferably, the warning device includes: a light warning device and/or a sound warning device.

Preferably, the embedded platform is further configured to perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information. Due to its inherent reasons, the wide-angle camera will form distortion of the image when imaging in a two-dimensional plane. For example, the straight lane line will be bent, and the pedestrian will be distorted. The straight line model can be used to correct the distorted image, so that the image on the image will be distorted. The curve is restored to the straight line in the real scene.

Compared with the prior art, the present invention has the following advantages:

(1) The early warning method and early warning system for comity pedestrians of the present invention, according to road traffic signs and pedestrian detection results, a set of logical judgment systems in line with new traffic regulations are designed, which can warn drivers, clarify the rules of comity pedestrians, and also protect the the safety of pedestrians crossing the road normally;

(2) The comity pedestrian warning method and warning system of the present invention adopts the environment perception based on deep learning, and the detection rate of pedestrians and road traffic signs is greatly improved compared with the traditional method, and the use of the embedded platform also makes the whole The volume of the system is controlled within a reasonable range;

(3) The early warning method and early warning system for courteous pedestrians of the present invention adopts a wide-angle camera, which overcomes the problem of blind spots on both sides of the road existing in the previous intelligent system.

Of course, it is not necessary for any product embodying the present invention to achieve all of the above-described advantages simultaneously.

Description of drawings

Embodiments of the present invention are further described below in conjunction with the accompanying drawings:

FIG. 1 is a flow chart of a method for early warning of courteous pedestrians according to an embodiment of the present invention

FIG. 2 is a schematic structural diagram of an early warning system for yielding pedestrians according to an embodiment of the present invention.

Label description: 1-sensor, 2-embedded platform, 3-early warning device.

Detailed ways

The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

With reference to FIG. 1 , some embodiments of the method for early warning of courteous pedestrians of the present invention are described in detail, as shown in FIG. 1 , which includes:

S11: Collect vehicle travel information, pedestrian information and traffic sign line information;

S12: Use a pedestrian detection algorithm based on deep learning to detect pedestrian information, and directly perform pedestrian position detection and category distinction on the original image;

In this step, a deep artificial neural network is used to unify the feature extraction network, the spatial transformation network and the detection network to perform the extraction, transformation and classification of pedestrian features end-to-end. Supervised training with massively annotated datasets accurately distinguishes pedestrians from backgrounds. By directly integrating the spatial transformer network into the existing network, the pedestrian distortion problem on large-view images is well resolved, and the detection accuracy is improved.

S13: Use the traffic sign line detection algorithm to detect the traffic sign line information, and extract the traffic sign line;

In this step, the traffic sign line detection algorithm includes, according to specific traffic rules, the detection of road traffic sign lines such as zebra crossings, lane lines, and dividing lines between motor vehicles and non-motor vehicles.

In some embodiments of the present invention, the Hough transform and the Canny operator are used to define the ROI area to detect various marking lines on the road. Hough transform transforms the image coordinate system into a new Hough coordinate system, so that all the lines in the original image converge in the new coordinate system, which can easily detect the lines in the original image. The Canny edge detection algorithm can be divided into the following 5 steps:

(1) Apply Gaussian filtering to smooth the image;

(2) Find the intensity gradient of the image;

(3) Apply non-maximum suppression techniques to eliminate edge false detections;

(4) Apply a double-threshold method to determine possible boundaries;

(5) Use the lag technology to track the boundary, and finally get all the edge information of the original image.

At the same time, perform HSV threshold segmentation on the image to distinguish the color information of different marking lines. After converting the original RGB image to HSV space, the contrast between white, yellow and the road is very obvious, and the threshold can be easily set to distinguish various marking lines on the road. Come out, so far the zebra crossing, lane line and motor vehicle non-motor vehicle dividing line on the road can all be extracted.

The numbers of S12 and S13 mentioned above in the present invention are only for better description of the entire technical solution. The above two operations do not have a strict sequence, and can be performed in any order or simultaneously.

S14: According to the results obtained in S12 and S13, determine whether the pedestrian has the intention to cross the road, specifically: by detecting and tracking the pedestrian, identifying the direction and position of the pedestrian, if the pedestrian has crossed the boundary between motor vehicles and non-motor vehicles and meet each other Towards the road, it is considered that there is an intention to cross the road. If there is an intention to cross the road, enter S15. If there is no intention to cross the road, it will end, and the driver can pass directly;

S15: Analyze the relative positions of the pedestrian and the vehicle, and when the vehicle meets the regulations of parking courtesy pedestrians, issue an early warning signal to prompt the vehicle driver to stop and wait for the pedestrian to pass.

In a preferred embodiment, after S15, it also includes: S16: in the process of pedestrian crossing the road, always track the position of the pedestrian, once the pedestrian leaves the safety area in front of the vehicle, the warning signal stops, which is more convenient for the driver, It eliminates the need for the driver to observe whether pedestrians pass by himself.

Preferably, the steps between S11 and S12 further include: S17: Perform distortion correction on vehicle travel information, pedestrian information and traffic sign line information, and the correction is more in line with the actual situation and the judgment is more accurate.

With reference to FIG. 2 , some embodiments of the polite pedestrian warning system of the present invention are described in detail. As shown in FIG. 2 , it includes: a sensor 1 , an embedded platform 2 and an early warning device 3 , wherein,

Sensor 1 is used to collect vehicle travel information, pedestrian information and traffic sign line information; embedded platform 2 is used to detect pedestrian information by using a pedestrian detection algorithm based on deep learning to complete pedestrian location detection and classification; The sign line detection algorithm detects the traffic sign line information and extracts the traffic sign line; it is also used to judge whether the pedestrian has the intention to cross the road according to the above results. When the vehicle complies with the regulations of parking courtesy pedestrians, the early warning device is controlled to issue an early warning signal; the early warning device is used to issue an early warning signal.

In a preferred embodiment, the embedded platform is also used to keep track of the pedestrian's position during the process of pedestrian crossing the road, and once the pedestrian leaves the safe area in front of the vehicle, the early warning device is controlled to stop sending out the early warning signal.

In a preferred embodiment, in the embedded platform, it is determined whether the pedestrian has the intention of crossing the road, specifically: by detecting and tracking the pedestrian, identifying the direction and position of the pedestrian, if the pedestrian crosses the boundary between the motor vehicle and the non-motor vehicle, and Facing the road, it is considered that there is an intention to cross the road.

In a preferred embodiment, the sensor adopts a wide-angle camera, and the wide-angle camera is installed on the front windshield of the car, and is attached to the mirror surface to avoid reflection. Wide-angle cameras can overcome the narrow field of view of ordinary cameras and monitor a large area in front of the car. However, due to its inherent reasons, the wide-angle camera will form distortion of the picture when imaging in a two-dimensional plane, such as the straight lane line will be bent, and the pedestrian will be distorted. In a preferred embodiment, a straight line model can be used to correct the distorted image, so that the curve on the image is restored to the straight line in the real scene.

In a preferred embodiment, the warning device includes: a light warning device and a sound warning device, and the sound and light warning is performed at the same time, and the effect is better.

It should be noted that the steps in the polite pedestrian warning method provided by the present invention can be implemented by using the corresponding modules, devices, units, etc. in the polite pedestrian warning system, and those skilled in the art can refer to the technology of the system. The solution describes the step flow for implementing the method, that is, the embodiments in the system can be understood as preferred examples for implementing the method, which will not be repeated here.

Those skilled in the art know that, in addition to implementing the system provided by the present invention and its respective devices in the form of pure computer-readable program codes, the system provided by the present invention and its respective devices can be made by logic gates, Switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers are used to achieve the same function. Therefore, the system and its various devices provided by the present invention can be regarded as a kind of hardware components, and the devices for realizing various functions included in the system can also be regarded as structures in the hardware components; The means for implementing various functions can be regarded as either a software module implementing a method or a structure within a hardware component.

Only preferred embodiments of the present invention are disclosed herein, and the present specification selects and specifically describes these embodiments to better explain the principles and practical applications of the present invention, rather than limiting the present invention. Any modifications and changes made by those skilled in the art within the scope of the description should fall within the protection scope of the present invention.

Claims (9)

1. a courtesy pedestrian early warning system, is characterized in that, comprises: sensor, embedded platform and early warning device, wherein, The sensor is installed on the front windshield of the vehicle, and is used to collect vehicle travel information, pedestrian information and traffic sign line information; The embedded platform is used to detect the pedestrian information by using a pedestrian detection algorithm based on deep learning to complete pedestrian position detection and category distinction; it is also used to detect the traffic sign line information by using a traffic sign line detection algorithm, Extract the traffic sign line; it is also used to judge whether the pedestrian has the intention to cross the road according to the above pedestrian detection and traffic sign line detection results. If there is an intention to cross the road, the relative position of the pedestrian and the vehicle is analyzed. When parking to give way to pedestrians, control the warning device to issue a warning signal; The pre-warning device is used to issue an early-warning signal; The embedded platform is also used to keep track of the pedestrian's position during the process of pedestrian crossing the road, and once the pedestrian leaves the safe area before the vehicle, the pre-warning device is controlled to stop sending out the pre-warning signal. 2 . The polite pedestrian warning system according to claim 1 , wherein the sensor is a wide-angle camera. 3 . 3. courtesy pedestrian warning system according to claim 1, is characterized in that, described embedded platform, wherein judging whether pedestrian has the intention of crossing the road refers to: by detecting and tracking pedestrian, identify the direction, position of pedestrian, A pedestrian is considered to have the intention to cross the road if he crosses the dividing line between motor vehicles and non-motor vehicles and faces the road. 4 . The polite pedestrian warning system according to claim 1 , wherein the embedded platform is further configured to perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information. 5 . 5. A polite pedestrian warning method implemented by the polite pedestrian warning system according to any one of claims 1 to 4, characterized in that, comprising: S11: Collect vehicle travel information, pedestrian information and traffic sign line information; S12: Use a pedestrian detection algorithm based on deep learning to detect the pedestrian information, and complete pedestrian position detection and category distinction; S13: use a traffic sign line detection algorithm to detect the traffic sign line information, and extract the traffic sign line; The S12 and S13 are performed in any order or simultaneously; S14: According to the results obtained in the steps S12 and S13, determine whether the pedestrian has an intention to cross the road, and if there is an intention to cross the road, go to step S15; S15: Analyze the relative positions of the pedestrian and the vehicle, and when the vehicle meets the regulations of parking courtesy pedestrians, issue an early warning signal to prompt the vehicle driver to stop and wait for the pedestrian to pass. 6. The early warning method for yielding pedestrians according to claim 5, characterized in that, after the S15, the method further comprises: S16: During the process of pedestrian crossing the road, the position of the pedestrian is always tracked, and the warning signal is stopped once the pedestrian leaves the safe area in front of the vehicle. 7. The early warning method for yielding pedestrians according to claim 5, characterized in that, between the S11 and S12, the method further comprises: S17: Perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information. 8 . The early warning method for yielding pedestrians according to claim 7 , wherein the step S17 is specifically: using a straight line model to perform distortion correction on the vehicle travel information, pedestrian information and traffic sign line information. 9 . 9. courtesy pedestrian warning method according to claim 5, is characterized in that, described S14 judges whether pedestrian has the intention of crossing the road, is specifically: by detecting and tracking pedestrian, identify the direction, position of pedestrian, if pedestrian crosses over If the dividing line between motor vehicles and non-motor vehicles is reached and faces the road, it is considered that there is an intention to cross the road.

Images