CN102765365B - Pedestrian detection method and pedestrian anti-collision warning system based on machine vision - Google Patents

Abstract

The invention discloses a pedestrian detection method and a pedestrian anti-collision early warning system based on machine vision. The anti-collision early warning system comprises an image acquisition unit, a pedestrian positioning unit, a pedestrian distance measurement unit and a collision possibility analysis unit, wherein the pedestrian classifier is adopted to detect pedestrians on a road, so that individual characteristics among the pedestrians are blurred, the influence of individual difference on a detection result is reduced, meanwhile, the influence of illumination on the detection result is reduced, the detection efficiency of the pedestrians is improved, the possibility of accidents is judged through the anti-collision early warning system, a warning signal is sent to a driver, and the safety of motor vehicle road driving is improved.

Description

Pedestrian detection method and pedestrian anti-collision warning system based on machine vision

technical field

The invention relates to a pedestrian detection method and a pedestrian anti-collision early warning system, in particular to a machine vision-based pedestrian detection method and a pedestrian anti-collision early warning system that can accurately determine the position of pedestrians and the possibility of dangerous accidents.

Background technique

Pedestrian detection and warning system means that through the detection function of certain sensors (such as radar and camera), the road traffic information in front of the driving car is obtained, including relatively moving and relatively stationary pedestrians or objects, and then through the background system, the sensor The acquired signals are processed and analyzed, and various physical parameter measurement and computer vision recognition technologies are used to realize the detection and tracking of pedestrians, and by calculating the relative displacement and distance, the possibility of collision between vehicles and pedestrians is predicted. If it is judged that there is a possibility of collision according to the current parameters, the alarm system will output an alarm signal to remind the driver to avoid the risk of collision and achieve the purpose of protecting pedestrian safety.

Existing pedestrian detection systems generally include two modules: ROI segmentation and object recognition. The purpose of ROI segmentation is to extract the window area that may contain pedestrians from the image for further verification, so as to avoid exhaustive search and improve the speed of the system. Target recognition is the core of the pedestrian detection system. It verifies the obtained region of interest to determine whether it contains pedestrians. Its performance determines the detection accuracy and robustness that the entire system can achieve. At present, pedestrian detection technology generally has the following methods: 1. methods based on motion; 2. methods based on explicit human models; 3. methods based on template matching; 4. methods based on statistical classification. The basic principles, advantages and disadvantages of the above methods are as follows:

The principle of the motion-based method is to identify pedestrians by analyzing the periodicity of pedestrian gait. Its advantage is that it is less affected by color and light and has good robustness. The disadvantage is that it can only identify moving pedestrians and requires multiple frames. Affect real-time performance;

Based on the method of clear human body model, the principle is to construct a clear human body parameter model to represent pedestrians. Its advantage is that it has a clear model, which is convenient for dealing with posture and occlusion problems. Its disadvantage is that the modeling and solution are more complicated;

The method based on template matching, the principle is to represent pedestrians through templates, the advantage is that the calculation method is simple, the system overhead is small, the disadvantage is that many templates are needed to deal with the pose problem, and the matching is time-consuming;

The principle of the method based on statistical classification is to identify pedestrians through a classifier. Its advantage is that it does not need to manually set a large number of parameters and has good robustness. Its disadvantage is that it requires a large amount of training data and the training period is long.

Contents of the invention

In view of this, the main purpose of the present invention is to provide a pedestrian detection method based on machine vision and a pedestrian anti-collision warning system that can improve the accuracy of pedestrian detection and accurately determine the possibility of dangerous accidents.

In order to achieve the above object, the present invention provides a pedestrian detection method based on machine vision, which includes the following steps:

(a) collect the image in front of the car, collect the real-time image in front of the car through the camera installed on the car, and transmit the image to the image processing module to preprocess the image;

(b) Pre-locate the pedestrian in the pre-processed image, and pre-locate the pedestrian area in the image;

(c) Judging the pre-positioned pedestrian area, removing false detection areas, and accurately locating the pedestrian area;

(d) measuring the distance between pedestrians and cars;

(e) Judging whether the pedestrian is in the dangerous area, and giving an alarm to the pedestrian in the collision dangerous area.

The image preprocessing in the step (a) includes anti-aliasing and histogram equalization on the image according to the requirements of the database format.

In said step (b), according to the image information of pedestrians in a specific scene, the HAAR feature classifier file suitable for pedestrian recognition in this scene is pre-trained, and the pedestrian area is located according to the pre-trained HAAR feature classifier when locating pedestrians.

In the step (c), by performing HOG feature extraction on the pre-located pedestrian area, and judging the extracted HOG feature according to the trained HOG feature classifier, the pedestrian area is accurately located.

In the step (d), the distance between the pedestrian and the car is calculated using the principle of camera calibration.

In the step (e), including the two situations of the car going straight and the car turning, the car's driving path is estimated according to the car's speed and the car's yaw rate, and the collision risk area is determined in combination with the distance between the pedestrian and the car.

Preferably, a step (c1) is also included between the steps (c) and (d), tracking the detected pedestrian, and expanding the detected pedestrian area of the previous frame image as the pedestrian of the current frame image In the area of interest, if no pedestrian is detected, then return to the step (a).

Preferably, a step (d1) is included between the step (d) and the step (e), which is to estimate the walking path of the pedestrian, continuously collect several frames of images of the pedestrian, calculate the position data of the pedestrian, and use a function fitting method to predict Estimate the walking path of the pedestrian, and determine the collision risk area according to the driving path of the car and the distance between the pedestrian and the car combined with the walking path of the pedestrian.

The present invention also provides a pedestrian anti-collision warning system based on machine vision, which includes:

An image acquisition and preprocessing unit, which is used to collect images in front of the car and preprocess the images;

A pedestrian positioning unit connected to the image acquisition unit, the pedestrian positioning unit is used to locate the position of the pedestrian in the image, and the pedestrian positioning unit pre-positions and accurately locates the pedestrian area in the image according to the pre-trained pedestrian feature classifier;

A pedestrian ranging unit connected to the pedestrian positioning unit, the pedestrian ranging unit measures the distance between the pedestrian and the car based on the position of the pedestrian in the image, and the pedestrian ranging unit calculates the position of the pedestrian according to the camera calibration method;

A collision possibility analysis unit connected to the pedestrian distance measuring unit, the collision possibility analysis unit is used to judge the possibility of a collision between a pedestrian and a car, the collision possibility analysis unit adopts area division, according to the vehicle speed and yaw The angle parameter is used to estimate the driving path of the car, and the possibility of collision between the pedestrian and the car is calculated and analyzed.

Preferably, the pedestrian collision avoidance warning system based on machine vision of the present invention also includes a pedestrian walking path unit connected to the pedestrian positioning unit, and the pedestrian walking path unit uses a dynamic object tracking prediction method to predict the pedestrian walking path through function fitting. Path, according to the vehicle speed and yaw angle parameters to estimate the car's driving path and combined with pedestrian path calculation to analyze the possibility of collision.

The present invention detects pedestrians on the road by using a pedestrian classifier, which blurs the individual characteristics of pedestrians, reduces the influence of individual differences on detection results, reduces the influence of light on detection results, and improves the detection efficiency of pedestrians. Furthermore, the possibility of an accident is judged by the anti-collision warning system, and a warning signal is sent to the driver, which improves the safety of motor vehicles on the road.

Description of drawings

Fig. 1 is a flow chart of the steps of the pedestrian detection method based on machine vision in the present invention;

Fig. 2 is the calculation schematic diagram of the HAAR matrix characteristic gray value in the present invention;

Fig. 3 is a schematic diagram of an embodiment of the pedestrian detection method based on machine vision of the present invention;

FIG. 4 is a second schematic diagram of an embodiment of the machine vision-based pedestrian detection method of the present invention;

Fig. 5 is the structural schematic diagram of Embodiment 1 of the pedestrian collision avoidance warning system based on machine vision of the present invention;

FIG. 6 is a structural principle diagram of Embodiment 2 of the machine vision-based pedestrian collision avoidance warning system of the present invention.

Detailed ways

In order to facilitate a further understanding of the method and system of the present invention, preferred embodiments are described in detail below in conjunction with the accompanying drawings.

The present invention obtains the real-time front image by installing the camera on the top of the car, extracts the region of interest in the image according to the requirements of pedestrian collision prevention, and then performs a series of image processing and calculation on the extracted region to realize the detection of pedestrians and combine the risk of pedestrian collision The zone judges whether to issue an alarm.

As shown in Figure 1, implementation of the present invention comprises the following steps:

Step 1, collect the image in front of the car. The real-time images in front of the car are collected by cameras installed on the car (such as infrared CCD cameras or CMOS cameras), and the images are properly processed, such as converting the acquired images into single-channel grayscale images according to the needs of the data format. According to the requirements of the database format, anti-aliasing operation and histogram equalization operation are performed on the image.

Step 2, pre-locating pedestrians in the image. According to the pedestrian image information in a specific scene, pre-train the HAAR feature classifier file suitable for pedestrian recognition in this scene, and locate the pedestrian area according to the pre-trained HAAR feature classifier when locating pedestrians.

In the second step, according to the current status of pedestrian detection and project conditions, the HAAR feature method is used, and the HAAR feature is calculated using an integral map. The integral map S of the grayscale image I is defined as:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \&Integral;</span>}_{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; u</span>}}{<span\; class="notranslate">\; \&Integral;</span>}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; v</span>}}\mathrm{<span\; class="notranslate">\; I</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; dxdy</span>}$

Among them, (u, v) represents the coordinates of the integral map, and I(x, y) represents the gray value of the point (x, y) of the original image.

The calculation of the gray value of the HAAR rectangular feature is shown in Figure 2. The value of point A1 represents the sum of the gray values of area A, which is abbreviated as A; the value of point A2 is A+B; the value of point A3 is A+C; The value of the point is A+B+C+D. Therefore, the sum of the gray levels of the rectangular area D surrounded by A1, A2, A3, and A4 can be expressed as: A1+A4-A2-A3. With the help of the integral image, the gray-level features of the rectangular template are calculated independently of the size of the rectangle.

Step 3, judging the pre-positioned pedestrian area. There are many false detections in the pedestrian areas extracted in step 2. By extracting HOG features from these areas, the extracted HOG features are judged according to the trained HOG feature classifier, and the false detection areas are removed, so as to accurately locate pedestrians. area.

Step 4, track the detected pedestrians. After accurately locating the pedestrian area in step 4, tracking pedestrians can greatly improve the real-time performance of the system, for example, appropriately expanding the detected pedestrian area of the previous frame image as the pedestrian area of interest in this frame image. If no pedestrian is detected, return to step 1.

Step five, measure the distance between the pedestrian and the car. According to the position of the pedestrian in the image, the invention uses the principle of camera calibration to calculate the distance between the pedestrian and the car. The formula is:

$\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; f</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&Delta;\; u</span>}<span\; class="notranslate">\; |</span>}<span\; class="notranslate">\; *</span>\mathrm{<span\; class="notranslate">\; h</span>}$

d is the distance between the car and the person, h is the distance between the camera and the ground, Δu is the pixel difference between the pedestrian’s position and the vanishing point on the ground plane, and f is the pixel focal length of the camera (that is, the product of the physical focal length of the lens and each unit size of the imager).

Among them, the principle of the camera calibration method is to use the method of projecting the world coordinates to the image coordinates. If the world midpoint Q (X, Y, Z) is projected to the image coordinates (x, y), then:

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; Z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}$

$\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; Y</span>}}{\mathrm{<span\; class="notranslate">\; Z</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; c</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}$

Where (c _x , _cy ) is the ground vanishing point; f _x , f _y is the pixel focal length.

Therefore, when the ground vanishing point of the camera and the pixel focal length are calculated by the calibration method, the horizontal distance between a point in the projected image world and the camera can be calculated.

Step 6: Estimate the walking path of pedestrians. The invention continuously collects several frames of images of pedestrians, calculates the position data of pedestrians, uses a function fitting method to estimate the future movement direction of pedestrians, and obtains the walking path of pedestrians.

Step 7, judging whether the pedestrian is in the danger zone, and giving an alarm to the pedestrian in the collision danger zone. Since it takes a certain reaction time for people to step on the brakes, set this period of time as t1; after stepping on the brakes, the car has a certain reaction time to stop, and set this period of time as t2, then the total time from stepping on the brakes to the stop of the car is t, then:

t=t1+t2

There are two situations for entering the danger zone. From step five, the distance between the pedestrian and the car can be obtained. Set the speed of the car as v, and introduce the parameter k (0<k≤1). The parameter k is to leave a little space to prevent the car from hitting people after braking.

The first case is that the car goes straight, as shown in Figure 3, when the car brakes until the speed is zero, the displacement traveled is l, then:

l=vt

When l≥kd, the pedestrian is in the dangerous area, and the car alarms at this time; when l<kd, the pedestrian is in the safe area, and the car only shows that the pedestrian is detected.

The second situation is that the car is turning, as shown in Figure 4. At this time, the track that the car has traveled is the curve MN, and the central angle between the curves MN is Φ, so the car displacement l,

$\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; v</span>}}{\mathrm{<span\; class="notranslate">\; w</span>}}\mathrm{<span\; class="notranslate">\; sin</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; wt</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>$

When l≥kd, the pedestrian is in the dangerous area, and the car alarms at this time; when l<kd, the pedestrian is in the safe area, and the car only shows that the pedestrian is detected.

The above-mentioned vehicle speed v and swing angular velocity w when turning can be measured in real time by the ECU (Electronic Control Unit) of the car itself and the built-in swing angle sensor of the car and sent to the data processing end for fusion and judgment.

As shown in Figure 5, the pedestrian collision avoidance warning system based on machine vision of the present invention includes: an image acquisition and preprocessing unit, which is used to collect the image in front of the car, and preprocesses the image; it is connected with the image acquisition unit The pedestrian positioning unit, the pedestrian positioning unit is used to locate the position of pedestrians in the image based on the pedestrian feature, and the pedestrian positioning unit is based on the pre-trained pedestrian feature classifier (such as HAAR feature classifier and HOG feature classifier) respectively in the image The pedestrian area is pre-located and accurately positioned; the pedestrian ranging unit connected to the pedestrian positioning unit, the pedestrian ranging unit locates the position of the pedestrian relative to the car based on the position of the pedestrian in the image, that is, measures the distance between the pedestrian and the car, the said The pedestrian ranging unit calculates the pedestrian position according to the method of camera calibration; the collision possibility analysis unit connected with the pedestrian ranging unit is used to judge the possibility of collision between the pedestrian and the car, and the collision possibility analysis unit Using regional division, the car's driving path is estimated according to the car's driving speed and yaw angle parameters, and the possibility of collision between pedestrians and cars is calculated and analyzed. As shown in Figure 6, the pedestrian collision avoidance warning system based on machine vision of the present invention can also include a pedestrian walking path unit connected to the pedestrian positioning unit, and the pedestrian walking path unit uses a dynamic object tracking prediction method to predict pedestrians through function fitting. Walking path, according to the vehicle speed and yaw angle and other parameters to estimate the car's driving path and combined with pedestrian path calculation to analyze the possibility of collision.

Wherein, the pedestrian positioning unit includes: a pre-trained HAAR feature classifier and a HOG feature classifier for pedestrian detection, which divides the area where pedestrians appear for the first time in the image, so that only this area is detected, and after detection, use The tracking algorithm tracks the target, reduces the amount of calculation when detecting pedestrians, and improves the operating speed of the system.

Using the pedestrian walking path unit and the collision possibility analysis unit, the pedestrian's movement path is estimated according to the position of the pedestrian in the previous few frames of images, and compared with the vehicle's travel path, the possibility of collision between the pedestrian and the car is analyzed.

The present invention detects pedestrians on the road by using a pedestrian classifier, which blurs the individual characteristics of pedestrians, reduces the influence of individual differences on detection results, reduces the influence of light on detection results, and improves the detection efficiency of pedestrians. Furthermore, the possibility of an accident is judged by the anti-collision warning system, and a warning signal is sent to the driver, which improves the safety of motor vehicles on the road.

The above content is a further detailed description of the present invention in conjunction with specific embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (7)

1. a pedestrian detection method based on machine vision, is characterized in that, it comprises the steps: (a) Collect the front image of the car, collect the real-time image of the front of the car through the camera installed on the car, and transmit the image to the image processing module to preprocess the image; (b) Pre-locate the pedestrian in the pre-processed image, and pre-locate the pedestrian area in the image; (c) Judging the pre-positioned pedestrian area, removing false detection areas, and accurately locating the pedestrian area; (d) measuring the distance between pedestrians and cars; (d1) Estimate the walking path of pedestrians, continuously collect several frames of images of pedestrians, calculate the position data of pedestrians, and use the function fitting method to estimate the walking path of pedestrians; (e) Determine whether pedestrians are in a dangerous area, and give an alarm to pedestrians in the collision risk area. The direction of the car includes two situations: the car is going straight and the car is turning. The car’s driving path is estimated by the car speed and the car’s yaw rate. According to the car’s The driving path and the distance between the pedestrian and the car are combined with the pedestrian walking path to determine the collision risk area. 2. The pedestrian detection method based on machine vision according to claim 1, wherein the image preprocessing in the step (a) includes de-aliasing and histogram equalization on the image according to the requirements of the database format operation. 3. The pedestrian detection method based on machine vision according to claim 1, wherein in the step (b), according to the image information of pedestrians in a specific scene, a HAAR feature classifier suitable for pedestrian recognition in this scene is pre-trained File, Locating Pedestrian Areas Based on a Pretrained HAAR Feature Classifier When Locating Pedestrians. 4. The pedestrian detection method based on machine vision according to claim 3, characterized in that, in the step (c), by performing HOG feature extraction on the pre-located pedestrian area, according to the trained HOG feature classification The detector judges the extracted HOG features and accurately locates the pedestrian area. 5. The pedestrian detection method based on machine vision according to claim 1, characterized in that, in the step (d), the distance between the pedestrian and the car is calculated by using the principle of camera calibration. 6. The pedestrian detection method based on machine vision according to claim 1, characterized in that, between the steps (c) and (d), there is also a step (c1) of tracking the detected pedestrians, and The detected pedestrian area of the previous frame image is expanded as the pedestrian interest area of the current frame image, and if no pedestrian is detected, return to the step (a). 7. A pedestrian collision avoidance warning system based on machine vision, characterized in that it includes: An image acquisition and preprocessing unit, which is used to collect images in front of the car and preprocess the images; A pedestrian positioning unit connected to the image acquisition unit, the pedestrian positioning unit is used to locate the position of the pedestrian in the image, and the pedestrian positioning unit pre-positions and accurately locates the pedestrian area in the image according to the pre-trained pedestrian feature classifier; A pedestrian ranging unit connected to the pedestrian positioning unit, the pedestrian ranging unit measures the distance between the pedestrian and the car based on the position of the pedestrian in the image, and the pedestrian ranging unit calculates the position of the pedestrian according to the camera calibration method; A collision possibility analysis unit connected to the pedestrian distance measuring unit, the collision possibility analysis unit is used to judge the possibility of a collision between a pedestrian and a car, the collision possibility analysis unit adopts area division, according to the vehicle speed and yaw The angle parameter estimates the driving path of the car, and calculates and analyzes the possibility of collision between pedestrians and cars; the pedestrian anti-collision warning system also includes a pedestrian walking path unit connected to the pedestrian positioning unit, and the pedestrian walking path unit adopts dynamic object tracking and prediction The method predicts the walking path of pedestrians through function fitting, estimates the driving path of the car according to the vehicle speed and yaw angle parameters, and combines the calculation of the pedestrian path to analyze the possibility of the collision of the car in the two situations of going straight and turning.