CN100447820C - Bus Passenger Flow Statistics Method and System Based on Stereo Vision - Google Patents

Abstract

The invention discloses a bus passenger flow statistics method and system based on stereo vision. The present invention adopts stereo vision device and algorithm. The stereo vision image acquisition device is placed on the top of the bus door to collect the images of passengers getting on and off the bus in real time. The processor uses the stereo vision algorithm to process the collected binocular images to obtain the image depth information, combined with the processing information of the monocular image to obtain the head of the passengers. The position, size, grayscale and other information of the top, and then the tracking part will track the information on the top of the passenger's head in real time, and the number of passengers getting on and off the bus can be known from the tracking results, so as to obtain real-time and accurate passenger flow information. The invention improves the accuracy rate of the bus passenger flow statistics system, and the obtained passenger flow information can be used as a basis for deploying urban bus frequency density and bus line planning and design, and plays an important role in promoting the healthy development of the automobile passenger transport business.

Description

Bus Passenger Flow Statistics Method and System Based on Stereo Vision

Technical field:

The invention relates to bus passenger flow statistics, in particular to a method and system for bus passenger flow statistics based on stereo vision.

Background technique:

In recent years, public places such as exhibition halls, stadiums, libraries, airports, subways and buses have installed intelligent monitoring and dispatching systems. For these systems, the importance of real-time and accurate passenger flow information is self-evident. At present, the main means of passenger flow information statistics are infrared occlusion system and pressure sensor system. The advantages of these two traditional methods are simple implementation and low cost, but their main disadvantages are inaccurate statistics, and they are powerless to solve the congestion situation during peak passenger flow.

Compared with other information, the capacity of image information is larger and richer. Therefore, the rise and continuous development of image processing technology has proposed many new methods for solving a series of problems faced by traditional passenger flow counting technology. At present, there are many methods based on image processing applied to the passenger flow statistics system, but the methods they adopt are all based on monocular two-dimensional images. When applied to the bus passenger flow statistics system, there are still many insurmountable problems. These problems There are:

1) The background changes complexly with the light, which is not conducive to the extraction of the foreground (human head);

2) In the case of peak passenger flow congestion, people are closely connected, and the method based on two-dimensional information is difficult to accurately distinguish several people who are crowded together, and the crowding situation is very frequent;

3) Due to the existence of the steps of the bus, the head of the same person appears to change in size from the monocular camera at a fixed position when getting on and off the bus, which is very unfavorable for recognition and tracking.

Invention content:

The purpose of the present invention is to provide a method and system for bus passenger flow statistics based on stereo vision.

Bus passenger flow statistics method based on stereo vision: the processor performs stereo vision processing on the binocular images acquired by the stereo vision device, obtains the distance between each point in the scene and the camera, and then sets a threshold on the distance to obtain a certain distance from the camera All points in the scene within the distance range, through denoising and fitting of these points, combined with the feature recognition method of the monocular image, the set of points in the scene that approximately form a circle is used as the head of the person, so as to realize After the detection of the human head, the position, radius, and gray level information of the human head detection result are handed over to the tracking algorithm for effective tracking, so that the movement direction of the passenger flow can be correctly judged, and the passenger flow information statistics can be completed.

Stereo vision processing: refers to the use of two slightly shifted cameras to obtain the depth information of the scene through triangulation operations. The steps of the stereo vision processing method are as follows: 1) Establish correspondence between image feature points between different images 2) Calculate the relative offset between the image feature point positions, 3) Calculate the three-dimensional position information of the feature points in the image through the known camera parameters.

Feature recognition method of monocular image: take the shape of the human head as a feature, take any image in the binocular image as a monocular image, perform image edge detection on it, and implement improved Hough transform on the edge detection result , fuzzy clustering is performed on the results of the Hough transform, so as to obtain the detection results of all circular objects in the scene as human heads.

Bus passenger flow statistics device based on stereo vision: the processor is connected to the control circuit, network transmission module, protocol conversion module, and memory, the control circuit is connected to the binocular image acquisition module and the network transmission module, and the network transmission module is connected to the network interface connected, the memory is connected with the binocular image acquisition module.

Binocular image acquisition module: the timing control circuit is connected to camera 1 and camera 2 respectively.

The present invention can overcome many problems existing in the existing bus passenger flow statistics system. For urban buses, the passenger flow statistics information has very important reference significance for the deployment of bus frequency density and the planning and design of bus lines; for long-distance passenger vehicles, the passenger flow Statistical information has a good supervisory effect on overloading, private toll collection and other undesirable phenomena that threaten the personal safety of passengers and cause national economic losses. In a word, the present invention can not only provide more reliable passenger flow data for public transportation planning and municipal construction, but also play an important role in the healthy development of automobile passenger transportation.

Description of drawings:

Fig. 1 is a block diagram of the bus passenger flow statistics system based on stereo vision;

Fig. 2 is a structural block diagram of the binocular image acquisition module of the present invention;

Fig. 3 is that the present invention realizes the human head detection algorithm software flowchart;

Fig. 4 is a software flow chart of the tracking algorithm of the present invention.

Detailed ways:

As shown in Figure 1, the bus passenger flow statistics system based on stereo vision: the processor is connected with the control circuit, network transmission module, protocol conversion module, and memory respectively, and the control circuit is connected with the binocular image acquisition module and the network transmission module respectively. The network transmission module is connected with the network interface, and the memory is connected with the binocular image acquisition module. In order to ensure real-time processing, the image resolution should not be too large, do not use color images, and directly obtain grayscale images from the camera. Use processor on-chip SRAM or external expansion SRAM. Compared with the general image processing algorithm, the stereo vision algorithm has a large amount of computation, many floating-point operations, and a large demand for memory. In order to ensure real-time requirements, the choice of processor is very important. TMS320C6000 series is a high-end DSP product produced by Texas Instruments. This kind of DSP can improve the cost performance of the system, reduce the development time, increase the reliability, and has been widely used. Its main features are: ① Fixed-point/floating-point series compatible with DSP, main frequency 100MHz-600MHz; ② VelciTI ^TM advanced VLIW structure core; ③ RISC-like instruction set; ④ On-chip integrated large-capacity SRAM, up to 8M; ⑤ High-efficiency coprocessor (C64x); ⑥ provides a variety of integrated peripherals on-chip. Including multi-channel buffer serial port McBSP, multi-channel audio serial port McASP and multi-channel DMA/EDMA controller, etc.; its characteristics determine that the C6000 series is especially suitable for developing image products, so as the central processor of the stereo vision passenger flow statistics system, the C6000 series is a suitable choice. In the development of embedded systems, efficient debugging methods are extremely important. Not only are debugging interfaces needed in the product development stage, but also in the initial stage of product launch, in order to repair various problems that have not been encountered in the development stage, debugging interfaces must also be retained. Considering the amount of image data, the debugging interface must ensure a high transmission rate. Considering the transmission rate and development cost comprehensively, the network interface is an ideal choice. Using a 100M Ethernet card chip (such as RTL8139) as a network interface, various necessary debugging data and even Real-time binocular image data provides guarantee for efficient debugging.

As shown in Figure 2, the binocular image acquisition module: the timing control circuit is connected to the camera 1 and the camera 2 respectively. The stereo vision system adopts binocular cameras, and the binocular cameras are placed horizontally, that is, the line connecting the centers of the lenses of the two cameras is on the same horizontal line. Considering the limited installation height of the cameras, in order to take into account that the two cameras have a larger common field of view, a wide-angle lens is required. The focal length of the lens is not greater than 600 pixels wide. Considering the real-time requirements, two cameras collect grayscale images, combined with the stereo vision accuracy requirements, the resolution of the collected images is 640*480; the baseline length between the two cameras is 15 cm.

As shown in Figure 3, the algorithm flow chart of the detection module, firstly, stereo vision processing is performed on the binocular image, and the image I(x, y, z) containing depth information is obtained by using the window-based matching algorithm, and the corresponding value is set in the Z direction. After the threshold value, the regions that may be human heads are extracted. Then randomly select one of the binocular images as a monocular image for plane circle-like object extraction. The extraction process first requires preprocessing such as edge detection and background removal on the monocular image, and then performs an improved Hough transform on the preprocessed foreground edge image, and introduces a fuzzy measure to the result of the Hough transform. Under the guidance, the false circle-like objects in the Hough transform result are removed, and the positions and radii of all circle-like objects in the monocular image are obtained. Finally, the results of the stereo vision algorithm are combined with the results of the Hough transform of the monocular image to extract the positions of the heads of all people in the foreground, and at the same time calculate the eigenvectors of each head required for the tracking part ( x, y, z, R, G). Since the head of the human body is obviously higher than other parts such as the shoulder of the human body, that is to say, the distance between the head of the human body and the camera is closer than other parts. The distance between each point in the scene and the camera can be obtained by using the method of stereo vision, so we can obtain all points in the scene within a certain range from the camera. Through the denoising and fitting process of these points, the set of points in the scene that approximately form a circle is used as the head of the person, thereby realizing the detection of the head, and then the detection results of the head (position, radius) , grayscale and other information) is effectively tracked by the tracking algorithm, and the direction of the person's movement can be correctly judged, thereby completing the counting. Stereo vision technology can use two slightly offset cameras to obtain the depth information of the scene through triangulation. The camera module acquires two images of the scene at the same time. The points in the scene have image points in the two images respectively. In different images, the position of the image point is different. This difference in position is called parallax. The distance between the point in the scene and the camera is different, resulting in different parallax. The distance is close, the parallax is large, and the distance is small. Stereo vision is based on different degrees of offset and uses triangulation to determine the distance from objects in the scene to the camera. Stereo vision can be roughly divided into the following three steps: 1. Establish the correspondence between image feature points between different images; 2. Calculate the relative offset between the image feature point positions; 3. Through the known camera The three-dimensional position information of the feature points in the parameter calculation graph. The most critical of these is the first step. In order to establish corresponding point pairs in the two images, we use a window-based matching method, which is suitable for hardware acceleration and has relatively stable output results. The similarity measurement function uses SAD, and the expression is as follows:

$\underset{\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; min</span>}}}{\overset{{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; max</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\overset{\frac{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\overset{\frac{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; right</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; left</span>}}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; |</span>$

where d _min and d _max are the minimum and maximum disparity values, respectively. m is the size of the template window. I _left and I _right are the left picture and the right picture respectively.

After the feature point pair is established, we can calculate its parallax for any point A in the scene, the formula is as follows: D(A)=x(A _left )-x(A _right ). Where x(A _left ) and x(A _right ) are the x-coordinates of the image points of point A in the left and right images respectively.

Finally, we perform 3D reconstruction of the points in the scene by the following formula:

$\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; Xc</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; d</span>}}\\ \mathrm{<span\; class="notranslate">\; Yc</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; d</span>}}\\ \mathrm{<span\; class="notranslate">\; Zc</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; d</span>}}\end{array}\right.$

Where U ₀ and V ₀ are the image center coordinates, f is the focal length, b is the camera baseline length, d is the parallax of the scene point, Xc, Yc, Zc are the three-dimensional coordinates of the scene point in the camera coordinate system. After obtaining the depth information of the scene, take any one of the binocular images as a monocular image, first perform edge detection on it, and implement an improved Hough transform on the edge detection results to detect the existence of circular objects, mainly The principle is: the formula for calculating the center of a circle: $\left\{\begin{array}{c}{x}_c=x_i-r\cos \mathrm{\&theta;}\\ {\mathrm{the\; y}}_c={\mathrm{the\; y}}_i-r\sin \mathrm{\&theta;}\end{array}\right.---\left(1\right),$

Among them, (x _i , y _i ) is the coordinate value of the edge point in the monocular image, (x _c , y _c ) is the coordinate value of the center of the circle corresponding to the radius r, θ=arctg(g _y /g _x ), g _y , g _x are the gradients in two directions, which can be obtained in advance during edge detection. In this way, given the radius range of the human head, according to formula (1) to calculate the corresponding center coordinates of all the edge points in the detection area under each radius, a mapping table of the center positions under all radii can be obtained. In this table, the clustering operation is performed according to certain clustering rules, and a threshold value of the number of points falling on the same circle can be given to obtain the center position and radius of all circle-like objects. Of course, there will be many false circles, including some cases of intersecting circles and compatible circles, and further processing should be done on these cases to eliminate false circles as much as possible, so that the results obtained in this step are as artificial as possible. head. For the results obtained by the Hough transform, there may be cases where the degree of approximation to the circle is uncertain, such as the curve is not closed, the edge information is incomplete, and such uncertain cases need to be dealt with by introducing fuzzy concepts. Two confidence levels are introduced here to measure the degree to which a circle-like object approximates a circle, and their membership functions are:

1) μ ₁ =T/2πr; 2) μ ₂ =A/πr ²

Where T is the number of points obtained by the Hough transform of this type of circle, which can be understood as the arc length, then when the arc length is equal to the circumference of the circle, that is, T=2πr, the degree of membership is 1, and in other cases, 0<μ ₁ <1. A is the area of the inscribed polygon obtained by sequentially connecting the points obtained by the Hough transformation in this type of circle, then in the limit case, A=πr ² , the degree of membership is 1, and in other cases, 0<μ ₂ <1 . It can be seen that μ ₁ and μ ₂ can well describe the degree to which a circle-like object approximates a circle under the condition that the curve is not closed and the edge information is incomplete, and these two confidence levels can be combined separately or in a certain linear combination (the coefficient can be determined by the neural network Network or support vector machine trained) acts on the result of Hough transformation, and the false circles can be eliminated and merged according to the size of the confidence, so that the output result is as real as possible.

As shown in Figure 4, the algorithm flow chart of the tracking module, the feature vectors extracted by the detection part each time each person appears in the process of entering and leaving the field of view constitute a sequence of feature vectors. In addition to recording the feature vector of each occurrence, each sequence also needs to predict the predicted value of the sequence feature vector in the next frame through the Kalman predictor. In this way, all the people appearing in the field of view each have a tracking sequence, and the tracking sequences of all people form a tracking sequence group or a tracking sequence matrix. The task of the tracking module is to extract the tracking sequence with the closest vector distance to the detection vector of the current frame from the tracking sequence matrix of the previous frame. If the distance of the closest vector is lower than the set threshold, the matching is successful, and the detection vector Add to the corresponding tracking sequence, otherwise, the matching fails. At this time, if the fuzzy confidence of the detection vector is higher than the given threshold, it means that a person who has not appeared before enters the field of view, and the new tracking sequence corresponding to this person Generated, and the detection vector is the first vector value in the new tracking sequence. It is worth mentioning that the vector distance is used here instead of the geometric distance in the usual sense. The reason is that when people get on and off the bus, especially when going up and down the steps, the movement is very irregular. If only the geometric distance is used, the probability of tracking errors is very high. However, if the vector distance is used, due to the consideration of the size of the head and the grayscale information, then The chances of tracking errors are greatly reduced. When a person disappears in the field of view (can be defined as two or more frames do not match), the corresponding tracking sequence task ends, and all the vector values recorded in it are taken out as the basis for judging whether he gets on or off the car.

Claims (4)

1. public traffice passenger flow statistical method based on stereoscopic vision, it is characterized in that, processor carries out stereopsis to the binocular image that stereo vision apparatus obtains, obtain the distance between the video camera of each point in the described stereo vision apparatus in the scene, threshold value is set on distance then, obtain apart from the scene in a certain distance range of video camera have a few, by denoising to these points, fit, again in conjunction with the characteristic recognition method of monocular image, the set of the scene mid point that those proximate compositions are round is as people's head, thereby realized head part's detection, the result's that the head part is detected position again, radius, half-tone information is transferred to track algorithm and is implemented effectively to follow the tracks of, just can judge the direction of motion of passenger flow, thereby finish the passenger flow information statistics.

2. a kind of public traffice passenger flow statistical method according to claim 1 based on stereoscopic vision, be primarily characterized in that, described stereopsis: refer to utilize two slightly the video camera of offset pass through triangulo operation, obtain the depth information of scene, the stereopsis method step is as follows: 1) set up the corresponding relation between the image characteristic point between different images, 2) relativity shift between the computed image characteristic point position, the 3) three dimensional local information by unique point in the known camera parameters calculating chart.

3. a kind of public traffice passenger flow statistical method according to claim 1 based on stereoscopic vision, be primarily characterized in that, the characteristic recognition method of described monocular image: with the shape of human body head as feature, get arbitrary order image in the binocular image as monocular image, it is carried out edge of image detects, on edge detection results, implement improved hough transform, in the result of hough transform, carry out fuzzy clustering, thereby all similar round targets are as the testing result of human body head in the acquisition scene.

4. bus passenger flow statistical system based on stereoscopic vision, it is characterized in that, processor respectively with control circuit, network transmission module, protocol conversion module, storer joins, control circuit respectively with the binocular image capture module, network transmission module joins, network transmission module and network interface join, storer and binocular image capture module join, described binocular image capture module: sequential control circuit joins with video camera 1 and video camera 2 respectively, processor is a flush bonding processor, processor carries out stereopsis to the binocular image that stereo vision apparatus obtains, obtain the distance between the video camera of each point in the described stereo vision apparatus in the scene, threshold value is set on distance then, obtain apart from the scene in a certain distance range of video camera have a few, by denoising to these points, fit, again in conjunction with the characteristic recognition method of monocular image, the set of the scene mid point that those proximate compositions are round is as people's head, thereby realized head part's detection, the result's that the head part is detected position again, radius, half-tone information is transferred to track algorithm and is implemented effectively to follow the tracks of, just can judge the direction of motion of passenger flow, thereby finish the passenger flow information statistics.

Images