JPH02502947A - Vehicle detection with image processing for traffic monitoring and control - Google Patents

Abstract

A vehicle detection system for providing data characteristic of traffic conditions includes a camera overlooking a roadway section for providing video signals representative of the field (traffic scene), and a digitizer for digitizing these signals and providing successive arrays of pixels (picture elements) characteristic of the field at successive points in space and time. A video monitor coupled to the camera provides a visual image of the field of view. Through use of a terminal and in conjunction with the monitor, an operator controls a formatter so as to select a subarray of pixels corresponding to specific sections in the field of view. A microprocessor then processes the intensity values representative of the selected portion of the field of view in accordance with spatial and/or temporal processing methods to generate data characteristic of the presence and passage of vehicles. This data can be utilized for real-time traffic surveillance and control, or stored in memory for subsequent processing and evaluation of traffic flow conditions.

Description

[Detailed description of the invention] name of invention Background of the invention: Vehicle detection by image processing for traffic monitoring and control 1. Field of invention The present invention relates generally to traffic detection and monitoring devices. specifically The present invention determines the presence and passage of vehicles, measures various traffic parameters, and Infrared images or in-situ images of highways and streets to facilitate traffic monitoring and control. vehicle detection system in which the visible image is processed by digital computing means; It is mu. The system can also be used as a vehicle counter/classifier, as well as for accident detection. Used for other traffic engineering applications such as transportation, safety analysis, and measurement of traffic parameters. can do.

2. Description of conventional technology Intersections in busy urban areas and where safety rather than capacity or efficiency is the primary concern to regulate traffic flow, both at low-traffic rural or suburban intersections where Therefore, traffic signals are widely used. Traffic control signal timing (i.e. , cycle time and length of green light given to each move/progress) Fixed by data or based on variable and real-time sensed data. Ru. The timing sequence of pre-timed traffic control signals is based on the required pattern. on the other hand, real-time traffic control decisions depend on actual traffic control decisions. Obtained from traffic flow information.

This information can be processed locally or centrally where decisions about signal settings are made. Can be remotely transmitted to a computer. Real-time traffic control signals are in rapid demand. has the ability to respond to fluctuations and is, in principle, preferable to pre-timed signals. , also efficient.

The equipment currently used for real-time traffic control signals is expensive and inaccurate. This is often the case. Effective traffic monitoring and regulation of highways and highways Detection includes vehicle detection, counting and classification, and other traffic parameters. Requires data measurement. The vast majority of such detection devices are of the inductive loop type; Wire loops placed on the pavement to detect the presence of vehicles by magnetic induction It consists of

The information obtained from such detection devices is very limited, so complex traffic control In order to obtain the necessary data for management and monitoring systems, such detection equipment is required. It is often necessary to install a large number of For example, measuring traffic volume per lane requires at least one detection device per lane, while speed measurement This requires at least two detection devices. The problems with the existing system are reliability and maintenance. In major cities, 25% to 30% induction loops are created. I haven't moved. Additionally, induction loops are expensive to install.

Electro-optical vehicle detection systems using visible or infrared sensors can It has been suggested as an alternative to Yaloup detection devices. Such as electronic cameras, The system's sensors focus on a field of view of the traffic scene and set a predetermined frame rate. generate images at degrees (like regular television). under computer control, Frame data with traffic images are captured, digitized, and stored on computer media. Memorized by Mori.

The computer then processes the stored data. Vehicle detection is performed on each selected window. By comparing the image of the doe with the background image of the window when no vehicle is present. can be done. If the intensity of the instantaneous image is greater than that of the background, the vehicle inspection An exit is made. After detection, the speed and characteristics of the vehicle can be extracted. to this traffic data can be obtained and used for traffic control and monitoring. .

To make this type of electro-optical vehicle detection system cost-effective, A large field of view intersection so that you can get all the necessary information from the captured image. One camera must be positioned in such a way that it covers the entire area.

In other words, the intersection approach area or the roadway area from which it is desired to obtain information. One camera must be able to provide images of all strategic locations for minutes. No. To process the frames of these statues, the combination is required: time is very important for real-time applications. Furthermore, the data representing the image is processed. The methods currently used to do so are not very effective.

It is clear that there is a continued need for improved traffic control and monitoring systems. Ru. To be commercially viable, a system must be reliable, cost-effective, and It must be accurate and capable of performing multiple functions. crowded street network and The need for traffic control on roads and expressways is increasing.

This can only be achieved with real-time detection and monitoring equipment. the Such a machine-visual device is proposed here. The ultimate goal is to replace human observers with The goal is to replace visual inspection with only machines for monitoring and organizing traffic. Finally, a suggestion The device does not require wires to be placed on the pavement, increasing reliability. and reduce maintenance.

Summary of the invention A vehicle detection system according to an embodiment of the present invention has a field of view of traffic. raffic) and detect pixel intensity values representing the characteristics of the visual field at successive points in time. Sensor means are provided for providing a continuous array. coupled to the sensor means; and the formatter means responsive to the subarray selection signal selects the desired portion of the traffic field of view. Select a subarray of intensity values representative of the characteristics of . Subarray selection means such as terminals generates a subarray selection signal representative of the desired portion of the traffic field of view. The processing means are Process the intensity values of the selected subarray and calculate the field of view represented by the subarray. Generate characteristic data representing characteristics of the vehicle within the section.

In one embodiment, the processing means spatially processes the array of image intensities and characterizes the presence of the vehicle. Generate data representing . In other embodiments, the processing means processes the array of image intensities over time. and generate data characterizing the presence of vehicles within the field of view. still other In embodiments, the processing means comprises spatially processed data and temporally processed data. data are logically combined to generate data that characterizes the presence of vehicles within the field of view. do.

The vehicle detection system of the present invention is both effective and cost effective. Fo - By using mattes, specific parts of the image obtained by the camera can be selected. and can be processed. Therefore, one camera can be used for multiple detection, That is, it can be used effectively to detect large numbers of points along a road. can. Parts of the image that do not need to be processed are not used, saving computer time. is saved. In addition, temporal and spatial data processing methods allow data to be processed quickly. control and produce accurate results. This allows for accurate real-time traffic control. It can be realized.

Brief description of the drawing FIG. 1 is a block diagram of a vehicle detection traffic control system according to the present invention.

Figure 2 shows a digitized frame of the image captured by the camera shown in Figure 1. FIG.

FIG. 3 is a diagram showing the operation of the formatter shown in FIG. 1.

Figure 4 shows the spatial data that can be implemented by the system shown in Figure 1. FIG. 3 is a block diagram of a data processing method.

Figure 5 shows the spatial averaging step performed in the spatial data processing method shown in Figure 4. FIG.

FIG. 6 shows the temporal data processing that can be performed by the system shown in FIG. It is a block diagram of the theory.

Figure 7 shows the image displayed by the monitoring device of Figure 1 and the terminal and formatter. It is a diagram showing the operation of ivy.

FIG. 8 is a diagram showing the logical processing steps shown in FIG. 4.

Figure 9 shows another process that can be realized by the system shown in Figure 1. FIG. 2 is a block diagram of the method.

FIG. 10 is a diagram showing a speed determination processing method.

Figure 11 shows Equation 1 Equation % executed by the system of Figure 1. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS A vehicle detection traffic control system 10 according to the present invention is schematically illustrated in FIG. As shown in the diagram The vehicle detection system 10 includes a sensor 12 such as a camera, a monitoring device 13, and a digital camera. Iza 14, Formatsuta 16, Computers such as microprocessors 18, related random access memory or RAM 17 and read-only memory or It includes a ROM 19, a terminal 20, a traffic signal control device 22, and a recording device 24. .

The camera 12 is attached to a street light pole, traffic light pole, building or other support structure (see Figure (not shown) and can be placed at a height of 25 to 40 feet, as shown in Figure 1. A desired traffic area on the roadway 26 can be focused on. The camera 12 is a camera Any device 0 is fine. Camera 12 uses normal television frame rate It can be operated in a conventional manner.

As shown in FIG. 2, each consecutive frame 29 (although only one is shown) is An image 30 of the traffic view at a certain moment in time is captured. The camera 12 sends a series of images 3゛0 An analog video signal characterized as a scan line 32 is generated. Each scan line consists of 48 Each frame of four scan lines lasts approximately 65 milliseconds, and represents the intensity of energy reflected from the field area encompassed by the field. The camera 12 , described as operating in the visible part of the spectrum; The analog video signal generated by the controller 12 is converted into a digital signal by the digitizer 14. be converted into The digitizer 14 converts the analog signal of the scanning line into the nth line shown in FIG. Pixels representing the intensity I of the image 3G at each position in the first row and fifth column of the frame! 0. Convert to As shown, the digitizer 14 is Divide the image 30 into (iXj) frames, ie, an array of pixels. Shown in Figure 2. In this example, it is 1-j-22, but typically larger arrays are used. .

Depending on the position and orientation of camera 12 relative to roadway 26 (FIG. 1), image 30 A fairly large traffic field of view may also be used. However, various types of information (e.g. For example, the length of the line in the left-most lane, the presence of vehicles at an intersection, or the Typically, specific parts of the image 30 are You only need to process it.

As shown in FIG. 1, the monitoring device 13 receives and receives video signals from the camera 12. , thereby providing a real-time display of the image 30. No. 7 corresponds to the images of FIGS. 2 and 3 displayed on the monitoring device 13. FIG. 3 is a graphical representation of image 30; Using the terminal 20, the operator performs the following steps for further processing. A window or desired portion of image 30 can be selected.

In one embodiment, an operator uses terminal 20 to select a desired window on monitoring device 13. An indicator, such as a cursor 15 (FIG. 7), is positioned at a location defining the cursor 15 (FIG. 7). end Through the terminal 20, the operator directs the formatter 16 to the portion of the image 30 within the window. To select from the digitizer 14 the pixels In, . Can be done. The selected pixel In,, is then processed by the microprocessor J 18 and stored in the RAM 17.

The above procedure can be explained in further detail with reference to FIGS. 3 and 7. . For example, the data in window 40 at the top of the leftmost lane in FIG. If desired, the operator should display the upper left and lower right corners of this window. The cursor 15 can be positioned at the desired location. In response, the former The ivy 16 represents the portion of the image 30 within the window 40 where 4≦i≦10 and 5≦ pixel for j≦8 This pixel is then stored in RAM1 via microprocessor 18. Rolled to 7. This procedure is repeated for successive frames 29. Similar person pixel 1n1j for 1-19.9≦j≦13 representing window 41, Or, pixel In for 8≦i≦14, j-12 representing window 43, can be selected.

l co Once selected and stored, it represents successive frames of the window portion of image 30. Pixel In, , has various temporal and spatial components. and/or other statistical methods by the microprocessor 18. The presence, passage, and speed of vehicles 28 within the selected window of roadway 26 are degree or other characteristics can be determined. This data is then processed using known methods is used by the traffic signal controller 22 to control the traffic signal according to the current traffic conditions. Traffic flow along road 26 can be optimized. Alternatively, post-processing and/or for evaluation, the data is recorded by a recording device 24. is also possible.

A microphone is used to determine the presence, passage and/or other characteristics of the vehicle 28. The spatial data processing method performed by the processor 18 is illustrated in connection with FIG. explain. The spatial data processing steps shown in Figure 4 are based on the traffic view at a certain moment. Allows the system 10 to determine characteristics of the vehicle 28 from a "-glance" of the field.

This determination is based on the measurements taken from the instantaneous image and the characteristics of the background data in the image. This is done on the basis of a comparison with the corresponding measured value. Therefore, the characteristics of the vehicle The determination is made that the selected window of the image 30 represented by that pixel 1n1j Based on the characteristics of the intensity profile. Assumptions underlying this processing method is the characteristic of the instantaneous intensity profile of the selected portion of the image 30 (s tgnat ure) is significantly changed when the vehicle 28 is within the field of view. Ru.

43 (Figure 3), in the nth frame (last) of the window. Pixel I09. First, step J As shown at 50, the microprocessor 18 processes the previous N frames. Time averaged with corresponding pixels. N is the parameter stored in RAM17 or ROM19. It is a parameter. In one embodiment, microprocessor 18 executes equation 1-3 to According to the defined recursive formula (recursive f’or+mula) Pixel "1j" represents the average background intensity of window 43 over N frames.

The time averaged pixel In1j is then added to the current array image, as shown in addition step 52. It is subtracted from the element In, , and the background is adjusted. Once again, the microprocessor 18 can perform the mathematical implementation. background By using adjustment pixels Tn, . From changes between asphalt and concrete, railway crossings, or signs on road surfaces It will be possible to compensate for any natural changes in the road surface that may occur.

After calculating the background adjustment pixel 〒n　, the microprocessor j 18 is the spatial average array A3 according to equation 5 or equation 6. 1 piece Occur. The size of the averaging window is the size of the vehicle 28. The position and orientation of the camera 12 relative to the roadway 26 It will change depending on the direction (Fig. 1).

Microprocessor 18 uses the (IXL) averaging window according to Equation 5. Therefore, the spatial average pixel for a (IXJ) horizontal window (Fig. 3) such as 41 A 0. Calculate can do. In a similar manner, using the (MXI) averaging window, 43 A vertical window equation 6 such as (Ixl) can be used. Using Equation 7 By doing so, microprocessor 18 uses the (MXL) averaging window. to generate a spatial average pixel A tj for a two-dimensional window such as 40. I can do it.

FIG. 5 shows that (I 30) An example is shown in which the spatially averaged surface n1j is generated for the horizontal window 44. . Therefore, Equation 5 becomes Equation 8 in the case of L-6. During its execution, the micro The processor 18 calculates a series of background adjustment intensity values Tn for the entire window 44. Average the consecutive groups of six. between background Next, a second group of background adjustment pixels T''++ (2≦j≦7) are averaged in a similar manner. This process is performed for the background adjustment pixel T''1j (2s≦j≦ 30) is repeated by microprocessor 18 until averaged. the As the result shows in spatial averaging step 56, microprocessor 18 then , background adjustment pixel in- and spatial average pixel A, and one column 1. Calculate the spatial variance v1j. this is , all values in the selected window, such as 43 of the nth frame.

1 piece and Anlj.

Variance value V 1. is the background adjustment value I1. is one distributed window Space plane within the variance window How much does it vary from the average value An? Give a measure of what to do. The variance window, like the spatial average window, is 28 is sized to represent a vehicle such as . The microprocessor 18 is For example, using the formula in Equation 9, the spatial variance value vehicle for the (IXL) variance window is The variance A” ij in the case where there is no Calculated. If there is a vehicle in the window in question and the nth frame AVni If logic 58 determines that j has not been updated, then . that window If the logic determines that there is no vehicle in , 'AVnij is updated according to Equation 9. be renewed.

Logic 58 operates on either the background adjustment strength "ij or the variance vn1j . If, or If , there is a possibility that a vehicle exists at location (i j), which means P1j″″1 Indicated by Logic 58 uses P9. for a window of length 6. Accumulate the values of Ru. Using the majority rule, if (IXk) (However, k-30) If it holds true somewhere in the window, the vehicle A decision is made that it exists.

Passage is determined by vehicle detection at the first pixel of presence detection.

This will be explained with reference to FIG. 8, which shows a vehicle 28 present. Pixel P, (6≦j≦11 ) is covered by these pixels Since the vehicle 28 was located in the area of the image, microprocessor 1 8 would have been set to "1".

The remaining pixels P, (1≦j≦5) and Plj (12≦j≦5) are images containing the vehicle. It will be set to ``0'' because it does not represent the part of . Detection window 72 is the (IX6) window in this example. By detection window 72 One of the included pixels (i.e. P08, where 5≦j≦10) The sum of the values is compared to a constant X-4, as described by Equation 11.

In this case, the sum is equal to 6, so microprocessor 18 issues a presence signal. will live. For example, the window 72 encompasses pixels P, (13≦j≦18). If included, the sum is 1 piece. 0 and the microprocessor generates a signal indicating the absence of a vehicle. Will.

The microprocessor 18 also includes a Bayesian processor for vehicle presence determination. Any other statistical criteria can be provided. A display indicating the passing of a vehicle. (e.g., a signal that toggles logic state on entry to the associated window). It can be determined in various ways. All of the above steps are performed for each new frame. is repeated continuously.

A microprocessor is used to determine presence, passage, and other vehicle characteristics such as speed. The temporal data processing method carried out by the sensor 18 is schematically illustrated in FIG. time This method estimates the background intensity of the road surface when no vehicles are present. It was a moment ( current frame) strength, and if the latter is statistically greater, the vehicle Existence judgment is made.

Croprocessor 18 first time averages the intensity values, pixel in,.

J generate a time-averaged array of . The time averaged pixel rn, , is according to formula 1-3. It is calculated using a method similar to spatial processing.

Microprocessor 18 then converts step 62 and equation 4 to 6' Therefore, instantaneous pixel 10. l code to subtract the time average l1j from For the nth frame, the background-adjusted pixel background-adjusted intensity Using the pixels of , microprocessor 18 then, as shown in step 64, R preceding frames are generated. The time variance value Qn, , is the background of the preceding R frame. 1 piece The average intensity M at the adjustment pixel Tn1j and the corresponding pixels in N previous frames 0. generated as a function of 1 micropro The processor 18 calculates the time variance value and the average value according to equations 12 and 13. Ru. In one example, R and N are equal to 20 frames.

Microprocessor 18 also performs the spatially distributed processing step 56 shown in FIG. As part of the time dispersion step 64, the vehicle Calculate the background variance Qn when there is no Qn. The background variance A” ij is the moving average 1 j It is calculated as a function of (running average) (Equations 12 and 13).

If logic 68 determines that the vehicle is not present, then the variance is given by Equation 12.13. Therefore, it will be updated. If it is determined that a vehicle is present according to logic 68 is. Comparator 66 operates as follows. background adjustment instant n The intensity I is compared with a function of the background variance according to equation 14 and ij Ru. The function f (AQ''sj> is, for example, the absolute value of the background variance value AQ1j or can be the square root. The constant will typically be between 1 and 4. The instantaneous background adjustment value If it is thicker than the functional relationship of the scene variance, the decision that the vehicle is present at pixel ij is , determined by the comparison mechanism. This is denoted by P,,-1. That's right J If there is no vehicle, it is indicated by P-, -0 (no vehicle).

l co P05 pixels with a value of 0 or 1 are input to logic 68 where J Processed to determine vehicle presence and passage. The logic processing at step 68 Step 58 of the spatial processing method shown in Figure 4 and described by Equation 11. The processing is similar to that described in connection with this. All of the above steps apply to each new Continuously repeat ij for an array of frames or pixels I returned.

The spatial data processing method explained in connection with FIG. 4 and the method explained in connection with FIG. Although temporal data processing methods provide accurate data related to vehicle detection, the system The performance of 10 can be improved by the simultaneous use of these methods.

As shown in Figure 9, the pixel intensity for the selected window of the nth frame is The degree value In, , is the spatial and temporal processing method J (steps 76 and 78, respectively), the microprocessor 18 can be processed simultaneously.

Results from these two processing methods (e.g., representing presence, passage, or other characteristics) data) is then processed logically or otherwise, as shown in step 80. may be combined to create a signal or data indicating the presence, passage, or other characteristic of generate data.

In one embodiment, microprocessor 18 performs spatial and temporal processing steps 76 and Perform rANDJ operation on each output of exists or passes through data only if it is caused by generate data.

Through execution of either the spatial processing method shown in Figure 4 or the temporal processing method shown in Figure 6. presence and/or passage data generated by microprocessor 18. further processing by microprocessor 18 to generate vehicle speed data. can do. This processing method will be explained in connection with FIG.

The velocity data spans several (N) frames and includes gates P1□2 and P1□6, etc. is calculated by monitoring the logic states assigned to the two gates of .

The spatial distance between pixels P1□2 and Pl,6 depends on the geometry and sensor parameters. corresponds to the actual distance in the traffic field based on the data.

The microprocessor 18 changes the logic state of the pixel Pi12 from logic "0" to logic "1". ” and the logic state represented by pixel P1□6 becomes logic “0”. ” to 7 frames switching to logic “1”. Ru.

The number N of frames separating these two events corresponds to the time Δt. micro Processor 18 can then calculate the speed using Equation 15. child The accuracy of the determination of can be improved by calculations involving several pairs.

Although the invention has been described in conjunction with preferred embodiments, those skilled in the art will understand the spirit and scope of the invention. be able to make changes in form and detail without departing from the framework; will recognize it.

7ζ3,11 x3 =, -γ'-Noroheji r, 7'-Rokuno (Senou Saragoshukyutsui; Do Tsuj1: Sentence 9 and 7) Zue.

% = st 4,7 shi’ (t-m>(j-’) @ K 7 A, j=:≦, Xl (j-J) (1 ru 1)-Arσ-ノ)) 2　　　　　　　■9ノ’ij　〉kf　( AQtj Yagi Ll<ki4. 1 tumor"h);I, international search report

Claims (15)

[Claims] 1. To sense the field of view of traffic and provide a continuous pixel array representing the characteristics of the field of view. a sub-array of pixels in the field of view of the traffic, coupled to said sensor means; A formatter means for selecting rays and processing the selected pixel subarray. , to provide characteristic data representing the presence or passage of vehicles within the field of view of traffic. a vehicle detection system comprising: processor means for detecting a vehicle; 2. The sensor means includes a camera for supplying a video signal representative of the field of view; and a digitizer for digitizing the signal to generate the pixel array. The system in the first term of the scope of demand. 3. The subarray selection means displays a desired portion of the field of view in response to an operation by an operator. A system according to claim 1, including a terminal for providing a subarray selection signal. Mu. 4. The processor means characterizes the presence of a vehicle within the selected portion of the field of view. 2. The system of claim 1, further comprising means for supplying data. 5. The processor means detects the presence or absence of a vehicle via a selected portion of the field of view. The system according to claim 1, comprising means for supplying data representative of the characteristics of the passage. stem. 6. said processor means for processing the pixel array according to a spatial processing method; A system according to claim 1, comprising the means of: 7. The above processor has a method for processing the pixel array according to a temporal processing method. The system of claim 1 comprising stages. 8. coupled to the sensor means on the road on which vehicle detection is to be carried out; of a view of traffic by which a portion or point can be selected by the user The system of claim 1 further comprising monitoring means for providing a visual indication. . 9. coupled to said processor means to direct/monitor traffic as a function of characteristic data; Traffic control/monitoring/classification/counting rveillance)/declassifying The system of claim 1 further comprising means. 10. Claim further comprising storage means for further processing and storing characteristic data The system in the first term of the range. 11. Generates a continuous pixel array representing the characteristics of the traffic field of view over time. an imaging means for processing the pixels to represent the presence and passage of a vehicle within the field of view; and a processor means for generating data to be processed. coupled between the image forming means and the processor means; Select a subarray of pixels representing the characteristics of the desired portion of the traffic field of view for processing. A system characterized by comprising a formatter means for. 12. coupled to said formatter means, said operator controlling said formatter means; and to make it possible to select pixels that represent the characteristics of a desired part of the traffic field of view. 12. The system of claim 11, further comprising terminal means. 13. a surveillance device coupled to said imaging means for generating a visible image of the field of view of the traffic; further comprising: an operator who uses the terminal means in conjunction with the monitoring device to monitor the 13. The system of claim 12, which selects target pixels of . 14. A traffic field of view to generate data representing characteristics of vehicles within the traffic field of view. Processing the pixel array representing spatially over time A method for operating a programmable computing means in order to receiving a continuous pixel array representing a field of view over time; The corresponding pixels of the consecutive arrays are time-averaged over time to provide a time-averaged array. supply, The background adjustment pixel array is created by adding the corresponding pixels of the time-averaged array with the pixels of the instantaneous array. occurs, Spatially average window groups of background adjustment pixels to generate a spatially averaged array death, generating a spatial variance of pixels as a function of a background adjustment pixel and a spatially averaged pixel; By logically comparing the instantaneous pixel array with the spatially distributed array, the presence of a vehicle can be determined. A method characterized in that the method comprises: generating data representative of calls and passages. 15. Traffic vision is used to generate data representing vehicle presence and passage characteristics. A continuous pixel array representing a field is processed temporally according to time. A method for operating a logrammable calculation means, the method comprising: receiving a continuous pixel array representing a field of view of the traffic over time; time-averaging corresponding pixels of the successive intensity value array to generate a time-averaged pixel array; The corresponding time-averaged pixel of the time-averaged array and the pixel of the instantaneous pixel array are added together to calculate the Generate a background control pixel array and generate a time-distributed pixel array based on the background control pixel array. occurs, Generate a threshold pixel array of thresholds as a function of the corresponding time-dispersed pixel array death, Vehicle presence and and generating data representative of the passage.