JP2001043352A - Outside monitoring device with fail-safe function - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To accurately detect a road surface condition in which it is difficult to recognize a traveling environment from a captured image. An external monitoring device that performs fail-safe when it is determined that a failure has occurred.
And the number of luminance edges in the horizontal direction of the monitoring area based on the image data in the monitoring area set in a predetermined area in the captured images obtained by the cameras 1 and 2, and the overall luminance of the monitoring area. And a fail judging unit 12 for judging a failure when the number of luminance edges is smaller than the number of judgments and the overall luminance is larger than the judgment value. . Here, the monitoring area extends in the horizontal direction so as to include a part of the road surface reflected in the captured image, and also extends in the horizontal direction so as to include the outside of both ends of the traveling road reflected in the captured image. Extends.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road surface condition in which it is difficult to recognize a road from a picked-up image, in particular, the entire road surface has snow (one-sided snow)
The present invention relates to a vehicle exterior monitoring device that detects a situation in which a vehicle is present from a captured image and performs fail-safe.

[0002]

2. Description of the Related Art In recent years, attention has been paid to a vehicle exterior monitoring device using a vehicle-mounted camera having a solid-state imaging device such as a CCD.
This device recognizes a traveling environment (for example, a distance between a traveling road or an object outside the vehicle and an own vehicle) based on an image captured by a vehicle-mounted camera, and pays attention to a driver as necessary. It is for arousing and controlling the vehicle such as deceleration by downshifting. For example, when recognizing a traveling environment using a stereo camera-based distance measurement technique, a positional difference (parallax) of the same object in a pair of captured images is obtained, and the target object is determined using the principle of triangulation. (Distance information) can be obtained.

[0003] In order to put such a vehicle exterior monitoring device into practical use, it is necessary to provide a fail-safe function in order to guarantee the safe operation of the device. One of the failures to be detected by this type of device is a situation in which snow is piled up over one surface of a traveling path (one-sided snow). If snow exists on the entire road, the center line, the traveling lane, or the road shoulder line, etc., marked on the road surface will be covered with snow. Therefore, image information sufficient to grasp the situation outside the vehicle (particularly, road recognition) cannot be obtained, and it may be difficult to perform accurate monitoring control.

Further, in a situation where snow exists on the road surface, the friction coefficient of the road surface is significantly reduced as compared with a dry road surface. Therefore, when performing vehicle control such as deceleration on such a road surface condition, it is required to control the vehicle with a gentler braking force than the normal control so that the vehicle does not slip.

[0005] From the viewpoint of ensuring the safety of the external monitoring device at a high level, in a driving environment such as snow on one side,
It is necessary to activate the fail-safe function. It should be noted that “fail-safe” in the present specification is a broad concept of performing monitoring control different from normal control. Therefore, not only the normal monitoring control is temporarily interrupted but also the control of performing the vehicle control with a gentler braking force than the normal control is included in the concept of the “fail safe” here.

[0006]

Heretofore, a fail-safe function, which is indispensable for the practical use of the vehicle exterior monitoring device, has not been established, and there remains a problem for ensuring the safety of the device at a high level.

An object of the present invention is to accurately detect, from a captured image, a road surface condition in which it is difficult to recognize a traveling environment.

Another object of the present invention is to ensure the safety of the vehicle exterior monitoring device at a high level by performing appropriate monitoring control according to the road surface condition.

[0009]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, the present invention provides an external monitoring apparatus that performs fail-safe when it is determined that a failure has occurred. Calculating means for calculating the number of luminance edges in the horizontal direction of the monitoring region and the magnitude of the overall luminance of the monitoring region, based on image data in the monitoring region set in a predetermined region in the obtained captured image; A determination unit for determining a failure when the number of luminance edges is smaller than the number of determinations and the overall luminance is larger than the determination value. Here, the monitoring area extends in the horizontal direction so as to include a part of the road surface reflected in the captured image, and also extends in the horizontal direction so as to include the outside of both ends of the traveling road reflected in the captured image. Extends.

Here, control means for controlling the braking force of the vehicle may be further added to the above configuration. This control means
When it is determined that the vehicle has failed, the vehicle control may be performed with a braking force that is less than the braking force exerted in the normal control.

[0011]

FIG. 1 is a block diagram of a vehicle exterior monitoring apparatus according to this embodiment. A pair of cameras 1 and 2 having a built-in image sensor such as a CCD are attached at predetermined intervals in a vehicle width direction of a vehicle such as an automobile, and capture an image of a scene in front of the vehicle. The main camera 1 captures a reference image (right image) necessary for performing stereo processing, and the sub camera 2 captures a comparison image (left image) in this processing. In a state where the cameras 1 and 2 are synchronized with each other,
Each of the analog images output from 2 is converted into a digital image of a predetermined luminance gradation (for example, 256 gray scales) by A / D converters 3 and 4. The digitized image is subjected to luminance correction, geometric conversion of the image, and the like in the image correction unit 5. Usually, since the mounting positions of the pair of cameras 1 and 2 have an error to some extent, a shift caused by the error exists in the left and right images. In order to correct this displacement, geometric transformation such as rotation or translation of the image is performed using affine transformation or the like. The reference image and the comparative image corrected in this way are stored in the original image memory 8.

On the other hand, the stereo image processing unit 6 calculates the three-dimensional position of the same object (including the distance from the vehicle to the object) in the image from the reference image and the comparison image corrected by the image correcting unit 5. calculate. This distance can be calculated based on the principle of triangulation from the relative displacement of the position of the same object in the left and right images. The distance information of the image thus calculated is stored in the distance data memory 7.
Is stored in

The microcomputer 9 recognizes a road condition or the like in front of the vehicle based on the information stored in the original image memory 8 and the distance data memory 7 (road recognition unit 10) or a three-dimensional object in front of the vehicle (running vehicle). (Three-dimensional object recognition unit 1)
1). Then, the processing unit 13 outputs these recognition units 10, 1
When it is determined from the information from 1 that an alarm is required, the driver is alerted by an alarm device 19 such as a monitor or a speaker, or the various control units 14 to 18 are controlled as necessary. For example, it instructs an AT (automatic transmission) control unit 14 to execute downshifting. Further, it may instruct the engine control unit 18 to reduce the engine output. In addition, an anti-lock brake system (ABS) controller 15, a traction control system (TCS) controller 16, or
Appropriate vehicle control can also be instructed to the vehicle behavior control unit 17 that controls the torque distribution and rotation speed of each wheel.

Further, the fail determining section 12 performs a fail determination based on the original image information stored in the original image memory 8 in accordance with a routine described later. During a period in which a failure is determined, that is, a period in which a fail flag NG is set to 1 by a later-described failure determination routine, fail-safe is executed by the processing unit 13. For example, the above-described vehicle control or the like is temporarily suspended in order to prevent a malfunction of the device in a situation where it is difficult to recognize a road. Further, the various braking parameters of the above-described control units 14 to 18 are changed so that a vehicle braking force that is gentler than that during the normal control is generated.

FIG. 2 is a flowchart showing a routine for determining a failure due to one-sided snow. This flowchart is repeatedly executed at a predetermined control cycle (for example, 100 ms). When the fail flag NG is set to 1 by this routine, the fail determination unit 12
The processing unit 13 is instructed to that effect, and the processing unit 13 executes fail-safe of the monitoring apparatus outside the vehicle.

First, in step 1, the fail judging section 12 reads the image data in the monitoring area R. FIG.
FIG. 5 is a diagram for explaining a monitoring area R set in an image area. In the image area having the size of 200 × 512 pixels, a rectangular area extending in the horizontal direction on the lower side of the image as shown in FIG. The determination as to whether or not snow is piled up over the entire road surface is made by the monitoring area R
It is performed based on. Therefore, it is necessary to set the monitoring area R so as to include an area where the road is projected in the captured image. At this time, it is preferable to set the monitoring region R on the lower side of the captured image as much as possible. Usually, the own vehicle travels at a certain distance from the preceding vehicle, so that the preceding vehicle tends to be displayed relatively above a captured image that is a two-dimensional plane. Therefore, if the monitoring area R is set as low as possible in the captured image, a situation in which the road surface is masked by the preceding vehicle can be avoided, and the road surface state can be monitored accurately. Also,
As shown in FIG. 3, the monitoring area R extends in the horizontal direction so as to include a situation outside the both ends (that is, edges) of the traveling road such as a center line, a traveling lane, or a road shoulder marked on the road surface. Must be kept. The horizontal length of the monitoring region R is set to be relatively long in anticipation of a traveling situation (for example, when traveling on a curve) in which the edge swings right and left of the captured image.

In step 2 following step 1, the number of edges N in the monitoring area R is counted. FIG. 4 is a diagram for explaining an edge calculation method of the monitoring region R.
First, the luminance change amount Δχ is calculated for a certain target pixel T existing in the monitoring region R. The luminance change amount Δχ is the difference (absolute value) between the luminance sum of two pixels (pixel blocks) adjacent on the left side of the target pixel T and the luminance sum of two pixels (pixel blocks) adjacent on the right side. In this calculation method, the left and right 2
Since the luminance change is observed using the pixel block as a comparison unit, a value that accurately reflects the state of the horizontal edge of the target pixel T can be obtained. Such a luminance change amount Δχ
Is calculated for all pixels existing in the monitoring area R. Thereby, the state of the luminance change in the horizontal direction in the entire monitoring region R can be grasped.

Next, among the luminance change amounts Δχ calculated for the number of pixels in the monitoring area R, the number of those having a value larger than a predetermined threshold value Δχth (for example, 30 during the day) is obtained. Regarding the threshold value Δχth, a value smaller than the daytime (for example, 20) is used to ease the edge criterion at night or the like when a change in luminance hardly occurs. Then, the luminance change amount Δχ larger than the threshold value Δχth
Are counted as the number N of edges.

In step 3 following step 2, the average of the luminance values of all the pixels in the monitoring area R, that is,
An average luminance A is calculated.

Then, the number of edges N calculated in step 2 is smaller than the appropriately set judgment number Nth (step 4), and the average luminance A calculated in step 3 is set appropriately. If it is larger than the determination threshold value Ath (step 5), the failure flag NG is set to 1
Is set to Otherwise, fail flag N
G is set to 0 (step 7).

In a situation where snow is present on the road surface, the number of luminance edges appearing in the horizontal direction tends to be smaller than in a dry road situation. This is because a white line such as a center line, a traveling line, or a road shoulder line projected on a dry road surface condition is masked by snow, so that there is no difference in luminance there. Therefore, in step 4, by comparing the calculated number of edges N with an appropriately set determination number Nth through an experiment or the like, the possibility of the presence of one-sided snow is evaluated. That is, the number of edges N
Is smaller than the determination number Nth, it is determined that there is a possibility that one-sided snow exists. On the other hand, if the number of edges N is equal to or greater than the determination number Nth, it is determined that there is no possibility.

However, in a situation such as a tollgate on an expressway where white lines or the like do not exist on the left and right sides of the travel road, or on a gravel road where there is not much difference in brightness, the number of edges N is almost the same as that of snow on one side. It may not be counted. Therefore, in step 5, it is determined whether or not the entire area of the monitoring area R is snow by looking at the overall luminance level (average luminance). That is, in the case of one-sided snow, the monitoring area R
Tends to increase. On the other hand, on a gravel road, a dry pavement, or the like, the overall brightness is not as large as in the case of one-sided snow. Therefore, in addition to the affirmative determination in step 4, if the calculated average luminance A is larger than a determination threshold value Ath appropriately set through an experiment or the like, it is possible to estimate one-sided snow.

In order to ensure the reliability of the monitoring outside the vehicle and the stability of the control, the switching of the NG flag is performed when the above-described normal / abnormal judgment is continued for a predetermined cycle. Is preferred. For example,
When the fail flag NG is set from 0 to 1, the change is made when abnormality (fail) determination continues for 5 cycles (corresponding to 0.5 seconds). Further, the fail flag NG is set from 1 to 0.
If the normal judgment is continued for 20 cycles (equivalent to 2.0 seconds), the setting is changed.

Further, in order to improve the detection accuracy of the failure situation, the above-mentioned failure determination routine may be executed according to the traveling situation. For example, the failure determination is performed only when the inter-vehicle distance between the own vehicle and the preceding vehicle is 13 m or more, the vehicle speed is 20 km / h or more, and the steering angle is ± 23 ° or less. When the inter-vehicle distance with the preceding vehicle is short, most of the road surface is masked by the preceding vehicle, and it is difficult to accurately detect the road surface condition. Further, in particular, when the method of switching the fail flag NG when the normal / abnormal determination is continued for a predetermined cycle is used, if the vehicle speed is low, the failure determination accuracy may be reduced. When the vehicle speed is low, the travel distance of the vehicle is small,
Only local roads can be monitored. If the local road surface is in an unpredicted state that accidentally matches the failure situation, an inappropriate failure determination may be made. If the high vehicle speed is used as a condition for executing the failure determination, it is possible to avoid erroneously determining a failure even if such a road surface exists locally. Further, the reason for taking the steering angle into consideration is to improve the determination accuracy by limiting the pattern of the scenery in the field of view of the camera to some extent.

Furthermore, if the outside air temperature obtained from the outside air temperature sensor is considered in addition to the determination result of the road surface condition based on the captured image, the presence or absence of snow can be more accurately estimated.

As described above, in the fail determination method based on one-sided snow according to the present embodiment, in order to monitor the condition of the road surface, in the captured image which is a two-dimensional plane, a specific region where the road surface is projected is set as a monitoring region. You have set. Then, the one-sided snow is detected based on the number of edges in the monitoring area and the average luminance in the area. As a result of the inventor actually repeating the running test, it was confirmed that a good judgment result can be obtained by the above judgment procedure. If control (including fail-safe) according to the determination result is performed, it is possible to ensure the safety of the vehicle exterior monitoring device at a higher level.

Further, in this embodiment, the failure judgment of the stereo camera is described. However, the present invention is not limited to this, and it is needless to say that the present invention can be applied to a case where a monocular camera is used.

[0028]

As described above, according to the present invention, it is possible to accurately detect a road surface condition in which it is difficult to recognize a traveling environment from a captured image. By performing fail-safe according to the road surface condition, the safety of the vehicle exterior monitoring device can be ensured at a high level.

[Brief description of the drawings]

FIG. 1 is a block diagram of a vehicle exterior monitoring device according to an embodiment;

FIG. 2 is a flowchart showing a routine for determining a failure due to one-sided snow;

FIG. 3 is a diagram for explaining a monitoring area set in an image area;

FIG. 4 is a diagram for explaining a method of calculating an edge of a monitoring area;

[Explanation of symbols]

1 main camera, 2 sub camera, 3,
4 A / D converter, 5 image correction unit, 6 stereo image processing unit, 7 distance data memory, 8 source image memory, 9 microcomputer, 10 road recognition unit, 11 three-dimensional object recognition unit, 12 failure determination unit, 13 processing unit, 14 AT control unit,
15 ABS control unit, 16 TCS control unit, 17 Vehicle behavior control, 18 Engine control unit, 19 Alarm device

Continued on the front page F term (reference) 5B057 AA16 BA02 CA08 CA12 CA16 CB08 CB13 CB16 CC01 CD01 CH08 DB02 DB09 DC16 DC25 DC36 5C054 AA01 AA05 CA04 CC03 EA01 EA05 ED17 FC04 FC12 FC15 FC16 FF07 HA30 5H180 AA01 CC04 CC24 LL07

Claims (2)

[Claims] An outside monitoring apparatus that performs fail-safe when it is determined that a failure has occurred, an imaging unit that captures a scene outside the vehicle, and a monitoring area set in a predetermined area in a captured image obtained by the imaging unit. Based on the image data in, the number of luminance edges in the horizontal direction of the monitoring area, a calculating means for calculating the overall luminance magnitude of the monitoring area, the number of luminance edges is less than the number of determination, And, when the magnitude of the overall luminance is greater than a determination value, the determination unit determines a failure, and the monitoring area includes a part of the road surface reflected in the captured image. An exterior monitoring device, which extends in the horizontal direction and extends in the horizontal direction so as to include the outside of both ends of the traveling path shown in the captured image. 2. The vehicle further comprises control means for controlling a braking force of the vehicle, wherein the control means performs the vehicle control with a braking force that is less than the braking force exerted in the normal control when the failure is determined. The vehicle exterior monitoring device according to claim 1, wherein: