JP4506573B2 - In-vehicle fog determination device - Google Patents

Description

  The present invention relates to an in-vehicle fog determination device that determines fog from an image captured by an in-vehicle camera.

  There is known an apparatus that performs fog determination by processing a captured image of an in-vehicle camera (for example, Patent Document 1). The fog determination method in Patent Document 1 is based on the degree of blurring of an image, and in the case of fog, the image is blurred. Therefore, the fog determination is performed by estimating the degree of blurring of the image. In estimating the degree of blur of an image, first, an edge amount of luminance is calculated by differential calculation for each pixel with respect to the entire image, and the degree of blur of the image is estimated based on the edge amount. As the image is blurred, the edge amount becomes smaller as a whole. Therefore, the degree of blurring of the image can be estimated based on the edge amount.

  The apparatus of this patent document 1 uses the result of fog determination for white line recognition. When white line recognition is performed from an image captured by an in-vehicle camera, fog is determined prior to white line recognition because the image is blurred and white line recognition becomes difficult if the image is fog. And when it determines with it being fog, it also describes lighting a fog lamp.

  In addition, the device of Patent Document 2 does not determine only fog, but if the image is blurred due to poor visibility due to fog or dirt on the windshield, it is possible to accurately monitor the outside of the vehicle (the vehicle and the object outside the vehicle). It is difficult to monitor the distance between them, so it is determined whether the image is blurred.

The method for determining image blur in Patent Document 2 calculates a brightness edge for an image of a fixed region set at the upper center of the image, and determines the degree of image blur based on the ratio of the edge exceeding a predetermined value. is doing.
Japanese Patent No. 3444192 Japanese Patent No. 3312729

  In each of Patent Documents 1 and 2, fog determination is performed by determining whether an image captured by a vehicle-mounted camera is blurred. However, as in Patent Document 1, the edge amount is set for the entire image. If it is calculated, there is a problem that the load of image processing is large. On the other hand, in Patent Document 2, since the edge is calculated only in the fixed region set at the upper center of the image, the load of image processing is less than that in Patent Document 1. However, since an object at a short distance from the own vehicle is captured relatively clearly even if it is fog, if an object located at a short distance from the own vehicle is captured in the fixed area, the fog determination There was a problem that could not be performed accurately. In addition, as a case where an object located at a short distance from the own vehicle is imaged in the fixed area set at the upper center of the image, for example, a case where a preceding vehicle on the same lane is traveling at a narrow inter-vehicle distance is considered. However, in addition to this, when traveling on a relatively steep curve, a roadside scenery located at a relatively short distance is imaged in the fixed area.

  The present invention has been made based on this situation, and the object of the present invention is to determine the fog while reducing the load of image processing for determining whether the image is a foggy image. This is to ensure the accuracy.

Invention of claim 1, wherein in order to achieve the object, an image taken by the onboard camera mounted on the vehicle, and the image fog state determining means for determining whether the misty image, the vehicle Based on the road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the camera, and from the vehicle in the image based on the road shape determined by the road shape determining means. A far road area determining means for determining a far road area that is an area on a road at a predetermined far distance, and the image determined by the image fog state determining means is a far distance determined by the far road area determining means An in- vehicle fog determination device that is an image of a road area ,
The road shape determination means determines a shape of a road on which the vehicle is traveling by detecting an on-road sign provided along the road from an image captured by the in-vehicle camera. Means for detecting the position of the preceding vehicle in the image and determining the shape of the road on which the vehicle is traveling based on the detected position of the preceding vehicle; and a steering angle sensor At least two of the third determining means for determining the shape of the road on which the vehicle is traveling based on at least one of the steering angle of the steering wheel and the yaw rate of the vehicle detected by the yaw rate sensor With
When the shape of the road can be determined by the first determining means, the shape of the road on which the vehicle is traveling is determined using the first determining means, and the shape of the road cannot be determined by the first determining means. However, when the shape of the road can be determined by the second determining means, the shape of the road on which the vehicle is traveling is determined using the second determining means, and depending on the first and second determining means, the road The shape of the road cannot be determined, but when the shape of the road can be determined by the third determining means, the shape of the road on which the vehicle is traveling is determined using the third determining means. .

  According to the invention of claim 1, since the image to be analyzed in the image fog state determination means is an image of the far road area determined by the far road area determination means, not the entire image captured by the in-vehicle camera. Compared to analyzing the entire image captured by the camera, the load of image processing is reduced. In addition, the region for analyzing whether the image is foggy is a region on the road at a predetermined distance from the vehicle in the image. In this way, if the area of the image to be analyzed is an area on the road, since the road usually continues far away, there is a high possibility that the far portion is imaged in that area. The determination in the determination means is ensured.

Further, according to the invention of claim 1, wherein, in one unit of the first to third determination means even if the shape of the road can not be determined, the shape of the road can be determined by other determination means If so, since the shape of the road on which the vehicle is traveling is determined, there is little possibility that the shape of the road cannot be determined. Further, when comparing the first to third determining means, the first determining means determines the shape of the road from the road signs provided along the road, so the most accurate shape of the road is determined. Since the second determining means determines the shape of the road based on the position of the vehicle actually traveling on the road ahead, the second determining means accurately determines the shape of the road next to the first determining means. Can be determined. Therefore, in the invention according to the second aspect, since the road shape is determined using the determination means having higher determination accuracy, the road shape can be determined with high accuracy.

According to a second aspect of the invention for achieving the above object, image fog state determination means for determining whether or not an image captured by an in-vehicle camera mounted on a vehicle is a foggy image. In the image based on the road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera, and the road shape determined by the road shape determining means A far road area determining means for determining a far road area which is an area on a road at a predetermined far distance from the vehicle, and the image determined by the image fog state determining means is determined by the far road area determining means. In-vehicle fog determination device that is an image of a distant road area,
The vehicle travels based on the road shape determined by the road shape determining means and a pre-stored far distance line indicating a predetermined far distance from the vehicle on the flat ground in the image. A far distance point determining means for determining a far distance point at a predetermined distance from the vehicle on the road, wherein the far road area determining means determines the far distance point determined by the far distance point determining means. The far road area is determined as a reference.

Further, preferably, the far road area determining means determines an outer frame of the far road area on the basis of the far distance point determined by the far distance point determining means as described in claim 3 , A region where at least one of the road portion in the outer frame and the front vehicle portion in the outer frame is removed is determined as the far road region. The road part is usually removed from the area within the outer frame of the far road area. Since the road part usually has a brightness change only at the boundary of the white line, it is determined whether or not it is fog from the road part image. This is because it is relatively difficult to do. In addition, the front vehicle part is removed because the distance of the front vehicle may be close to the distant background that you want to focus on, and the brightness may change greatly due to the influence of the tail lamp. This is because it may be difficult to determine whether or not it is fog from the image of the front vehicle portion. By removing a portion that is relatively difficult to determine whether or not it is fog, determination accuracy based on the area after removal is improved.

According to a fourth aspect of the present invention for achieving the above object , there is provided an image fog state determination means for determining whether an image captured by an in-vehicle camera mounted on a vehicle is a foggy image. In the image based on the road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera, and the road shape determined by the road shape determining means A far road area determining means for determining a far road area which is an area on a road at a predetermined far distance from the vehicle, and the image determined by the image fog state determining means is determined by the far road area determining means. In the in-vehicle fog determination device that is an image of the far road area, the image fog state determination means, in the edge intensity distribution diagram of the luminance of the image of the far road area, Based on the ratio of the constant reference intensity below the edge strength, characterized in that to determine whether the fog took images.

As described above, the image fog state determination means is a fogged image based on a ratio of edge intensity below a predetermined reference intensity in the luminance edge intensity distribution diagram of the image of the far road area. Judgment is made. In the case of a foggy image, since the edge strength is reduced as a whole, it is possible to determine whether the image is foggy based on the ratio of the edge strength below the reference strength.

  In addition to the method based on the edge intensity distribution of the luminance of the image, other methods such as determination based on the average value of the luminance of the image are possible as the determination method in the image fog state determination unit. The reason why the fog can be determined based on the average value of the luminance of the image is that the image is bright as a whole when it is fog.

  In addition, the image fog state determination means analyzes only the image of the far road area determined by the far road area determination means, thereby determining whether the image of the far road area is fogged. You may judge. However, in this case, the edge strength is generally small as in the case of fog because the image of the far road area is not fog because of traveling on flat terrain. It may be an image or an overall bright image. In that case, there is a possibility that it may be erroneously determined as fog although it is not fog.

Therefore, in the invention 請 Motomeko 5, wherein the image taken by the onboard camera mounted on the vehicle, and the image fog state determining means for determining whether the misty image, captured by the onboard camera A road shape determining means for determining a shape of a road on which the vehicle is traveling in the image, and a predetermined far distance from the vehicle in the image based on the shape of the road determined by the road shape determining means. A far road area determining means for determining a far road area that is an area on the road in which the image fog state determining means determines the image of the far road area determined by the far road area determining means. In-vehicle fog determination device,
The image fog state determination means includes an image of a distant road area determined by the distant road area determination means, and a close road area that is preset as a range in which a roadside near the vehicle is imaged in the image. by comparing the images, the image of the distant road area, determine whether the fog took images.

  In this way, when comparing the image of the preset close road area and the image of the far road area in the image, the image of the close road distance is not easily affected by fog and is compared even if it is fog. Since the image is clearly captured, when it is not foggy, the image of the far road area shows the same luminance or edge intensity distribution as the image of the close road area. On the other hand, in the case of fog, the image of the far road area is different in brightness and edge intensity distribution from the image of the closest road area. Therefore, comparing the image of the far road area and the image of the closest road area improves the accuracy of fog determination.

When the fog determination is performed by comparing the image of the distant road area and the image of the close road area as described in claim 5 , for example, the edge strength of the image of the close road area is described in claim 6. A threshold value for determination is determined based on the distribution. That is, the invention according to claim 6 is the in-vehicle fog determination device according to claim 5 , wherein the image fog state determination means is equal to or higher than a predetermined reference intensity in the edge intensity distribution diagram of the luminance of the image of the far road area. A strong edge amount that is a ratio is calculated, a threshold value is calculated based on an edge intensity distribution map of luminance of the image of the closest road area, and a fog is calculated based on a comparison between the strong edge amount and the threshold value. It is determined whether or not the image is covered.

The image fog state determination means may perform the fog determination using only the image picked up at the temporary point. However, when only the temporary point image is used, the image happens to be erroneously determined. It may be an image. Therefore, preferably, as described in claim 7, it is determined whether or not the image is a fogged image based on the images of the distant road area at a plurality of time points captured by the in-vehicle camera. To. In this way, the accuracy of fog determination is improved.

Preferably, as described in claim 8 , the image fog state determination means repeatedly determines whether or not the image is a fogged image, and the fog has not been detected in the previous determination. When it is determined that the image is fogged, it is easier to determine that the image is fogged than when the previous determination determines that the image is not fogged. .

  In the case of repeatedly determining whether or not the image is foggy, if the same criterion is used regardless of the previous determination result, the determination result fluctuates within a short time when the fog is relatively thin. However, this causes inconvenience when the determination result is used for subsequent control (for example, automatic lighting of a fog lamp). On the other hand, according to the tenth aspect, since the determination result is less likely to fluctuate even when the mist is relatively thin, the inconvenience in the case where the control is further performed based on the determination result is solved. Is done.

  Hereinafter, an embodiment of an in-vehicle fog determination device according to the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an in-vehicle fog determination device 10 to which the present invention is applied.

  The in-vehicle fog determination device 10 includes an in-vehicle camera 12, an image processing ECU 14, a yaw rate sensor 16, a steering sensor 18, a millimeter wave radar 20, and a vehicle speed sensor 22, which are connected to each other by an in-vehicle LAN 24. In addition, a driving support control ECU 26 and a light control ECU 28 are also connected to the in-vehicle LAN 24.

  The in-vehicle camera 12 is constituted by a CCD camera, and its mounting position is the interior of the vehicle, for example, the ceiling near the driver's seat. The vehicle-mounted camera 12 continuously captures images in front of the vehicle, and the captured image data is processed in the image processing ECU 14.

  The image processing ECU 14 is a computer having a CPU, a ROM, a RAM, and the like inside (not shown), and the RAM temporarily stores image data for a predetermined time continuously captured by the in-vehicle camera 12. The Then, the CPU executes the process shown in FIG. 2 on the image data stored in the RAM according to the program stored in the ROM. The processing of this CPU will be described later.

  The yaw rate sensor 16 sequentially detects the yaw rate of the vehicle, and the steering sensor 18 sequentially detects the steering angle of the steering. The millimeter wave radar 20 outputs a millimeter wave having a predetermined frequency toward the front of the vehicle and receives a reflected wave from the object. The driving support ECU 26 also functions as a forward vehicle detection unit, and based on the reflected wave received by the millimeter wave radar 20, there is a forward vehicle (a vehicle traveling ahead in the same lane as the host vehicle). Are continuously determined, and when a preceding vehicle is detected, a distance from the preceding vehicle, a relative direction, and a relative speed are calculated. Furthermore, based on the calculated information, driving support control such as vehicle speed control is executed.

  The light control ECU 28 performs control of turning on / off a fog lamp (not shown) when the image processing ECU 14 determines that it is fog.

  FIG. 2 is a flowchart showing a main part of the control function of the image processing ECU 14. The control shown in FIG. 2 is executed at a predetermined cycle during traveling, and images of the front of the vehicle are continuously taken by the in-vehicle camera 12 during execution of this control. 3 and 4 are image examples captured by the in-vehicle camera 12.

  As shown in FIG. 2, the image processing ECU 14 first executes a road shape determination process corresponding to the road shape determination means (step S100), and then performs a far road area determination process corresponding to the far road area determination means. After that, the fog determination image processing (step S300) corresponding to the image fog state determination means is executed.

  The road shape determination process in step S100 is shown in detail in FIG. In FIG. 5, first, in step S <b> 110, white line recognition processing for recognizing a white line that is a road sign provided along the road is executed based on images continuously captured by the in-vehicle camera 12. As this white line recognition processing, various known processing methods can be used. For example, an image captured by the in-vehicle camera 12 is binarized, and a white portion is extracted from the binarized image. The white line is recognized. Note that the white line here includes not only the white line but also the yellow line as in the normal white line recognition process.

  In the white line recognition process in step S110, the white line cannot always be recognized. For example, the white line may not be recognized because the white line is not drawn on the road. Therefore, in the subsequent step S120, it is determined whether or not the white line has been recognized. The determination here is to determine whether or not the white line has been recognized to the extent that the road shape can be determined in the subsequent step S130, and it is not necessary to recognize the white line on both sides, and from the vehicle position to the vanishing point. There is no need to be able to recognize white lines continuously. That is, in step S120, it is determined whether or not at least one white line can be recognized continuously for a predetermined length or intermittently.

  If the determination in step 120 is affirmed, the road shape is determined in the subsequent step S130. The road shape here is a single line indicating the degree of bending (shape) of the road extending forward from the host vehicle, for example, a one-dot chain line shown in FIGS. Note that the alternate long and short dash line shown in FIGS. 3 and 4 is the center line in the width direction of the lane in which the host vehicle is traveling, but instead of either the left or right side of the lane in which the host vehicle is traveling. The white line may be used as the road shape as it is, and when there are a plurality of lanes, the center line in the width direction of the entire road may be used as the road shape. The above steps S110 to S130 correspond to the first determining means.

  On the other hand, if the determination in step S120 is negative, it is determined in step S140 whether or not a forward vehicle is detected in the driving assistance ECU 26. If this determination is affirmative, the driving assistance ECU 26 has calculated the distance and relative direction from the preceding vehicle, so in step S150, the driving assistance ECU 26 calculates the distance from the preceding vehicle. Based on the distance and relative orientation, the position of the vehicle ahead in the image is determined.

  In the subsequent step S160, the steering angle is detected based on the signal from the steering sensor 18. In step S170, a predetermined point in the image that is predetermined as an end point on the own vehicle side of the line indicating the road shape (for example, the center point in the vehicle width direction of the boundary line of the bonnet in the image) is determined in step S150. The road shape is determined by connecting the position of the vehicle ahead in the image with an arc having the radius of curvature as the steering angle determined in step S160. Steps S140 to S170 correspond to the second determination unit.

  When the above-described determination in step S140 is also denied, steps S180 to S190 corresponding to the third determination unit are executed to determine the road shape. First, in step S180, the steering angle is detected based on the signal from the steering sensor 18, and in the subsequent step S190, at a predetermined point in the image that is predetermined as an end point on the own vehicle side of the line indicating the road shape, An arc whose tangent is parallel to the vehicle longitudinal direction is determined as a line indicating the road shape. Note that the yaw rate may be detected from the yaw rate sensor 16 instead of detecting the steering angle in one or both of the above-described steps S160 and S180.

  When the road shapes determined in steps S130, S170, and S190 are compared, step S130 determines the road shape based on the white line provided along the road. Therefore, the road shape can be determined most accurately. Moreover, since step S170 uses the position of the preceding vehicle in addition to the steering angle, the road shape can be determined more accurately than step S190 based only on the steering angle. On the other hand, step S130 cannot determine the road shape when the white line cannot be recognized, and step S170 cannot determine the road shape when the preceding vehicle is not detected. However, step S190 reliably determines the road shape. Can be determined.

  When the road shape is determined in this way, the far road area determination process (step S200 in FIG. 2) is executed. This far road area determination process is a process shown in detail in FIG.

  In FIG. 6, first, in step S210, a far distance point in an image at a predetermined far distance from the own vehicle on the road on which the own vehicle is traveling is determined. This step S210 corresponds to a long distance point determination means. Here, the predetermined far distance is set to 100 m here, but is not limited to 100 m, and may be far enough that the image is blurred in the case of fog of a certain degree of density.

  Since the in-vehicle camera 12 is fixed to the vehicle, it is possible to determine in advance the actual distance to a point on the flat ground and where the point is located in the image. Therefore, a 100 m line where a point 100 m away from the own vehicle is located on the flat ground in the image can be determined in advance. FIG. 7 shows the 100 m line Lf in the image. In the present embodiment, the 100 m line Lf is a long distance line, and the 100 m line Lf is stored in the ROM or other storage device in the image processing ECU 14. In step S210, the intersection of the 100 m line Lf and the line indicating the road shape determined in step S100 is set as a far distance point on the road on which the vehicle is traveling (that is, a point 100 m ahead here). decide.

  In subsequent step S220, the outer frame OF of the far road area is determined based on the far distance point determined in step S210. The outer frame OF of the far road area shown in FIGS. 3 and 4 is set so that the far distance point determined in step S210 is the center of the lower side of the outer frame OF. Further, the size of the outer frame OF is set to a sufficiently small size with respect to the size of the entire image captured by the in-vehicle camera 12.

  Note that the position of the outer frame OF may be determined so that the far distance point is the center of the far road region, not limited to the examples of FIGS. 3 and 4. Further, depending on whether the road shape is the center line in the width direction of the traveling lane, the center line in the width direction of the entire road, or the white line on the left or right of the traveling lane, the far distance point of the far road area with respect to the outer frame OF of the far road area The position may be changed. For example, when the road shape (the line indicating the road) is the white line on the right side of the traveling lane, the outer frame OF of the outer road OF is set so that a point ¼ from the right of the lower side of the outer frame OF of the far road area becomes a far distance point. The position may be determined.

  In the subsequent step S230, the road portion is removed from the area within the outer frame OF of the far road area determined in step S220. In the present embodiment, the preset fixed road portion 30 shown in FIG. 8 is removed. This road portion is a triangular portion having the center O of the outer frame OF as the apex and the lower side of the outer frame OF as the base.

  Considering that the road portion actually imaged in the outer frame OF differs in shape depending on the width of the road being traveled, whether it is a straight road or a curved road, and so on. If the portion 30 is a fixed region, it is difficult to accurately remove the road portion from the image actually captured in the outer frame OF. However, whatever the shape of the road, in the far road area, the road portion becomes narrower as it goes upward from the lower side. The road portion can be removed, and the processing load can be reduced by fixing the road portion 30 in this way. When the road portion is removed in step S230, the ratio of the roadside portion in the area after removal becomes relatively large.

  In addition, it is also possible to determine a road part based on the road shape determined by step S100, without fixing the road part 30. FIG. In this case, for example, when the line indicating the road shape and the outer frame OF of the far road area intersect only at one point (see FIG. 3), the vanishing point is the vertex and the lower side of the outer frame OF is the bottom side. If the triangle to be used is a road portion and the line indicating the road shape and the outer frame OF of the far road area intersect at two points (see FIG. 4), it is not the lower side of the outer frame OF at the two intersections. What is necessary is just to let the right-angled triangle which makes the vertex the intersection of the side which crosses in a part (or a point above predetermined coordinate) and apex the bottom of the outer frame OF as a road part. Alternatively, a white line may be recognized by image processing and a road portion may be determined based on the white line.

  In the subsequent step S240, it is determined whether or not there is a vehicle portion in the outer frame OF of the far road area determined in step S220. This determination is made because if the image determined as the far road area includes a vehicle located at a short distance, fog determination becomes difficult. It is for removing from the image inside. Therefore, in step S240, it is first determined whether or not the presence of the forward vehicle is detected by the driving assistance ECU 26. When the presence of the preceding vehicle is detected, an image processing range that is considered to include the vehicle is determined based on the position and relative distance of the preceding vehicle specified by the driving support ECU 26, and the image processing is performed. For example, the vehicle portion is determined by executing known image processing such as determining a vehicle contour line based on a change in luminance, and the determined vehicle portion is determined in step S220. Compare with the outer frame OF.

  If the determination in step S240 is negative, the routine shown in FIG. In this case, the area obtained by removing the road portion 30 from the area in the outer frame OF determined in step S220 is the far road area. On the other hand, if the determination in step S240 is affirmative, in step S250, a region obtained by removing the vehicle portion determined in the determination in step S240 from the region in the outer frame OF determined in step S220 is a far road. Decide on an area.

  In step S300, fog determination image processing is performed on the far road area determined as described above. Therefore, it can be said that the far road region is an image processing region in which fog determination image processing is performed.

  The fog determination image processing in step S300 is processing shown in detail in FIG. In FIG. 9, first, in step S310, the edge value two-dimensional distribution map is obtained by continuously differentiating the luminance value in the X direction and the Y direction with respect to the image of the far road area determined in step S200. create. In step S320, a strong edge amount that means a ratio exceeding a predetermined edge strength (reference strength) in the two-dimensional distribution chart created in step S310 is calculated.

  In the subsequent step S330, the vehicle speed is detected from the vehicle speed sensor 22, and in step S340, the roadside portion that was in the far road area moves to the nearest roadside area 32 set in advance based on the detected vehicle speed. Calculate the required travel time. Here, the nearest roadside area 32 is a fixed area set as an area where a roadside at a short distance is imaged to such an extent that the image is not so blurred even in fog. For example, FIG. 3 and FIG. As shown, it is a rectangular region of a predetermined size that is located at one of the left and right end portions of the image captured by the in-vehicle camera 12 and at the center in the vertical direction of the image.

  As described above, the close road area 32 is a fixed area, and since the distance from the host vehicle is fixed in the far road area, an object on the road side portion in the far road area is the close road area 32. The moving distance to move to can be determined in advance. Therefore, in step S340, the travel time required for the roadside portion in the far road area to move to the closest road area 32 set in advance from the predetermined travel distance and the vehicle speed detected in step S330 is calculated. calculate.

  In subsequent step S350, after the image used to create the two-dimensional distribution map of the edge strength in step S310 is captured, the image of the closest road area after the movement time calculated in step S340 has elapsed, A two-dimensional distribution map of luminance edge strength is created in the same manner as in step S310.

  Then, in step S360, a strong edge amount is calculated for the two-dimensional distribution map created in step S350, as in step S320, and in step S370, the closest road area calculated in step S360 is calculated. Based on the strong edge amount of the image, a threshold value used in step S380 described below is determined. For example, the threshold value is determined by multiplying the strong edge amount calculated in step S360 by a predetermined value α (for example, 0.7) smaller than 1. However, when the previous fog determination, that is, when the routine of FIG. 9 that is repeatedly executed is executed last time, the threshold value is changed depending on whether or not the fog is determined in step S395 described later. For example, if it is determined that it is not fog in the previous determination, the predetermined value α is set to 0.7, but if it is determined to be fog in the previous determination, the predetermined value is set to 0.6. To do.

  In the subsequent step S380, the strong edge amount calculated in step S320 is compared with the threshold value determined in step S370 by comparing the strong edge amount of the image of the far road area calculated in step S320 described above. If it is greater than the threshold value, it is determined that the image is not fogged. If the strong edge amount calculated in step S320 is equal to or smaller than the threshold value, it is determined that the image is fogged. In addition, since the threshold value is changed based on the previous determination result as described above, if it is determined as fog in the previous determination, it is not fog in the previous determination. It is easier to determine as fog than when it is determined. Therefore, even in the case of a thin mist, fluctuations in the determination result within a short period are prevented.

  Then, in the subsequent step S390, it is determined whether or not the final determination timing has been reached with the time when the first fog determination (S380) is executed as the timing start point. This final determination time is set to 1 minute, for example. If this determination is negative, the above steps S310 and after are repeated.

  On the other hand, if the determination in step S390 is affirmed, final fog determination is executed in step S395. The final fog determination here is that if the ratio determined to be fog among the plurality of determination results determined in step S380 by repeating steps S310 to 390 is equal to or greater than a predetermined ratio set in advance. On the other hand, if it is determined that it is fog, if it is determined that the ratio is not fog, the ratio is determined not to be fog. As described above, if the final determination is performed using the determination results obtained in a plurality of times in step S380, even if the temporary image is erroneously determined, the final determination is made. As an incorrect determination, an erroneous determination can be prevented, so that the determination accuracy is improved.

  In addition, when the predetermined ratio is 50%, it is always determined that it is not fog or mist, but the predetermined ratio may be other than 50%. In this case, it may not be determined whether it is fog or not, but in this case, the previous determination result is maintained.

  As described above, according to the present embodiment described above, the image to be analyzed in step S300 (image fog state determination unit) is determined in step S200 (far road area determination unit), not the entire image captured by the in-vehicle camera 12. Since it is a far road area image, the load of image processing is reduced compared to analyzing the entire image captured by the in-vehicle camera 12. In addition, the far road area for analyzing whether or not the image is foggy is an area on the road at a predetermined far distance from the vehicle in the image. In this way, if the area of the image to be analyzed is an area on the road, since the road usually continues far away, there is a high possibility that the far portion is imaged in that area. The determination in the image fog state determination means) is ensured.

  Further, according to the present embodiment, the image of the far road area is a fogged image by comparing the image of the preset close road area 32 and the image of the far road area in the image. It is not fog because it is judged whether or not it is fog, but even if it is judged as fog only from the image of the far road area because it is traveling on flat terrain, Is prevented from being determined.

  As mentioned above, although embodiment of this invention was described, this invention is not limited to the above-mentioned embodiment, The following embodiment is also contained in the technical scope of this invention, and also the summary other than the following is also included. Various modifications can be made without departing from the scope.

  For example, in the above-described embodiment, a white line is recognized as a road sign for determining the shape of the road. However, in addition to the white line, the road sign is a continuous line for dividing the lane in the same manner as the white line. There are protrusions (such as hemispherical or rod-shaped), median strips, roadside gutters, ridges, etc., which may be recognized to determine the shape of the road.

  In the above-described embodiment, the travel time for the roadside portion that was in the far road area to move to the close road area is calculated, and the image of the close road area compared with the image of the far road area is The image was taken at the time point after the moving time since the image of the road area was taken, but it is the same as the image of the far road area as the image of the close road area compared with the image of the far road area A point-in-time image may be used. In this case, the same part (landscape) is not compared, but the above-described erroneous determination that may occur when traveling on flat terrain is prevented without comparing the same part. I can.

  In the above-described embodiment, a strong edge amount is calculated for each image of the distant road area at the temporary point (S320), and fog determination is performed for each distant road image at the temporary point based on the strong edge amount (S380). The final determination is performed using a plurality of the determination results (S395). The strong edge amount calculated for each temporary image is averaged, and one determination result (based on the average strong edge amount ( Final determination) may be determined.

  In the above-described embodiment, the road portion is removed from the outer frame OF of the far road region, and if there is a vehicle portion, the region from which the vehicle portion is also removed is the far road region. The process of removing the part is not essential, and the fog determination may be performed using all the images in the outer frame OF of the far road area, or one of the road part and the vehicle part may not be removed.

  In the second determining means (S140 to S170) of the above-described embodiment, a road is obtained by connecting a predetermined end point on the vehicle side in the image and the position of the preceding vehicle with an arc having a steering angle as a curvature radius. Although the shape has been determined, when the millimeter wave radar 20 detects only a vehicle in a relatively narrow range with respect to the vehicle front-rear direction line, the predetermined end point on the own vehicle side and the vehicle ahead The road shape may be determined by connecting the lines with straight lines. In this case, step S160 (detection of the steering angle) is not necessary. In order to detect the vehicle ahead, a laser radar may be provided in place of the millimeter wave radar 20, and the vehicle is detected by detecting a characteristic shape of the vehicle such as a tail lamp or a license plate from the image. You may make it detect a vehicle.

  In the above-described embodiment, the outer frame OF of the far road area and the closest road area 32 are both rectangular, but they need not be rectangular, and may be other shapes such as a circle. .

It is a block diagram which shows the structure of the vehicle-mounted fog determination apparatus 10 with which this invention was applied. It is a flowchart which shows the principal part of the control function which image processing ECU14 of FIG. 1 performs. It is an example of an image captured by the in-vehicle camera 12. It is an example of an image captured by the in-vehicle camera 12. It is a flowchart which shows the road shape determination process of step S100 of FIG. 1 in detail. It is a flowchart which shows the far road area | region determination process of step S200 of FIG. 1 in detail. It is a figure which shows the 100 m line Lf set beforehand in the image imaged with the vehicle-mounted camera. It is a figure which shows the fixed road part 30 set beforehand. It is a flowchart which shows the fog determination image process of step S300 of FIG. 1 in detail.

Explanation of symbols

10: Vehicle-mounted fog determination device 12: Vehicle-mounted camera 16: Yaw rate sensor 18: Steering sensor 32: Closest road area Lf: 100m line (far distance line)
OF: Outer frame of far road area

Claims (8)

Image captured by the vehicle-mounted camera that is mounted on the vehicle, and the image fog state determining means for determining whether the fog took images,
Road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera;
Far road area determining means for determining a far road area that is an area on the road at a predetermined far distance from the vehicle in the image based on the shape of the road determined by the road shape determining means,
The image foggy state determining means determines an image, an image der Ru car Nokiri determination apparatus of the distant road area distant road area determined by the determining means,
The road shape determining means is
First determining means for determining a shape of a road on which the vehicle is traveling by detecting a road sign provided along the road from an image captured by the in-vehicle camera;
Second determining means for detecting a position of a preceding vehicle in the image and determining a shape of a road on which the vehicle is traveling based on the detected position of the preceding vehicle;
And third determining means for determining the shape of the road on which the vehicle is traveling based on at least one of the steering angle of the steering detected by the steering angle sensor and the yaw rate of the vehicle detected by the yaw rate sensor, With at least two of these decision means,
When the shape of the road can be determined by the first determining means, the shape of the road on which the vehicle is traveling is determined using the first determining means, and the shape of the road cannot be determined by the first determining means. However, when the shape of the road can be determined by the second determining means, the shape of the road on which the vehicle is traveling is determined using the second determining means, and depending on the first and second determining means, the road The shape of the road cannot be determined, but when the shape of the road can be determined by the third determining means, the shape of the road on which the vehicle is traveling is determined using the third determining means. In-vehicle fog determination device . Image fog state determination means for determining whether an image captured by a vehicle-mounted camera mounted on a vehicle is a foggy image;
  Road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera;
  Far road area determining means for determining a far road area that is an area on the road at a predetermined far distance from the vehicle in the image based on the shape of the road determined by the road shape determining means,
  The image determined by the image fog state determination means is an in-vehicle fog determination device that is an image of a far road area determined by the far road area determination means,
  The vehicle travels based on the road shape determined by the road shape determining means and a pre-stored far distance line indicating a predetermined far distance from the vehicle on the flat ground in the image. Further comprising a long distance point determination means for determining a long distance point at a predetermined long distance from the vehicle on the road,
  The in-vehicle fog determination device characterized in that the far road area determining means determines the far road area on the basis of the far distance point determined by the far distance point determining means. The far road area determining means determines an outer frame of the far road area on the basis of the far distance point determined by the far distance point determining means, and further, a road portion in the outer frame and in the outer frame The in-vehicle fog determination device according to claim 2 , wherein an area from which at least one of the front vehicle portions is removed is determined as the far road area. Image fog state determination means for determining whether an image captured by a vehicle-mounted camera mounted on a vehicle is a foggy image;
Road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera;
Far road area determining means for determining a far road area that is an area on the road at a predetermined far distance from the vehicle in the image based on the shape of the road determined by the road shape determining means,
The image determined by the image fog state determination means is an in-vehicle fog determination device that is an image of a far road area determined by the far road area determination means,
The image fog state determination means determines whether or not the image is fogged based on a ratio of edge intensity below a predetermined reference intensity in the luminance edge intensity distribution diagram of the image of the far road area. A vehicle-mounted fog determination device characterized by being . Image fog state determination means for determining whether an image captured by a vehicle-mounted camera mounted on a vehicle is a foggy image;
  Road shape determining means for determining the shape of the road on which the vehicle is traveling in the image captured by the in-vehicle camera;
  Far road area determining means for determining a far road area that is an area on the road at a predetermined far distance from the vehicle in the image based on the shape of the road determined by the road shape determining means,
  The image determined by the image fog state determination means is an in-vehicle fog determination device that is an image of a far road area determined by the far road area determination means,
The image fog state determination means includes an image of a distant road area determined by the distant road area determination means, and a close road area that is preset as a range in which a roadside near the vehicle is imaged in the image. A vehicle-mounted fog determination device characterized by determining whether or not the image of the far road area is a foggy image by comparing with an image. The image fog state determination means calculates a strong edge amount that is a ratio equal to or greater than a predetermined reference intensity in the luminance edge intensity distribution diagram of the image of the far road area, and also the luminance edge of the image of the closest road area calculating a threshold based on the intensity distribution diagram, the strong based on a comparison between the edge amount and the threshold value, claim and characterized in that to determine whether a foggy image 5 The vehicle-mounted fog determination device described in 1. The image fog state determination means determines whether or not the image is a foggy image based on images of distant road areas at a plurality of time points captured by the vehicle-mounted camera. The on-vehicle fog determination device according to any one of claims 1 to 6 . The image fog state determination means repeatedly determines whether the image is a foggy image, and when it is determined that the image is a fogged image in the previous determination. , than when it is determined not to be image took fog previous determination, of claims 1 to 7, characterized in that it is likely to be determined that fog is taken for image according to any one In-vehicle fog determination device.

Images