JP2007128427A - Driving support device and driving support method - Google Patents

Abstract

An object of the present invention is to prevent a driver's attention from being lowered by both issuing an obstacle alarm and presenting a route, and to avoid giving a sense of annoyance.
In a steering scene, a driving load is high or a driving skill level is low based on a ratio between an area occupied by a gaze object and the number of gaze points with respect to the gaze object, and the gaze object of the driver is an obstacle. If it is determined that there is an obstacle outside the visual field area, an obstacle alarm is issued (steps S5 to S7). From the above ratio, when the driving load is low or the driving skill level is high and the gaze target satisfying the cue information is equal to or greater than the threshold value, it is determined that the gaze target is an object that can be a cue, and the visual field When the area is narrow, route presentation is performed (steps S11 to S13). When there are fewer gaze objects that satisfy the clue information than the threshold value, it is determined that the gaze object is an obstacle, and an obstacle alarm is issued when an obstacle exists outside the visual field area.
[Selection] Figure 4

Description

  The present invention can perform a plurality of supports such as an obstacle notification support for notifying the presence of an obstacle ahead of the host vehicle, a route display support for displaying an optimum travel route of the host vehicle on a curved road, etc. The present invention relates to a driving support apparatus and a driving support method that provide only the support information that the driver currently needs in the driving support apparatus.

2. Description of the Related Art Conventionally, as a driving support device, a so-called obstacle approach warning device that notifies a driver that the vehicle is approaching an obstacle is known. As a problem when mounting this obstacle approach warning device on a vehicle, there is a problem that the driver is bothered by the frequent occurrence of warnings. In other words, the obstacle alarm has a contradictory element that the more an obstacle is detected and the more frequently the notification is given to the driver, the more the driver feels bothered by frequent notification. For this reason, development of a technique for solving this contradiction is being pushed forward. For example, basically, once the driver gives his eyes to the obstacle, it is determined that the driver has recognized the obstacle, and thereafter the alarm control is performed so as not to perform the obstacle alarm for the obstacle. Has been proposed (see, for example, Patent Document 1). By performing such alarm control, it is possible to reduce the frequency of issuing warnings and to reduce annoyance for the driver.
Japanese Patent Laid-Open No. 2004-210213

  By the way, as a driving support device for providing information to the driver, not only an obstacle approach warning device, but also a number of support devices such as a route information presentation device that presents information on the optimum route of the host vehicle on a curved road, etc. However, in providing these multiple driving support devices, not only the individual driving support devices but also the relationship of all the driving support devices mounted on the vehicle is considered. There is a need to do. That is, the driving support device that originally issues a warning or the like attracts the driver's attention. Even if an obstacle warning is presented in a form that matches the driver's cognitive information, if the driver is inadvertently provided with other information at the same time, the driver In addition to being conscious of the annoyance with respect to the driving support device, attention to the “driving support system”, which is an aggregate of a plurality of driving support devices, is required, and the driver's annoyance increases. In some situations, the total amount of attention that can be allocated by the driver may be exceeded.

  Therefore, the present invention has been made paying attention to the above-mentioned conventional unsolved problems, and when performing a plurality of different types of driving assistance, a plurality of driving assistance information is presented at the same time. An object of the present invention is to provide a driving support device and a driving support method capable of avoiding giving annoyance to the vehicle.

  In order to achieve the above object, a driving support apparatus according to the present invention is a driving support apparatus including a plurality of driving support means for providing driving support information having different support contents. The object is detected and its type is discriminated, and driving support information currently required by the driver is estimated based on the discriminated type of the gaze object. And only the driving assistance means which provides the driving assistance information which the driver | operator needs among several driving assistance means is operated.

  According to the driving support apparatus according to the present invention, the driving support information currently required by the driver is estimated based on the type of the driver's gaze target, and the estimated driving support information required by the driver is estimated. Since only the driving support means that requires the vehicle is operated, the driver assistance information that the driver needs is provided securely, and the driver is bothered by the multiple driving support. Can be avoided.

Embodiments of the present invention will be described below.
First, a first embodiment will be described.
FIG. 1 is a block diagram showing a functional configuration of a driving support apparatus 100 according to the present invention.
In the figure, 1 is a line-of-sight detection means for detecting the driver's line-of-sight direction, and 2 is a driver's line-of-sight action with respect to the gaze object that the driver is gazing at from the line-of-sight direction detected by the line-of-sight detection means. Gaze target judging means for judging what the gaze target object of the person is, 3 is a steering scene judging means for judging whether or not the driver is in a driving state that needs to be steered, and 4 is a presentation information determining means Based on the detection result of the gaze target determination unit and the determination result of the steering scene determination unit, the route presentation unit 5 for presenting the optimum route of the host vehicle on a curved road and the presence of an obstacle ahead of the host vehicle It is determined which of the obstacle notification means 6 for notifying the driver which information is to be presented.

FIG. 2 is a functional block diagram showing a more specific functional configuration of the driving support device 100. As shown in FIG.
In FIG. 3, reference numeral 11 denotes a line-of-sight detection device. As shown in FIG. 3, the line-of-sight detection device 11 includes a pair of eye cameras 11a for imaging a driver's eyeball and the like, and a captured image of the eye camera 11a. And an image processing device 11b for detecting the driver's line-of-sight direction according to a known procedure. Note that reference numeral 41 in FIG. 3 denotes a steering wheel. The driver's line-of-sight direction detected by the line-of-sight detection device 11 is input to the gaze target determination device 12.

  The gaze target determination device 12 superimposes a front camera 12a for capturing a travel path ahead of the host vehicle, a captured image of the front camera 12a and a visual line direction detected by the visual line detection device 11, and a travel route on the captured image. An obstacle on the road, an object on the road such as a road white line, a guardrail, and the like (hereinafter also referred to as an extracted object) is extracted, a point where the driver's line of sight stays is detected as a gazing point, and The gaze point distribution extraction circuit 12b for detecting the distribution state of the viewpoint and the driver's line-of-sight range detected from the driver's line-of-sight direction detected by the line-of-sight detection device 11 are obtained. A visual field extraction circuit 12c for extraction is provided.

  Further, the vehicle is equipped with a navigation device 13, and this navigation device 13 includes a map matching method or a vehicle position detection means for detecting the current position of the vehicle such as GPS and road map information. The road map information around the own vehicle detected by the vehicle position detection means is displayed on the display device 13a, and the current position of the own vehicle and the road map information around the current position of the own vehicle are output to the steering scene determination device 14. To do. This steering scene determination device 14 is based on the road map information of the host vehicle, and there is a curved road such as a curve, a right turn point, and a left turn point in front of the host vehicle, and whether or not the host vehicle needs to be steered. Judging.

  The determination result of the steering scene determination device 14 and the determination result of the gaze target determination device 12 are input to the presentation information determination circuit 15, and the presentation information determination circuit 15 needs to perform steering from the steering scene determination device 14. Judgment whether to issue an obstacle alarm or display a route based on whether it is in the situation, the driving skill level of the driver, the driving load of the driver, etc., and depending on the result, the route presentation device 16 or the obstacle Only one of the object alarm devices 17 is activated.

  The route presentation device 16 clearly indicates to the driver the optimum travel route for enabling traveling with stable vehicle behavior when traveling on this curved route. For example, in the display device 13a of the navigation device 13, instead of the road map information around the host vehicle, the optimum route of the host vehicle on the curved road is displayed in an enlarged manner, or the host vehicle is used by using a head-up display (HUD). The route display is performed by directly superimposing the optimal route to be traveled on with the scenery in front.

The optimum route of the host vehicle includes, for example, road map information from the navigation device 13 including road information indicating the shape of a curved road such as a turning radius of the curved road, and road map information from the navigation device 13. Based on the acquired curve information, the traveling speed of the host vehicle, the steering angle of the host vehicle, and the like, an ideal route on which the host vehicle can travel with a stable vehicle behavior is estimated by a known procedure, This may be displayed as the optimum route.
The obstacle alarm device 17 issues an obstacle alarm when the presentation information determination circuit 15 determines that an obstacle alarm should be issued.

Next, operation | movement of the driving assistance apparatus 100 is demonstrated based on the flowchart of FIG.
The navigation device 13 detects the current position of the host vehicle, extracts road map information around the current position of the host vehicle, displays the road map information on the display device 13a, and allows the driver to recognize the current position of the host vehicle. At the same time, the current position of the host vehicle and road map information around the host vehicle are output to the steering scene determination device 14. Based on the road map information, the steering scene determination device 14 determines whether there is a curved road ahead of the host vehicle and it is necessary to perform steering (step S1). When it is not necessary to perform steering, it is not necessary to display a route because the vehicle is running substantially straight. In addition, since the vehicle is traveling substantially straight and the driver needs to pay attention to only the obstacle, it is determined that an obstacle alarm is not required, and the process is terminated.

On the other hand, the line-of-sight detection device 11 detects the driver's line-of-sight direction from the captured image by, for example, imaging the driver's eyeball with the eye camera 11a (step S2).
Subsequently, the gaze target determination device 12 extracts a gaze point distribution that is a distribution of the driver's gaze point (step S3). Usually, when recognizing an object, a person recognizes the object by moving his / her line of sight on the object to be recognized. As a result, a gaze point, which is a point where the line of sight stays in the process until the driver recognizes the target, is distributed on the target. Therefore, the front camera 12a captures a road ahead of the host vehicle, and performs image processing according to a known procedure to detect an obstacle on the front side of the host vehicle, a road white line, a guardrail, a curbstone, or the like as an extracted object. The driver's line of sight is extracted from the detected extracted objects on the road by superimposing the captured image of the scenery in front of the vehicle and the driver's line-of-sight direction per predetermined time calculated in step S2. The extracted object is detected and used as a gaze target, and the gaze of the driver who is focused on at least one gaze object in front of the host vehicle is poured into each gaze target. The viewpoint distribution is set as a gaze point distribution, which is extracted by the gaze point distribution gaze circuit 12b. For example, as shown in FIG. 5, a point at which the line of sight stays is detected as a gazing point from the change state of the driver's line of sight in the forward scenery during a predetermined time. Then, the extracted object that overlaps the gazing point is recognized as a gazing target, and the distribution of the gazing point with respect to each gazing target is further detected.

For example, as shown in FIG. 5, when there is an obstacle on the traveling path of the host vehicle, the driver recognizes the obstacle by moving his / her line of sight, so that the gazing point appears on the obstacle. Also, when driving on a curved road, the driver recognizes the position of the vehicle in the driving lane by recognizing the position of the guardrail and the road white line, etc. Since the vehicle travels on a curve by selecting a route that can be traveled by steering, a gaze point appears on the guardrail or the white road line.
Note that the extracted object may be detected by a known procedure based on, for example, a luminance change state in the captured image.

  When the gaze point distribution is detected in this way, the process proceeds to step S4. Next, the gaze target determination device 12 extracts a visual field region that is the width of the driver's field of view. That is, when a person recognizes the environment, he / she usually moves his / her line of sight to the entire field of view, but the range of this field of view changes depending on driving skill, driving load, and the like. Therefore, in the present embodiment, the visual field region extraction circuit 12c of the gaze target determination device 12 is shown in FIG. 6 based on the driver's gaze direction at the predetermined time detected by the gaze detection device 11 in step S2. As described above, the vertical viewing angle (FIG. 6A) and the horizontal viewing angle (FIG. 6B) of the driver are extracted, and the vertical viewing area obtained from the maximum value and the minimum value of the viewing angle The left and right visual field regions obtained from the left and right end angles in the left and right direction are extracted.

  In step S5, it is determined whether or not the number of gazing points is large or small in the driver's gazing point distribution with respect to the individual gazing objects ahead of the host vehicle calculated in the gazing target determination device 12 in step S3. In other words, when a driver recognizes an object, usually, the higher the driving skill level of the driver and the lower the driving load given to the driver, the lower the point of gaze for this object. It tends to decrease. Therefore, in the present embodiment, the gazing point number ratio (= occupied area / (Point of gazing point) is calculated, and when the gazing point number ratio is equal to or less than a preset gazing point number ratio threshold value P, it is determined that the number of gazing points for the gazing target is large. And when there are one or more points of interest below the threshold P among the attention point ratios for each object to be watched, it can be predicted that the driver's driving skill level is low or the driving load is high. Therefore, it is determined that there is a high possibility that the driver's gaze target is an obstacle, and the process proceeds to step S6.

  On the other hand, when the gaze point ratio with respect to the gaze object is larger than the threshold value P of the gaze point ratio, it is determined that the number of gaze points with respect to the gaze object is small. When the gazing point ratio for all the gazing objects is larger than the gazing point ratio threshold P, it is determined that the driver's driving skill level is high or the driving load is low. The process proceeds to S11.

In addition, what is necessary is just to estimate the occupation area in the captured image about the extraction object extracted at the said step S3.
Here, the level of the driver's driving skill is calculated when the gaze point ratio is calculated from the number of gaze points for each gaze object and the gaze number ratio of any gaze object is equal to or less than the threshold value P. However, the present invention is not limited to this. For example, the number of gazing points for a plurality of gazing objects is not limited to this. Whether the level of the driving skill of the driver is low or the driving load is high may be estimated from the distribution state of the number of gazing points with respect to the gazing target such as the average value of the ratio.

  In step S6, the driver's gaze target is likely to be an obstacle, but the driver's driving skill level is low or the driving load is high, and the obstacle ahead of the host vehicle is Since there is a possibility that attention to an object may be insufficient, it is determined whether or not an object exists outside the driver's field of view. In other words, the driver's level of driving skill is lower and the driver's load is higher, so the visual field area becomes narrower and the possibility of overlooking the presence of an obstacle outside the visual field area increases. Therefore, in this embodiment, whether or not to issue an obstacle alarm is determined according to whether or not there is an obstacle outside the visual field area of the driver detected in step S4. In other words, when an obstacle exists outside the driver's field of view, the driver's driving load is high or the driving skill level is predicted to be low, and there is an obstacle outside the field of view. Since it is predicted that the presence of an obstacle outside the visual field area may be overlooked, the process proceeds to step S7, where the obstacle alarm device 17 is activated to issue an obstacle alarm. On the other hand, when there is no obstacle outside the field of view, the obstacle is included in the driver's field of view, and it is predicted that the driver has recognized all the obstacles in front of the host vehicle. Therefore, it is determined that no obstacle alarm is required and the process is terminated. That is, the obstacle warning is not issued.

  On the other hand, in step S11, when it is determined that the gaze point ratio is larger than the threshold value P and the number of gaze points on the gaze target is small, the driver's driving skill level is high or the driving load is low. Therefore, it is determined whether or not the plurality of gaze objects extracted in step S3 include a gaze target corresponding to preset “clue” information. In other words, as the driving skill level is higher and the driving load is lower, in addition to acquiring information about obstacles on the route, the driver acquires information that is a clue to follow an appropriate route. To do. That is, information such as road white lines, guardrails, curbs, reflectors, vehicles up to several vehicles, etc. is acquired. Therefore, in the present embodiment, the object to be watched, such as the characteristics for identifying the road white line, the guard rail, the curb stone, the reflector, the vehicle up to several vehicles, etc., is the road white line, the guard rail, the curb, Information that can identify the reflector or the vehicle is set in advance as clue information and stored in a predetermined storage area. Specifically, as the clue information, for example, “a width of about 150 mm” or “edges on both sides of the white line” is set as the clue information. For the curb, “step difference of about 200 mm in height on the left side of the host vehicle” or “having a color different from the road surface” is set as clue information.

  Then, based on the gazing point distribution detected by the gazing point distribution extraction circuit 12b, it is detected whether there is a gazing target object corresponding to the clue information, and the number of the gazing target objects corresponding to the clue information is calculated. For example, in the case of FIG. 5, a total of six gazing point distributions including two gazing point distributions corresponding to two obstacles on the traveling lane and four gazing point distributions corresponding to road white lines, guardrails, etc. That is, there are gaze objects, of which four correspond to the clue information, and thus the number of gaze objects corresponding to the clue information is four.

When the gaze point distribution corresponding to the clue information is one or zero, it is determined that the driver's gaze target is not an object that can be a clue but an obstacle, and the process proceeds to step S6. When an obstacle exists outside the visual field area, an obstacle alarm is issued.
On the other hand, if at least two or more gaze objects corresponding to the clue information are included, the driver has acquired the cue information, so the gaze object is predicted to be an object that can be a clue, It is determined whether the driver's visual field calculated in step S4 is wide. That is, even if the driver has a high driving skill level or the driving load is low, the visual field region may be narrowed. In such a case, by actively providing the clue information, that is, the information on the optimum route, the route on which the host vehicle should travel becomes clear and stable driving can be achieved. So, in this embodiment, when the driver | operator's visual field area is narrow, it transfers to step S13 and presents route information.

  Whether or not the visual field area is narrow is determined by the following procedure. First, the visual field area when traveling in the past on the road corresponding to the steering scene calculated in step S1 is collected in advance as travel history information. Then, based on the travel history information, an average value of the visual field area when traveling in the past on the road corresponding to the steering scene is calculated, and this average value is compared with the current visual field area for determination. That is, when the current visual field area is smaller than the average value of the past visual field areas, it is determined that the visual field area is narrow.

When it is determined that the visual field area is narrow, the process proceeds to step S13, where the visual field area is narrow and it is determined that route presentation is necessary, and information is presented by the route presentation device 16. For example, as shown in FIG. 7, in the display device 13a of the navigation device 13, an image clearly showing the optimum route of the host vehicle is displayed on the curved road instead of the normal road map around the host vehicle. The head up display displays the optimum route superimposed on the scenery in front of the vehicle.
On the other hand, if the field of view is not narrow, the driver knows the optimum route of the vehicle sufficiently and determines that route presentation is unnecessary, and ends the processing without performing information presentation.

Next, the operation of the first embodiment will be described.
The gaze detection device 11 detects the driver's gaze direction at a predetermined cycle, and the gaze target determination device 12 superimposes the driver's gaze direction and the captured image of the front camera 12a, and distributes the driver's gaze point. Is detected. Further, the steering scene determination device 14 determines whether or not the steering situation is in a state where steering is performed based on the road map information from the navigation device 13.

  At this time, when the driver is in a state of steering and the number of gazing points on the gaze target to which the driver's line of sight is directed is large (steps S1 to S4), the driving skill level of the driver is low. Or because it is predicted that the driving load is high, the driver's gaze target is predicted to be an obstacle, and the process proceeds from step S5 to step S6, where the obstacle is outside the driver's field of view target area. Determine if an object exists. If there is no obstacle outside the visual field area, the obstacle alarm is not issued.

  Therefore, it is possible to avoid giving the driver annoyance by issuing an obstacle alarm even though the driver recognizes a gaze target in front of the host vehicle, that is, an obstacle or the like. Also, when the driver's driving skill level is low or when the driver's driving load is high, an obstacle warning is issued, and in some cases, attention to obstacles ahead of the host vehicle However, as described above, when there is a gaze target in the driver's field of view and it can be considered that the driver is sufficiently aware of the gaze target Since the obstacle alarm is not issued, it is possible to avoid adversely affecting the driver by issuing an unnecessary obstacle alarm. At this time, although it is a steering scene, the route display will not be performed, but in this case, the driver's driving skill level is low or the driver's driving load is high, It is predicted that the driver cannot afford to search for a suitable route for smooth route selection and easy steering.Therefore, there is no problem even if the route is not displayed. It is possible to avoid removing the driver's attention to the obstacles by displaying an unnecessary route while watching the vehicle.

  On the other hand, if an obstacle exists outside the driver's field of view, the process proceeds from step S6 to step S7, and an obstacle alarm is issued. Therefore, when the driver's driving skill level is low or when the driver's driving load is high, there is a high possibility that the obstacle will be overlooked. When there is an obstacle that is not recognized by the driver, an obstacle alarm is issued, so that the driver can be surely recognized of the presence of the obstacle.

On the other hand, for example, when the driver's driving skill level is relatively high, or when the driver's driving load is relatively low, the driver reliably recognizes the presence of the gaze target and Since the number of times that the line of sight is directed toward the object is also relatively small, the number of points of gaze at the object to be watched is reduced.
For this reason, when the ratio of the number of gazing points with respect to all the gazing objects becomes larger than the threshold value P, the process proceeds from step S5 to step S11. At this time, the driver turns a curved road such as a road white line, a guardrail, or a curbstone. If the information of the clues for traveling is acquired, the process proceeds to step S12. If the driver's visual field at this time is predicted to be narrower than before, the process proceeds to step S13. Then, route information presentation is performed to display the optimum route of the host vehicle.

  In other words, the driver has acquired information on the clues for driving along a road such as a road white line, guardrail or curb, and the driver is not looking for obstacles on the route for safety confirmation. On the other hand, when it is predicted that a suitable route for realizing smooth route selection and easy steering is expected, route information presentation for displaying the optimum route of the host vehicle is performed. Therefore, the driver can clearly recognize the optimum route on which the host vehicle should travel by referring to the displayed route information, and the driver can perform more stable driving behavior.

  At this time, even if there is an obstacle, an obstacle alarm is not issued, but in this way, the driver wants to present the route information rather than the obstacle alarm. It can be regarded as an obstacle that is fully recognized or does not require much attention, so there is no problem even if an obstacle alarm is not issued, and by issuing an obstacle alarm On the contrary, it can be avoided that the driver feels bothered.

On the other hand, if the driver's field of view is the same as or wider than before, the smooth field will be the same field of view as when driving on a point corresponding to this steering scene before traveling on a curved road. Since it is predicted that the route suitable for realizing selection and easy steering operation is sufficiently recognized, it is predicted that there is no need to display the route, and the processing is terminated and the route display is not performed. .
Therefore, when the driver has a relatively low driving load or has a high driving skill level, he / she is fully aware of the route when traveling in the front steering scene, and does not require a route display. By displaying the information, it is possible to avoid giving the driver an annoyance.

  As described above, in the first embodiment, in the driving support device, whether the driver is looking for an obstacle on the route for safety confirmation when the steering is being performed. Or, determine whether you are looking for a suitable route to achieve smooth route selection and easy steering, and if you are looking for a route, only display the route and look for obstacles Occasionally, since an obstacle alarm is issued only when there is an unrecognized obstacle, it is possible to provide information that matches the information desired in the situation where the driver steers, and it is bothersome. It can be realized without giving a feeling.

  In particular, the driver should pay attention to the obstacle when an obstacle warning is issued, and pay attention to the route display when the route is displayed. become. When only one of these obstacle warnings and route indications is performed, as shown in FIG. 8, the amount of attention that the driver pays for the obstacle warnings or route indications is the driving amount. However, if both an obstacle warning is issued and a route is displayed, it is necessary to pay attention equivalent to the sum of the warnings. In some cases, such as when the load is relatively high, the total amount of attention paid by the driver to the obstacle alarm and route display exceeds the driver's allowable range. The driver may not be able to pay sufficient attention to the route display, and the amount of attention to the obstacle may be reduced, or the amount of attention to the route display may be reduced. Driving by doing Also enough attention perform the assistance in some cases or not used effectively or not paid.

However, as described above, since only one of the obstacle alarm and the route display is presented, the driver can pay sufficient attention to both the obstacle alarm and the route display. Driving assistance.
Further, when extracting the gaze point distribution, based on a method in accordance with a general driver's line-of-sight distribution behavior based on an image captured in front of the host vehicle captured by the front camera 12a and the line-of-sight direction of the driver. Since the gaze target is determined, the gaze behavior that changes according to the driving skill level of the driver and the driving load imposed on the driver can be properly captured.

  At this time, in consideration of how to distribute the line of sight to the driver's gaze target, the visual field area, etc., the driver is in a state of wanting an obstacle alarm or in a state of wanting to display a route. Since it is determined whether or not to provide such information, it is possible to make a suitable determination from time to time according to the level of the driving skill of the driver and the driving load of the driver. Thus, it is possible to provide information more accurately and to provide driving assistance effectively without giving a sense of inconvenience.

Further, since the driver's gaze object is determined based on the driver's gaze behavior, the driver's gaze object can be detected easily and accurately.
Further, since it is determined whether there is a curved road on the basis of the road map information around the host vehicle held by the navigation device 13, it is easily and accurately determined whether the vehicle is in a steering situation. can do.

  In the first embodiment, the route presentation device 16 and the obstacle alarm device 17 correspond to driving support means, the route presentation device 16 corresponds to route information providing means, and the obstacle alarm device 17 is an obstacle. It corresponds to the information provision means. 4 corresponds to the steering scene detection unit, the processing from step S2 to step S5 and step S11 corresponds to the gaze target determination unit, and the processing of step S6 and step S12 corresponds to the provided information determination unit. It corresponds.

4 corresponds to the line-of-sight direction detection means, the front camera 12a corresponds to the front imaging means, the process of extracting the extracted object in step S3 corresponds to the object extraction means, and the process of step S5 Corresponds to the gaze ratio detection means, and the processing in step S11 corresponds to the clue count detection means.
Further, in step S5 in FIG. 4, the process of detecting the occupied area on the captured image of the extracted object to which the driver's line of sight is directed corresponds to the occupied area detecting means, and the process of measuring the number of gazing points is gazing. It corresponds to the point measuring means.
Further, the process of step S4 in FIG. 4 corresponds to the visual field area estimation means, and the process of step S6 corresponds to the obstacle detection means outside the visual field area.

Next, a second embodiment of the present invention will be described.
FIG. 9 is a functional configuration diagram of the driving support device 200 according to the second embodiment.
As shown in FIG. 9, the driving support device 200 according to the second embodiment includes a line-of-sight detection device 11 used in the first embodiment, a front camera 21 that images a road ahead of the host vehicle, Based on the captured image of the front camera 21 and the detection result of the line-of-sight detection device 11, a gaze target determination device 22 that detects the presence / absence and position of the gaze target object being watched by the driver, the visual field region of the driver, and the like And an image processing device 23 that performs image processing on an image captured by the front camera 21 to detect a road linear shape ahead of the host vehicle, a determination result of the gaze target determination device 22, and processing in the image processing device 23. Provided is a presentation information determination circuit 24 that determines whether to notify alarm information and display route information based on the result, and the route presentation device 16 and obstacle alarm device 17 used in the first embodiment. There.

  The gaze target determination device 22 in the second embodiment detects an object on the road as an extracted object in the captured image of the front camera 21 as in the first embodiment, and also detects the line-of-sight detection device. 11 is a time in which the driver's line of sight is distributed to the gaze object, that is, the gaze target, with respect to the extracted object by superimposing the gaze direction detected in 11 and the captured image detected by the front camera 21. A gaze time extraction circuit 22a that extracts a gaze time and a visual field region extraction circuit 22b equivalent to the visual field region extraction circuit 12c in the first embodiment are provided.

FIG. 10 is a flowchart showing the operation of the driving support device 200.
The front camera 21 images a road ahead of the host vehicle at a preset cycle. The image processing device 23 performs image processing on the captured image of the front camera 21, and detects the road shape in front of the host vehicle. For example, an edge extraction process is performed by a known procedure to detect a road white line, and a road shape is detected from the shape of the road white line. Then, it is predicted whether or not the driver needs to steer from the road shape, that is, whether or not there is a curved road ahead of the host vehicle (step S21). If it is not predicted, the process is terminated as it is, but if it is predicted that a situation that requires steering is predicted, the process proceeds to step S22.

  In this step S22, the line-of-sight detection device 11 detects the driver's line-of-sight direction in the same procedure as the process of step S2 in the first embodiment, and then the process proceeds to step S23, and the gaze time extraction circuit 22a Thus, the extracted object on the road is detected and the driver's gaze time for the extracted object is calculated. Here, normally, when a person recognizes an object, it recognizes it by moving a line of sight on the object. As a result, a gazing point, which is a point at which the line of sight stays in the process until the driver recognizes the object, is distributed on the object for a certain time. Therefore, in the present embodiment, at least one or more included in the landscape ahead of the vehicle by superimposing the driver's line-of-sight direction for a predetermined time detected by the line-of-sight detection device 11 on the captured image of the front camera 21. In the line of sight of the driver poured into the target object, the dwell time of the gaze point distributed to each target object is extracted, and this is extracted as the gaze time.

  Next, the process proceeds to step S24, where the visual field area extraction circuit 22b extracts the driver's visual field area in the same procedure as the visual field area extraction circuit 12c in the first embodiment, and then the process proceeds to step S25. It is determined whether or not the driver's gaze time for each gaze object detected in (1) is long. That is, when the driver recognizes an object, the gaze time is usually shorter as the driver's driving skill level is higher, and the gaze time is shorter as the driving load given to the driver is lower. It is in. Therefore, in this embodiment, a gaze time ratio (= occupied area / gazing time) is calculated based on the occupation area of each gaze object in the captured image of the front camera 21 and the gaze time for this gaze object. When the gaze time ratio is smaller than the preset threshold value Q, it is determined that the gaze time for the gaze object is long. Then, the gaze time ratio is calculated for each gaze object, and when any one of the gaze time ratios is smaller than the threshold value Q, the gaze time for the gaze object is long, that is, the driver's driving Since it is predicted that the skill level is low or the driving load on the driver is high, the driver's gaze target is likely to be an obstacle, and the process proceeds to step S26.

  Here, when any one of the gaze time ratios for each gaze object is smaller than the threshold value Q, the level of the driver's driving skill is low or the driver's Although the driving load is high and the gaze target is likely to be an obstacle, the process proceeds to step S26. However, the present invention is not limited to this. For example, the average gaze time ratio for a plurality of gaze objects Whether the driver's driving skill level is low or the driving load is high may be estimated from the distribution state of the gaze time ratio such as a value.

  In the process of step S26, an obstacle exists outside the driver's field of view in the captured image of the front camera 21 in the same procedure as the process of step S7 of FIG. 4 in the first embodiment. If there is an obstacle, the process proceeds to step S27, where the obstacle alarm device 17 is activated to issue an obstacle alarm regarding the obstacle that exists outside the driver's field of view.

  In other words, the driver's field of view in a state where the gaze time for the gaze object is relatively long and the driver's driving skill level is low or the driver's driving load is predicted to be high. If there is an obstacle outside, the driver predicts that the driver may not be aware of the obstacle and issues an obstacle warning. At this time, although it is a steering scene, the route display is not performed, but as described above, the driver is in a state where the driving skill level is low or the driving load is high, and the driver Because it is predicted that no action will be taken such as smooth route selection or route selection to realize easy steering, there is no problem even if the route display is not performed, along with the issuance of obstacle warning By displaying the route, it is possible to avoid the driver's attention being dispersed.

Conversely, when there is no obstacle outside the driver's field of view (step S26), the processing is terminated as it is and no obstacle information is issued. In other words, since there are no obstacles outside the driver's field of view, the driver can be regarded as having recognized all obstacles ahead of the host vehicle. Therefore, by issuing an alarm even though the driver recognizes an obstacle, the driver may feel annoyed or distract attention from an obstacle that has already been recognized. Can be avoided.
On the other hand, when it is determined in the process of step S25 that the gaze time for all the gaze objects is equal to or greater than the threshold value Q, the gaze time for the gaze object is short, that is, the level of the driver's driving skill is low. It is determined that the driving load is high or the driving load on the driver is low, and the process proceeds to step S31.

  In step S31, the presentation information determination circuit 24 determines whether or not the driver's gaze target extracted in step S21 includes a gaze target corresponding to the clue information. This determination is the same as in the first embodiment, and clue information is held in advance, and determination is made based on this. When there is a target object corresponding to the clue information, the number of target target objects is counted, and as a result of gaze detection in a predetermined time, when at least two target objects correspond to the clue information, driving is performed. It is determined that the person's gaze target is an object that can be a clue, and the process proceeds to step S32. Then, in the same procedure as the process of step S12 of FIG. 4 in the first embodiment, the current driving is performed based on the average value of the visual field area when traveling on the road corresponding to the steering scene in the past. It is determined whether or not the driver's field of view is narrow, and when it is predicted that the field of view is narrow, the process proceeds to step S33, and the route presentation device 16 is operated to display the route to the driver. .

  Therefore, in a steering situation in which it is determined that the gaze time for the gaze object is relatively short and the driver's driving skill level is relatively high, or the driver's driving load can be considered small. When the driver is gazing at an object corresponding to multiple pieces of clue information, the driver can select a smooth route and perform easier steering than searching for an obstacle on the route for safety confirmation. Therefore, since it is predicted that a suitable route is being searched for, the route display is performed, so that the driver becomes clear about the optimum route to travel and stable driving is achieved. At this time, the obstacle warning is not issued, but at this time, the driver is in a state where the driving skill level is high or the driving load is low, and the driver is acquiring the clue information than the obstacle. Is in a state where it can be assumed that it is fully aware of obstacles, so there is no problem even if an alarm is not generated. On the other hand, the driver fully recognizes obstacles and acquires clue information. It is possible to avoid giving the driver a sense of incongruity by issuing an obstacle warning.

  Therefore, the second embodiment can also obtain the same operational effect as the first embodiment, and in the second embodiment, the driver's gaze time for the gaze object is detected. However, since the determination of information presentation is performed based on this, as compared with the case where the determination of information presentation is performed based on the gaze point distribution as in the first embodiment, the information presentation is performed more easily. Can do. Moreover, since the road shape ahead is detected based on the image captured by the front camera 21, it can be realized without providing a navigation device, and the number of components can be reduced accordingly. It can be realized at low cost.

In the second embodiment, the route presentation device 16 and the obstacle alarm device 17 correspond to driving support means, the route presentation device 16 corresponds to route information providing means, and the obstacle alarm device 17 is an obstacle. It corresponds to the information provision means.
Further, the process of step S21 in FIG. 10 corresponds to the steering scene detection means, the processes of steps S22 to S25 and step S31 correspond to the gaze target determination means, and the processes of steps S26 and S32 correspond to the provided information determination means. is doing.

Further, the process of step S22 corresponds to the line-of-sight direction detection means, the front camera 21 corresponds to the front imaging means, the process of detecting the extracted object in step S23 corresponds to the object extraction means, and the process of step S25 is the gaze ratio. Corresponding to the detection means, the process of step S31 corresponds to the clue hit count detection means.
Further, in the process of step S25 of FIG. 10, the process of detecting the occupied area on the captured image of the extracted object to which the driver's line of sight is directed corresponds to the occupied area detecting means, and the gaze time for the extracted object in step S23 The process of measuring S corresponds to the gaze time measuring means, the process of step S24 corresponds to the visual field area estimation means, and the process of step S26 corresponds to the obstacle detection means outside the visual field area.

Further, the camera 21 corresponds to a traveling road shape imaging means, and the image processing device 23 corresponds to a traveling road shape detection means.
In each of the above embodiments, when the driver is not in a situation where steering is necessary, the processing is terminated and the obstacle alarm is not issued. However, the present invention is not limited to this. When the person is not in a situation that requires steering, only the obstacle alarm device may be activated in accordance with the presence or absence of the obstacle.

  Further, in each of the above embodiments, the case has been described in which one of the route display device and the obstacle alarm device is selected and information is presented by only one of them, but the present invention is limited to the route display device and the obstacle alarm device. For example, a parking assist device of the kind that instructs the driver to perform parking procedures, a rear side warning device that notifies the presence of obstacles on the back and side, the speed of approach to the curve is exceeded The present invention can also be applied to a combination of driving support devices that present other information, such as a curve approach speed warning device that notifies when the vehicle is driving and a lane departure warning device that notifies the lane departure. Moreover, it is not restricted to two driving assistances, You may make it select one driving assistance apparatus operated from three or more driving assistance apparatuses.

It is a block diagram which shows the function structure of the driving assistance device of this invention. It is a functional block diagram which shows the more detailed functional structure of FIG. It is a figure where it uses for description of a gaze detection apparatus. It is a flowchart which shows an example of operation | movement of a driving assistance device. It is explanatory drawing for demonstrating gaze point distribution. It is explanatory drawing for demonstrating a visual field area | region. It is an example of the presentation content by a route presentation apparatus. It is explanatory drawing for demonstrating the attention amount requested | required of a driver | operator. It is a functional block diagram in a 2nd embodiment. It is a flowchart which shows an example of operation | movement of the driving assistance device in 2nd Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Gaze detection apparatus 11a Eye camera 11b Image processing apparatus 12 Gaze object determination apparatus 12a Front camera 12b Gaze point distribution extraction circuit 12c Field of view area extraction circuit 13 Navigation apparatus 14 Steering scene determination apparatus 15 Presentation information determination circuit 16 Path presentation apparatus 17 Obstacle Alarm device 21 Front camera 22 Gaze target determination device 22a Gaze time extraction circuit 22b Field of view area extraction circuit 23 Image processing device 24 Presentation information determination circuit

Claims (12)

In the driving support device that provides a plurality of driving support means for providing driving support information to the driver, and the driving support means provides driving support information with different support contents,
Gaze target discriminating means for detecting the driver's gaze target and discriminating its type;
Based on the type of the gaze target determined by the gaze target determination unit, the driver assistance information currently required by the driver is estimated, and the driving support information required by the driver is selected from the driving support units. Provided information determining means for operating only the driving assistance means to be provided, and a driving assistance device comprising: Route information providing means for providing route information indicating an appropriate route when traveling on the curved road when traveling on a curved road;
Obstacle information providing means for providing obstacle information around the vehicle;
Gaze direction detecting means for detecting the gaze direction of the driver;
Gaze target determining means for detecting the driver's gaze target from the gaze direction detected by the gaze direction detecting means and determining the type thereof;
Steering scene detection means for detecting whether the vehicle is in a situation of traveling on a curved road,
Based on the type of the gaze target determined by the gaze target determination unit and the detection result of the steering scene detection unit, which information is currently required by the driver among the route information and the obstacle information is determined. Driving assistance comprising: providing information determining means for operating only the means for providing information that the driver currently needs out of the route information providing means and the obstacle information providing means. apparatus.   Whether the gaze target discriminating means is an object that can be used as a clue to determine the driver's route when traveling along the curved path, or whether the driver's gaze target is an obstacle on the travel path of the host vehicle The driving support apparatus according to claim 2, wherein: Forward imaging means for imaging the front of the vehicle;
An object extraction means for extracting an object on the traveling road of the host vehicle in the front image picked up by the front image pickup means;
The gaze target discriminating means is an extraction extracted by the object extraction means based on the gaze direction detected by the gaze direction detection means when a point on the captured image to which the driver's gaze is directed is taken as a gaze point. Gaze degree detection means for detecting a distribution situation of the gaze point with respect to the object and detecting a driver's gaze degree with respect to the extracted object based on the distribution state;
A clue for detecting the number of extracted objects that satisfy preset clue information for determining whether the extracted object is an object that can be the clue among the extracted objects to which the driver's line of sight is directed And a corresponding number detection means,
From the gaze degree detected by the gaze degree detection means, it is estimated that the driver's gaze degree with respect to the extracted object is small, and the number of clue hits detected by the clue hit count detection means is a predetermined number of clue hits. When the threshold value is greater than or equal to a threshold value, it is determined that the gaze target is an object that can be a clue,
When it is estimated from the gaze degree that the driver's gaze degree with respect to the extracted object is large, or when the number of clues is smaller than the threshold value, it is determined that the gaze target is the obstacle. The driving support apparatus according to claim 3, wherein The gaze degree detection means includes an occupied area detection means for detecting an occupied area on the captured image of the extracted object to which the driver's line of sight is directed,
Gaze number measuring means for measuring the number of gaze points on the extracted object to which the driver's line of sight is directed, and
The gaze number ratio (occupied area / number of gaze points), which is a ratio between the occupied area of the extracted object detected by the occupied area detection unit and the number of gaze points measured by the gaze number measurement unit, Calculated as a degree,
The gaze target determination unit determines that the driver's gaze degree with respect to the extracted object is large when the gaze point ratio is equal to or less than a preset threshold value, and the gaze ratio is greater than the threshold value. The driving support device according to claim 4, wherein the gaze degree is determined to be small. The gaze ratio detecting means includes an occupied area detecting means for detecting an occupied area on the captured image of the extracted object to which the driver's line of sight is directed.
Gaze time measuring means for measuring a gaze time for the extracted object to which the driver's line of sight is directed, and
A gaze time ratio (occupied area / gaze time), which is a ratio between the occupied area of the extracted object detected by the occupied area detection unit and the gaze time measured by the gaze time measurement unit, is calculated as the gaze degree. ,
The gaze target determination unit determines that the gaze degree of the driver with respect to the extracted object is large when the gaze time ratio is equal to or less than a preset threshold value, and the gaze time ratio is greater than the threshold value. The driving support device according to claim 4, wherein the gaze degree is determined to be small. Visual field region estimation means for estimating the driver's visual field region from the distribution state of the gazing point on the captured image;
A visual field region obstacle detection means for detecting whether an obstacle exists outside the visual field region estimated by the visual field region estimation unit,
The provided information determining means determines that the driver's gaze target is an object that can serve as the clue when the gaze target determination means detects that the steering scene is detected by the steering scene detection means. And when the visual field area estimated by the visual field area estimation means is narrower than a preset visual field area, the route information providing means is operated,
The gaze target determining means determines that the driver's gaze target is an obstacle on the travel path, and the out-of-view area obstacle detection means detects that an obstacle exists outside the view area. The driving support device according to any one of claims 4 to 6, wherein the obstacle information providing means is operated. A navigation device that holds road map information around the host vehicle,
The steering scene detection means is based on road map information around the host vehicle held by the navigation device.
The driving support device according to any one of claims 2 to 7, wherein it is detected whether or not the host vehicle is in a situation of traveling on a curved road. Traveling road shape imaging means for imaging the traveling road shape in front of the host vehicle;
Road shape detecting means for detecting the road shape in front of the host vehicle based on the captured image of the road shape imaging means,
3. The steering scene detection unit detects whether or not the host vehicle is traveling along a curved path based on the travel path shape detected by the travel path shape detection unit. The driving support device according to any one of 7. A traveling road shape detecting means for detecting a traveling bypass shape in front of the host vehicle based on a captured image of the front imaging means;
The said steering scene detection means detects whether the own vehicle is in the condition which drive | works a curved path based on the travel path shape detected by the said travel path shape detection means. The driving support device according to any one of 7. In the driving support method that provides a plurality of driving support information with different support contents to the driver,
The driver's gaze object is detected and its type is discriminated, and the driver assistance information currently required by the driver is estimated from the type of the gaze object, and the driving among the plurality of driving assistance information A driving support method characterized by providing only driving support information currently required by a person. In the driving support method that provides the driver with route information that represents the optimum route when driving on a curved road and obstacle information around the vehicle,
When the driver's gaze object is detected and its type is determined, and the host vehicle is traveling along a curved road, the route information and the obstacle are determined based on the determination result of the type of the gaze object. Guess what information you need,
Of the route information and the obstacle information, only the information estimated to be required by the driver is provided.

Images