JP5925163B2 - Driving assistance device - Google Patents

Description

  The present invention relates to a driving assistance device that assists in safe driving by detecting the line of sight of a driver of a vehicle.

  Conventionally, drivers who drive vehicles such as automobiles may lead to serious accidents by driving side-by-side, such as gazing at speedometers and car navigation systems in the car. Has been done. For example, Patent Document 1 discloses a device that detects the direction of the driver's line of sight and guides the driver to see the line of sight.

  On the other hand, a head-up display (HUD) that has been commercialized in recent years is a device that displays various information superimposed on a transmissive display in the direction of the user's line of sight, that is, displays various information while being superimposed on the user's field of view. It is a display device to be performed, and thereby, a speedometer, a car navigation system, and the like can be displayed in front of the vehicle which is the user's field of view.

JP 2010-126135 A

  However, when a display device that displays various types of information in a state of being superimposed on the user's field of view such as HUD is used, the user's line-of-sight direction is the front of the vehicle. Although it is not a side-view driving, watching attention to a speedometer, car navigation system, etc. significantly reduces attention to vehicle front information other than the contents displayed on the HUD due to the difference in focal length. There was a problem that could lead to a serious accident.

  The present invention has been made to solve the above problems, and even when a display device that displays various information in a state of being superimposed on the user's field of view, such as HUD, is used. It is an object of the present invention to provide a driving support device that guides a user's line of sight in an arbitrary appropriate direction that a person should watch closely.

  In order to achieve the above object, the present invention is a driving support device that is mounted on a vehicle and outputs driving support information from an output device provided in the vehicle, and detects a gaze direction and a gazing point of the driver. And a gaze target determination unit that determines a gaze target that the driver is gazing based on the driver's gaze direction and gaze point detected by the detection unit, and a driving operation of the vehicle. Based on a vehicle state determination unit that determines an operation state of the vehicle based on vehicle information, a gaze object determined by the gaze object determination unit, and a vehicle operation state determined by the vehicle state determination unit. Determining a degree that the driver's line of sight is not in an appropriate direction in the operating state of the vehicle, and determining a danger level according to the determined degree, and the danger situation It is determined whether or not to guide the driver's line of sight and gaze point according to the danger level determined by the fixing unit, and when it is determined to perform the guidance, the gaze object determination unit determines A gaze direction guidance processing unit that determines a guidance direction for guiding the driver's gaze and gaze point in the appropriate direction based on a gaze target, and determines a guidance method according to the danger level; and the gaze direction guidance An output control unit that outputs driving support information for directing the driver's line of sight and gaze point in the guidance direction determined by the line-of-sight direction guidance processing unit from the output device in a guidance method determined by the processing unit; It is characterized by that.

  According to the present invention, it is possible to accurately detect where the driver's line of sight is directed even when a display device that displays various information in a state of being superimposed on the user's field of view such as HUD is used. Since the driver can determine the gaze target that the driver is gazing at and guide the driver's gaze and gaze point in any appropriate direction that the driver should gaze at, the driver gazes more than necessary Or the focus is directed to a display device such as a car navigation system or a HUD, and it is possible to prevent a lack of attention in the direction of travel or forward.

1 is a block diagram illustrating a configuration of a driving support device in Embodiment 1. FIG. In Embodiment 1, it is an example of the correspondence table | surface which determined the danger level based on time T [second] when the driver | operator does not look at the road condition outside the vehicle in the traveling direction, and the vehicle state. 3 is a flowchart showing the operation of the driving support apparatus in the first embodiment. 6 is a diagram illustrating an example of a guidance display screen using arbitrary characters according to Embodiment 1. FIG. It is a figure which shows the example of a screen of the guidance display by an arrow. FIG. 10 is a block diagram illustrating a configuration of a driving support apparatus in a second embodiment. In Embodiment 2, it is an example of the correspondence table | surface which determined the danger level based on time T [second] when the driver | operator is not seeing the road condition outside a vehicle in a driving direction, and a vehicle state. 6 is a flowchart showing the operation of the driving support apparatus in the second embodiment. 10 is a diagram illustrating an example of a guidance display screen using an arbitrary character according to Embodiment 2. FIG. FIG. 10 is a diagram illustrating another example of a guidance display screen using arbitrary characters according to the second embodiment. FIG. 10 is a block diagram illustrating a configuration of a driving support device in a third embodiment. 10 is a flowchart showing the operation of the driving support apparatus in the third embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of the driving support apparatus according to Embodiment 1 of the present invention. This driving support device is mounted on a vehicle such as an automobile, and outputs driving support information from an output device provided in the vehicle, and includes a gaze detection unit 1, a gaze target determination unit 2, a vehicle state determination unit 3, A danger situation determination unit 4, a time acquisition unit 5, a line-of-sight direction guidance processing unit 6, a display control unit (output control unit) 7, and a display device (output device) 8 are provided.

  The line-of-sight detection unit 1 detects the direction of the driver's face and the movement of the eyeball (both eyes or one eyeball) while driving the vehicle, using one or more optical cameras, infrared cameras, eye cameras, or the like. Thus, the driver's line-of-sight direction and gaze point are detected. Then, the detected gaze direction and the gazing point of the driver are notified to the gaze target determination unit 2.

  The gaze target determination unit 2 calculates the distance from the driver's position to the gaze point that the driver is gazing from the driver's gaze direction and gaze point detected by the gaze detection unit 1, and The gaze target that the driver is gazing is determined from the gaze direction and the calculated distance to the gaze point.

  Then, the information on the determined gaze target is notified to the danger situation determination unit 4 and the gaze direction guidance processing unit 6. The distance from the driver's position to the object being watched can be obtained by estimating the distance of the depth of the object being watched from the convergence angle of the second eye. Since it is a technique, detailed description is omitted here.

  The vehicle state determination unit 3 acquires a gear signal, a vehicle speed signal, a back signal, and the like, and based on vehicle information corresponding to the driving operation of the vehicle, such as a traveling direction, a traveling speed, and whether or not the vehicle is stopped, Determine the operating state. Then, the determined operation state of the vehicle is notified to the dangerous situation determination unit 4.

  The dangerous situation determination unit 4 is based on the gaze target information determined by the gaze target determination unit 2 and the vehicle operating state determined by the vehicle state determination unit 3, and the driver's gaze direction during traveling When the driver is in a dangerous situation where the driver is not concentrating on driving the vehicle, such as facing the direction other than the driving direction, or the driver is gazing at the display content from the display device superimposed in the driving direction, That is, the degree to which the driver's line of sight is not in the proper direction to be turned in the vehicle operating state is determined, and the danger level is determined according to the determined degree.

  At this time, the danger level is determined according to the degree of time in which the driver's line of sight is not in the proper direction, or the time degree in which the driver's line of sight is not in the proper direction and the operation of the vehicle. Decide according to the state. For example, when the danger level is determined according to the degree of time when the driver's line of sight is not in the proper direction, the driver determines the road condition in the direction of travel outside the vehicle for a predetermined time (dangerous). (Determination time) Whether or not it has been viewed for more than a predetermined time (danger determination time), if the driver's line of sight is not in the appropriate direction to be turned in the vehicle operating state, the danger level for guidance is set. Then, the determined danger level is notified to the line-of-sight direction guidance processing unit 6.

  As for the danger level, for example, the level is set to 0 to 5, and if the time when the road direction outside the vehicle in the driving direction is not seen is less than a certain time, the danger level is 0. The longer you are not watching, the higher the danger level. Further, depending on the time during which the vehicle road condition in the traveling direction is not viewed, or the combination of the time during which the vehicle road condition in the traveling direction is not viewed and the operation state of the vehicle determined by the vehicle state determination unit 3 Set the level. The risk level is determined only by the degree of time that the driver's line of sight is not in the proper direction, and depending on the time degree that the driver's line of sight is not in the proper direction and the vehicle operating state. Either of the above may be adopted.

FIG. 2 is an example of a correspondence table in which the danger level is determined based on the time T [seconds] when the driver does not look at the road direction outside the vehicle and the vehicle state in the first embodiment.
Here, in order to simplify the explanation, only the vehicle speed V [km / h] is considered as the vehicle state.

  As shown in FIG. 2, no matter what the vehicle speed V is, if the time T when the driver is not looking at the road outside the vehicle is 0, the danger level is 0 and the vehicle speed V is 20 km / h or less. For example, the danger level is 0 even if the time T is 1 second, and the danger level is increased as the time T becomes longer thereafter. Even if the time T is as short as 1 second, the danger level is increased as the vehicle speed V increases, and the danger level is set to a high value of 4 when the vehicle speed V is faster than 80 km / h. And, as the time T becomes longer and as the vehicle speed V becomes faster, the danger level is set higher.

Although the description here assumes that the danger level is from 0 to 5, the risk level may be from 0 to 10 or may be further reduced.
In addition, when the video from the rearview mirror, the side mirror, and the rear camera video display device other than the back travel direction during back travel is being watched, the danger level may be set to zero.

  The time acquisition unit 5 receives a GPS signal by a GPS (Global Positioning System) receiver or the like, and acquires the current time. The acquired current time information is notified to the line-of-sight direction guidance processing unit 6.

  The line-of-sight direction guidance processing unit 6 determines whether or not to guide the driver's line of sight and gaze point according to the danger level determined by the danger situation determination unit 4. When it is determined that guidance of the line of sight and the gaze point is to be performed, based on the information on the gaze target determined by the gaze target determination unit 2 and the current time information acquired by the time acquisition unit 5, the driver's A display device that performs a gaze and gaze point guidance process, a guidance direction that induces the gaze and gaze point in an appropriate direction, and a guidance method according to the danger level are determined.

  Regarding the gaze and gaze point guidance method, the display device in the gaze direction or the display device that is gazing, that is, the display device such as a car navigation system arranged in front of the driver's seat, or the state superimposed on the driver's field of view The method of displaying a guidance display on a display device such as a HUD that performs display or the method of turning off the screen display of the display screen in the display device being watched is selected. In addition, when displaying a guidance display on a display device, the guidance display content is changed depending on the danger level. Further, the brightness of the display may be changed depending on the time. Further, the guidance method according to the danger level may be changed according to the time zone. Then, the determined guidance method is notified to the display control unit 7.

  The display control unit (output control unit) 7 is a guidance method determined by the line-of-sight direction guidance processing unit 6 and is a display device such as a car navigation system or a head-up display (HUD) in the direction of the driver's line of sight or the direction of the target object. (Guide direction determined by the line-of-sight direction guidance processing unit 6 by displaying arbitrary characters, images, and videos from the (output device) 8 or by erasing the displayed content (screen display of the display screen). Driving assistance information that directs the driver's line of sight and gaze point.

  In other words, the guidance method in this case is directed to the guidance direction in which the driver's line of sight and gaze point are determined in accordance with the danger level expressed using at least one of characters, images, and video. Either the driving support information to be replaced is displayed on the display device 8 or the screen display of the display device 8 is turned off.

  Then, by causing the driver's line of sight and gaze point to be directed in the guidance direction in this way, the driver can be prevented from gazing at the display device 8, or the driver's line of sight and gaze point can be moved forward or It can be arbitrarily guided in an appropriate direction.

Next, operation | movement of this driving assistance device is demonstrated using the flowchart shown in FIG. FIG. 3 is a flowchart showing the operation of the driving support apparatus according to the first embodiment.
First, when the vehicle key is turned ON (YES in step ST1), the line-of-sight detection unit 1 detects the driver's line of sight (step ST2), and the vehicle state determination unit 3 determines the vehicle state (step ST3), the time acquisition unit 5 acquires the current time (step ST4).

In step ST <b> 2, the line-of-sight detection unit 1 acquires the driver's face orientation and eyeball movement using an optical camera, an infrared camera, an eye camera, and the like, and notifies the gaze target determination unit 2.
In step ST3, the vehicle state determination unit 3 obtains a gear signal, a vehicle speed signal, and a back signal from the vehicle, and performs vehicle operation operations such as whether the vehicle is traveling speed, whether the vehicle is stopped, whether the vehicle is traveling in reverse, or the like. Based on the corresponding vehicle information, the operation state of the vehicle is determined and notified to the dangerous situation determination unit 4.

In step ST <b> 4, the time acquisition unit 5 receives a GPS signal by a GPS receiver or the like, acquires the current time, and notifies the line-of-sight direction guidance processing unit 6.
Next, the gaze target determination unit 2 determines the driver's line-of-sight direction and gaze point based on the driver's face orientation, eyeball movement, convergence angle of the two eyeballs, etc. acquired in steps ST2 and ST3. The object to be calculated is determined (step ST5). Then, the information on the calculated gaze direction and the determined gaze target is notified to the dangerous situation determination unit 4.

  The dangerous situation determination unit 4 determines that the driver's line-of-sight direction is other than the driving direction during driving from the line-of-sight direction calculated in step ST5 and the determined gaze target information and the vehicle operating state determined in step ST3. Or the driver is gazing at the display content of the display device 8 superimposed in the traveling direction, and determines whether or not the vehicle road condition in the traveling direction has not been seen for a certain period of time. From this, the danger level is determined (step ST6). Then, the determined danger level is notified to the line-of-sight direction guidance processing unit 6.

  The line-of-sight direction guidance processing unit 6 determines whether or not the danger level determined in step ST6 is a level for guiding the driver's line of sight / gaze point (step ST7). For example, if a danger level is 3 or more, a gaze / gaze point is guided, a predetermined threshold is set in advance as a gaze / gaze point guidance danger level, and the danger level is a predetermined threshold. Judgment is made based on whether or not the above. Needless to say, if the danger level is other than 0 (1 or more), guidance may be set.

  When the danger level is a level for guiding the driver's gaze / gaze point (in the case of YES in step ST7), the danger level determined in step ST6, the gaze direction calculated in step ST5, and the determination Based on the information on the gaze target and the current time information acquired in step ST4, a display device that performs gaze / gaze point guidance processing and any direction that guides the driver's gaze or gaze point, A guidance point guidance method is determined (step ST8). The determined display device, guidance direction, and guidance method are notified to the display control unit 7.

  Based on the guidance direction and guidance method notified from the gaze direction guidance processing unit 6, the display control unit 7 displays arbitrary characters, images / videos, or display contents being watched (display screen). Display control is performed so as to turn off the (screen display), and an instruction is given to the display device 8 (display device determined in step ST8) such as a car navigation system or HUD in the direction of the driver's line of sight or the direction of the gaze target (step ST8) ST9 to ST11).

  That is, when the guidance method determined in step ST8 is guidance display (in the case of YES in step ST9), the display device 8 is instructed to display arbitrary characters, images / videos for guidance display. If the guidance method is not guidance display but display erasure (NO in step ST9), the display contents displayed on the display device 8 (screen display of the display screen) are displayed. Is instructed to be erased (step ST11).

  4 and 5 are diagrams illustrating examples of guidance display screens according to the first embodiment. As the guidance display, for example, as shown in FIG. 4, an arbitrary character such as “Danger! Look ahead” or an arrow image as shown in FIG. 5 is displayed. It is displayed on a display device 8 such as a car navigation system or HUD in the direction of the object.

The display contents when characters as shown in FIG. 4 are displayed may be changed in accordance with the danger level so that the driver's attention is appropriately urged.
In addition, the image of the arrow shown in FIG. 5 shows the direction in which the driver's gaze / gaze point is desired to be guided. In this case, the arrow color is changed to yellow or red according to the danger level. It is recommended to change the display contents by flashing or flashing to prompt the driver's attention appropriately.
Furthermore, not only the display contents are changed according to the danger level, but if the danger level becomes high, for example, when the driver does not look forward even if guidance display is performed, the guidance display is turned off and the driver turns the display device. You may make it stop watching.

And the process after step ST2 is repeated until a danger level becomes less than a predetermined threshold value (in the case of NO of step ST7). The processes after step ST2 are repeated until the vehicle key is turned OFF (NO in step ST12). That is, even when the danger level is less than the predetermined threshold (in the case of NO in step ST7), the driver continues to process the gaze / gaze point of the driver and guides the gaze / gaze point.
Thereafter, when the danger level is less than the predetermined threshold value (in the case of NO in step ST7) and the vehicle key is turned off (in the case of YES in step ST12), the process is terminated.

  As described above, according to the first embodiment, a gaze point is detected from the driver's face direction, gaze direction, focal length, and the like using a camera or the like, and based on the driver's gaze direction and gaze point. By determining the gaze object that the driver is gazing at and guiding the driver's line of sight or gaze point in any appropriate direction that the driver should gaze at safely, It is possible to prevent the attention to the traveling direction and the forward direction from being lost because the line of sight and focus are directed to a display device such as a car navigation system or HUD more than necessary.

Embodiment 2. FIG.
FIG. 6 is a block diagram showing the configuration of the driving support apparatus according to Embodiment 2 of the present invention. In addition, the same code | symbol is attached | subjected to the structure similar to what was demonstrated in Embodiment 1, and the overlapping description is abbreviate | omitted. In the second embodiment described below, compared to the first embodiment, the vehicle surrounding detection unit 9, the sensor unit 10, and the surrounding situation determination unit 11 are further provided, and the situation of people and vehicles around the host vehicle is also taken into consideration. Thus, the danger level is determined or the line of sight is guided.

The vehicle periphery detection unit 9 acquires a vehicle periphery image using one or a plurality of front detection cameras, a back camera, a side camera, or the like, and notifies the surrounding state determination unit 11 of the acquired image.
The sensor unit 10 has one or more sensors such as millimeter wave radars and ultrasonic radars attached to the front, side, and rear of the vehicle body, and vehicles surrounding objects such as vehicles around the host vehicle, people, light vehicles, and motorcycles. The surrounding state determination unit 11 is notified of the presence / absence of (a vehicle external object around the vehicle) and the detection results such as the distance between the peripheral object (vehicle external object) and the host vehicle.

  The surrounding state determination unit 11 performs image analysis from an image around the vehicle captured from the vehicle periphery detection unit 9, and combines the image analysis result with the detection result acquired from the sensor unit 10, thereby generating a vehicle, a person, or a light vehicle. The presence or absence of objects outside the vehicle such as motorcycles and obstacles is determined. When the vehicle outside object is detected, the direction of the vehicle outside vehicle and the distance from the vehicle outside vehicle to the vehicle are calculated, the relationship between the vehicle outside object such as the direction and the distance and the vehicle is determined, Information about the relationship with the vehicle is notified to the danger situation determination unit 4.

  The dangerous situation determination unit 4 includes information on the gaze target determined by the gaze target determination unit 2, the operation state of the vehicle determined by the vehicle state determination unit 3, and an out-of-vehicle object determined by the surrounding status determination unit 11. The driver's line-of-sight direction is not in the driving direction or the driver is gazing at the display content from the display device superimposed in the driving direction. The degree to which the driver is in a dangerous situation that is not concentrating on driving the vehicle, i.e., the degree to which the driver's line of sight is not in the proper direction that should be directed in relation to the operating state of the vehicle and the relationship between the vehicle and the vehicle. Determine the risk level according to the determined degree.

  At this time, the danger level is determined according to the degree of time in which the driver's line of sight is not in the proper direction, or the time degree in which the driver's line of sight is not in the proper direction and the operation of the vehicle. Decide according to the state. For example, when the danger level is determined according to the degree of time when the driver's line of sight is not in the proper direction, the driver determines the road condition in the direction of travel outside the vehicle for a predetermined time (dangerous). Judgment time) or not, and whether or not the presence or dangerous state of an object outside the vehicle is recognized or confirmed for a predetermined time (danger determination time) or more, that is, a predetermined time (danger determination time) or more When the driver's line of sight is not in the proper direction that should be directed due to the operation state of the vehicle and the relationship between the object outside the vehicle and the vehicle, the danger level for guidance is set.

In addition, the length of a predetermined time (danger determination time) used as an index for risk level determination is changed (fluctuated) depending on the presence of an object outside the vehicle and a dangerous state, that is, depending on the relationship between the object outside the vehicle and the vehicle. The risk level may be increased when the host vehicle is in a dangerous state with respect to an object outside the vehicle around the vehicle by changing the risk level or changing the risk level itself.
Then, the determined danger level is notified to the line-of-sight direction guidance processing unit 6.

FIG. 7 is an example of a correspondence table in which the danger level is determined based on the time T [seconds] when the driver does not look at the road conditions outside the vehicle and the vehicle state in the second embodiment.
Here, in order to simplify the explanation, only the distance X [m] between the vehicle ahead and whether or not there is a vehicle is considered as the vehicle state.

  As shown in FIG. 7, when there is no vehicle ahead, or when the distance X to the vehicle ahead is 50 m or more, it is dangerous if the time T during which the driver does not look at the road outside the vehicle is zero. The risk level is increased as the level is 0 and the time T is longer. Even if the time T is short, if the distance X to the preceding vehicle is 10 m or 20 m, the danger level is set to a high value such as 4 or 3.

  The line-of-sight direction guidance processing unit 6 determines whether or not to guide the driver's line of sight and gaze point according to the danger level determined by the danger situation determination unit 4. When it is determined that guidance of the line of sight and the gaze point is to be performed, information on the gaze target determined by the gaze target determination unit 2, and an outside vehicle and the vehicle around the vehicle determined by the surrounding state determination unit 11 Based on the information on the relationship between the driver and the current time information acquired by the time acquisition unit 5, a display device that performs a driver's gaze and gaze point guidance process, and a guidance direction that guides the gaze and gaze point in an appropriate direction. Determine the guidance method according to the danger level.

FIG. 8 is a flowchart showing the operation of the driving support apparatus according to the second embodiment.
First, when the vehicle key is turned on (in the case of YES at step ST21), the line-of-sight detection unit 1 detects the driver's line of sight (step ST22), and the vehicle periphery detection unit 9 acquires an image around the vehicle. (Step ST23), the sensor unit 10 detects the presence or absence of an object around the vehicle (a vehicle outside the vehicle) (Step ST24), the vehicle state determination unit 3 determines the vehicle state (Step ST25), and the time acquisition unit 5 Acquires the current time (step ST26).

  In step ST <b> 23, the vehicle periphery detection unit 9 acquires a vehicle periphery image using a vehicle periphery detection camera such as a front detection camera, a back camera, or a camera again, and notifies the surrounding state determination unit 11 of the acquired image.

  In step ST24, the sensor unit 10 uses a sensor such as a millimeter wave radar or an ultrasonic radar attached to the front, side, or rear of the vehicle body, and surrounding objects such as a vehicle, a person, a light vehicle, and a motorcycle around the host vehicle. The presence / absence of (object outside the vehicle) and the distance between the surrounding object (object outside the vehicle) and the host vehicle are detected, and the detection result is notified to the surrounding state determination unit 11.

Steps ST25 to ST27 are the same as steps ST3 to ST5 in the flowchart shown in FIG.
Next, the surrounding situation determination unit 11 performs image analysis from the image around the vehicle acquired in step ST23, and based on the image analysis result and the detection result detected in step ST24, the surrounding vehicle When the presence or absence of a vehicle surrounding object such as a person, a light vehicle, a motorcycle, or an obstacle, and the presence of the vehicle surrounding object are confirmed, the distance and direction to the surrounding object are determined (step ST28). Then, the danger situation determination unit 4 is notified of information on these peripheral objects.

  The dangerous situation determination unit 4 includes information on the gaze direction calculated in step ST27 and the determined gaze target, the presence / absence of the vehicle peripheral target determined in step ST28, and the distance and direction to the target. From the information on the object and the vehicle operating state determined in step ST25, the display content of the display device 8 in which the driver's line-of-sight direction faces other than the traveling direction during traveling or the driver is superimposed on the traveling direction. And whether or not you have not seen the road conditions in the direction of travel for more than a certain period of time, and whether you have recognized or confirmed the presence or danger state of objects around the vehicle for a certain period of time, A danger level is determined from the determination result (step ST29). Then, the determined danger level is notified to the line-of-sight direction guidance processing unit 6.

  The line-of-sight direction guidance processing unit 6 determines whether or not the danger level determined in step ST29 is a level for guiding the driver's line of sight / gaze point (step ST30). Since the determination of the danger level is the same as that of the first embodiment, a detailed description thereof is omitted here, but the danger level is also determined in addition to the conditions for the presence or absence of a vehicle peripheral object (an external object around the vehicle). You may do it.

  When the danger level is a level for guiding the driver's gaze / gaze point (in the case of YES in step ST30), the danger level determined in step ST29, the gaze direction calculated in step ST27, and the determination Based on the information on the gaze target that has been determined, information on the peripheral target such as the presence or absence of the vehicle peripheral target determined in step ST28, and the distance and direction to the target, and the current time information acquired in step ST26 Thus, the display device that performs the gaze / gaze point guidance process, the driver's gaze / gaze point arbitrary direction, and the gaze / gaze point guidance method are determined (step ST31). The determined display device, guidance direction, and guidance method are notified to the display control unit 7.

The subsequent processing of steps ST32 to ST35 is the same as steps ST9 to ST12 of the flowchart shown in FIG.
In the second embodiment, since the presence / absence of a peripheral object of the host vehicle, the direction and distance of the target object, and the like can be determined, in addition to the screen examples shown in FIGS. 4 and 5, for example, FIG. As shown in (a) to (c), use letters such as “Look forward”, “Please keep the distance from the vehicle ahead”, “Danger! The distance from the vehicle ahead is approaching” There are also examples of guidance display screens that change the display according to the danger level, or display characters such as “the vehicle is changing from the right lane” on the display device 8 as shown in FIG. It is done.

  As described above, according to the second embodiment, in addition to the effects in the first embodiment, a gaze point is detected from the driver's face direction, line-of-sight direction, focal length, etc. using a camera or the like, and the driver In order to drive the driver safely, not only to determine the target of the vehicle, but also to acquire information about the vehicle's surroundings using devices that detect the vehicle's surroundings, such as cameras and sensors, and to take into account the surrounding conditions of the vehicle. By guiding the driver's line of sight and gaze point in any appropriate direction that should be watched, the driver will turn his / her line of sight and focus more than necessary to a display device such as a car navigation system or a HUD, thereby causing a dangerous state around the vehicle. Even when the vehicle is not recognized or confirmed, the driver can recognize the dangerous state around the vehicle and can prevent the driver from being cautioned about the direction of travel or forward.

Embodiment 3 FIG.
FIG. 11 is a block diagram showing the configuration of the driving support apparatus according to Embodiment 3 of the present invention. In addition, the same code | symbol is attached | subjected to the structure similar to what was demonstrated in Embodiment 1, 2, and the overlapping description is abbreviate | omitted. In the third embodiment described below, as compared with the first embodiment, instead of the display control unit (output control unit) 7 and the display device (output device) 8, an audio output control unit 12 and an audio output device such as a speaker are provided. 13. A light output control unit 14 and a light output device 15 are provided, and the line of sight is guided not by display but by sound or light.

  The line-of-sight direction guidance processing unit 6 determines whether or not to guide the driver's line of sight and gaze point according to the danger level determined by the danger situation determination unit 4. When it is determined that guidance of the line of sight and the gaze point is to be performed, based on the information on the gaze target determined by the gaze target determination unit 2 and the current time information acquired by the time acquisition unit 5, the driver's An output device that performs a gaze and gaze point guidance process, a guidance direction that guides the gaze and gaze point in an appropriate direction, and a guidance method according to the danger level are determined.

  For the gaze and gaze point guidance method, the driver assistance information that directs the driver's gaze and gaze point in the guidance direction is applied to the sound output device such as a speaker installed in the gaze or gaze direction according to the danger level. The output mode and the content of the sound, or the method of causing the light output device such as the LED set in the direction of the line of sight or the gaze point to output with the light of the output mode corresponding to the danger level. To do. In addition, you may output not by any one but by both guidance methods.

Output in the form or content according to the danger level is to output a voice or light that easily recognizes that the danger or urgency is high if the danger level is high. If it is high, increase the volume of the sound, change the sound type, increase the brightness of the light, shorten the blinking interval of the light, etc. . Moreover, you may change the brightness of light with time.
The determined guidance method is notified to the audio output control unit 12 and the light output control unit 14.

  The audio output control unit 12 is a guidance method determined by the line-of-sight direction guidance processing unit 6, and is output from the voice output device 13 such as a speaker in the direction of the driver's line-of-sight direction or the gaze target by the line-of-sight direction guidance processing unit 6. The driving support information that directs the driver's line of sight and gaze point to the determined guidance direction is determined with the determined sound volume and voice guidance sound, that is, with the output mode and content voice (guidance sound) according to the danger level. Output.

  The light output control unit 14 is a guidance method determined by the line-of-sight direction guidance processing unit 6, and is output from the light output device 15 such as an LED in the direction of the driver's line of sight or the direction of the gaze target by the line-of-sight direction guidance processing unit 6. The driving support information for directing the driver's line of sight and the gazing point in the determined guidance direction is output with the light with the determined brightness and blinking interval, that is, with the light in the output mode corresponding to the danger level.

Next, operation | movement of this driving assistance device is demonstrated using the flowchart shown in FIG. FIG. 12 is a flowchart showing the operation of the driving support apparatus in the third embodiment.
The processing in steps ST41 to ST47 is the same as steps ST1 to ST7 in the flowchart shown in FIG.

  When it is determined in step ST47 that the danger level is a level for guiding the driver's gaze / gaze point (in the case of YES in step ST47), the gaze direction guidance processing unit 6 is determined in step ST46. The gaze / gaze point guidance method, any direction to be guided based on the danger level, the gaze direction calculated in step ST45 and the determined gaze target information, and the current time information acquired in step ST44, The speaker and light output device for outputting sound and light, the type and volume of sound output from the speaker, the brightness and color of light, and the blinking interval are determined (step ST48). The determined guidance method is notified to the audio output unit 12 and the light output unit 14 that perform guidance processing.

  When the guidance method is not audio output (NO in step ST49), the light output unit 14 determines the brightness and blinking with respect to the light output device in any direction that guides the driver's line of sight or gaze point. An instruction is issued to output the guide light at intervals (step ST50).

  On the other hand, when the guidance method is voice output and not light output (when step ST49 is YES and step ST51 is NO), the voice output unit 12 is an arbitrary one that guides the driver's line of sight or gaze point. The determined volume and voice guidance sound are output from the direction speaker (step ST52).

  When the guidance method is both voice output and light output (when both step ST49 and step ST51 are YES), the voice output unit 12 is from a speaker in an arbitrary direction that guides the driver's line of sight or gaze point. The light output unit 14 outputs a guide light of the determined brightness and blinking interval from a light output device in an arbitrary direction in which the light output unit 14 guides the driver's line of sight and the gazing point. (Step ST53).

And the process after step ST42 is repeated until a danger level becomes less than a predetermined threshold value (in the case of NO of step ST47). The processes after step ST42 are repeated until the vehicle key is turned off (NO in step ST54). In other words, even if the danger level is less than the predetermined threshold (in the case of NO in step ST47), the driver continues to process the gaze / gaze point of the driver and guides the gaze / gaze point.
Thereafter, when the danger level is less than the predetermined threshold (in the case of NO in step ST47) and the vehicle key is turned off (in the case of YES in step ST54), the process is terminated.

  As described above, according to the third embodiment, the gaze point is detected from the driver's face direction, line-of-sight direction, focal length, etc. using a camera or the like, and the driver's gaze target is determined, By guiding the driver's line of sight and gaze point with sound and light from any appropriate direction that the driver should originally watch in order to drive safely, the driver can focus on the line of sight and focus more than necessary, such as car navigation and HUD. It is possible to prevent a lack of attention in the direction of travel or forward.

  Although the third embodiment has been described as including the audio output control unit 12 and the light output control unit 14 instead of the display control unit 7 in the first embodiment, the display control unit 7 in the second embodiment is described in the second embodiment. The same effect can be obtained even if the audio output control unit 12 and the light output control unit 14 are provided instead.

  Further, all of the display control unit 7, the audio output control unit 12, and the light output control unit 14 are provided, and the guidance method is output as any one or a combination thereof to guide the driver's line of sight or gaze point. By doing so, a higher effect can be expected.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Gaze detection part, 2 Gaze object determination part, 3 Vehicle state determination part, 4 Danger state determination part, 5 Time acquisition part, 6 Gaze direction guidance processing part, 7 Display control part (output control part), 8 Display apparatus ( Output device), 9 vehicle periphery detection unit, 10 sensor unit, 11 ambient state determination unit, 12 audio output control unit, 13 audio output device, 14 light output control unit, 15 light output device.

Claims (10)

A driving support device that is mounted on a vehicle and outputs driving support information from an output device provided in the vehicle,
A detection unit for detecting the driver's gaze direction and gazing point;
Based on the driver's line-of-sight direction and gaze point detected by the detection unit, a gaze target determination unit that determines a gaze target that the driver is gazing at;
A vehicle state determination unit that determines an operation state of the vehicle based on vehicle information corresponding to a driving operation of the vehicle;
Based on the gaze object determined by the gaze object determination unit and the operation state of the vehicle determined by the vehicle state determination unit, the driver's line of sight is directed in an appropriate direction that should be directed in the operation state of the vehicle. A dangerous situation determination unit that determines the degree of non-compliance and determines a danger level according to the determined degree;
It is determined whether or not to guide the driver's line of sight and gaze point according to the danger level determined by the danger situation determination unit, and when it is determined to perform the guidance, the gaze object determination unit A gaze direction guidance processing unit for determining a guidance direction for guiding the driver's gaze and gaze point in the appropriate direction based on the determined gaze object, and determining a guidance method according to the danger level;
Output control for outputting driving support information for directing the driver's line of sight and gazing point in the guidance direction determined by the line-of-sight direction guidance processing unit from the output device with the guidance method determined by the line-of-sight direction guidance processing unit A driving support device comprising: a driving unit. A surrounding state determination unit that detects an external object around the vehicle, calculates a direction of the external object with respect to the vehicle and a distance from the external object to the vehicle, and determines a relationship between the external object and the vehicle; In addition,
The danger situation determination unit includes a gaze object determined by the gaze object determination unit, an operation state of the vehicle determined by the vehicle state determination unit, and the outside object determined by the surrounding state determination unit. Based on the information on the relationship with the vehicle, it is determined whether or not the driver's line of sight is not facing the appropriate direction that should be directed due to the operation state of the vehicle and the relationship between the vehicle and the vehicle. Depending on the risk level,
When the gaze direction guidance processing unit determines whether to guide the driver's gaze and gaze point according to the danger level determined by the danger situation determination unit, Based on the gaze target determined by the gaze target determination unit and the relationship between the vehicle and the outside object determined by the surrounding situation determination unit, the driver's gaze and gaze point are set in the appropriate direction. The driving support device according to claim 1, wherein a guidance direction to be guided is determined, and a guidance method according to the danger level is determined. 3. The driving according to claim 1, further comprising: a display device disposed in front of a driver's seat as the output device, or a display device that performs display in a state of being superimposed on the driver's field of view. Support device. The said danger situation determination part determines the said danger level according to the time degree when the said driver | operator's eyes | visual_axis has not faced the said appropriate direction. Any one of the Claims 1-3 characterized by the above-mentioned. The driving support device according to claim 1. The output control unit, as the guidance method, sets the driver's line of sight and gazing point in the guidance direction with contents corresponding to the danger level expressed using at least one of characters, images, and videos. The driving support device according to claim 3, wherein driving support information to be directed is displayed on the display device. The line-of-sight direction guidance processing unit is based on the gaze target determined by the gaze target determination unit, and the driver is gazing at the screen of the display device and the line of sight is not facing the appropriate direction. If determined, determine to turn off the screen display of the display device as the guidance method,
The operation according to claim 3, wherein the output control unit instructs the display device to turn off the screen display of the display screen according to the guidance method determined by the line-of-sight direction guidance processing unit. Support device. The output device includes an audio output device and an optical output device,
The output control unit outputs, as the guidance method, driving support information for directing the driver's line of sight and gazing point in the guidance direction from the voice output device in an output manner and content according to the danger level. The driving support device according to any one of claims 1, 2 and 5, wherein the driving power is output from the light output device with light having an output mode corresponding to the danger level. . The driving assistance apparatus according to claim 1, wherein the danger situation determination unit sets the danger level to be guided when the driver's line of sight is not in the appropriate direction for a danger determination time or longer. . The danger situation determination unit sets the danger level to be guided when the driver's line of sight is not in the appropriate direction for a danger determination time or more, and the outside condition determined by the surrounding situation determination unit The driving support device according to claim 2, wherein the danger determination time is changed according to a relationship between an object and the vehicle. The driving assistance device according to claim 1, wherein the line-of-sight direction guidance processing unit changes a guidance method according to the danger level depending on a time zone.

Images