KR101986734B1 - Driver assistance apparatus in vehicle and method for guidance a safety driving thereof - Google Patents

Abstract

The vehicle driving assistant apparatus according to the present invention includes a photographing section for photographing a peripheral image of a driver's face and a traveling vehicle, a memory for storing a program for judging whether or not the running state is safe from the photographed image, Lt; / RTI > At this time, as the program is executed, the processor analyzes driver's gaze information from the photographed driver's face, analyzes driving information of the vehicle from the photographed peripheral image, and compares the gaze information with the analyzed traveling The driver's visibility information includes direction information of the face and the gaze, and the driving information includes information on the direction of the face and the gaze, Information on the driving lane, and information on the objects positioned on the front, rear, left, and right sides of the driving lane based on the running speed information, the running direction information, the driving lane information, and the like.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a driving assisting device,

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle driving assist system and a safe driving induction method thereof.

Recently, various sensors or cameras such as LDWS (Lane Departure Warning), LKAS (Lane Keeping Assistance) camera, SCC (Smart Cruise Control) and BSD (Blind Spot Detection) radar have been applied to implement intelligent safety functions. ADAS (Advanced Driver Assistance System) that assists the safe driving of the vehicle has been commercialized and applied.

In addition, a technology for analyzing the driver's condition through a camera for recognizing the face is provided to detect the driver's attention, thereby providing a technique for preventing drowsiness driving.

However, there is a problem that ADAS and driver 's face recognizing technologies are not interrelated and provided separately, and their utilization is very limited.

That is, the ADAS only guides the collision state with the front vehicle, the lane departure state, and the like, and the face recognition technology is still at a level to warn of situations such as drowsiness.

In this way, the information provided to the driver is only individual information, and it is necessary to provide the driver with more direct alarm by fusing the driver's state information with the information provided by ADAS.

In this regard, Korean Patent Laid-Open Publication No. 10-2017-0017616 (title of the invention: driving assist system and method) discloses a technique for warning when a driver is in a drowsy state or is not watching forward.

In the embodiment of the present invention, the driver's gaze information is obtained by photographing the driver's face, the driving information of the vehicle is analyzed from the surrounding image, and the safety of the driving state of the vehicle based on whether or not the gaze information corresponds to the driving information And a method of inducing a safe driving of the vehicle.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

As a technical means for achieving the above technical object, a vehicle driving assist system according to a first aspect of the present invention includes a photographing section for photographing a peripheral image of a driver's face and a traveling vehicle, And a processor for executing a program stored in the memory. At this time, as the program is executed, the processor analyzes driver's gaze information from the photographed driver's face, analyzes driving information of the vehicle from the photographed peripheral image, and compares the gaze information with the analyzed traveling The driver's visibility information includes direction information of the face and the gaze, and the driving information includes information on the direction of the face and the gaze, Information on the driving lane, and information on the objects positioned on the front, rear, left, and right sides of the driving lane based on the running speed information, the running direction information, the driving lane information, and the like.

The processor can distinguish the directions that the driver can see in the vehicle into a plurality of areas and obtain an area matching the direction information of the driver's face and sight line among the plurality of areas as the sight information of the driver have.

The processor may output a warning notification message to the driver when the area matching the direction information of the driver's face and the sight line does not correspond to the information of the driving lane among the driving information.

When the possibility of collision between the object and the vehicle is determined to be within a predetermined time based on the information of the object, the traveling direction information, and the traveling speed information of the traveling information, the processor calculates the direction information of the face and the eyes of the driver, If the matching area does not correspond to the information of the object, the warning notification message may be output to the driver.

The processor may calculate the movement direction and speed of the object among the information of the object, and may output the warning notification message when it is analyzed that there is a possibility of collision between the object and the vehicle within the predetermined time as a result of calculation.

Wherein the processor measures the driver's viewing time of the matching area and acquires the viewing time of the driver as the sight line information, and when the matching time does not correspond to the driving information, Messages can be output.

The processor may accumulate the gaze information in the memory in real time or at preset intervals and generate predictive gaze information of the driver from the accumulated gaze information through a previously learned algorithm.

The processor may generate the predictive sight line information by distinguishing the predictive line information for each driver, and may generate the predictive line information when the matching area does not correspond to the running information but the determined number of times that the traveling state is safe satisfies a predetermined number of times or more .

The processor may apply different weights to the plurality of areas, and may apply different warning levels for outputting the warning notification messages according to the applied weights.

The processor can calculate the safe driving index of the driver based on the determination result of the safety state of the driving state.

According to a second aspect of the present invention, there is provided a safe driving induction method in a vehicle driving assist system, comprising the steps of: capturing a face of a driver and analyzing sight line information including direction information of the face and eyes of the driver; From the peripheral images of the surroundings of the vehicle, information on the traveling speed of the vehicle, information on the traveling direction, information on the driving lane, and information on objects located on the front, rear, left and right sides of the traveling lane Analyzing the information; Determining whether the driving state of the vehicle is safe or not based on whether or not the sight line information corresponds to the driving information, and outputting an alarm notification message to the driver when the driving state of the vehicle is determined to be unsafe .

According to any one of the above-described objects of the present invention, it is possible to provide a driver with a more direct driving condition and a warning to the driver by judging the driving state of the vehicle by combining the driver's state information and the driving information of the vehicle.

1 is a block diagram of a vehicle driving assist system according to an embodiment of the present invention.
2 is a view for explaining a plurality of viewable areas in a vehicle.
3 is an exemplary view of a peripheral image.
4 is a flowchart of a safe driving induction method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly explain the present invention in the drawings, parts not related to the description are omitted.

Whenever a component is referred to as " including " an element throughout the specification, it is to be understood that the element may include other elements, not the exclusion of any other element, unless the context clearly dictates otherwise.

Hereinafter, a vehicle driving assistance apparatus 100 according to an embodiment of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram of a vehicle driving assistance apparatus 100 according to an embodiment of the present invention. 2 is a view for explaining a plurality of viewable areas in a vehicle. 3 is an exemplary view of a peripheral image.

The vehicle driving assistance apparatus 100 according to an embodiment of the present invention includes a photographing unit 110, a memory 120, and a processor 130. [

The photographing unit 110 includes a driver photographing unit for photographing the driver's face and a surrounding image photographing unit for photographing a surrounding image of the driving vehicle.

At this time, the driver photographing part is not limited to a specific kind of camera such as a general camera, a stereo camera, and a depth camera. In the present invention, the camera refers to a camera capable of acquiring a line of sight through a driver's face and a pupil of a driver. In addition, in order to acquire an image of a driver's face and a line of sight even in a dark room, an infrared camera may be provided or an infrared camera may be provided together with the camera.

Further, the driver's photographing section can photograph not only the face of the driver but also the posture information of the driver. Therefore, according to an embodiment of the present invention, for example, even when the height of the driver's face or the distance from the camera is changed according to the change of the position of the seat, the zoom of the camera is adjusted or the image is corrected, You can shoot faces or acquire accurate images.

The peripheral image capturing unit captures the front, both sides or the rear of the vehicle. At this time, the vehicle driving assistant apparatus 100 according to an embodiment of the present invention may be provided with a side camera and a rear camera, respectively, installed in a front camera, a side mirror, or the like so as to photograph the front, , Or may be provided with only one of the cameras.

The peripheral image capturing unit is not necessarily implemented by a camera. The peripheral image capturing unit is not necessarily implemented by a camera, but may be an LDWS (Lane Departure Warning), a LKAS (Lane Keeping Assistance) camera, a Smart Cruise Control (SCC) Of radar and the like.

The memory 120 stores a program for judging whether or not the running state is safe from the photographed image. Herein, the memory 120 is collectively referred to as a nonvolatile storage device and a volatile storage device which keep the stored information even when power is not supplied.

For example, the memory 120 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD) A magnetic computer storage device such as a NAND flash memory, a hard disk drive (HDD) and the like, and an optical disc drive such as a CD-ROM, a DVD-ROM, etc. .

The vehicle driving assistant apparatus 100 according to an embodiment of the present invention may further include a communication module 140. The communication module 140 receives the images captured by the photographing unit 110 . The communication module 140 may include both a wired communication module and a wireless communication module. The wired communication module may be implemented by a power line communication device, a telephone line communication device, a cable home (MoCA), an Ethernet, an IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module can be implemented with a wireless LAN (WLAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60 GHz WPAN, Binary-CDMA, wireless USB technology and wireless HDMI technology.

More preferably, the communication module 140 can transmit and receive data through the CAN (Controller Area Network) communication with the photographing unit 110 and the memory 120.

The processor 130 executes the program stored in the memory 120. [ As the program 130 is executed, the processor 130 analyzes the driver's gaze information from the driver's face photographed through the driver's photographing section. At this time, the driver's gaze information includes direction information of the driver's face, direction of the driver's face, or direction of the eyes according to the pupils.

After acquiring the line of sight information of the driver, the processor 130 divides the direction that the driver can see in the vehicle into a plurality of areas, as shown in FIG. An area matching with the direction information of the driver's face and sight line among the plurality of areas can be obtained as the sight line information of the driver.

For example, when the vehicle is traveling in a straight lane, the driver mainly looks at the first region or the third region in the direction of the face and the line of sight. Thus, the first region or the third region Is identified as an area matching with the direction information of the driver's face and the sight line, and is obtained as sight line information.

In addition to normal driving, for example, when the driver is driving a sleeping driver while the vehicle is driving in a straight lane, the direction of the driver's face is looking at the eighth area, and the eighth area is the direction of the driver's face and sight line The information is matched with the information, and it is acquired as sight information.

In the case of FIG. 2, a limited number of regions are set for the front side and the lateral side. However, the present invention is not limited thereto. For example, the plurality of regions may be divided and divided more precisely, Of course, can be divided and distinguished.

As the area is divided more precisely and over a wide range, the driver's gaze information can be further refined and the driver's habit described later can be learned more concretely.

Next, the processor 130 analyzes the traveling information of the vehicle from the surrounding image photographed by the surrounding image photographing unit. At this time, the running information of the vehicle includes at least one of the information of the objects located at the front, rear, left, and right based on the running speed information, the running direction information, the running lane information and the driving lane of the vehicle.

At this time, the running information of the vehicle may be obtained by analyzing from the surrounding image, or may be obtained by analyzing and acquiring information sensed by each sensor included in the vehicle, or by using information sensed by the ECU, It can also be obtained by analyzing the diagnosed information.

After acquiring the sight line information and the driving information of the driver in this way, the processor 130 determines whether the driving state of the vehicle is safe or not based on whether the sight line information and the driving information correspond to each other, Can be output.

Specifically, the processor 130 may output a warning notification message to the driver when the area matching the direction information of the driver's face and the sight line does not correspond to the information of the driving lane in the driving information. At this time, an area matched with the sight line information and an area corresponding to the travel information may be stored in advance in the memory 120.

In the case of the area corresponding to the running information, it may be an intersecting area when the direction of the line of sight of the driver is extended with respect to the object or the lane, and when a single object is divided into a plurality of areas, And may be an area corresponding to the travel information.

For example, when the vehicle is traveling in a straight lane, it is normal that the direction of the driver's gaze is looking at the first area or the third area, and when looking at the surroundings, the second area or the fourth area It is normal. Therefore, when the current running information of the vehicle is running straight ahead, the corresponding area corresponds to the first to fourth areas.

However, in the case where the vehicle is traveling in the straight lane, when the area matched with the direction information of the driver's face and the sight line is the fifth to eighth areas other than the first to fourth areas, In this case, the processor 130 may output a warning notification message to the driver.

As another example, when the vehicle is making a lane change to the left lane, the driver's gaze direction is the front area of the first area and the third area, the seventh area as the left side mirror area, and the fifth area It is normal to look at. Therefore, when the running information of the current vehicle is changing to the left, the corresponding area corresponds to the first, third, fifth, and seventh areas.

However, when the vehicle does not acquire line-of-sight information looking at the area corresponding to the lane change in the left lane and acquires line-of-sight information that is in another area, for example, the eighth area, If the area and the running information do not correspond to each other, the processor 130 may output a warning notification message to the driver.

When the processor 130 determines that there is a possibility of collision between the object and the vehicle within a predetermined time based on the information of the object, the traveling direction information, and the traveling speed information in the traveling information, And the area matched with the information of the object does not correspond to the information of the object, the warning notification message can be outputted to the driver.

At this time, the processor 130 calculates the movement direction and speed of the object among the information of the object, and may output a warning notification message when the object is analyzed as having a possibility of collision between the object and the vehicle within the predetermined time.

Here, the object is a concept that includes other vehicles, pedestrians as well as other obstacles located in the lane around the vehicle. Also, the moving direction of the object may be, for example, a crossing direction including a vertical direction, a diagonal direction, and the like with respect to a vehicle, such as an endless crossing of a pedestrian, It is possible to include all these directions.

For example, when a vehicle is traveling in a forward direction with a straight lane of 60, and there is a stopped vehicle on the left side of the lane, the corresponding area may be a first area, a second area and a fifth area have.

When the processor 130 analyzes that there is a possibility that the processor 130 collides with the objects within 5 seconds through a distance measuring sensor such as an ultrasonic sensor or a Lidar sensor, The processor 130 determines that the corresponding area does not correspond to the driving information, and outputs the warning notification message to the driver if the area is not the second area and the fifth area, for example, the seventh area.

For example, when the pedestrian endlessly traverses from left to right, the processor 130 calculates the moving direction of the pedestrian (moving from left to right) and the moving speed, If it is determined that there is a possibility of collision with the pedestrian after 5 seconds based on the information, the driver can output a warning notification message.

As described above, according to an embodiment of the present invention, not only when the object is in a stationary state, but also the direction and speed of the moving object are calculated and the possibility of collision is calculated based on the traveling direction and speed, There is an advantage that the risk of an accident can be minimized even when the driver's gaze does not correspond to the direction of the object.

3, the vehicle driving assistant apparatus 100 according to an embodiment of the present invention may extract only an area directly corresponding to surrounding vehicles from an image photographed through the photographing unit 110 The region of interest, and the information of the object, that is, the moving direction and speed of the object, can be calculated based on the set region of interest.

If the area matching the sight line information of the driver does not correspond to the area of interest, the warning alert message may be output to the driver.

For example, in FIG. 3, when the vehicle is running straight, the processor 130 sets the region of interest P1 to P3 from the photographed image. In this case, when the vehicle, which is the area of interest P2, approaches the front of the vehicle through the lane change, it is normal for the driver to see the first area or the third area in the area matching the sight line information, P2 also correspond to the first area or the third area. However, at this time, when the area is different from the area corresponding to the area of interest P2 such as the eighth area or the sixth area, the processor 130 may output a warning notification message to the driver.

Accordingly, an embodiment of the present invention can quickly and accurately detect and analyze an object using only the region of interest, without analyzing the entire captured image, so that it can cope with the risk of an accident in real time.

Meanwhile, as shown in FIG. 3, an area of interest P4 is set for each of a plurality of distances in which the surrounding image photographing unit is divided into distances, and interest areas P1 to P3 for the corresponding objects are set By mapping a plurality of ROIs to the ROIs P1 to P3, an embodiment of the present invention can acquire the distance and relative position information of the ROI to the ROIs P1 to P3 more accurately and quickly can do.

For example, in the case of the region of interest P 1, a plurality of relatively interesting regions P 4 located at the center of the region P 4 of interest and relatively wide in the region of interest P 2 The processor 130 can accurately and quickly obtain the distance and the relative position between the vehicle and the vehicle corresponding to the area of interest P1, the area of interest P2, and so on.

In determining whether the area matching the driver's sight line information corresponds to the area corresponding to the driving information, the area of interest includes information (position and distance information) of the area of interest P4 per distance, Even if the regions of interest corresponding to the plurality of objects are included in the same area of FIG. 3, information of one or more objects of which the user is currently interested can be accurately extracted from among a plurality of objects based on the sight line information of the user.

For example, if the area of interest P2 and the area of interest P3 in FIG. 3 correspond to the same third area of FIG. 2 and the user is looking at the object, which is the area of interest P3, In an embodiment of the present invention, information on all the regions of interest (P2, P3) included in the third region is not acquired, and the region of interest P3 corresponding to the user's gaze information, i.e., Can be obtained only.

Meanwhile, the vehicle driving assist system 100 according to an exemplary embodiment of the present invention may measure the driver's watch time with respect to the area where the processor 130 matches the direction information of the driver's face and the sight line, have. That is, the gaze information may include face direction information, gaze direction information, and watch time information.

Accordingly, the processor 130 can output a warning notification message to the driver when the viewable time exceeds a predetermined time when the matching area does not correspond to the driving information.

That is, if the driver does not look at the area corresponding to the driving lane information or the area corresponding to the information of the object, and if the unobserved area is viewed over a certain period of time, it is determined that the accident risk is high, .

For example, in a state where the vehicle is traveling in a straight lane, when the sight line direction of the driver is looking at the first to fourth regions, the viewing time for each region is measured.

When the area matched with the direction information of the driver's face and sight line is the fifth to eighth areas other than the first to fourth areas while the vehicle is traveling in the straight lane, At this time, if the viewing time for the fifth to eighth areas exceeds 5 seconds, for example, the processor 130 may determine that the risk of an accident is high and output an alarm notification message to the driver.

In addition, the processor 130 may store the driver's gaze information in the memory 120 in real time or at predetermined intervals. And predicted sight line information of the driver can be generated from the accumulated sight line information through the previously learned algorithm.

At this time, the processor 130 distinguishes the predicted sight line information for each driver, and generates the predicted sight line information based on the predicted sight line information, which is generated when the area that matches the direction information of the face and sight line of the driver does not correspond to the running information, If the number of times or more is satisfied, it can be generated.

The processor 130 divides the gaze information of the driver, that is, the direction information of the face, the gaze direction information, and the gaze time information for each region, and accumulates the gaze information in the memory 120.

The accumulated gaze information is set as an input value, and the predicted gaze information of the driver can be generated through the previously learned algorithm. At this time, the algorithms already learned are not limited to algorithms based on Artificial Neural Network, Hidden-Markov-Model (HMM), but also algorithms such as CNN (Convolutional Neural Networks) and RNN (Recurrent Neural Network) A learning-based algorithm may be applied.

For example, when the driver desires to move the vehicle backward, the user views the seventh area and the eighth area, which are the rear mirror area or the rear mirror area, but only the ninth area when the rear camera is installed Reverse can be possible.

In this case, the processor 130 generates a predetermined number or more of the ninth region only, and if there is no accident such as a collision, the matching region (ninth region) It is generated as visual information, and in this case, it does not output a warning message.

As another example, when it is desired to change the lane to the left lane while driving the straight lane, it is common to look at the seventh area as the left side mirror area and change the lane in a complicated road with many vehicles, In this case, it is possible to change the lane even if only the fifth area, which is the rearview mirror area, is viewed.

In this case, the processor 130 generates a lane change more than a predetermined number of times by looking only at the fifth area. In the case where there is no accident such as a collision, the matching area (fifth area) does not correspond to the driving information, And generates the predicted visual information, and in this case, the warning notification message is not output.

As described above, the driving assistance apparatus 100 according to the embodiment of the present invention analyzes the pattern of the driver's driving habits and the like, generates the prediction line information according to the analyzed result, and outputs the warning notification message, , It is possible to prevent the driver from interfering with the operation due to the alarm notification message even in the normal operation state.

On the other hand, the warning notification message can be output in the form of voice or display on the screen through the instrument panel, HUD (Head Up Display) or AVN (Audio Video Navigation) system of the vehicle, An alert notification message may be output. Or an alert notification message may be output through a smart terminal such as a smart phone, a tablet PC, or a smart watch paired with the vehicle.

The processor 130 may apply a different weight to each of a plurality of areas that the driver can watch in the vehicle and apply different warning levels for outputting the warning notification message according to the applied weight.

For example, the weight of the first to fourth areas, which are the front areas mainly viewed by the driver, is set relatively low, and the fifth to seventh areas used when the lane is changed are set as intermediate levels. The eighth to ninth regions, which are the regions to be looked at in the situation, can be set to have a high weight. Accordingly, the above-described warning notification message is output only at pop-ups or flickers on the screen in a low weighted level, and is outputted at a low volume in a middle level, and at a high level, Can be output.

In addition, the processor 130 may apply different weights to a plurality of areas, and may set weights differently in real time by reflecting driving information of the vehicle. That is, one embodiment of the present invention considers whether the running state of the vehicle is straight, whether the lane is being changed, whether the vehicle is in the U-turn or backward, and the speed of the vehicle and the information of the object are also applied as parameters in setting the weights .

For example, in the case where the vehicle is straight ahead, the first to fourth regions may be regions having the lowest weight, but when the vehicle is moving backward, the first to fourth regions are regions that are not easily overlooked, Can be set a little higher.

As another example, when the lane change is made in the left lane, the weight is set higher for the sixth area as the right side mirror than the seventh area as the left side mirror area, so that vehicles, pedestrians, It is possible to output a warning message more aggressively for the risk of collision.

Further, according to an embodiment of the present invention, not only an alarm notification message but also an alarm notification message may be output or, when there is no action by the driver despite a certain time, The risk of accidents can be prevented through.

That is, if there is a risk of a collision with a pedestrian, the processor 130 outputs a warning notification message or if the speed of the vehicle does not decelerate after a predetermined time, the processor 130 decelerates the speed of the vehicle directly, It is possible to change the steering angle of the steering wheel of the vehicle so as to directly prevent the occurrence of the collision.

Meanwhile, the vehicle driving assist system 100 according to the embodiment of the present invention can calculate the safe driving index of the driver based on the judgment result of the safety state of the driving state.

That is, the processor 130 comprehensively considers the number of times that the driving information such as the sight line information, the driving lane information, and the object information are not matched, the number of times the warning notification message is output, The safe driving index of the driver can be calculated.

In this case, when the driving information and the area do not correspond to each other as the predicted sight line information is generated, the processor 130 does not output the warning notification message. Therefore, the processor 130 treats the driving information as a correspondence between the driving information and the area, .

Such a safe driving index can be calculated at regular intervals such as every day or one month, and the calculated safe driving index can be reflected when providing a descent such as incentive or driver's penalty such as driver's mileage.

By providing the incentive or the descent through the safe driving index, it is possible to more aggressively prevent drowsiness and inactivity in the driving of the vehicle, which can be expected to greatly reduce the traffic accident.

1 may be implemented in hardware such as software or an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform predetermined roles can do.

However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors.

Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, memory, data structures, tables, arrays, and variables.

The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, a method for inducing safe driving in the vehicle driving assistant 100 according to an embodiment of the present invention will be described with reference to FIG.

4 is a flowchart of a safe driving induction method according to an embodiment of the present invention.

In the safe driving induction method according to an embodiment of the present invention, first, a face of a driver is photographed and sight line information including direction information of a face and eyes of a driver is analyzed (S110).

Next, it is assumed that at least one of the information of the vehicle traveling speed information, the traveling direction information, the information of the driving lane, and the information of the front, rear, left, and right objects based on the driving lane is obtained from the peripheral image of the front of the vehicle And analyzes the included driving information (S120).

Next, it is determined whether or not the running state of the vehicle is safe based on whether or not the sight line information and the running information correspond to each other (S130). If it is determined that the running state of the vehicle is not safe, S140).

On the other hand, in the above description, steps S110 to S140 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed. In addition, the contents already described with respect to the vehicle driving assistant device 100 in Figs. 1 to 3 apply to the safe driving inducing method in Fig. 4 even if other contents are omitted.

One embodiment of the present invention may also be embodied in the form of a computer program stored on a medium executed by a computer or a recording medium including instructions executable by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: vehicle driving assist device
110:
120: Memory
130: Processor
140: Communication module

Claims (11)

A vehicle driving assist system,
A photographing unit for photographing a periphery image of the driver's face and the driving vehicle,
A memory for storing a program for judging whether or not the running state is safe from the photographed image,
And a processor for executing a program stored in the memory,
The processor analyzes the driver's gaze information from the photographed driver's face and analyzes the driving information of the vehicle from the photographed peripheral image and displays the gaze information and the analyzed driving information And outputs an alarm notification message to the driver,
Wherein the driver's gaze information includes direction information of the face and the gaze,
The driving information includes at least one of information of the traveling speed information, the traveling direction information, the information of the driving lane, and the information of the front, rear, left, and right objects based on the driving lane,
Wherein the processor sets a region of interest for a plurality of distances classified by distance from the peripheral image and a region of interest for an object directly corresponding to surrounding vehicles, And extracts only information on a region of interest of the object matching the sight line information based on the calculated distance and position information, And outputs the warning notification message if the matching area does not match the area of interest for the object. The method according to claim 1,
Wherein the processor divides the driver's viewable direction into a plurality of areas and obtains an area matching the direction information of the driver's face and sight line among the plurality of areas as the sight line information of the driver Vehicle driving assistance device. 3. The method of claim 2,
Wherein the processor outputs an alarm notification message to the driver when an area matched with the direction information of the driver's face and sight line does not correspond to the information of the driving lane of the driving information. 3. The method of claim 2,
When the possibility of collision between the object and the vehicle is determined to be within a predetermined time based on the information of the object, the traveling direction information, and the traveling speed information of the traveling information, the processor calculates the direction information of the face and the eyes of the driver, And outputs a warning notification message to the driver if the matching area does not correspond to the information of the object. 5. The method of claim 4,
Wherein the processor calculates the moving direction and the speed of the object among the information of the object and outputs the warning notification message when it is analyzed that there is a possibility of collision between the object and the vehicle within the predetermined time as a result of calculation, Auxiliary device. 6. The method according to any one of claims 3 to 5,
Wherein the processor measures the driver's viewing time of the matching area and acquires the viewing time of the driver as the sight line information, and when the matching time does not correspond to the driving information, And outputs a message. The method according to claim 6,
Wherein the processor accumulates and stores the gaze information in the memory in real time or at preset intervals and generates predictive gaze information of the driver from the accumulated gaze information through a previously learned algorithm. 8. The method of claim 7,
Wherein the processor generates the predictive sight line information for each driver by generating the predictive line information when the number of times that the matched area does not correspond to the driving information but the driving state is determined to be safe is greater than a predetermined number Vehicle driving assistance device. 3. The method of claim 2,
Wherein the processor applies a different weight to each of the plurality of areas and differently applies a warning level for outputting the warning notification message according to the applied weight. 3. The method of claim 2,
Wherein the processor calculates the safe driving index of the driver based on a result of the determination as to whether or not the driving state is safe. A method for inducing safe driving in a vehicle driving assist system,
Analyzing the gaze information including the direction information of the driver's face and gaze by photographing the driver's face;
From the peripheral images of the surroundings of the vehicle, information on the traveling speed of the vehicle, information on the traveling direction, information on the driving lane, and information on objects located on the front, rear, left and right sides of the traveling lane Analyzing the information;
Determining whether the running state of the vehicle is safe or not based on whether or not the sight line information corresponds to the running information;
And outputting a warning message to the driver when the driving state of the vehicle is determined to be unsafe,
Wherein the step of analyzing the running information comprises:
A region of interest for an object corresponding to a plurality of distances classified by distance from the surrounding image and an object directly corresponding to vehicles located in the vicinity, and mapping an interest region for the object to the interest region for each of the plurality of distances Extracting only information on a region of interest of the object matching the sight line information based on the calculated distance and position information,
Wherein the step of outputting the warning message comprises:
And outputting the alert notification message if an area matching the sight line information does not match an area of interest for the object.

Images