KR101511858B1 - Advanced Driver Assistance System(ADAS) and controlling method for the same - Google Patents

Abstract

The present invention relates to a radar module mounted on an automobile and detecting an object in front of the automobile and acquiring information of the object, a vision module mounted on the automobile and acquiring image information of the automobile front, A communication module for performing bidirectional communication with the object, a type and position of the front object through the bidirectional communication, and based on the information of the object, the image information and the position of the object, The present invention relates to a driving assistance system for recognizing a pedestrian or a motorcycle including a controller for judging whether or not an obstacle to be avoided is to be avoided.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a driving assist system for recognizing a pedestrian or a motorcycle,

The present invention relates to a driving assist system for recognizing a pedestrian or a motorcycle and a control method thereof, and more particularly to a driving assist system for recognizing a pedestrian or a motorcycle, Or a two-wheeled vehicle) or a preceding vehicle, and a control method thereof.

The driver assistance system uses a state-of-the-art sensor to detect the risk of collision in the same way as the driver recognizes through visual, auditory, and tactile factors, alerting the driver of the risk of an accident, slowing down the speed for avoiding front / Or actively performing emergency braking. In addition, the driving assistance system can perform lane departure warning, blind spot monitoring, and improved rearward surveillance.

The driving assist system is divided into various types according to its functions. The Forward Collision Warning System (FCW) is a system that provides visual, auditory, and tactile warning to drivers for the purpose of avoiding collision with the forward vehicle by detecting the vehicle in the same direction ahead of the driving lane to be.

The Advanced Emergency Braking System (AEBS) detects the possibility of collision with an automobile located in front of the driving lane and warns the driver. If the driver does not respond or it is determined that a collision is inevitable, This is a system for automatically braking the vehicle for the purpose of stopping the vehicle.

Adaptive Cruise Control (ACC) is a system in which the vehicle autonomously travels at a speed set by the driver. This system interrupts the traffic flow when a preceding vehicle that runs at a speed lower than the driver's set speed appears during self-driving. So that the vehicle can follow the preceding vehicle.

In addition, the driving assistance system includes a lane departure warning system (LDWS), a lane keeping assist system (LKAS), blind spot detection (BSD), and a rear collision warning system Rear-end Collision Warning System (RCW), and Smart Parking Assist System (SPAS).

On the other hand, according to the characteristics of sensors used in the conventional driving assistance system, it is difficult to precisely understand whether the object ahead of the vehicle should follow the obstacle or the obstacle to avoid is the pedestrian or the two- I needed a solution to do it.

SUMMARY OF THE INVENTION [0006] In order to solve the above problems, an object of the present invention is to provide a method of detecting a pedestrian or a motorcycle for avoiding a collision by grasping an object vulnerable to collision such as a pedestrian or a two- A driving assist system and a control method thereof.

The problems of the present invention are not limited to the above-mentioned problems, and other problems not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a driving assist system for recognizing a pedestrian or a motorcycle, comprising: a radar module mounted on an automobile and detecting an object in front of the automobile and acquiring information on the object; A vision module mounted on the vehicle and acquiring image information of the front of the automobile, a communication module mounted on the automobile and performing bidirectional communication with the object in front of the vehicle, a position of the front object through the bidirectional communication, Based on the information, the image information, and the position of the object, whether the object ahead is an obstacle to avoid collision.

The details of other embodiments are included in the detailed description and drawings.

According to the driving assistance system and the control method for recognizing the pedestrian or the motorcycle according to the present invention, one or more of the following effects can be obtained.

First, the recognition rate for pedestrians, two-wheeled vehicles, and following vehicles improves even in bad weather or nighttime through bidirectional communication performed with objects in front of the car.

Second, the performance of the Advanced Emergency Braking System (AEBS) or the Adaptive Cruise Control (ACC) is improved because recognition rates for pedestrians, two-wheeled vehicles and forward-facing vehicles are improved.

Third, the driving assist system and its control method of the present invention grasps the driving situation (stop, low-speed driving, high-speed driving, left / right turn, etc.) of another vehicle and predicts how the movement of the other vehicle will change It is possible to prevent dangerous situations from occurring, thus enabling safe driving.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram of a driving assistance system according to an embodiment of the present invention.
2 is a diagram illustrating the operation of a vision module and a radar module of a driving assistance system according to an embodiment of the present invention.
3 is a block diagram of a driving assistance system recognizing a pedestrian according to an embodiment of the present invention.
4 is a configuration diagram for explaining communication between an automobile and an object in front according to an embodiment of the present invention.
5 is a flowchart according to an embodiment of the present invention.
6 is a flowchart according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. To fully disclose the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described with reference to the drawings for explaining a driving assistance system for recognizing a pedestrian according to embodiments of the present invention and a control method thereof.

The suffix "module" and "part" for constituent elements used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

FIG. 1 is a block diagram of a driving assistance system according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating an operation of a vision module and a radar module of a driving assist system according to an embodiment of the present invention.

The driving assist system according to an embodiment of the present invention includes an acceleration input module 10, a brake input module 20, a vision module 30, a radar module 40, a velocity sensing module 50, 60, a position sensing module 70, a communication module 210, a controller 100, a memory 150, a warning module 110, and a power / braking module 120.

The acceleration input module 10 is a user's operation device for increasing the speed of the vehicle. The acceleration input module 10 increases the power of the power / braking module 120 to increase the speed of the vehicle. Generally, the acceleration input module 10 increases the speed of the vehicle by increasing the rotation of the vehicle engine. The acceleration input module 10 is generally provided as an accelerator pedal in the form of a pedal below the driver's seat of the vehicle.

Acceleration input module 10 may be inputted with an acceleration degree in accordance with a user's operation. If the acceleration input module 10 is an accelerator pedal, the degree of acceleration may be input according to the pedal pressure.

When the user operates the acceleration input module 10, the acceleration input module 10 outputs an acceleration signal including the degree of acceleration to the controller 100. The controller 100 controls the power / braking module 120 according to the input acceleration signal to accelerate the vehicle. According to the embodiment, the acceleration input module 10 can directly control the power / braking module 120 to accelerate the vehicle.

The braking input module 20 is a user's operating device for reducing the speed of the vehicle or for stopping the vehicle. The braking input module 20 reduces the power of the power / braking module 120 or generates a braking force to decelerate or stop the vehicle. Generally, the braking input module 20 operates a brake that applies a frictional force to a disk of a wheel of a vehicle to reduce the speed of the vehicle. Depending on the embodiment, the braking input module 20 may directly reduce the rotation of the vehicle engine or operate a reduction device such as a retarder. The braking input module 20 is generally provided as a brake pedal in the form of a pedal below the driver's seat of the vehicle.

The braking input module 20 can be inputted with the degree of deceleration according to the operation of the user. If the braking input module 20 is a brake pedal, the degree of deceleration may be inputted according to the pedal pressure.

When the user operates the braking input module 20, the braking input module 20 outputs the braking signal including the degree of deceleration to the controller 100. The controller 100 controls the power / braking module 120 according to the input braking signal to decelerate or stop the vehicle. The braking input module 20 can directly control the power / braking module 120 to decelerate or stop the vehicle.

The speed sensing module 50 senses the current speed of the vehicle. The speed sensing module 50 senses the rotation speed of the wheels of the vehicle or senses the rotation speed of the output shaft of the transmission connected to the engine of the vehicle to calculate the current speed of the vehicle. The speed sensing module 50 may include a speed sensor for sensing a rotation speed and a processor for calculating a current speed value of the vehicle.

The speed sensing module 50 outputs the sensed speed value of the vehicle to the controller 100.

The posture sensing module 60 senses the posture variation of the vehicle. The posture sensing module 60 senses a variation of at least one of a pitch axis, a yaw axis, and a roll axis, and senses a yaw rate in the present embodiment. That is, in the present embodiment, the attitude sensing module 60 senses the yaw rate of the vehicle and senses the degree of rotation of the vehicle. The attitude detection module 60 may include a gyro sensor or an acceleration sensor for detecting the attitude change, a processor for calculating the variation value, and the like.

The attitude detection module (60) outputs the detected attitude variation value of the vehicle to the controller (100).

The position sensing module 70 senses the position of the vehicle. The position sensing module 70 is a receiver of a global positioning system (GPS), which receives a distance and a signal from a satellite and calculates the position of the vehicle. According to the embodiment, the position sensing module 70 can replace the speed sensing module 50 by calculating the speed of the vehicle.

The vision module 30 is a device for recognizing an object outside the vehicle by photographing an image outside the vehicle and discriminating the type of the object. The vision module 30 is generally disposed at the front end of the vehicle to capture an image of the front of the vehicle.

As shown in FIG. 2 (a), the vision module 30 can distinguish the road R and recognize various objects O on the road R to recognize and distinguish the objects. The vision module 30 recognizes whether the object O is a vehicle, a person, or a simple obstacle by recognizing the shape of the object O. In the case of a vehicle, the vision module 30 can distinguish whether the object O is a passenger car, a truck or a two-

The vision module 30 can recognize the lane L on the road R and can distinguish whether the lane L is a general lane, a center line, a curb line, or a divided lane. In addition, the vision module 30 can recognize and distinguish a curb or a walk on the road.

The vision module 30 can recognize the lane N between the lane L and the lane L through the recognized lane L. [ The vision module 30 can recognize the lane N in which the subject vehicle H equipped with the vision module 30 is running. In addition, the vision module 30 may recognize which lane N the recognized object O is located on, or whether the recognized object O lies on the lane L.

The recognition and identification of the object O can be performed in the vision module 30 itself or in the controller 100 through the image captured by the vision module 30. [

The vision module 30 has a constant field of view. As shown in Fig. 2 (a), the vision module 30 photographs the objects O in the clock F. Fig. According to the embodiment, the vision module 30 can change the photographing direction up and down and / or right and left. That is, the vision module 30 can change the center of the clock F up and down and / or right and left.

The vision module 30 may include a camera that captures an image, a processor that processes the captured image, and a memory that stores data. According to the embodiment, the vision module 30 may include a driving device capable of changing the photographing direction of the camera.

The vision module 30 may output photographed image data to the controller 100 or output information of the recognized and discriminated object O to the controller 100. [

The radar module 40 is a device that emits an electromagnetic wave to a specific object O and then receives an electromagnetic wave reflected from the object O to sense the distance, position, direction, speed, etc. with the object O. The radar module 40 is generally disposed at the front end of the vehicle to calculate the distance to a specific object O in front of the vehicle, and the like. According to an embodiment, the radar module 40 may be a lidar that fires a laser at the object O.

The radar module 40 detects the distance, position, direction, speed, and the like to the target vehicle T, which is a specified one among various objects O as shown in FIG. 2 (b). The target vehicle T is generally selected from the data detected by the radar module 40 and the vision module 30 as the object O to be tracked by the controller 100 and is generally set to the radar module 40 and the vision module 30. [ Is an object O recognized as a vehicle at a distance closest to the front of the subject vehicle H on the lane N in which the subject vehicle H is installed.

The radar module 40 may include a radar that emits electromagnetic waves, a processor that processes information about the electromagnetic waves received by the radar, and a memory that stores data.

The radar module 40 outputs information such as the distance to the object O, the position, the direction, and the velocity to the controller 100.

The communication module 210 performs communication with an object in front of the automobile. The communication module 210 can be used to determine what is ahead when the object is difficult to detect by the radar module 40 and the vision module 30 when the vehicle is traveling in bad weather or at night. The communication module 210 performs communication with an object in front, and the communication information is transmitted to the controller 100. Based on the information received from the communication module 210, the controller 100 determines whether an object in front of the automobile is an obstacle (e.g., a pedestrian or a motorcycle) to be avoided or a follow-up vehicle. The communication module 210 continuously transmits an interrogation signal (for example, within 20 msec) to the mobile terminal carried by the pedestrian or the motorcycle driver. At this time, the mobile terminal may be provided with an application for receiving current location information of the mobile terminal and transmitting and receiving the location information. Here, the communication between the communication module 210 and the mobile terminal may be bi-directional communication.

The controller 100 comprehensively determines cognitive information transmitted from the radar module 40, the vision module 30, and the communication module 210 to determine the type, speed, relative distance, and avoidance plan of the forward object.

The warning module 110 is a device for giving a warning to a driver who drives the vehicle, and can warn the driver visually, audibly, and tactually according to the embodiment. The warning module 110 may display a visual warning through the instrument panel of the driver's seat, head-up display, navigation, integrated information display device, and the like. The warning module 110 can make audible warning through the speaker of the vehicle. The warning module 110 may alert the driver by tilting the handle of the vehicle or tightening the seat belt.

The warning module 110 may operate according to the control of the controller 100 to alert the driver.

The power / braking module 120 is a device that accelerates, decelerates, or stops the vehicle. The power / braking module 120 may include an engine and / or a motor that generates a rotational force to rotate the wheels of the vehicle, and a transmission that changes the rotation ratio of the engine and / or the motor. The power / brake module 120 may include brakes and / or retarders that generate braking forces or reduce rotation of the engine and / or motor.

The power / braking module 120 may operate under the control of the controller 100 or may be operated by the acceleration input module 10 or the braking input module 20.

The controller 100 includes an acceleration input module 10, a brake input module 20, a vision module 30, a radar module 40, a velocity sensing module 50, a position sensing module 60, and a position sensing module 70 And controls the warning module 110 and the power / braking module 120 according to the processed result. The controller 100 may include a CAN (Controller Area Network) for data communication with each module. On the other hand, the scope of the present invention is not limited to the communication using the CAN (Controller Area Network), and the communication method used in the automobile network is included in the scope of the present invention.

Memory 150 stores programs, instructions, and data. The controller 100 stores data in the memory 150 or calls programs, commands, or data stored in the memory 150. [

3 is a block diagram of a driving assistance system recognizing a pedestrian according to an embodiment of the present invention.

3, the driving assistance system recognizing a pedestrian includes a radar module 40, a vision module 30, a communication module 210, a warning module 110, a power / braking module 120, a controller 100, . ≪ / RTI >

The radar module 40 is mounted on the vehicle. The radar module 40 detects an object in front of the automobile and acquires information about an object in front of the automobile. Here, the acquired information may be distance, position, direction, and speed with respect to an object in front of the automobile.

The radar module 40 emits an electromagnetic wave to the preceding vehicle, and receives electromagnetic waves reflected from an object in front of the automobile to sense distance, position, direction, speed, etc. with the object. The radar module 40 transmits information about the detected leading vehicle to the controller 100.

The vision module 30 is mounted on a vehicle. The vision module 30 is disposed at the front end of the automobile and captures an image of the front of the automobile. That is, the image information of the front of the automobile is acquired. The vision module 30 may include a plurality of cameras. The vision module 30 including a plurality of cameras can measure the distance to the object included in the image information. The vision module 30 transmits the obtained image information of the front of the automobile to the controller 100. [

The communication module 210 is mounted on a vehicle. The communication module 210 performs communication with an object in front of the automobile. At this time, the communication performed with the object in front of the automobile may be bi-directional or unidirectional communication.

On the other hand, the object ahead of the vehicle in which the communication module 210 performs communication may be a pedestrian, a two-wheeled vehicle, or an automobile. The communication module 210 can perform communication with an object in front of the automobile at regular intervals. For example, when the object ahead of the automobile is a pedestrian or a two-wheeled vehicle, the communication module 210 may communicate with a mobile terminal equipped with an application for transmitting and receiving position information held by a pedestrian or a motorcycle driver. The communication module 210 may continuously transmit an interrogation signal (for example, within 20 msec) to the mobile terminal for communication with the mobile terminal carried by the pedestrian or the motorcycle driver.

The communication module 210 may be a GPS (Global Positioning System) or a DGPS (Differential Global Positioning System) based communication device. DGPS is a GPS positioning method of relative positioning method, which corrects the elements causing error by using known reference point coordinates and obtains accurate position by minimizing error.

4 is a graph for explaining communication between an automobile and an object in front according to an embodiment of the present invention.

Referring to FIG. 4, a radar module 40 mounted on a vehicle 310 detects an object in front of the vehicle 310. In this case, the forward object includes the preceding vehicles 330, 331 and 332 traveling in a stopped state, a pedestrian 320, a two-wheeled vehicle (not shown), a guard rail (not shown) ). The vision module 30 mounted on the automobile 310 includes image information including preceding cars 330, 331 and 332, a pedestrian 320, a motorcycle (not shown), a guardrail or a sign (not shown) . The communication module 210 transmits a query signal at a predetermined period in order to attempt communication with an object in front of the automobile 310. For example, the self communication module 210 can perform a bidirectional communication by transmitting a query signal with the following leading vehicle (330) or an obstacle in front of the automobile. The communication module 210 performs communication with the object when a response signal is received from an object ahead of the vehicle 310. [ The communication module 210 transmits information related to communication performed with an object in front of the automobile 310 to the controller 100. The controller 100 determines the position of an object ahead of the vehicle 310 based on communication between the communication module 210 and an object ahead of the vehicle 310. [

On the other hand, when it is difficult to detect an object ahead by the radar module 40 and the vision module 30 when the vehicle is traveling in bad weather or at night, the communication module 210 can be used to determine what is ahead have. For example, when a pedestrian 320 or a two-wheeled vehicle (not shown) is located in front of the automobile 310, the communication module 210 communicates with a mobile terminal carried by the pedestrian 320 or a mobile terminal Can be performed. The controller 100 receives the communication related information from the communication module 210 and can determine the position of the pedestrian 320 or the motorcycle (not shown) based on the communication.

For example, the communication module 210 mounted on the vehicle 310 may perform communication with the mobile terminal 321 carried by the pedestrian 320. The mobile terminal 321 may include a GPS module (not shown). The mobile terminal 321 may be provided with an application for transmitting and receiving the current location information of the mobile terminal 321. The communication module 210 may receive the current location information transmitted by the mobile terminal 321. The communication module 210 transmits the received positional information of the mobile terminal 321 to the controller 100 and the controller 100 transmits the position information of the mobile terminal 321 to the pedestrian 320 or the motor vehicle 321 carrying the mobile terminal 321, (Not shown) can be grasped.

For example, suppose the communication module 210 is a DGPS-based communication device. The mobile terminal 321 carried by the pedestrian 320 or the motorcycle driver (not shown) transmits the current position information to the satellite 350. [ The GPS receiver 340 is installed at a reference point that knows the location of the mobile terminal 321, receives the signal from the satellite, corrects the error, and transmits the corrected location information of the mobile terminal 321 to the communication module 210 via the terrestrial wireless communication network . The communication module 210 transmits the location information of the received mobile terminal 321 to the controller 100. The controller 100 can determine the position of the pedestrian 320 or the motorcycle (not shown) carrying the mobile terminal 321 through the received location information.

Meanwhile, the communication module 210 can compensate for the disadvantages of the radar module 40 or the vision module 30. For example, if the radar module 40 detects the vision module 30, or if the vision module 30 does not detect the object, or if the vision module 30 senses that the object ahead of the vehicle 310 is the pedestrian 320, The controller 100 synthesizes the information received from the communication module 210, the radar module 40 and the vision module 30 to determine the type and position of the object in front of the vehicle Can be determined.

Referring again to FIG. 3, the warning module 110 detects the possibility of collision with an object in front of the automobile and outputs a warning message to the driver. The warning message may be a visual, auditory, or tactile message. The warning module 110 outputs a warning message under the control of the controller 100 based on the predicted collision time with an object ahead of the vehicle calculated by the controller 100. [

The power / braking module 120 detects the possibility of collision with an object in front of the automobile and automatically decelerates or stops the automobile when it is determined that a collision is inevitable. The power / braking module 120 decelerates or stops the vehicle under the control of the controller 100 based on the predicted collision time with an object ahead of the vehicle calculated by the controller 100.

Also, the power / braking module 120 may accelerate or decelerate the automobile so that the object ahead of the automobile is the following automobile, so that the following automobile can follow the following automobile.

The controller 100 receives information on the object ahead of the vehicle detected by the radar module 40. [ The controller 100 receives the image information acquired by the vision module 30. The controller 100 receives communication information with an object ahead of the vehicle performed by the communication module 210. Based on the information received from the communication module 210, the controller 100 determines the position of an object in front of the automobile. The operation of locating the object in front of the automobile is as described with reference to FIG. Based on the information of the object ahead of the car received from the radar module 40, the image information received from the vision module 30, and the position information of the object received from the radar module 40, the controller 100 determines whether an object ahead of the vehicle is an obstacle to avoid collision The obstacle may be a pedestrian or a motorcycle. Here, the controller 100 can determine whether an object ahead of the vehicle is an obstacle based on the velocity of the object among the information of the object received from the radar module 40. [ For example, when the speed of an object ahead of the automobile is 60 km / h, the controller 100 can judge the object as an automobile. If the speed of the object ahead of the vehicle is 5 km per hour, the controller 100 can determine the object as a pedestrian. In addition, the controller 100 may determine whether an object ahead of the vehicle is an obstacle based on the size or shape of an object ahead of the vehicle extracted from the image information received from the vision module 40. In this case, the controller 100 can perform the operation of interpreting the image information received from the vision module 30. [ The controller 100 can determine whether the object ahead of the vehicle is an obstacle based on the direction in which the object moves based on the position information. For example, when an object in front of the automobile travels across a road on which the automobile travels, it can be determined that the object is a pedestrian crossing the road.

When the object ahead of the automobile is not an obstacle to avoid a collision, the controller 100 determines, based on the information of the object received from the radar module 40, the image information received from the vision module 30, It is judged whether or not the object ahead is the following vehicle. The determination as to whether or not the vehicle is a preceding leading vehicle can be made based on the speed of the object among the information of the object received from the radar module 40, The determination as to whether the vehicle is a preceding leading vehicle can be made based on the size or shape of an object ahead of the vehicle extracted from the image information received from the vision module 40. In addition, the determination as to whether the vehicle is a preceding leading vehicle may be based on the direction in which the object moves, based on the position information.

When the object ahead of the automobile is the following automobile, the controller 100 controls the power / braking module 120 to follow and follow the following automobile. At this time, the power / braking module 120 can accelerate or decelerate the automobile following the following predecessor under the control of the controller 100.

When the object ahead of the vehicle is an obstacle, the controller 100 calculates a time to collision (TTC) with the object based on the information of the object ahead of the car received from the radar module 40.

Generally, the collision prediction time with an object in front of the vehicle is obtained based on the relative distance between the object ahead and the relative speed with respect to the object ahead of the object detected by the radar module 40. In other words,

The collision prediction time with an object in front of the automobile can be calculated.

The controller 100 receives image information on the front of the automobile from the vision module 30. [ The controller 100 can extract an image corresponding to an object ahead of the vehicle detected by the radar module 40 from the image information acquired by the vision module 30 and calculate the collision prediction time based on the extracted object image . That is, the measurement of the relative distance with the extracted object can be performed in the vision module 30 including a plurality of cameras. The relative distance measurement from the vision module 30 to the preceding vehicle can be made by a disparity map. (First relative distance) with an object in front of the vehicle measured through the radar module 40 or a relative distance (second relative distance) with an object in front of the vehicle measured through the vision module 30 to the mathematical expression The collision prediction time can be obtained by substituting the equation (1). At this time, the collision prediction time can be calculated using the average value of the first relative distance and the second relative distance to increase the accuracy of the relative distance with respect to the object ahead of the automobile.

The controller 100 can recognize the shapes of the objects ahead of the vehicle detected by the radar module 40 through the vision module 30. [ The controller 100 can recognize the shape that the vision module 30 can not recognize through the communication module 210. [ The information on the obstacle (for example, a pedestrian or a motorcycle) obtained through the communication module 210 may compensate for the disadvantages of the radar module 40 or the vision module 30 in the case of bad weather or night driving.

The controller 100 controls the warning module 110. The controller 100 controls the warning module 110 to output a warning message based on the calculated collision prediction time. That is, when the collision prediction time is equal to or less than the first reference time, the controller 100 controls the warning module 110 to output a warning message. At this time, the first reference time may be a time set by the automobile manufacturer, or a time defined by the euro cap. For example, if the first reference time is set to 5 seconds, the warning module 110 does not operate if the collision prediction time calculated by the controller 100 is 7 seconds. If the collision prediction time calculated by the controller 100 reaches 4 seconds after the vehicle approaches the front object, the warning module 110 outputs a warning message under the control of the controller 100. The warning message may be a visual, auditory, or tactile message as described above.

The controller 100 controls the power / brake module 120. The controller 100 controls the power / braking module 120 to decelerate or stop the vehicle based on the calculated collision prediction time. That is, when the collision prediction time is equal to or less than the second reference time, the controller 100 controls the power / braking module 120 to decelerate or stop the vehicle. At this time, the second reference time may be a time set by the automobile manufacturer. For example, assuming that the second reference time is set to 3 seconds, the power / braking module 120 does not operate when the collision prediction time calculated by the controller 100 is 5 seconds. If the collision prediction time calculated by the controller 100 reaches 2 seconds, the power / braking module 120 decelerates or stops the vehicle under the control of the controller 100.

On the other hand, the automobile may include the driving assistance system described above. Here, the automobile may be an internal combustion engine automobile, an electric automobile or a hybrid automobile.

5 is a flowchart according to an embodiment of the present invention.

Referring to FIG. 5, the radar module 40 is mounted in a vehicle. The radar module 40 detects an object in front of the automobile and acquires information about an object in front of the automobile. Here, the acquired information may be distance, position, direction, and speed with respect to an object in front of the automobile.

The radar module 40 emits an electromagnetic wave to the preceding vehicle, and receives electromagnetic waves reflected from an object in front of the automobile to sense distance, position, direction, speed, etc. with the object. The radar module 40 transmits information about the detected leading vehicle to the controller 100.

The controller 100 receives information on the object ahead of the vehicle detected by the radar module 40 (S310).

The vision module 30 is mounted on a vehicle. The vision module 30 is disposed at the front end of the automobile and captures an image of the front of the automobile. That is, the image information of the front of the automobile is acquired. The vision module 30 may include a plurality of cameras. The vision module 30 including a plurality of cameras can measure the distance to the object included in the image information. The vision module 30 transmits the obtained image information of the front of the automobile to the controller 100. [

The controller 100 receives the image information acquired by the vision module 30 (S320).

The communication module 210 is mounted on a vehicle. The communication module 210 performs communication with an object in front of the automobile. At this time, the communication performed with the object in front of the automobile may be bi-directional or unidirectional communication.

On the other hand, the object ahead of the vehicle in which the communication module 210 performs communication may be a pedestrian, a two-wheeled vehicle, or an automobile. The communication module 210 can perform communication with an object in front of the automobile at regular intervals. For example, when the object ahead of the automobile is a pedestrian or a two-wheeled vehicle, the communication module 210 may communicate with a mobile terminal equipped with an application for transmitting and receiving position information held by a pedestrian or a motorcycle driver. The communication module 210 may continuously transmit an interrogation signal (for example, within 20 msec) to the mobile terminal for communication with the mobile terminal carried by the pedestrian or the motorcycle driver.

The communication module 210 may be a GPS (Global Positioning System) or a DGPS (Differential Global Positioning System) based communication device. DGPS is a GPS positioning method of relative positioning method, which corrects the elements causing error by using known reference point coordinates and obtains accurate position by minimizing error.

The controller 100 receives communication information with an object ahead of the vehicle performed by the communication module 210. The controller 100 determines the position of an object ahead of the vehicle based on the information received from the communication module 210 (S330). The operation of locating the object in front of the automobile is as described with reference to FIG.

Based on the information of the object ahead of the car received from the radar module 40, the image information received from the vision module 30, and the position information of the object received from the radar module 40, the controller 100 determines whether an object ahead of the vehicle is an obstacle to avoid collision (S340). The obstacle may be a pedestrian or a motorcycle. Here, the controller 100 can determine whether an object ahead of the vehicle is an obstacle based on the velocity of the object among the information of the object received from the radar module 40. [ For example, when the speed of an object ahead of the automobile is 60 km / h, the controller 100 can judge the object as an automobile. If the speed of the object ahead of the vehicle is 5 km per hour, the controller 100 can determine the object as a pedestrian. In addition, the controller 100 may determine whether an object ahead of the vehicle is an obstacle based on the size or shape of an object ahead of the vehicle extracted from the image information received from the vision module 40. In this case, the controller 100 can perform the operation of interpreting the image information received from the vision module 30. [ The controller 100 can determine whether the object ahead of the vehicle is an obstacle based on the direction in which the object moves based on the position information. For example, when an object in front of the automobile travels across a road on which the automobile travels, it can be determined that the object is a pedestrian crossing the road.

When the object ahead of the vehicle is an obstacle, the controller 100 calculates a time to collision (TTC) with the object based on the information of the object ahead of the car received from the radar module 40 (S350) .

Generally, the collision prediction time with an object in front of the vehicle is obtained based on the relative distance between the object ahead and the relative speed with respect to the object ahead of the object detected by the radar module 40.

The controller 100 determines whether the collision prediction time is shorter than a first reference time (S360).

If the collision prediction time is less than the first reference time, the controller 100 controls the warning module 110 to output a warning message (S370). At this time, the first reference time may be a time set by the automobile manufacturer. For example, if the first reference time is set to 5 seconds, the warning module 110 does not operate if the collision prediction time calculated by the controller 100 is 7 seconds. If the collision prediction time calculated by the controller 100 reaches 4 seconds after the vehicle approaches the front object, the warning module 110 outputs a warning message under the control of the controller 100. The warning message may be a visual, auditory, or tactile message as described above.

The controller 100 determines whether the collision prediction time is shorter than a second reference time (S380).

If the collision prediction time is less than the second reference time, the controller 100 controls the power / braking module 120 to decelerate or stop the vehicle (S390). At this time, the second reference time may be a time set by the automobile manufacturer. For example, assuming that the second reference time is set to 3 seconds, the power / braking module 120 does not operate when the collision prediction time calculated by the controller 100 is 5 seconds. If the collision prediction time calculated by the controller 100 reaches 2 seconds, the power / braking module 120 decelerates or stops the vehicle under the control of the controller 100.

If it is determined in step S340 that the object ahead of the vehicle is not an obstacle that should be avoided from collision, the controller 100 determines whether or not the object information received from the radar module 40, the image information received from the vision module 30, Based on the information, it is determined whether the object ahead of the automobile is the following automobile (S342). The determination as to whether or not the vehicle is a preceding leading vehicle can be made based on the speed of the object among the information of the object received from the radar module 40, The determination as to whether the vehicle is a preceding leading vehicle can be made based on the size or shape of an object ahead of the vehicle extracted from the image information received from the vision module 40. In addition, the determination as to whether the vehicle is a preceding leading vehicle may be based on the direction in which the object moves, based on the position information.

When the object ahead of the automobile is the following automobile, the controller 100 controls the power / braking module 120 to follow the following automobile (S345). At this time, the power / braking module 120 can accelerate or decelerate the automobile following the following predecessor under the control of the controller 100.

6 is a flowchart illustrating pedestrian recognition and supplementation through a communication module according to an embodiment of the present invention.

Referring to FIG. 6, the mobile terminal installs a pedestrian position tracking application according to a user's command (S460). The mobile terminal turns on the position transmission function of the pedestrian position tracking application (S470). The mobile terminal transmits the position of the pedestrian in real time (S480). Here, the position tracking or position transmission can be performed through a satellite or through a base station installed on the ground.

On the other hand, the operation performed in the vehicle is performed separately from the mobile terminal. The ignition is turned on (S410).

The controller 100 determines whether the AEB (Advanced Emergency Braking) and ACC (Adaptive Cruise Control) functions are on (S415).

If the AEB and ACC functions are on, the controller 100 determines whether an object ahead of the vehicle is an obstacle based on the information received from the vision module 30 or the radar module 40 (S420). Here, if the object ahead of the automobile is not an obstacle, the controller 100 controls the automobile to travel at a constant speed (S450).

If the object ahead of the vehicle is an obstacle, the controller 100 determines whether the object ahead of the vehicle is a pedestrian (S425). The determination as to whether the pedestrian is a pedestrian can be made by communication with the mobile terminal performed by the communication module 210. [ The communication between the communication module 210 and the mobile terminal is as described with reference to FIG. 3 and FIG.

In step S425, when the obstacle is the following vehicle, the controller 100 controls to follow the following vehicle ahead (S430).

In step S425, if the obstacle is a pedestrian, the controller 100 determines the possibility of collision with the pedestrian (S435). Here, the determination of the possibility of collision may be performed based on the time to collision (TTC) as described above.

If it is determined in step S435 that there is a possibility of collision, the controller 100 controls the warning module 110 to perform a warning.

If it is determined in step S435 that the emergency braking is required, the controller 100 controls the power / braking module 120 to perform emergency braking (S445).

At this point, it will be appreciated that the combinations of blocks and flowchart illustrations in the process flow diagrams may be performed by computer program instructions. These computer program instructions may be loaded into a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, so that those instructions, which are executed through a processor of a computer or other programmable data processing apparatus, Thereby creating means for performing functions. These computer program instructions may also be stored in a computer usable or computer readable memory capable of directing a computer or other programmable data processing apparatus to implement the functionality in a particular manner so that the computer usable or computer readable memory The instructions stored in the block diagram (s) are also capable of producing manufacturing items containing instruction means for performing the functions described in the flowchart block (s). Computer program instructions may also be stored on a computer or other programmable data processing equipment so that a series of operating steps may be performed on a computer or other programmable data processing equipment to create a computer- It is also possible for the instructions to perform the processing equipment to provide steps for executing the functions described in the flowchart block (s).

In addition, each block may represent a module, segment, or portion of code that includes one or more executable instructions for executing the specified logical function (s). It should also be noted that in some alternative implementations, the functions mentioned in the blocks may occur out of order. For example, two blocks shown in succession may actually be executed substantially concurrently, or the blocks may sometimes be performed in reverse order according to the corresponding function.

Here, the term " module " used in this embodiment means a hardware component such as software or an FPGA or an ASIC, and the module performs certain roles. However, a module is not limited to software or hardware. A module may be configured to reside on an addressable storage medium and configured to play back one or more processors. Thus, by way of example, a module may include components such as software components, object-oriented software components, class components and task components, and processes, functions, attributes, procedures, Microcode, circuitry, data, databases, data structures, tables, arrays, and variables, as will be appreciated by those skilled in the art. The functionality provided within the components and modules may be combined into a smaller number of components and modules or further separated into additional components and modules. In addition, the components and modules may be implemented to play back one or more CPUs in a device or a secure multimedia card.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

30: Vision module
40: Radar module
100: controller
110: Warning Module
120: Power / Brake Module
210: Communication module

Claims (18)

A radar module for detecting an object ahead and acquiring first speed information of the object;
A vision module having a plurality of cameras and acquiring image information on the object;
A communication module that communicates with the mobile terminal held by the object and receives the position of the object and the second rate information; And
A warning module for warning a collision with the object; And
Based on the first speed information and the shape of the object extracted from the image information,
Determining whether the object is an obstacle based on the position and the second speed information when it is impossible to determine whether the obstacle is an obstacle based on the first speed information and the shape,
Calculating a collision prediction time with the obstacle based on the first or second speed information and a disparity map obtained through the plurality of cameras when the object is an obstacle,
And a controller for controlling the warning module to output a warning message if the collision prediction time is equal to or less than a first reference time. The method according to claim 1,
Wherein the object is a pedestrian or a motorcycle driver walking on the road on which the automobile travels,
The communication module includes a driving assist system that performs bidirectional communication with a mobile terminal equipped with an application for transmitting and receiving position information held by the pedestrian or the motorcycle driver The method according to claim 1,
The communication module includes:
And performs communication with the object at regular intervals. The method according to claim 1,
Wherein the communication module is a GPS or DGPS-based communication device. The method according to claim 1,
The controller comprising:
If the object is not an obstacle,
And recognizes a pedestrian or a motorcycle that determines whether the object is a preceding vehicle based on the first speed and the shape. 6. The method of claim 5,
Further comprising a power / brake module for accelerating or decelerating the automobile under the control of the controller,
If the object is a preceding vehicle,
And the controller controls the power / braking module to follow and follow the following preceding-following vehicle. delete The method according to claim 1,
Wherein the warning message is a visual, audible or tactile message. The method according to claim 1,
Further comprising a power / braking module for accelerating, decelerating or stopping the automobile under the control of the controller,
If the object is an obstacle,
The controller calculates a collision prediction time with an obstacle,
If the collision prediction time is equal to or less than the second reference time,
And the controller automatically brakes the vehicle by controlling the power / braking module. Receiving first speed information of an object ahead of the vehicle detected by the radar module;
Receiving image information on the object obtained by a vision module having a plurality of cameras;
Communicating with a mobile terminal carried by the object through a communication module, receiving a position of the object and second velocity information;
Determining whether the obstacle is an obstacle based on the speed information and the shape of the object extracted from the image information;
Determining whether the object is an obstacle based on the position and the second speed information received through the communication module when it is impossible to determine whether the obstacle is an obstacle based on the speed information and the shape;
Calculating a collision prediction time with the obstacle through the first or second velocity information and the parallax map obtained through the plurality of cameras when the object is an obstacle; And
And outputting a warning message through the warning module when the collision prediction time is equal to or less than a first reference time. 11. The method of claim 10,
Wherein the object is a pedestrian or a motorcycle driver walking on the road on which the automobile travels,
Wherein the communication module performs bidirectional communication with a mobile terminal equipped with an application for transmitting and receiving location information held by the pedestrian. 11. The method of claim 10,
The communication module includes:
Wherein the communication is performed with the object at a constant cycle. 11. The method of claim 10,
Wherein the communication module is a GPS or DGPS-based communication device. 11. The method of claim 10,
If the object is not an obstacle,
And determining whether the object is a preceding vehicle based on the first speed and the shape. 15. The method of claim 14,
If the object is a preceding vehicle,
Further comprising the step of following and following said following preceding vehicle. delete 17. The method of claim 16,
Wherein the warning message is a visual, audible or tactile message. 11. The method of claim 10,
If the object is an obstacle,
The collision prediction time with the obstacle is calculated,
If the collision prediction time is equal to or less than the second reference time,
Further comprising the step of automatically braking the vehicle.

Images