KR102485318B1 - System and Method for Recognizing Cut-in Vehicle on the basis of Monocular Image - Google Patents

Abstract

The present invention discloses a system and method for overtaking and merging vehicle recognition based on a monocular image. An overtaking and merging (cut-in) vehicle recognition system according to an aspect of the present invention includes an IPM image generation unit that converts a monocular image captured by a vehicle front or side camera into an IPM (Inverse Perspective Mapping) image; a motion detection unit detecting a motion component in consecutive frames of the IMP image; and calculating a longitudinal speed and a lateral speed from the motion vector of each motion component, and determining a possibility of collision with an object intervening into the driving lane of the vehicle based on at least one of the calculated longitudinal speed and lateral speed. It is characterized in that it comprises a collision determination unit to.

Description

System and Method for Recognizing Cut-in Vehicle on the basis of Monocular Image}

The present invention relates to an overtaking and merging vehicle detection technology, and more particularly, to a monocular image-based overtaking and merging vehicle recognition system and method capable of recognizing an overtaking and merging vehicle using a monocular image.

A conventional vehicle detection method is a method of extracting feature points of a vehicle and then detecting a vehicle shape based on machine learning. However, it is difficult to detect a vehicle located in the front side or front and rear of the vehicle in the above-described manner because it cannot be seen.

On the other hand, in the conventional radar-based vehicle detection method that can secure this, due to the nature of the sensor, it is not until the L-shaped structure with the rear or rear side of the merging vehicle merging in the direction of the host vehicle is identified that the merging vehicle is detected. was recognizable.

Therefore, the conventional radar-based vehicle detection method cannot recognize a merging vehicle at a point in time before the merging vehicle cuts in or before the rear of the merging vehicle is visible, which is the moment when the merging vehicle cuts in.

In addition, the conventional lane recognition-based cutting-in detection method cannot predict cut-in when the lane is not recognized, and has a problem in that it is difficult to recognize the lane itself in a low-speed driving situation such as congestion.

An object of the present invention is to provide a monocular image-based overtaking and merging vehicle recognition system and method capable of recognizing an overtaking and merging vehicle using a motion component of a monocular image.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

An overtaking and merging (cut-in) vehicle recognition system according to an aspect of the present invention includes an IPM image generation unit that converts a monocular image captured by a vehicle front or side camera into an IPM (Inverse Perspective Mapping) image; a motion detection unit detecting a motion component in consecutive frames of the IMP image; and calculating a longitudinal speed and a lateral speed from the motion vector of each motion component, and determining a possibility of collision with an object intervening into the driving lane of the vehicle based on at least one of the calculated longitudinal speed and lateral speed. It is characterized in that it comprises a collision determination unit to.

According to another aspect of the present invention, a method for recognizing overtaking and merging vehicles by at least one processor includes converting a monocular image captured by a front or side camera of the vehicle into an Inverse Perspective Mapping (IPM) image; detecting a motion component in consecutive frames of the IMP image; calculating longitudinal and lateral velocities from the motion vectors of each motion component; and determining a possibility of collision with a target object intervening into the driving lane of the vehicle based on at least one of the calculated longitudinal speed and lateral speed.

According to the present invention, a vehicle attempting to enter the driving lane of the host vehicle can be detected using motion components of the image.

1 is a configuration diagram illustrating a system for recognizing an overtaking and merging vehicle according to an embodiment of the present invention;
2 is a picture showing an IPM image and a corrected image according to an embodiment of the present invention.
3 is a diagram for explaining motion vectors and longitudinal/lateral velocities according to an embodiment of the present invention;
4 is a flowchart illustrating a method for recognizing overtaking and merging vehicles according to an embodiment of the present invention;

The foregoing and other objects, advantages and characteristics of the present invention, and a method of achieving them will become clear with reference to the detailed embodiments described below in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" means that a stated component, step, operation, and/or element is the presence of one or more other components, steps, operations, and/or elements. or do not rule out additions.

Now, embodiments of the present invention will be described in detail with reference to the accompanying drawings. 1 is a configuration diagram showing an overtaking and merging vehicle recognition system according to an embodiment of the present invention, FIG. 2 is a picture showing an IPM image and a corrected image according to an embodiment of the present invention, and FIG. It is a diagram for explaining motion vectors and longitudinal/lateral velocities according to an embodiment.

As shown in FIG. 1 , the overtaking and merging vehicle recognition system 10 according to an embodiment of the present invention includes a first image acquisition unit 111, a first motion detection unit 121, a validity detection unit 130, a second Image acquisition unit 112, IPM image generator 140, lane recognition unit 150, rotation correction unit 160, second motion detection unit 122, lane-based collision determination unit 170, motion-based collision plate It includes a government 180 and an alarm unit 190.

The first image acquisition unit 111 acquires a front image of a vehicle equipped with the overtaking and merging vehicle recognition system 10 according to the present invention. In this case, the first image acquisition unit 111 may be a monocular camera mounted on the front of the vehicle to photograph the front of the vehicle, or may be a receiving means for acquiring an image from the monocular camera. In this specification, a case in which the first image acquisition unit 111 is a monocular camera will be described as an example.

The first motion detection unit 121 detects motion components in consecutive frames of the image acquired by the first image acquisition unit 111 .

At this time, the first motion detector 121 extracts only a partial region of the image frame, for example, a region of interest (ROI), calculates an optical flow within the region of interest, or calculates a motion component through a difference between two adjacent images. can be detected. For example, the ROI may be a left-side area or a right-side area of the vehicle.

Accordingly, in the present invention, motion components can be calculated at high speed. In the following specification, a case in which the first motion detection unit 121 detects a motion component only for a region of interest will be described as an example.

Upon receiving the detected motion component of the ROI, the validity detector 130 checks whether the corresponding motion component is a valid motion component. Here, the effective motion component may be a motion component that is likely to be a vehicle that is close to the host vehicle or is attempting to overtake or merge.

In this case, the validity detection unit 130 may determine whether or not the motion component is valid based on the number of motion components within a predetermined area in the region of interest. More specifically, the validity detection unit 130 may determine that the motion is valid if there are more than a predetermined threshold number of motion components within a certain area.

Additionally, the validity detection unit 130 may determine that the motion component is a valid motion when recognizing a periodic color change within an area where a motion component exists or a lane of interest (left room, right room). For reference, side mirrors or direction indicators of a vehicle have a characteristic of being turned on and off periodically when lights are turned on. In the present invention, effective movement can be determined using this characteristic.

When the validity detection unit 130 determines that the detected motion component is a valid motion, the validity detection unit 130 may provide a first warning of approaching surrounding vehicles through the warning unit 190 .

The second image acquisition unit 112 acquires a front image, a left room image, or a right room image of a vehicle equipped with the overtaking and merging vehicle recognition system 10 according to the present invention.

At this time, the second image acquisition unit 112 may be a monocular camera mounted on the front, left side or right side of the vehicle to capture the front side, left side or right side of the vehicle, and may be a receiving means for acquiring an image from the monocular camera. may be In this specification, a case in which the second image acquisition unit 112 is a monocular camera will be described as an example.

When the second image acquisition unit 112 acquires an image in the same direction as the first image acquisition unit 111, the second image acquisition unit 112 may be integrated into the first image acquisition unit 111. there is.

The IPM image generator 140 converts the front image from the first image acquisition unit 111 or the front, right, or left room images from the second image acquisition unit 112 into an Inverse Perspective Mapping (IPM) image, respectively. do. Here, the IPM image generator 140 may generate the IPM image by targeting the entire region of the image frame, not the region of interest in which motion information is primarily detected.

In this case, the IPM image generation unit 140 may generate an IPM image, which is a top-view image, as a result of performing IPM conversion based on intrinsic parameters and extrinsic parameters of the camera.

The lane recognizing unit 150 recognizes a lane from the IPM image using components of a straight line on the IPM image. For reference, the own vehicle and the side vehicle do not balance exactly at every moment even when the lane is not changed, but repeatedly move closer or farther slightly. However, if an approach warning occurs in each of these cases, the driver's driving may be hindered. Therefore, in the present invention, lane recognition is performed to more reliably determine a situation in which a collision is concerned.

The rotation correction unit 160 determines the driving direction and direction of the vehicle by setting the center of rotation (red dot in FIG. A correction image is generated by rotating the IPM image so as to be parallel to the lane recognized from the IPM image.

For example, the lane on the IPM image may be tilted to the left by a predetermined angle (θ) with respect to the driving direction of the vehicle, as shown in the left photo of FIG. 2 .

In this case, the rotation correction unit 160 performs correction to rotate the IPM image to the right by a predetermined angle θ to generate a correction image as shown in the right picture of FIG. 2 .

At this time, the rotation correction unit 160 moves and arranges each pixel of the original image (x, y) to the position (x', y') calculated by Equation 1 below to perform rotation correction to obtain a corrected image ( Right photo of FIG. 2) can be output.

However, the lane recognizing unit 150 may not recognize the lane when the vehicle is not instantaneously parallel to the driving lane of the host vehicle in motion due to road curvature, slope, or driving behavior. In this case, the rotation correction unit 160 may determine the lane position based on the previously recognized lane and steering information of the vehicle. At this time, since the lane recognition technology is a configuration that can be derived from the prior art, a detailed description thereof will be omitted.

For example, the rotation correction unit 160 may determine n ₁ and n ₂ , which are x values of a straight line representing a lane, based on at least one of a lane and a vehicle width (see the right photo of FIG. 2 ).

Returning to FIG. 1 again, the second motion detection unit 122 detects motion components in consecutive frames of the corrected image. In this case, the second motion detection unit 122 may project the effective motion component calculated by the first motion detection unit 121 onto the corrected image and use it as additional motion information.

The second motion detection unit 122 may be the same component as the first motion detection unit 121 .

The lane-based collision determination unit 170 uses the motion component detected by the second motion detection unit 122 to determine whether there is a vehicle intending to overtake or merge into the driving lane of the host vehicle.

The lane-based collision determination unit 170 determines that the x-coordinate of the motion vector existing in the left-side room of the vehicle among the motion vectors according to the detected motion components is greater than the x-coordinate of the left-side lane of the driving lane of the vehicle recognized from the correction image. Check if this is an interrupt situation. In addition, the lane-based collision determining unit 170 detects a vehicle on the right side where the x-coordinate of a motion vector existing on the right side of the vehicle among the detected motion components is smaller than the x-coordinate of the right lane of the driving lane of the vehicle recognized from the correction image. Check if a lifting situation occurs.

When it is confirmed that the vehicle on the left side or the vehicle on the right side cuts in, the lane-based collision determination unit 170 issues a secondary warning of vehicle approach through the warning unit 190 .

The motion-based collision determination unit 180 calculates the longitudinal speed and the lateral speed from each motion vector, and based on at least one of the longitudinal speed and the lateral speed, the possibility of collision with the target object intervening in the driving lane of the host vehicle to judge

를 x, y 성분으로 분해하고, 하기의 수학식 2를 이용하여 종방향 속도 Vy와 횡방향 속도 Vx를 계산한다.Specifically, referring to FIG. 3 , the motion-based collision determination unit 180 determines the motion vector detected by the second motion detection unit 122.

is decomposed into x and y components, and the longitudinal speed Vy and the lateral speed Vx are calculated using Equation 2 below.

If the lateral speed is equal to or greater than the preset reference speed, the motion-based collision determination unit 180 instructs at least one of outputting a warning and controlling vehicle deceleration. Specifically, if the lateral speed Vx is greater than or equal to the preset reference speed (Vth), a side vehicle enters the own lane before the own vehicle passes and a collision is expected, so at least one of the 3rd warning and emergency deceleration control can be instructed. there is.

In this case, the reference speed is a reference value for determining whether the side vehicle collides with the host vehicle by considering the speed of the own vehicle and the longitudinal speed of the side vehicle based on Time to Correlation (TTC), and may be experimentally determined.

When deceleration control is performed, the motion-based collision determination unit 180 may determine a deceleration amount for deceleration control based on the longitudinal speed Vy. For example, the motion-based collision determination unit 180 may determine the amount of deceleration so that the vehicle speed is equal to or less than the longitudinal speed.

Alternatively, if the lateral speed is less than the preset reference speed, the motion-based collision determination unit 180 does not issue a separate control instruction or instructs a control for collision prevention depending on whether or not the secondary warning is output.

Specifically, if the lateral speed is less than the predetermined reference speed, the motion-based collision determining unit 180 determines whether the secondary warning is instructed by the lane-based collision determining unit 170 .

As a result of the check, if the secondary warning is indicated, the motion-based collision determination unit 180 instructs at least one of vehicle deceleration control, warning control, and lane change control. More specifically, if the lateral speed is less than the preset reference speed and the secondary warning has been previously instructed, the side vehicle has already entered the driving lane of the host vehicle, but slowly enters the front of the host vehicle, and a collision with the host vehicle is expected in case it becomes Accordingly, the motion-based collision determination unit 180 may issue a third warning through the warning unit 190 while instructing the host vehicle to gently decelerate or change lanes to the opposite lane of the approaching side vehicle.

On the other hand, if the secondary warning is not instructed, the motion-based collision determination unit 180 may not perform special control (deceleration, warning, and lane change).

The warning unit 190 is a component that outputs sounds such as a speaker and a buzzer, and sounds to the driver according to instructions from at least one of the validity determination unit, the lane-based collision determination unit 170, and the motion-based collision determination unit 180. warn of danger Alternatively, the alarm unit 190 may be a display or a haptic.

In this way, the embodiment of the present invention can detect an overtaking or merging vehicle before a side vehicle attempts to enter the own vehicle's lane, thereby significantly reducing the accident rate due to cut-in.

In addition, when an overtaking or merging vehicle is detected, the exemplary embodiment of the present invention supports active control such as deceleration control or lane change control as well as warning, so that an accident can be prevented even when the driver lacks attention.

In addition, the embodiment of the present invention can detect a vehicle attempting to cut in using a monocular image, thereby reducing the cost and effort required for installing additional sensors such as stereo cameras, lidars, or radars. savings can be made

Hereinafter, a method for recognizing overtaking and merging vehicles according to an embodiment of the present invention will be described with reference to FIG. 4 .

Referring to FIG. 4 , the first image acquisition unit 111 captures the front of the vehicle and generates an image frame (S400).

The first motion detection unit 121 receives continuous image frames from the first image acquisition unit 111 and detects motion components (S410). In this case, the first motion detector 121 may designate a region of interest (eg, a left room or a right room) in each image frame and detect a motion component only for the corresponding region of interest.

The validity detection unit 130 checks whether there is a valid movement within the region of interest (S420).

At this time, the validity detection unit 130 may recognize whether a periodic color change occurs within the ROI. Alternatively, the validity detection unit 130 may determine that there is a valid movement if there is a certain number of movements within the region of interest.

Here, when the validity detection unit 130 detects a valid movement within the region of interest, it may provide a primary warning of approaching surrounding vehicles through the warning unit 190 (S510).

Thereafter, the second image acquisition unit 112 captures the front of the vehicle or the left or right side of the vehicle to generate a left or right image frame (S430).

At this time, when the second image acquisition unit 112 photographs the front of the vehicle like the first image acquisition unit 111, the second image acquisition unit 112 may be replaced with the first image acquisition unit 111. .

The IPM image generation unit 140 converts each image frame generated by the second image acquisition unit 112 into an IPM image (S440).

Next, the rotation correction unit 160 rotates the IPM image based on the center of the first lane, and generates a corrected image such that the lane is parallel to the vertical axis of the corrected image (S450).

The second motion detection unit 122 detects motion components from consecutive frames of the corrected image (S460). In this case, the second motion detection unit 122 may project the effective motion detected by the first motion detection unit 121 onto the corrected image and use it as additional motion information.

The lane-based collision determination unit 170 checks whether there is an attempt to enter the driving lane of the host vehicle in at least one of the left lane and the right lane of the host vehicle from the secondly extracted motion components (S470).

Regardless of whether there is an attempt to enter the driving lane of the host vehicle, the motion-based collision determination unit 180 detects motion vectors of all motion components and calculates lateral/longitudinal speeds of each motion vector (S480).

Subsequently, the motion-based collision determination unit 180 checks whether the calculated lateral speed is equal to or greater than a preset reference speed (S490).

If the calculated lateral speed is equal to or greater than the reference speed, the motion-based collision determining unit 180 performs deceleration control (S500). At this time, the motion-based collision determination unit 180 may instruct the warning unit 190 to issue a third warning (S510).

On the other hand, if the calculated lateral speed is less than the reference speed, the motion-based collision determination unit 180 may perform deceleration control or not perform any other control depending on whether there is an attempt to enter the driving lane of the host vehicle. .

Specifically, if there is an attempt to enter the driving lane of the host vehicle and the lateral speed is less than the reference speed, the lane-based collision determination unit 170 warns the vehicle approaching through the warning unit 190 (S510).

In addition, if there is no attempt to enter the driving lane of the host vehicle and the lateral speed is less than the reference speed, the lane-based collision determination unit 170 does not instruct any control.

In this way, the embodiment of the present invention can detect an overtaking or merging vehicle before a side vehicle attempts to enter the own vehicle's lane, thereby significantly reducing the accident rate due to cut-in.

In addition, when an overtaking or merging vehicle is detected, the exemplary embodiment of the present invention supports active control such as deceleration control or lane change control as well as warning, so that an accident can be prevented even when the driver lacks attention.

In addition, the embodiment of the present invention can detect a vehicle attempting to cut in using a monocular image, thereby reducing the cost and effort required for installing additional sensors such as stereo cameras, lidars, or radars. savings can be made

In the above, the configuration of the present invention has been described in detail with reference to the accompanying drawings, but this is only an example, and various modifications and changes within the scope of the technical idea of the present invention to those skilled in the art to which the present invention belongs Of course this is possible. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be defined by the description of the claims below.

10: Overtaking and merging vehicle recognition system
111, 112: first and second image acquisition units 121, 122: first and second motion detection units
130: validity determination unit 140: IPM image generation unit
150: lane recognition unit 160: rotation correction unit
170: lane-based collision determining unit 180: motion-based collision determining unit
190: alarm department

Claims (15)

a first motion detection unit detecting a motion component in continuous frames of a monocular image captured by a front or side camera of the vehicle;
an validity detection unit for giving an alarm through an alarm unit when there is a valid motion in the motion component detected from the monocular image;
an IPM image generating unit that converts the monocular image into an IPM (Inverse Perspective Mapping) image;
a second motion detector detecting a motion component in consecutive frames of the IPM image;
Calculating a longitudinal speed and a lateral speed from the motion vector of each motion component, and determining a possibility of collision with an object intervening into the driving lane of the vehicle based on at least one of the calculated longitudinal speed and lateral speed Including; motion-based collision determination unit;
The validity detection unit,
The overtaking and merging (Cut-in) vehicle recognition system, characterized in that it is determined that there is the effective motion when a periodic color change is recognized in the region of interest where the motion component detected from the monocular image exists. delete In claim 1, the validity detection unit,
wherein the overtaking and merging vehicle recognition system determines that there is valid motion when motion components detected from the monocular image exist within a preset predetermined area of more than a threshold number of motions. In paragraph 1,
A rotation correction unit for rotating the IPM image so that the driving direction of the vehicle and the lane recognized from the IPM image are parallel with the center of the front or side camera as the center of rotation
Overtaking and merging vehicle recognition system further comprising a. In paragraph 1,
Among the motion vectors, the x-coordinate of the motion vector existing on the left side of the vehicle is greater than the x-coordinate of the left lane of the driving lane of the vehicle recognized from the IPM image, or the x-coordinate of the motion vector existing on the right side of the vehicle a lane-based collision determination unit that checks whether a cut-in situation occurs where the x-coordinate is smaller than the x-coordinate of the right lane of the driving lane of the vehicle recognized from the IPM image, and if the cut-in situation occurs, issues an alarm through an alarm unit;
Overtaking and merging vehicle recognition system further comprising a. In claim 5, the motion-based collision determining unit,
If the lateral speed is less than a predetermined reference speed, checking whether the cut-in situation is determined by the lane-based collision determination unit, and if the cut-in situation is not determined, not performing deceleration and warning control. overtaking and merging vehicle recognition system. In claim 5, the motion-based collision determining unit,
If the lateral speed is less than a predetermined reference speed, it is determined whether the cut-in situation is determined by the lane-based collision determination unit, and if the cut-in situation is determined, during vehicle deceleration control, warning control, and lane change control An overtaking and merging vehicle recognition system that indicates at least one. In claim 1, the motion-based collision determining unit,
and instructing at least one of an alarm output and vehicle deceleration control when the lateral speed is greater than or equal to a predetermined reference speed. A method of overtaking and merging vehicle recognition by at least one processor, comprising:
detecting, by a first motion detection unit, a motion component in continuous frames of a monocular image captured by a front or side camera of the vehicle;
alerting, by a validity detection unit, through an alarm unit when there is a valid motion in the motion component detected from the monocular image;
Converting, by an IPM image generation unit, a monocular image captured by a front or side camera of a vehicle into an Inverse Perspective Mapping (IPM) image;
detecting, by a motion detection unit, a motion component from consecutive frames of the IPM image;
calculating, by a motion-based collision determination unit, a longitudinal speed and a lateral speed from motion vectors of each motion component;
determining, by the motion-based collision determination unit, a possibility of collision with a target object intervening into the driving lane of the vehicle based on at least one of the calculated longitudinal speed and lateral speed;
The step of alerting the validity detection unit through the warning unit,
Overtaking and merging (cut-in) vehicle recognition method characterized in that it is performed based on the occurrence of periodic color change in the region where the motion component detected from the monocular image exists. delete 10. The method of claim 9, wherein the step of alerting the validity detection unit through the warning unit,
confirming whether or not motion components detected from the monocular image exist within a predetermined area of a threshold number or more; and
The overtaking and merging vehicle recognizing method further includes the step of alerting through the warning unit when there is more than the threshold number of movements. In paragraph 9,
generating a correction image by a rotation correction unit by rotating the IPM image so that the driving direction of the vehicle is parallel to the lane recognized from the IPM image with the center of the front or side camera as a rotation center;
The lane-based collision determination unit determines that the x-coordinate of a motion vector existing in the left side of the vehicle among motion vectors generated from the corrected image is greater than the x-coordinate of the left lane of the driving lane of the vehicle recognized from the IPM image; checking whether a cut-in situation occurs in which the x-coordinate of the motion vector existing in the right side of the vehicle is smaller than the x-coordinate of the right lane of the driving lane of the vehicle recognized from the IPM image; and
Alerting, by the lane-based collision determination unit, through a warning unit when the cut-in situation occurs
Overtaking and merging vehicle recognition method further comprising a. In claim 12, the step of determining the possibility of collision with the target object,
checking whether the cut-in situation is determined by the lane-based collision determination unit when the lateral speed is less than a predetermined reference speed; and
and not performing deceleration and warning control if the cut-in situation is not determined. In claim 12, the step of determining the possibility of collision with the target object,
checking whether the cut-in situation is determined by the lane-based collision determination unit when the lateral speed is less than a predetermined reference speed; and
and instructing at least one of vehicle deceleration control, warning control, and lane change control if the cut-in situation is determined. In claim 9, the step of determining the possibility of collision with the target object,
Instructing at least one of a warning output and vehicle deceleration control when the lateral speed is equal to or greater than a preset reference speed
Overtaking and merging vehicle recognition method comprising a.

Images