KR102267256B1 - Method and apparatus for detecting obstacle, and computer readable medium the same - Google Patents

Abstract

Disclosed are an obstacle detecting method, apparatus, and a computer-readable recording medium for identifying an obstacle by identifying a movement thereof.
To this end, the present embodiment clusters at least one object movement detected in an image, sub-blocks the clustered object to be placed in at least one block, and from this, at least one object movement trajectory and at least one object A mechanism is provided for extracting a motion prediction line, determining whether a trajectory coincides between the extracted at least one object motion trajectory and the at least one object motion prediction line, and recognizing it as any one of a virtual object, a stationary object, and a moving object.
Accordingly, the present embodiment can prevent an accident in advance by detecting all kinds of obstacles.

Description

Obstacle detection method, apparatus, and a computer-readable recording medium therefor {METHOD AND APPARATUS FOR DETECTING OBSTACLE, AND COMPUTER READABLE MEDIUM THE SAME}

The present embodiment relates to an obstacle detection technology, and more particularly, to an obstacle detection method, apparatus, and computer-readable recording medium for identifying an obstacle by distinguishing the movement.

The existing obstacle detection technology detects an obstacle by analyzing the position and brightness change of a moving object through image difference or optical flow calculation.

In this case, when the camera of the driving vehicle moves, the existing obstacle detection technology can detect the movement of a moving object and the movement of the vehicle at the same time. If removed from , only the movement of the moving object is detected, thereby detecting the moving object.

At this time, the movement of the surrounding background does not have a height such as road markings or cracks, but an imaginary obstacle whose image changes according to the movement of a vehicle or camera and a real obstacle in which movement is expressed with an actual height exist at the same time. , in the case of a distance sensor or a binocular camera, virtual obstacles and real obstacles were classified based on the distance to the object and ground surface conditions.

However, when a monocular camera is used, only a moving object is detected because it is difficult to distinguish the aforementioned obstacles.

As such, the existing obstacle detection technology is useful only for detecting a moving object when the vehicle is moving, but has a problem in that it cannot detect a stationary object at the same time.

An object of the present embodiment is to provide an obstacle detecting method and apparatus for simultaneously recognizing a stationary obstacle or a virtual obstacle as well as a moving obstacle when a vehicle is moving, and a computer readable recording medium therefor, in order to solve the above problems.

According to one embodiment, there is provided an obstacle detection method performed in an obstacle detection apparatus, the method comprising: detecting at least one object movement captured in an image captured by a camera; clustering the detected at least one object movement; sub-blocking the clustered object to be placed in at least one block, extracting at least one object movement trajectory based on the center of each block within the sub-block using object tracking, and using the vehicle movement prediction line extracting at least one object motion prediction line from within the lower block, and determining whether a trajectory coincides between the extracted at least one object motion trajectory and the at least one object motion prediction line to determine whether a virtual object, a stationary object, or a moving object It provides an obstacle detection method comprising the step of recognizing any one.

In the sub-blocking step, when there is no clustered object, the corresponding block may be excluded from the sub-block.

The lower block may refer to a block composed of a plurality of blocks arranged by dividing the clustered object.

The object tracking may refer to tracking the movement of the center of each block.

The vehicle movement prediction line may be extracted from at least one of steering information, vehicle information, and bird's eye view.

The determining may include comparing the motion of the at least one object with the prediction line of the at least one motion of the at least one object, and when the trajectory is partially identical, the stationary obstacle may be recognized.

The determining may include comparing the movement of the at least one object with the prediction line of the movement of the at least one object and recognizing it as the moving obstacle when the trajectories do not match.

The determining may include comparing the motion of the at least one object with the prediction line of the at least one motion of the object, and when all of the trajectories are identical, it may be recognized as the virtual obstacle.

According to one embodiment, there is provided an obstacle detection method performed by an obstacle detection apparatus, the method comprising: detecting movement of at least one object attached to an image captured by a camera; clustering the movement of the detected at least one object; sub-blocking the clustered object to be placed in at least one block, extracting at least one object movement trajectory based on the center of each block within the sub-block using object tracking, and using the vehicle movement prediction line Extracting the object motion prediction line of each block from within the sub-block, and after tracking and updating at least one sub-blocked block at every cycle, from the center of each block, the extracted object motion prediction line of each block Provided is an obstacle detection method comprising the step of determining whether the sum of the shortest distances exceeds a limit value.

In the sub-blocking step, when there is no clustered object, the corresponding block may be excluded from the sub-block.

The object tracking tracks the position movement of the center of each block, and the vehicle movement prediction line may be extracted from at least one of steering information, vehicle information, and bird's eye view.

The lower block may be composed of a plurality of blocks arranged by dividing the clustered object.

In the determining step, if the threshold is less than the threshold value, the object motion prediction line of each block may be regarded as coincident with the at least one object motion trajectory, and may be recognized as a virtual obstacle.

In the determining step, if the threshold value is greater than the threshold value, the object motion prediction line of each block may be regarded as a movement trajectory inconsistent with the at least one object motion trajectory and recognized as a moving obstacle.

In the determining step, if some are above the threshold and some are below the threshold, it is considered that the object motion prediction line of each block partially coincides with the at least one object motion trajectory, and it can be recognized as a stationary obstacle. .

As described above, in the present embodiment, unlike the method of detecting the movement of a moving object by removing the movement of the camera from all movement of objects in the image, the movement of the camera and movement of the moving object are distinguished from movement of the camera in all movement of objects in the image, By additionally analyzing the movement of the vehicle to distinguish virtual obstacles (road markings, etc.) from real obstacles (moving obstacles, stationary obstacles), it is possible to prevent accidents in advance by detecting all kinds of obstacles.

The present embodiment can prevent a collision with an obstacle hit when parking and/or exiting the vehicle and while driving the vehicle by accurately notifying the user of the location of the obstacle through the above-described obstacle detection technique.

This embodiment deviates from the fact that the obstacle recognition of the existing monocular camera is applied only to an object with a certain shape or an obstacle that is moving by itself. It has the effect of recognizing obstacles.

1 is a flowchart illustrating an example of an obstacle detecting method according to an embodiment.
2 to 6 and 10 are views illustrating image screens processed in each step of the method of detecting an obstacle of FIGS. 1 and 7 .
7 is a flowchart illustrating another example of an obstacle detection method according to an embodiment.
8 is a view showing a state of a trajectory used in step S240 of the obstacle detection method of FIG.
9 is a diagram illustrating an example of the configuration of an obstacle detecting apparatus according to an embodiment.

Various methods and devices to which the following embodiments are applied will be described in more detail with reference to the drawings. The suffix “part” disclosed in the present specification below is given or used in consideration of ease of writing the specification, and does not have a meaning or role distinct from each other by itself.

It should be understood that 'and/or' disclosed in the examples below includes any and all possible combinations of one or more of the listed related items.

Terms such as "include", "consist of" or "have" disclosed in the examples below, unless otherwise stated, mean that the corresponding component may be embedded, and other components It should be understood as further including other components rather than excluding them.

<Example of obstacle detection method>

1 is a flowchart illustrating an example of an obstacle detection method according to an embodiment, and FIGS. 2 to 6 are views illustrating image screens processed in each step of the obstacle detection method of FIG. 1 .

2 to 6 are referred to as ancillary for each step in FIG. 1 .

Referring to FIG. 1 , an obstacle detecting method S100 according to an embodiment realizes a mechanism for classifying and detecting states of obstacles such as stationary obstacles, moving obstacles, and/or virtual objects in the obstacle detecting apparatus.

The obstacle detecting device applied here may be an electronic control unit (ECU) installed in a vehicle, a head unit, a navigation device, and a telematics terminal cluster display device, at least one of these devices, or a vehicle terminal interworking with at least one of these devices.

Such an obstacle detecting device is connected to a camera installed in a vehicle to acquire an image from the camera, and is used to accurately detect all kinds of obstacles through the acquired image.

The camera may be, for example, at least one of a rear camera, a front camera and/or a left and right camera of the vehicle.

The obstacle detecting method ( S100 ) performed by the obstacle detecting apparatus may include steps S110 to S140 .

First, in step S110, at least one piece of image information photographed through a camera is acquired by the obstacle detection device, and at least one object movement captured in the acquired image information is detected.

As a technique for detecting the motion of an object, a technique for detecting a difference between image frames containing an object in an image, a technique for calculating an optical flow, or the like may be used.

Since these techniques are all known techniques, a description thereof will be omitted. However, the present invention is not limited to the above-described known technique, and if there is another object detection technique, it is of course applicable to the present embodiment.

In an embodiment, step S120 clusters at least one object movement detected by step S110. A clustered example may be represented as shown in FIG. 2 . That is, at least one moving object 101 shown in FIG. 2 may be bundled to form one cluster, and at least one cluster region 102 may be provided.

As a clustering technique, a clustering technique including k-means and c-means, a motion segmentation technique such as snake, etc. may be used.

Since these techniques are known techniques, their description is omitted, but the same is applied in the present embodiment, and in addition, various known techniques may be used without being limited to the aforementioned clustering technique.

Subsequently, in step S120 , a process of forming the clustered cluster region 102 into a lower block 104 is performed so that the clustered cluster region 102 is disposed in at least one block 103 . The lower block 104 may be composed of a plurality of blocks 103 arranged by dividing the clustered object 101 in an arbitrary clustering area 102 .

However, not all blocks 103 having clustered areas 102 have moving objects 101 . That is, when there is no clustered object, the corresponding block is excluded from the lower block 104 .

For example, when the clustering area 102 covering the moving object 101 shown in FIG. 2 spans within four blocks 103, as shown in FIG. 3 , the moving object 101 does not exist. The two blocks 103a covering the non-clustered area 102 may be excluded from the lower block 104 . That is, the two blocks 103a on the right may be excluded.

In this case, step S120 may calculate a center point for each block 103 that is not excluded as in FIG. 3 , and convert the calculated coordinates of the center point into coordinates of a viewpoint transformation (a bird's eye view). However, if the original image is initially view-converted, this process may be omitted.

In the embodiment, step S130 utilizes an object tracking technique that tracks the position movement of the center of each block 103 in the lower block 104 for every frame using color, feature point, motion vector, etc., and the center of each block 103 can be extracted from the image information.

As an object tracking method, a method using a feature point-optical flow such as KLT, a clustering-based tracking method such as block matching and cam-shift, etc. can be used.

Since these techniques are all known techniques, a description thereof will be omitted. However, it is not limited to the above-described known technique, and if there is another object tracking technique, it is of course applicable to the present embodiment.

Accordingly, in step S130, after accumulating the positional movement of the center of each block 103 extracted within the lower block 104 in one image, at least one object movement trajectory may be extracted based on this. An example of the extracted at least one object movement trajectory may be represented as shown in FIG. 4 .

For example, as shown in FIG. 4 , in each of the two blocks 103b in the lower block 104, one object movement trajectory 108 passing through a central position may be generated. Similarly, three object movement trajectories 109 passing through the central position of each of the three blocks 103c in the lower block 104 may be generated. The other blocks shown in FIG. 4 are also omitted because the same principle is applied.

Subsequently, in step S130, a vehicle movement prediction line used for steering interlocking of a camera, for example, a rear camera, etc. may be extracted from at least one of steering information, vehicle information, and viewpoint conversion (Bird's eye view). Since such a vehicle movement prediction line is a known technology, a description thereof will be omitted.

Through this, step S130 may extract at least one object motion prediction line in the lower block 104 using the extracted vehicle motion prediction line. An example of the extracted at least one object motion prediction line may be represented as shown in FIG. 5 .

For example, as shown in Fig. 5, in each of the two blocks 103b in the lower block 104, one object motion prediction line 105 passing through the center position is generated, and each of the three blocks 103c in the lower block 104 is generated. ), three object motion prediction lines 106 passing through the center position are generated, and the other blocks are also generated according to the same principle, so a description thereof will be omitted.

Here, if the vehicle motion prediction line is defined as f=a(ｘ), as shown in FIG. 6 , the first object motion prediction line in the first block 103b is f=b(ｘ− _Xb1 )+ _Ya1 (A1 block). the number of days the center _{(C A1))} with respect to a first block within a (second object motion prediction line y = f _{(x-x a2)} + _{Y a2} (a2 block center _{(C a2} of the 103b)) with respect to) have.

Similarly, as in FIG. 6 , the first object motion prediction line y = f(xX _b1 ) + Y _b1 (with respect to the center of block B1 (C _B1 )) in the second block 103d, and the second block ( The second object motion prediction line in 103d) is y = f(xX _b2 ) + Y _b2 (with respect to the center of block B2 (C _B2 )), and the third object motion prediction line in the second block 103d is y = f( xX _b3 ) + Y _b3 (relative to the center of block B3 (C _B3 )).

Similarly, the object motion prediction lines in the third block 103c are y = f(xX _cn ) + Y _cn (with respect to the center of the Cn block (C _Cn ), n=1,2,3), and the fourth block ( The object motion prediction lines in 103e) are y = f(xX _dn ) + Y _dn (for the center (C _Dn ) of the Dn block, n=1,2), and the object motion prediction lines y = f(xX dn ) in the fifth block 103f f(xX _en ) + Y _en (for the center (C _En ) of the En block, n=1,2).

In this way, it is possible to obtain at least one object motion prediction line in all sub-blocks.

Finally, in operation S140 , it is possible to determine whether the trajectories match by comparing the at least one object movement trajectory extracted by the above-described operation S130 with the at least one object movement prediction line.

For example, in step S140, as shown in FIGS. 4 to 6, the first object motion trajectory 109 in the corresponding block 103c is compared with the first object motion prediction line y, 106 in the corresponding block 103c. to determine the trajectory mismatch, and compare the second object movement trajectory in the same block 103c with the second object motion prediction line in the same block 103c to determine the trajectory coincidence, since the trajectories are partially identical, the corresponding block The object in 103c can be recognized by the obstacle detecting device as a stationary obstacle.

As another example, in step S140, as in FIGS. 4 to 6, the first object movement trajectory in the block 103d is compared with the first object movement prediction line in the block 103d to determine the trajectory mismatch, and the same The second object movement trajectory in the block 103d is compared with the second object movement prediction line in the same block 103d to determine the trajectory coincidence, and the third object movement trajectory in the same block 103d is determined to be the same in the same block 103d. When it is determined that the trajectory coincides by comparing the third object motion prediction lines, since the trajectories are partially coincident, the object in the block 103d can be recognized as a stationary obstacle in the obstacle detection apparatus.

As another example, in step S140, as in FIGS. 4 and 6, the first object motion trajectory in the corresponding block 103e is compared with the first object motion prediction line in the block 103e, and it is determined as a trajectory mismatch, and the same When the second object motion prediction line in the block 103e is compared with the second object motion trajectory in the block 103e and it is determined that the trajectory is inconsistent, the trajectories are all inconsistent, so the object in the block 103e is set as a moving obstacle. It can be recognized by the obstacle detection device.

As another example, in step S140, as in FIGS. 4 to 6, the first object movement trajectory in the corresponding block 103f is compared with the first object movement prediction line in the block 103f to determine the trajectory match, and the same When the second object motion prediction line in the block 103f is compared with the motion trajectory of the second object in the block 103f and it is determined that the trajectory is coincident, the trajectories are all identical, so the object in the block 103f is regarded as a virtual obstacle. It may be recognized by the obstacle detection device.

As such, in this embodiment, by recognizing any one of a virtual object, a stationary object, and a moving object through comparison of the object movement trajectory and the prediction line, not only all moving obstacles, but also all of the stationary obstacles and/or virtual obstacles It can help with accurate identification.

The above-described obstacle detection method S100 may be implemented in the form of program instructions that may be executed through hardware components and recorded in a computer-readable recording medium.

The mentioned recording medium may be a ROM, a magnetic disk or a compact disk, but is not limited thereto.

<Another example of an obstacle detection method>

7 is a flowchart illustrating another example of an obstacle detection method according to an embodiment, and FIG. 8 is a diagram illustrating a trajectory state used in step S240 of the obstacle detection method of FIG. 7 .

Here, the aforementioned FIGS. 2 to 6 and FIG. 8 will be referred to in the corresponding steps when describing each step of FIG. 7 .

Referring to FIG. 7 , the obstacle detecting method S200 according to an embodiment realizes a mechanism for classifying and detecting states of obstacles such as stationary obstacles, moving obstacles, and/or virtual objects in the obstacle detecting apparatus.

The obstacle detecting device applied here may be an electronic control unit (ECU) installed in a vehicle, a head unit, a navigation device, and a telematics terminal cluster display device, at least one of these devices, or a vehicle terminal interworking with at least one of these devices.

Such an obstacle detecting device is connected to a camera installed in a vehicle to acquire an image from the camera, and is used to accurately detect all kinds of obstacles through the acquired image.

The camera may be, for example, at least one of a rear camera, a front camera and/or a left and right camera of the vehicle.

The obstacle detecting method ( S200 ) performed by the obstacle detecting apparatus may include steps S210 to S240 .

First, in step S210, at least one piece of image information captured by the camera is acquired by the obstacle detection device, and at least one object movement captured in the acquired image information is detected.

As a technique for detecting the motion of an object, a technique for detecting a difference between image frames containing an object in an image, a technique for calculating an optical flow, or the like may be used.

Since these techniques are all known techniques, a description thereof will be omitted. However, the present invention is not limited to the above-described known technique, and if there is another object detection technique, it is of course applicable to the present embodiment.

In an embodiment, step S220 clusters at least one object movement detected by step S210. A clustered example may be represented as shown in FIG. 2 . That is, at least one moving object 101 shown in FIG. 2 may be bundled to form one cluster, and at least one cluster region 102 may be provided.

As a clustering technique, a clustering technique including k-means and c-means, a motion segmentation technique such as snake, etc. may be used.

Since these techniques are known techniques, their description is omitted, but the same is applied in the present embodiment, and in addition, various known techniques may be used without being limited to the aforementioned clustering technique.

Subsequently, in step S220 , the process of forming the clustered cluster area 102 into a lower block 104 to be disposed in at least one block 103 is performed. The lower block 104 may be composed of a plurality of blocks 103 arranged by dividing the clustered object 101 in an arbitrary clustering area 102 .

However, not all blocks 103 having clustered areas 102 have moving objects 101 . That is, when there is no clustered object, the corresponding block is excluded from the lower block 104 .

For example, when the clustering area 102 covering the moving object 101 shown in FIG. 2 spans within four blocks 103, as shown in FIG. 3 , the moving object 101 does not exist. The two blocks 103a covering the non-clustered area 102 may be excluded from the lower block 104 . That is, the two blocks 103a on the right may be excluded.

In this case, step S220 may calculate a center point for each block 103 that is not excluded as in FIG. 3 , and convert the calculated coordinates of the center point into coordinates of a viewpoint transformation (a bird's eye view). However, if the original image is initially view-converted, this process may be omitted.

In the embodiment, in step S230, the position of the central point of the block may be extracted from the image using an object tracking standard that tracks the positional movement of the central point of a lower block in every frame using a color, a feature point, a motion vector, and the like.

As an object tracking method, a method using a feature point-optical flow such as KLT, block matching, and a clustering-based tracking method such as cam-shift can be used.

Since these techniques are all known techniques, a description thereof will be omitted. However, it is not limited to the above-described known technique, and if there is another object tracking technique, it is of course applicable to the present embodiment.

Accordingly, in step S230, after accumulating the positional movement of the center of each block 103 extracted within the lower block 104 in one image, at least one object movement trajectory may be extracted based on this. An example of the extracted at least one object movement trajectory may be represented as shown in FIG. 4 .

For example, as shown in FIG. 4 , in each of the two blocks 103b in the lower block 104 , one object movement trajectory 108 passing through a central position may be generated. Similarly, three object movement trajectories 109 passing through the central position of each of the three blocks 103c in the lower block 104 may be generated. The other blocks shown in FIG. 4 are also omitted because the same principle is applied.

Subsequently, in step S230, a vehicle movement prediction line used for steering interlocking of a camera, for example, a rear camera, etc. may be extracted from at least one of steering information, vehicle information, and viewpoint conversion (Bird's eye view). Since such a vehicle movement prediction line is a known technology, a description thereof will be omitted.

Through this, step S230 may extract at least one object motion prediction line in the lower block using the extracted vehicle movement prediction line. An example of the extracted at least one object motion prediction line may be represented as shown in FIG. 5 .

For example, as shown in Fig. 5, in each of the two blocks 103b in the lower block 104, one object motion prediction line 105 passing through the center position is generated, and each of the three blocks 103c in the lower block 104 is generated. ), three object motion prediction lines 106 passing through the center position are generated, and the other blocks are also generated according to the same principle, so a description thereof will be omitted.

Here, if the vehicle motion prediction line is defined as f=a(ｘ), as shown in FIG. 6 , the first object motion prediction line in the first block 103b is f=b(ｘ− _Xb1 )+ _Ya1 (A1 block). the number of days the center _{(C A1))} with respect to a first block within a (second object motion prediction line y = f _{(x-x a2)} + _{Y a2} (a2 block center _{(C a2} of the 103b)) with respect to) have.

Similarly, as in FIG. 6 , the first object motion prediction line y = f(xX _b1 ) + Y _b1 (with respect to the center of block B1 (C _B1 )) in the second block 103d, and the second block ( The second object motion prediction line in 103d) is y = f(xX _b2 ) + Y _b2 (with respect to the center of block B2 (C _B2 )), and the third object motion prediction line in the second block 103d is y = f( xX _b3 ) + Y _b3 (relative to the center of block B3 (C _B3 )).

Also, as in FIG. 6 , the object motion prediction lines in the third block 103c are y = f(xX _cn ) + Y _cn (with respect to the center (C _Cn ) of the Cn block, n=1,2,3), and , object motion prediction lines in the fourth block 103e y = f(xX _dn ) + Y _dn (with respect to the center (C _Dn ) of the Dn block, n=1,2), and the object in the fifth block 103f The motion prediction line y = f(xX _en ) + Y _en (for the center (C _En ) of the En block, n=1,2) may be.

In this way, it is possible to obtain at least one object motion prediction line in all sub-blocks.

Finally, in step S240, for example, t0, t0+Δ, t0+2Δ, . At least one block may be updated in the obstacle detecting apparatus by tracking every period of t0+nΔ.

This step S240 determines whether the sum (E) of the shortest distance (eg, en, n=1,2 ...) to the object motion prediction line of each block extracted from the center of each updated block exceeds a threshold. to determine whether

For example, in step S240, as in FIG. 8, the sum (E) of the shortest distance (eg, en, n=1,2 ...) with the corresponding object motion prediction line 107 of each block 103 . ) is less than or equal to the threshold, by considering that the corresponding object motion prediction line of each block extracted in step S230 coincides with the corresponding at least one object motion trajectory, the corresponding object in the lower block The object may be recognized by the obstacle detection device as a virtual obstacle.

As another example, in step S240, as in FIG. 8, the sum (E) of the shortest distance (eg, en, n=1,2 ...) with the corresponding object motion prediction line 107 of each block 103 . ) is greater than or equal to the threshold, the object movement prediction line of each block extracted in step S230 is considered to be inconsistent with at least one object movement trajectory, so that the object in the lower block is moved can be perceived as an obstacle.

As another example, in step S240, the sum of the shortest distance (eg, en, n=1,2 ...) with the corresponding object motion prediction line 107 of each block 103 is a part as shown in FIG. 8 . If the obstacle detecting device determines that is above the threshold and the remaining part is below the threshold, the object motion prediction line of each block extracted in step S230 is considered to be partially consistent with at least one object motion trajectory, so that Objects can be recognized as stationary obstacles.

In this embodiment, by recognizing any one of a virtual object, a stationary object, and a moving object by comparing the object movement trajectory and the prediction line, not only all moving obstacles, but also all moving obstacles, as well as stationary obstacles and/or virtual obstacles, are identified. It can be helpful.

The above-described obstacle detection method S200 may be implemented in the form of program instructions that may be executed through hardware components and recorded in a computer-readable recording medium.

The mentioned recording medium may be a ROM, a magnetic disk or a compact disk, but is not limited thereto.

For reference, if FIGS. 4 and 5, which are the comparison objects of steps S140 and S240, are synthesized, it can be shown as in FIG. 10 as an example.

As shown in FIG. 10 , as a result of comparing the object motion trajectory with the object motion prediction line, it can be seen that reference numerals A1, C1, C2, C3, D1, and D2 in FIG. 6 do not match, A2, E2, E3, It can be confirmed that B2 and B3 match.

9 is a diagram illustrating an example of the configuration of an obstacle detecting apparatus according to an embodiment.

Referring to FIG. 9 , the obstacle detecting apparatus 100 according to an embodiment may algorithmize the above-described obstacle detecting method and process it.

To this end, the obstacle detecting apparatus 100 may include an input unit 110 , a processor 120 processing at least one core, a memory 130 , a display unit 140 , and an output unit 150 .

The input unit 110 may include a touch panel 111 and/or other input devices 132 .

The touch panel 111 is a touch operation related to the above-described obstacle detection method (for example, a user performs a touch on the touch sensitive surface or the vicinity of the touch sensitive surface using any suitable object or accessory such as a finger or a touch pen. operation) and at the same time, according to a preset program, the related connection device can be driven.

The touch panel 111 may include at least one of a touch detection means and a touch controller. The touch detection means detects a user's touch position and simultaneously detects a signal by a touch operation and transmits the touch information to the touch controller. The touch controller receives touch information from the touch detection means, converts the corresponding information into contact point coordinates, transmits it to the processor 140 , and simultaneously receives and executes a command from the processor 140 .

The other input device 112 uses at least one of a physical keyboard, a function key (eg, a volume button, a switch button, etc.), a track ball, a mouse, a joystick, etc. for recognizing a user input related to the above-described obstacle detection method. including, but not limited to. Such an input device 112 may be used as an auxiliary means of the touch panel 111 .

In an embodiment, the processor 120 is the control center of the obstacle detection device 100 and is connected to each part via various interfaces and/or circuits, and operates a software program and/or module stored in the memory 130; At the same time, by calling data stored in the memory 130 , it is possible to help the obstacle detection apparatus 100 to execute various functions and process data.

Here, the software program refers to the result of coding the obstacle detecting method described in FIGS. 1 to 8 , and the module may refer to each component of the obstacle detecting apparatus 100 described above.

For example, the processor 120 detects at least one object movement captured in an image captured by a camera, clusters the detected at least one object movement, and divides the clustered cluster area into at least one block. sub-blocking so as to be disposed within, extracting at least one object motion trajectory based on the center of each block within the sub-block using object tracking, and predicting at least one object motion within the sub-block using a vehicle motion prediction line An obstacle detection method comprising the steps of extracting a line and determining whether a trajectory coincides between the extracted at least one object motion trajectory and the at least one object motion prediction line, and recognizing it as any one of a virtual object, a stationary object, and a moving object can be algorithmically processed.

Alternatively, the processor 120 may perform the steps of detecting at least one object movement captured in an image captured by the camera, clustering the detected at least one object movement, and placing the clustered object in at least one block. sub-blocking, extracting at least one object motion trajectory based on the center of each block within the sub-block using object tracking, and using the vehicle motion prediction line to determine the object motion prediction line of each block in the sub-block After extracting and updating at least one sub-blocked block every period, it is determined whether the sum of the shortest distance from the center of each block to the object motion prediction line of each block exceeds a limit value. The obstacle detection method including the steps can be processed by algorithmizing it.

Here, the lower block means a block composed of a plurality of blocks arranged by dividing the clustered object, and in the step of sub-blocking, if there is no clustered object, the corresponding block may be excluded from the lower block.

In addition, the above-described object tracking refers to a technique for tracking the position movement of the center of each block in the lower block 104 for every frame using color, feature point, motion vector, etc., and the vehicle movement prediction line includes steering information, vehicle information and It can be extracted through bird's eye view.

In an embodiment, the processor 120 compares the extracted at least one object motion with the at least one object motion prediction line and, if the trajectory is partially identical, recognizes it as a stationary obstacle and matches the extracted at least one object motion with at least one When one object motion prediction line is compared and all trajectories do not match, it is recognized as a moving obstacle. When the trajectories are identical by comparing at least one object motion with the at least one object motion prediction line, it is recognized and processed as a virtual obstacle. can do.

Alternatively, if the processor 120 is less than or equal to the above-described threshold, the object motion prediction line of each block is regarded as coincident with the at least one object movement trajectory and is recognized as a virtual obstacle. If it is greater than or equal to the above-described threshold, each block The object motion prediction line of the block is considered to be inconsistent with the at least one object motion trajectory and recognized as a moving obstacle. At least one object movement trajectory may be regarded as partially coincident, and it may be recognized and processed as a stationary obstacle.

As such, in this embodiment, by recognizing any one of a virtual object, a stationary object, and a moving object through comparison of the object movement trajectory and the prediction line, not only all moving obstacles, but also all of the stationary obstacles and/or virtual obstacles are identified, It can help with accurate identification.

For such processing, the processor 120 may optionally include one or a plurality of cores. For example, an operation processor and a module processor for processing the above-described obstacle detection method may be integrated and configured, and the following memory 130 may be built-in for processing.

The operation processor mainly processes the OS, the user interface, and the above-described obstacle detection method, and the module processor may process camera interworking. However, the modular processor may not be integrated into the processor 130 .

In an embodiment, the memory 140 may store data processed by the processor 120 described above, and may store background data required for each processing, for example, vehicle information and steering information.

Such a memory 130 is a flash memory type (flash memory type), a hard disk type (hard disk type), a multimedia card micro type (multimedia card micro type), card type memory (for example, SD or XD memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM), Magnetic Memory, It may be a storage medium of at least one type of a magnetic disk and an optical disk.

However, the present invention is not limited thereto. For example, the memory 130 may be a computer-readable recording medium for the above-described obstacle handling method.

In one embodiment, the display unit 140 is for displaying information input by the user or information provided to the user, and various types of graphic user interfaces (GUIs) of the obstacle detection apparatus 100, such graphic user interfaces include figures, It may consist of text, icon, video, and any combination thereof.

The display unit 150 may include a display panel 141 . The display panel 141 may be at least one of a liquid crystal display (LCD) and an organic light-emitting diode (OLED).

Here, the touch panel 111 and the display panel 141 may be configured to realize the output and input functions as two independent parts, but may be configured as one to realize the input and output functions.

Finally, the output unit 150 recognizes the result of processing by the above-described processor 120, for example, recognizes an obstacle and issues an alert or informs the recognition state in the form of a voice, and an audio circuit 151 and a speaker ( 152) may be included.

However, the present invention is not limited thereto, and the output unit 150 may output the alert voice content in the form of text.

As described above, the embodiments of the present invention have been described with reference to the accompanying drawings, but those of ordinary skill in the art to which the present invention pertains are implemented in other specific forms without changing the technical spirit or essential features of the present invention You will understand that you can do it. Accordingly, the embodiments described above are illustrative in all respects and not restrictive.

1. Applicable to rear camera-based autonomous parking/SPAS, etc. For example, as an application of a rear camera, it can be applied to obstacle warning or obstacle recognition during autonomous parking.

2. Applicable to front camera-based cut-in recognition. For example, the moment of being cut-in after overtaking on the left and right sides of the driving vehicle (when the shape recognition-based vehicle recognition function cannot be performed because the overall shape of the vehicle is not yet grasped) can be applied to obstacle recognition.

100: obstacle detection device 110: input unit
120: processor 130: memory
140: display unit 150: output unit

Claims (17)

An obstacle detection method performed by an obstacle detection apparatus, comprising:
detecting movement of at least one object captured in an image captured by a camera;
clustering the detected at least one object movement, and sub-blocking the clustered cluster area to be disposed in at least one block;
After accumulating the positional movement of the center of each block in one image using the object tracking technique, at least one object movement trajectory is extracted based on this, and the lower block using the extracted vehicle movement prediction line extracting at least one object motion prediction line from within; and
Recognizing as any one of a virtual obstacle, a stationary obstacle, and a moving obstacle by determining whether or not the trajectory coincides between the extracted at least one object movement trajectory and the at least one object movement prediction line and the trajectory coincidence degree
An obstacle detection method comprising a. According to claim 1,
The step of sub-blocking is,
When there is no clustered object, the corresponding block is excluded from the lower block. 3. The method of claim 2,
The sub-block is an obstacle detection method consisting of a plurality of blocks arranged by dividing the clustered object. According to claim 1,
The object tracking method is an obstacle detection method that tracks the position movement of the center of each block. According to claim 1,
The vehicle movement prediction line is an obstacle detection method that is extracted from at least one of steering information, vehicle information, and a bird's eye view. 3. The method of claim 2,
The determining step is
An obstacle detecting method for recognizing the stationary obstacle by comparing the at least one object motion trajectory with the at least one object motion prediction line and partially matching the trajectory. 3. The method of claim 2,
The determining step is
An obstacle detecting method for comparing the at least one object motion trajectory with the at least one object motion prediction line and recognizing the moving obstacle when the trajectories do not match. 3. The method of claim 2,
The determining step is
An obstacle detecting method for recognizing the virtual obstacle by comparing the at least one object motion trajectory with the at least one object motion prediction line, and when all trajectories match. An obstacle detection method performed by an obstacle detection apparatus, comprising:
detecting movement of at least one object captured in an image captured by a camera;
clustering the detected at least one object movement, and sub-blocking the clustered object to be placed in at least one block;
After accumulating the positional movement of the center of each block in one image using the object tracking technique, at least one object movement trajectory is extracted based on this, and the lower block using the extracted vehicle movement prediction line extracting an object motion prediction line of each block within; and
Determining whether the sum of the shortest distance from the center of each block to the object motion prediction line of each block exceeds a threshold value after tracking and updating the at least one sub-blocked block at each period
An obstacle detection method comprising a. 10. The method of claim 9,
The step of sub-blocking is,
When there is no clustered object, the corresponding block is excluded from the lower block. 10. The method of claim 9,
The object tracking method is an obstacle detection method that tracks the position movement of the center of each block. 11. The method of claim 10,
The sub-block is an obstacle detection method consisting of a plurality of blocks arranged by dividing the clustered object. 10. The method of claim 9,
The vehicle movement prediction line is an obstacle detection method that is extracted from at least one of steering information, vehicle information, and a bird's eye view. 11. The method of claim 10,
The determining step is
If it is less than the threshold, the object motion prediction line of each block is regarded as coincident with the at least one object motion trajectory, and is recognized as a virtual obstacle. 11. The method of claim 10,
The determining step is
If the threshold is greater than the threshold, the object movement prediction line of each block is considered to be inconsistent with the at least one object movement trajectory, and is recognized as a moving obstacle. 11. The method of claim 10,
The determining step is
An obstacle detection method for recognizing as a stationary obstacle by considering that the object motion prediction line of each block partially coincides with the at least one object motion trajectory when a part is greater than the threshold value and the remaining part is less than or equal to the threshold value. Memory; and a processor;
The processor processes the obstacle detection method according to any one of claims 1 to 16,
The memory is an obstacle detecting device for storing data processed by the processor.

Images