KR100938194B1 - Method for detecting an object and apparatus for the same - Google Patents

Abstract

The present invention relates to an object detecting apparatus and an object detecting method. The present invention obtains a depth map by matching a stereo vision image based on a distance from an object, converts the depth map horizontally, and then obtains a vertical parallax map, thereby detecting a target object not only in a distance but also in a short distance. There is an effect of estimating accurate position, distance, speed, etc. based on the detected object. When the above object detection method and apparatus are applied to a vehicle, objects around the vehicle such as short distance left, right, left, and the like as well as a long distance can be detected accurately.

Such a detection method has an effect of accurately detecting an object by applying to various fields. For example, when applied to a robot or a car, it can be applied to a driving system such as collision prevention of a car, driving, or a safe driving system. In addition, the present invention can be applied to an object detection field not only in a moving object such as a robot or a car but also in a daily space such as a factory or a home.

Description

Method for detecting an object and apparatus for the same

The present invention relates to an object detecting apparatus and an object detecting method, and more particularly, to an apparatus and method for detecting an object included in an image by stereo matching a captured image.

There are several ways to measure the distance of an object away from the observer. Among them, there is a method of capturing an image of the object and measuring a distance to an object included in the image by using the photographed image.

One example of a technique of measuring a distance to a target object included in an image by using a captured image is stereo vision technology. The stereo vision technology may measure the position, distance, and speed of an object included in an image using a left image and a right image obtained from a plurality of image capturing apparatuses, for example, cameras.

In a conventional distance measuring method using stereo vision technology, a corresponding point for an object is found from left and right images obtained from a plurality of cameras, and the distance of the object is measured using disparity with respect to the corresponding point.

However, in the conventional stereo vision technology, a depth map using a vertical distance under the assumption that an object is far away in front of the vehicle has a problem that a distance error may occur when an obstacle exists at a close distance. .

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object of the present invention is to provide a distance object detection method and an object detection apparatus capable of estimating the exact position and distance of an object not only at a distance but also at a short distance.

In the object detecting method according to the present invention, a depth map centered on a center point is calculated by matching a photographed stereo image based on a distance from an object, and the depth map centered on the center point is horizontally aligned along a vertical line. Converting a depth map as much as possible, forming a vertical parallax map using the converted depth map, and detecting an object included in an image using the vertical parallax map.

In another aspect, an object detecting apparatus according to the present invention includes a matching unit configured to calculate a depth map centered on a center point by matching a photographed stereo image with respect to a distance from an object, and a depth line centered on the center point is a vertical line. And a transform unit for converting the depth map to be horizontal along the form, and an object detector for forming a vertical parallax map using the converted depth map and detecting an object included in the image.

According to the object detecting method and the object detecting apparatus of the present invention, it is possible to detect a target object not only in the distance but also in the short distance in the stereo vision system, and to estimate the exact position, distance, speed, etc. based on the detected object. It works. When the above object detection method and apparatus are applied to a vehicle, it is possible to accurately detect not only a long distance but also an object around the vehicle such as near, right, left, and rear, so as to change an auto lane, a lane keeping system (LKS), and a side obstacle. It is effective to provide safety driving functions such as warning.

Such a detection method has an effect of accurately detecting an object by applying to various fields. For example, when applied to a robot or a car, it can be applied to a driving system such as collision prevention of a car, driving, or a safe driving system. In addition, the present invention can be applied to an object detection field not only in a moving object such as a robot or a car but also in a daily space such as a factory or a home.

The objects, features and advantages of the present invention will become apparent from the following detailed description of embodiments taken in conjunction with the accompanying drawings. In addition, the terms used in the present invention was selected as a general term widely used as possible now, but in certain cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in detail in the corresponding description of the invention, It is to be clear that the present invention is to be understood as the meaning of terms rather than names.

Operation of the object detecting apparatus and the object detecting method according to the present invention configured as described above will be described in detail with reference to the accompanying drawings. 1 is a diagram schematically illustrating a method of estimating distance according to a stereo vision method according to an embodiment of the present invention.

In the case of estimating distance using a stereo vision system, a depth map is generated by finding a corresponding point using images captured from the left and right image capturing apparatus. The moving object is detected using the depth map and the left and right images, and a parallax average in the detected moving object is obtained. In the stereo vision system, the distance of an object may be estimated using the obtained parallax. The distance between the parallax and the target object will be described in detail using Equation 1 below and various parameters of FIG. 1. In FIG. 1, B (reference line) is the distance between the centers of the left and right imaging apparatuses, F is the focal length of the imaging apparatus (the distance between the lens and the imaging apparatus (eg, CCD, CMOS, etc.)), x _l is the distance from the center of the shooting device to the object on the left shooting device, x _r is the distance from the center of the shooting device to the object on the right shooting device, Z _d is the vertical distance between the object and the image shooting device, Z _r is The actual distance of the object, X _d , refers to the distance from the middle point of the left image capture device and the right image capture device to the point where the horizontal line of the image pickup device meets the Z _d line.

As shown in Equation 1, the vertical distance Z _d is a parallax value (x _r -x _l ) obtained by multiplying the distance B between the centers of the left and right imaging apparatuses by the focal length F of the imaging apparatus. It can be obtained by dividing (Equation 1). Since the difference between Z _r and Z _d is not large when the object is in the front, although the actual distance from the object is Z _r as shown in FIG. 1, in the case of Equation 1, the object and the reference line B The vertical distance Z _d can be found.

FIG. 2 is a diagram schematically illustrating a depth map obtained using a vertical distance according to an embodiment of the present invention. The depth map is a representation of distance information of a captured image as a gray value. In the case of the depth map, the closer the distance is represented by the bright value, the farther distance is represented by the dark value.

When the depth map is created using the vertical distance as shown in Equation 1, the depth map has the same distance value horizontally with respect to the reference line. That is, as shown in FIG. 2, the same brightness value may be horizontally provided.

3 is a diagram schematically illustrating a process of detecting an object using a vertical parallax map according to an embodiment of the present invention. In the depth map as shown in FIG. 2, as the distance increases, the planar region without the object decreases at a constant rate. However, when an object such as an obstacle is placed, the brightness value of the object portion is increased due to the object unlike the plane. The object can be detected using the above phenomenon.

The horizontal component of the vertical parallax map V-disparity of FIG. 3 represents a parallax value (0 to 255, when the gray value is 2 ⁸ ), and the vertical component represents a vertical line of the depth map. The vertical parallax map reads each pixel brightness value of the horizontal line corresponding to each vertical line of the depth map and maps the value to the vertical parallax map for each vertical line. For example, a high brightness value may be mapped to a vertical parallax map for each vertical line of the depth map.

As a result of the above mapping, as shown in FIG. 3, the planar component has a higher brightness value as the vertical line value decreases, and decreases at a constant rate as the vertical line value increases. The brightness value may be converted into a parallax value in the inverse of the process of creating the depth map.

However, the obstacle or the object may have a value different from the tendency of the planar component. For example, in FIG. 3, the obstacle object has a specific brightness value different from the tendency of the planar component, and the objects included in the image may be detected by detecting a part having the specific brightness value. The distance of the object may be estimated by reading the parallax value of the detected object and applying the parallax value to Equation (1).

4 is a diagram schematically illustrating a method of estimating an actual distance of an object according to a stereo vision method according to an embodiment of the present invention. In the case of Equation 1 and FIG. 1, since a vertical distance between the reference line and the object is obtained, an error may occur in measuring the distance between the short distance object such as a side or front and back. Therefore, the error as described above can be reduced by obtaining the actual distance from the object as shown in FIG. 4, and the object can be detected accurately.

The actual distance Z _r of the object is represented by Equation 2 below. Each parameter of Equations 2 and 4 is as described in Equations 1 and 1 above.

The actual distance (Z _r ) of the object is multiplied by the root value of the square of the parallax value (x _r -x _l ) and the square of the focal length twice (2F) and the 1/2 value (B / 2) of the baseline, This can be expressed as the value divided by the parallax value.

FIG. 5 is a diagram schematically showing a depth map obtained using an actual distance as an embodiment according to the present invention. When the depth map is created using the actual distance as shown in Equation 2, the depth map has the same distance value in the form of concentric circles with respect to the center. That is, as shown in FIG. 5, concentric circles at the same distance from the center of the center may have the same brightness value.

However, in the case of the depth map as described above, the distance of the object can be accurately reflected, but there is a problem that the vertical parallax map cannot be applied as it is when the object is detected. Therefore, rearrangement or transformation (hereinafter, referred to as transformation) in a form in which a vertical parallax map may be applied to the depth map as described above.

6 is a diagram schematically illustrating a process of creating a vertical parallax map by converting a depth map according to an embodiment of the present invention. In the depth map of FIG. 5, brightness values are arranged in the form of concentric circles, and thus vertical parallax maps for mapping the brightness values for each vertical line may not be directly generated. Accordingly, the depth map as shown in FIG. 6 (or FIG. 5) is converted to make the brightness value horizontal for each vertical line as shown in FIG. 6.

In order to convert the depth map as described above, a polar coordinate transformation method or a mapping method using a reading order of pixel values may be used. For example, in the polar coordinate transformation method, a log-polar mapping method or the like may be used. Alternatively, by using the Look-Up Table (LUT), which stores the order of reading the brightness values of pixels in the depth map, the brightness values of the pixels at the same distance from the center point are read in order and the horizontal lines are vertically aligned. You can map as much as possible.

6 of FIG. 6 schematically shows a map obtained by converting a depth map. Since the length of the outer line of the nth circle and the inner line of the n + 1th circle is the same as in 1 of FIG. 6, each section of the converted depth map should be displayed in a rhombus shape. Displayed. In addition, in the case of 1 of FIG. 5 or 6, only a part of the depth map is displayed for convenience of description, but in the actual depth map, concentric circles centered on the center point may be displayed on the entire inside of the rectangular screen. Thus, as the distance from the center of gravity increases as shown in 2 of FIG. 6, the number of pixels located at the same distance increases, but as the concentric circles begin to be cut by the left, right, or top surfaces of the rectangular screen, the number of pixels gradually decreases.

When the depth map converted as shown in 2 of FIG. 6 is obtained, a vertical parallax map may be obtained using the depth map. FIG. 3 is a diagram schematically illustrating a process of creating a vertical parallax map and detecting an object by using the transform depth map as shown in FIG. 6.

In the case of the transform depth map as shown in FIG. 6, the number of pixels differs depending on the vertical line, unlike in FIG. 3. That is, the middle portion of the transform depth map has a large number of pixels, but the upper and lower portions do not have many pixels. Therefore, when generating a vertical parallax map, noise may be removed using a thresholding value. For example, an adaptive threshold value or an adaptive threshold value is used for each horizontal line to remove noise.

For example, a mean value or a medium value of the horizontal line may be used as the adaptive threshold value, and the most frequent value among the pixel values of the horizontal line is determined in consideration of the number of pixels per horizontal line. It can also be used as a threshold. The threshold value may be set differently according to an implementation example. When the threshold is determined, the threshold is mapped to a vertical parallax map to generate a vertical parallax map along a vertical line.

In the case of the vertical parallax map, as described above, the planar region without the object decreases at a constant rate as the distance increases. However, when an object such as an obstacle is placed, the brightness value of the object portion is increased due to the object unlike the plane. Therefore, the object can be detected using the above phenomenon. The process of detecting the object and estimating the distance using the parallax value of the object is as described above.

If different parallax values appear within the same object, such as when the object is large enough or when the side part of the object is taken, etc., the distance can be estimated using the largest value of the parallax (the closest distance). have. For example, when applied to the field of driving a car, such as collision prevention, it is possible to use the value of the nearest distance estimated above for safe driving. However, depending on the needs of the applied field, the smallest parallax value (the farthest distance) or the middle value may be used.

FIG. 7 is a diagram schematically illustrating an object detection using a vertical distance and an object detection result using an actual distance according to an embodiment of the present invention. In the left figure of FIG. 7, the result is the object detection using the vertical distance. Since the object is detected and the distance is estimated in units of the horizontal plane based on the photographing position, A is the closest, and B and C are the same distance.

However, when the object is detected using the actual distance, since the object is detected and the distance is estimated in a circular unit based on the photographing position as shown in the right figure of FIG. 7, A and B are estimated to be the same distance, and C is It is assumed to be the farthest object.

8 is a diagram schematically illustrating a configuration of an object detecting apparatus according to an embodiment of the present invention. The object detecting apparatus includes a first image capturing apparatus 800, a second image capturing apparatus 805, a matching unit 810, a changing unit 820, and an object detecting unit 830.

The first image capturing apparatus 800 and the second image capturing apparatus 805 capture a left image and a right image, respectively, required for object detection, estimation of distance, speed, etc. in the stereo vision system. For example, a camera or the like may be used as the image capturing apparatus. The left image and the right image obtained by the first image capturing apparatus 800 and the second image capturing apparatus 805 are output to the matching unit 810.

The matching unit 810 may obtain a depth map of the image by matching the input left image with the right image. The depth map may obtain a depth map of the stereo image using the actual distance. For example, a depth map according to a distance with respect to a center may be obtained using the equation as shown in Equation 2 above. The obtained depth map information is transmitted to the converter 820.

The converter 820 converts the received depth map so that the brightness value of the depth map is horizontal for each vertical line. The conversion unit 820 may convert the depth map using a polar coordinate conversion method or a mapping method using a pixel reading order for the above conversion.

The object detector 830 detects the object included in the image by receiving the converted depth map information as described above from the converter 820. The object detector 830 removes noise by using an adaptive threshold value or an adaptive threshold value for each horizontal line. The object detector 830 may generate a vertical parallax map along the vertical line of the converted depth map, and detect an object included in the image. The distance, speed, etc. of the object may be estimated using the detected object.

9 is a flowchart schematically illustrating a distance estimation method according to an embodiment of the present invention.

The left and right images are photographed using the image capturing apparatus of the stereo vision system, and the captured left and right images are matched based on the actual distance (S900). As a result of the matching in step S900, a concentric depth map centered on the midpoint may be generated. When the depth map is generated as described above, the depth map is converted such that the depth map of the concentric circles becomes a horizontal depth map along a vertical line (S910). For example, the depth map can be converted using a polar coordinate conversion method, a mapping method using a pixel reading order, or the like.

When the converted depth map is generated as described above, a vertical parallax map may be calculated using the converted depth map (S920). In this case, the vertical parallax map may be calculated using an adaptive threshold value for each horizontal line in consideration of the difference in the number of pixels of the converted depth map. When the vertical parallax map is calculated, the objects included in the image may be detected using the vertical parallax map.

When the object is detected, the parallax may be calculated for the detected object, and the distance of the target object may be estimated using the parallax. When the distance of the object is estimated, the distance to the object may be tracked in units of frames by filtering the estimated distance information by using a dynamic filter such as a Kalman filter or a particle filter.

The present invention is not limited to the above-described embodiments, and as can be seen in the appended claims, modifications can be made by those skilled in the art to which the invention pertains, and such modifications are within the scope of the present invention.

1 is a diagram schematically illustrating a method of estimating distance according to a stereo vision method according to an embodiment of the present invention.

FIG. 2 is a diagram schematically illustrating a depth map obtained using a vertical distance as an embodiment according to the present invention.

3 is a diagram schematically illustrating a process of detecting an object using a vertical parallax map according to an embodiment of the present invention.

4 is a diagram schematically illustrating a method of estimating an actual distance of an object according to a stereo vision method according to an embodiment of the present invention.

5 is a diagram schematically illustrating a depth map obtained using an actual distance as an embodiment according to the present invention.

6 is a diagram schematically illustrating a process of creating a vertical parallax map by converting a depth map according to an embodiment of the present invention.

7 is a diagram schematically illustrating an object detection using a vertical distance and an object detection result using an actual distance according to an embodiment of the present invention.

8 is a diagram schematically illustrating a configuration of an object detecting apparatus according to an embodiment of the present invention.

9 is a flowchart schematically illustrating a distance estimation method according to an embodiment of the present invention.

* Explanation of symbols for the main parts of the drawings

800: first image capturing apparatus 805: second image capturing apparatus

810 matching unit 820 conversion unit

830: object detector

Claims (13)

Calculating a depth map centered on a midpoint by matching the photographed stereoscopic image based on a distance from the object; Converting a depth map such that the depth map centered on the midpoint is horizontal along a vertical line; Forming a vertical parallax map by mapping parallax values of pixels of the converted depth map to corresponding vertical lines; And And detecting an object included in an image according to a distribution of parallax values mapped to the vertical parallax map. The method of claim 1, The depth map centered on the above point, A method of detecting an object in which pixels at points separated by the same distance from the center of gravity are depth maps represented by the same brightness. The method of claim 1, Converting the depth map, An object detection method for transforming a depth map using a polar transformation method. The method of claim 1, Converting the depth map, An object detection method of converting a depth map by reading brightness values of pixels at equal distances from a midpoint in order and mapping them horizontally by vertical lines. The method of claim 1, Forming the vertical parallax map, An object detection method for removing noise by using a threshold value for each horizontal line of the converted depth map and forming a vertical parallax map. The method of claim 5, wherein Forming the vertical parallax map, An object detection method for removing noise by using an adaptive threshold value considering the number of pixels per horizontal line. The method of claim 1, Estimating the distance of the object by calculating the parallax of the detected object. In the object detecting apparatus, A matching unit which calculates a depth map centered on a midpoint by matching the photographed stereoscopic image with respect to a distance from an object; A conversion unit converting the depth map such that the depth map centered on the midpoint is horizontal along a vertical line; And A parallax map formed by mapping parallax values of pixels of the converted depth map to a corresponding vertical line, and detecting an object included in an image according to a distribution of parallax values mapped to the vertical parallax map. Object detection device. The method of claim 8, The matching unit, An object detecting apparatus for calculating a depth map in which pixels at points separated by the same distance from a center point are expressed with the same brightness. The method of claim 8, The conversion unit, An object detection apparatus for converting a depth map using a polar coordinate conversion method. The method of claim 8, The conversion unit, An object detection device that converts a depth map by reading brightness values of pixels at equal distances from a midpoint in order and mapping them horizontally by vertical lines. The method of claim 8, The object detector, An object detecting apparatus for removing noise by using a threshold value for each horizontal line of the converted depth map and forming a vertical parallax map. The method of claim 12, The object detector, An object detection apparatus for removing noise by using an adaptive threshold value considering the number of pixels per horizontal line.

Images