KR100955483B1 - 3D grid map making method and control method of automatic driving device using same - Google Patents

Abstract

Disclosed are a three-dimensional grid map preparation method and a control method of an automatic traveling device using the same. In creating a three-dimensional map for grasping the current position of the unmanned vehicle or the mobile robot and the surrounding topography, the two-dimensional positioning and the three-dimensional restoration are appropriately used to enable the accurate three-dimensional lattice map to be created in a shorter time.

3D, Coordinates, Navigation, SLAM, Localization, Mapping

Description

METHOD OF BUILDING 3D GRID MAP AND METHOD OF CONTROLLING AUTO TRAVELLING APPARATUS USING THE SAME}

The present invention relates to a 3D grid mapping method, and more particularly, to a 3D grid mapping method for creating a 3D map by grasping the surrounding terrain using a stereo omni-directional camera.

The omnidirectional camera is a camera system that photographs all directions at one time by using a rotating reflector, a condenser lens, and an imaging device, and is applied to security facilities, surveillance cameras, and robot vision. The shape of the reflector of the rotating body may vary from a hyperbolic surface, a spherical surface, a cone, a complex type, and the like, and a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) is used. The image projected onto the image pickup surface of the image pickup device (i.e., the omnidirectional image) is a distorted image that is reflected by the rotating reflector and is not suitable for human observation. Therefore, in order to accurately observe the image, the output of the image pickup device is converted into coordinates through an external microprocessor or the like to create a new panoramic image.

The omnidirectional image obtained by this omnidirectional camera provides two-dimensional information about the surroundings of the omnidirectional camera. If a plurality of omnidirectional images taken from different directions through a plurality of omnidirectional cameras are used, three-dimensional information about the omnidirectional cameras may be obtained. An imaging device composed of a plurality of omnidirectional cameras is called a stereo omnidirectional camera. Such a stereo omnidirectional camera may be mounted on an unmanned vehicle or a mobile robot, and the omnidirectional image captured by the stereo omnidirectional camera may be used for location recognition and mapping of an unmanned vehicle or a mobile robot.

SUMMARY OF THE INVENTION An object of the present invention is to enable accurate 3D location recognition and map creation in a short time in 3D location recognition and map making to grasp the current location and surrounding terrain of an unmanned vehicle or mobile robot.

Three-dimensional grid mapping method according to an aspect of the present invention, by using a stereo omnidirectional camera to obtain the surrounding omnidirectional image; Confirm the current two-dimensional position by using the omnidirectional image; Perform 3D reconstruction of the omnidirectional image; The 3D grid map around the stereo omnidirectional camera is created by combining the 2D positioning result and the 3D reconstruction result.

In addition, the above-described two-dimensional position confirmation is performed by performing two-dimensional position confirmation and map preparation using an omnidirectional image.

In addition, the two-dimensional positioning using the above-mentioned omnidirectional image is to perform two-dimensional positioning using the feature points on the epipolar line of the omnidirectional image.

In addition, the feature points on the epi-rolla line of the above-mentioned omnidirectional image are obtained through color edge extraction of the omnidirectional image.

The method may further include obtaining a degree of occupancy of the grating using the above-described three-dimensional reconstruction result.

The degree of occupancy of the above-described grating is determined by determining that the corresponding voxel is occupied when the occupancy count of the voxel of the three-dimensional grating exceeds a predetermined number.

According to another aspect of the present invention, there is provided a control method of an automatic traveling device, the method including: obtaining an omnidirectional image of a surrounding using a stereo omnidirectional camera; Confirm the current two-dimensional position by using the omnidirectional image; Perform 3D reconstruction of the omnidirectional image; Combine a two-dimensional positioning result with a three-dimensional reconstruction result to create a three-dimensional grid map around the stereo omnidirectional camera; Determine a driving route from the current position to the target point based on the three-dimensional grid map; Control the vehicle to travel to the target point along the driving route.

In addition, the above-described two-dimensional position confirmation is performed by performing two-dimensional position confirmation and map preparation using an omnidirectional image.

In addition, the two-dimensional positioning using the above-mentioned omnidirectional image is to perform two-dimensional positioning using the feature points on the epipolar line of the omnidirectional image.

In addition, the feature points on the epirolla line of the omnidirectional image are obtained by extracting color edges of the omnidirectional image.

The method may further include obtaining a degree of occupancy of the grating using the above-described three-dimensional reconstruction result.

The degree of occupancy of the above-described grating is determined by determining that the corresponding voxel is occupied when the occupancy count of the voxel of the three-dimensional grating exceeds a predetermined number.

According to the above, in the 3D location recognition and map creation for grasping the current location and the topography of the unmanned vehicle or mobile robot, it is possible to accurately recognize the 3D location and maps within a short time.

Referring to FIGS. 1 to 6, a preferred embodiment of a method for preparing a 3D grid map and a method for controlling an autonomous driving apparatus using a 3D grid map according to the present invention for this purpose is as follows. 3D grid mapping method according to an embodiment of the present invention can be applied to the determination of the moving path by grasping the current location and the surrounding terrain in an unmanned vehicle or mobile robot. It is also applicable to fields such as security devices and monitoring devices.

1 is a view showing the configuration of a stereo omnidirectional camera according to an embodiment of the present invention. As shown in FIG. 1, the first omnidirectional camera 102 and the second omnidirectional camera 104 are arranged in a line so as to face the same direction to form one stereo omnidirectional camera 100. Although not shown in FIG. 1, the first omnidirectional camera 102 and the second omnidirectional camera 104 may be fixed by separate fixing devices.

The first omnidirectional camera 102 is provided such that the camera module 102a and the rotating reflector 102b face each other. The rotating reflector 102b reflects light incident at a 360 ° angle around the first omnidirectional camera 102 to be transmitted to the camera module 102a. In the camera module 102a, the CCD 102c forms an image plane and converts the light reflected by the reflector reflector 102b into an electrical signal to generate an image signal.

The second omnidirectional camera 104 is also composed of a camera module 104a and a rotating reflector 104b. In the case of the camera module 104a of the second omnidirectional camera 104, the CCD 104c forms an image plane and converts the light reflected by the rotating reflector 104b into electrical energy to generate an image signal.

Each of the first omnidirectional camera 102 and the second omnidirectional camera 104 creates an image by capturing an image of a surrounding 360 ° angle of view (that is, an omnidirectional image). Since only one omnidirectional image can know two-dimensional information of the subject, three-dimensional information of the subject is obtained using two omnidirectional images photographed at different positions.

FIG. 2 is a diagram illustrating a control system of the stereo omnidirectional camera shown in FIG. 1. As shown in FIG. 2, the first omnidirectional camera 102 and the second omnidirectional camera 104 of the stereo omnidirectional camera 100 are electrically connected to an input side of the controller 202. The controller 202 converts an electrical signal generated by each of the first omnidirectional camera 102 and the second omnidirectional camera 104 of the stereo omnidirectional camera 100 into an image signal, and converts the stereo omnidirectional signal from an image implemented through the image signal. A three-dimensional map around the camera 100 is generated. The generated 3D map is stored in the memory 204, and the 3D map is updated whenever the position of the stereo omnidirectional camera 100 is changed. If the stereo omnidirectional camera 100 is mounted on an unmanned vehicle or a mobile robot, the position control or motion control is performed by grasping the current position of the unmanned vehicle or the mobile robot and the terrain of the surrounding robot from the 3D map.

3 is a view showing an image captured by the stereo omnidirectional camera shown in FIG. As shown in FIG. 3, when the surrounding is photographed by the stereo omnidirectional camera 100, a first omnidirectional image 302 is obtained through the first omnidirectional camera 102, and a second omnidirectional image is obtained through the second omnidirectional camera 104. 304 is obtained. In an embodiment of the present invention, the epipolar line 306 of the first omnidirectional image 302 obtained through the first omnidirectional camera 102 is obtained, and the feature points on the epipolar line 306 ( 308 is extracted to obtain two-dimensional coordinates of these feature points 308. The two-dimensional coordinates of these feature points 308 are used to generate a three-dimensional grid map. In an embodiment of the present invention, a color edge extraction method is used to extract these feature points 308.

FIG. 4 is a diagram illustrating a panoramic image and a stereo matching image of two omnidirectional images shown in FIG. 3. In FIG. 4, 402 is a panoramic image of the first omnidirectional image 302, that is, a first panoramic image, 404 is a panoramic image of the second omnidirectional image 304, that is, a second panoramic image, and 406 is a first panoramic image 402. ) And a stereo matching image (Disparity Map) of the second panoramic image 404. In the stereo matched image 406 shown in FIG. 4, the subject located closer to the stereo omnidirectional camera 100 appears relatively brighter, and the subject located farther from the stereo omnidirectional camera 100 appears relatively darker. Using this brightness information (ie, depth of an image), the distance between the stereo omnidirectional camera 100 and the subjects can be estimated.

5 is a diagram illustrating a 3D coordinate extraction concept of a stereo omnidirectional camera according to an embodiment of the present invention. In FIG. 5, <b> is a base line which is the distance between the first omnidirectional camera 102 and the second omnidirectional camera 104, and <r> is the radius of the rotor reflectors 102c and 104c. And <p> and <p '> are characteristic points in each omnidirectional image, and <d'> and <d> are from the first omnidirectional camera 102 and the second omnidirectional camera 104 to the subject P, respectively. Is the distance. <102c> and <104c> are the imaging surfaces of each of the first omnidirectional camera 102 and the second omnidirectional camera 104 as described above with reference to FIG. 1.

The distance d between the second omnidirectional camera 104 and the subject P is obtained by the following equation (1). Similarly, the distance d 'between the first omnidirectional camera 102 and the subject may be obtained.

Formula (1)

In this way, the distance d between the stereo omnidirectional camera 100 and the subject P is obtained, and the azimuth of the subject P centered on the stereo omnidirectional camera 100 is determined from the panoramic image of FIG. 4. In this case, the three-dimensional coordinates P (X, Y, Z) of the subject P represented by the following equation (2) are obtained.

Formula (2)

6 is a diagram illustrating a localization and mapping method according to an embodiment of the present invention. Simultaneous Localization and Mapping (hereinafter referred to as SLAM) is performed simultaneously with localization and mapping. SLAM according to an embodiment of the present invention is a two-dimensional SLAM 600a and a three-dimensional SLAM ( 600b) together. Since the three-dimensional SLAM has a large amount of calculation, the two-dimensional SLAM and the three-dimensional SLAM are mixed to reduce the total calculation amount, thereby speeding up the positioning and mapping. In other words, if the necessary data can be obtained with only the 2D SLAM, only the 2D SLAM is performed, and if the 3D SLAM is performed where the 3D SLAM is required, the amount of computation is greatly reduced than if the all data is obtained through the 3D SLAM. . Significantly reduced amount of computation for positioning means that the required positioning can be done in a shorter time, which means that an automatic driving device (a device such as an unmanned vehicle or a mobile robot) equipped with a stereo omnidirectional camera 100 is more suitable. It means you can move at a high speed.

First, in the two-dimensional SLAM 600a, the first omnidirectional image 302 and the second omnidirectional image 304 using the first omnidirectional camera 102 and the second omnidirectional camera 104 of the stereo omnidirectional camera 100, respectively. (602). The epipolar line 306 of the first omnidirectional image 302 obtained using the first omnidirectional camera 102 located below the center is obtained, and the feature points 308 in the epipolar line 306 are obtained. Extract (604). In the two-dimensional SLAM, two-dimensional coordinates of the feature points 308 are obtained. For this, the three-dimensional position recognition result of the three-dimensional SLAM 600b is used.

In the 3D SLAM 600b, a stereo matching image 406 is generated through stereo matching of the first panoramic image 402 and the second panoramic image 404 (606). The stereo matched image 406 enables 3D reconstruction of the image around the stereo omnidirectional camera 100 using the same method as described in the description of FIG. 5 (608).

Returning to the two-dimensional SLAM 600a, two-dimensional coordinates of the extracted feature points 308 are obtained by applying the following equations (3), (4), and (5) to the stereo omnidirectional image (610).

Formula (3)

식 (5)

Formula (4)

Equation (5)

는 상태 벡터이고,

는 입력 벡터이며,

는 입력 오차이고,

는 측정 값이며,

는 측정 오차이다. 식 (3)의 상태 벡터

는 식 (4)에 나타낸 것과 같은 스테레오 전방위 카메라(100)의 위치와 특징점들(308)의 2차원 좌표들로 구성되며, 식 (5)의 측정 값은 스테레오 전방위 카메라(100)에서 측정한 특징점의 각도(방위)이다. 이와 같은 식 (3), (4), (5)를 이용하면 에피폴라 라인(306)의 특징점들(308)의 위치 와 스테레오 전방위 카메라(100)의 위치 추정이 가능하다.Equation (3) above is for a two-dimensional SLAM,

Is the state vector,

Is the input vector,

Is the input error,

Is the measured value,

Is the measurement error. State vector of equation (3)

Is composed of the position of the stereo omnidirectional camera 100 and two-dimensional coordinates of the feature points 308 as shown in equation (4), and the measured value of equation (5) is the feature point measured by the stereo omnidirectional camera 100. Is the angle (orientation) of. Using equations (3), (4) and (5), it is possible to estimate the position of the feature points 308 of the epipolar line 306 and the position of the stereo omnidirectional camera 100.

If the stereo omnidirectional camera 100 according to an embodiment of the present invention is mounted and operated in an unmanned vehicle or a mobile robot, the stereo omnidirectional camera 100 may be moved. Accordingly, the motion of the stereo omnidirectional camera 100 is predicted, and the previously measured two-dimensional coordinate values are newly corrected and updated (612).

When the result of obtaining the 2D coordinates in the 2D SLAM 600a and the 3D position recognition result in the 3D SLAM 600b are combined, a 3D grid map around the stereo omnidirectional camera 100 may be generated (614). . If the occupancy count of each voxel is above a certain level, the voxel is occupied, and thus a three-dimensional lattice map can be constructed. If a voxel exists in the field of view of the stereo omnidirectional camera 100 (when the occupancy or non-occupancy can be confirmed), if the occupancy count is exceeded for a predetermined number of times, the voxel is determined to be occupied.

1 is a view showing the configuration of a stereo omnidirectional camera according to an embodiment of the present invention.

2 is a view showing a control system of the stereo omnidirectional camera shown in FIG.

3 is a view showing an image taken by the stereo omnidirectional camera shown in FIG.

FIG. 4 is a view illustrating a panoramic image and a stereo matching map of two omnidirectional images shown in FIG. 3; FIG.

5 is a view showing a three-dimensional coordinate extraction concept of a stereo omnidirectional camera according to an embodiment of the present invention.

6 is a diagram illustrating a localization and mapping method according to an embodiment of the present invention.

* Description of the symbols for the main parts of the drawings *

100: stereo omnidirectional camera

102, 104: omnidirectional camera

102a, 104a: Camera Module

102b, 104b: Rotor reflector

102c, 104c: imaging surface

302, 304: omnidirectional image

306: Epipolar Line

308 features

402, 404: panoramic image

406: stereo matching image

Claims (12)

Acquire a surrounding omnidirectional image using a stereo omnidirectional camera; Identifying a current two-dimensional position by using the omnidirectional image; Perform 3D reconstruction of the omnidirectional image; 3D grid map creation method of combining the two-dimensional positioning result and the three-dimensional reconstruction result to create a three-dimensional grid map around the stereo omnidirectional camera. The method of claim 1, The two-dimensional positioning is a three-dimensional grid map creation method made by performing a two-dimensional positioning and map creation using the omnidirectional image. The method of claim 2, 2D positioning using the omnidirectional image, the 2D positioning method using the feature points on the epipolar line of the omnidirectional image. The method of claim 3, wherein The feature points on the epipolar line of the omnidirectional image are obtained by extracting color edges of the omnidirectional image. The method of claim 1, 3. The method of claim 3, further comprising obtaining an occupancy degree of the grid by using the 3D reconstruction result. The method of claim 5, Obtaining the occupancy of the grating, if the occupancy count of the voxel (Voxel) of the three-dimensional grating exceeds a predetermined number is determined that the corresponding voxel is occupied. Acquire a surrounding omnidirectional image using a stereo omnidirectional camera; Identifying a current two-dimensional position by using the omnidirectional image; Perform 3D reconstruction of the omnidirectional image; Combine the two-dimensional positioning result with the three-dimensional reconstruction result to create a three-dimensional grid map around the stereo omnidirectional camera; Determine a driving route from a current position to a target point based on the 3D grid map; And a control method for controlling the automatic traveling device to travel to the target point along the driving route. The method of claim 7, wherein The two-dimensional positioning is a control method of the automatic driving device is performed by performing the two-dimensional positioning and map creation using the omnidirectional image. The method of claim 8, The two-dimensional positioning using the omnidirectional image is to perform the two-dimensional positioning using the feature points on the epipolar line of the omnidirectional image. The method of claim 9, And the feature points on the epipolar line of the omnidirectional image to obtain the feature points through color edge extraction of the omnidirectional image. The method of claim 7, wherein The control method of the automatic traveling device for obtaining the occupancy of the grid using the three-dimensional reconstruction result. The method of claim 11, Obtaining the occupancy of the grating, if the occupancy count of the voxel (Voxel) of the three-dimensional grating exceeds a certain number, the control method of the automatic driving apparatus determines that the voxel is occupied.

Images