JPH1185981A - Distance measurement origin recognition device for moving objects - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To accurately detect an absolute position and an absolute azimuth in a three-dimensional space when measuring a distance from a self-position from an origin. SOLUTION: At a distance measurement origin, a landmark of a pattern having no same pixel pattern on the same scanning line and having a plurality of intersections of line segments is set, an image is taken by a stereo camera 10, and stereo matching is performed by a stereo processing unit. To generate a distance image. Then, the recognition processing unit 50 extracts a line segment by differentiating the luminance of the original image of the mark, and determines the line segment in the three-dimensional space by using the distance image, thereby detecting the absolute position and the absolute azimuth.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a distance measuring origin recognizing device for recognizing a mark set at a distance measuring origin and detecting a position and an azimuth at the origin.

[0002]

2. Description of the Related Art Conventionally, various technologies such as movement control, route detection, route detection, and location detection have been developed for mobile objects that move autonomously, such as unmanned robots, autonomous mobile work vehicles, and unmanned helicopters. Among these, self-location recognition is one of the important technologies.

[0003] As this self-position recognition technology, for example,
For mobile objects that travel autonomously on the ground, such as autonomous vehicles, two-dimensional angular velocities are detected by vibrating gyros or optical gyros, and translational speeds are detected by sensors that measure ground speed, and movement from a reference position is performed. There is a technology to calculate the self-position by calculating the amount, and for flying objects such as unmanned helicopters, the inertial navigation device detects the gravitational acceleration to detect the acceleration of the flying object and integrates this acceleration. There is a technology to know the amount of movement.

Furthermore, in recent years, a technique for recognizing the self-position of a moving object by applying image processing technology has been developed. In particular, a camera is imaged on the moving object to image the surrounding environment, and the timing of the imaging timing is adjusted. The technology for calculating the velocity component by detecting the motion of the moving body by calculating the optical flow between two different images and calculating the navigation trajectory from the ranging start point (origin) to recognize the self-position is enormous. It is possible to analyze the surrounding environment unique to an image having a large amount of information, and realize accurate autonomous navigation by determining complicated terrain.

[0005]

However, in the technique of self-position recognition by image processing as described above, when it is necessary to know the absolute position and the absolute azimuth of the self, some additional methods for recognizing the distance measurement origin are required. Equipment is required.

For this reason, it is conceivable to take measures such as separately mounting a geomagnetic sensor or the like and detecting its own attitude angle based on an absolute azimuth angle. However, accuracy or reliability is insufficient, or the apparatus becomes large. There are problems that it is unsuitable for mounting on a moving body and that the cost increases.

The present invention has been made in view of the above circumstances, and provides a distance measuring origin recognizing device capable of accurately detecting an absolute position and an absolute azimuth in a three-dimensional space when measuring the distance from the origin. It is intended to be.

[0008]

According to the first aspect of the present invention,
A pair of stereo cameras for capturing a mark placed at the distance measurement origin, and a pair of images captured by the stereo camera, a corresponding position is searched for, and a pixel shift amount generated according to a distance to an object. And a stereo processing unit that generates a distance image in which the perspective information to the object obtained from the pixel shift amount is quantified, and a line segment shape is extracted by differentiating the brightness of the captured image of the mark. A recognition processing unit that recognizes a three-dimensional position and orientation with respect to the distance measurement origin based on the shape and the distance image.

According to a second aspect of the present invention, in the first aspect of the present invention, the line segment shape is extracted based on an interval between peak values and a frequency of a histogram centered on an angle of the luminance differential vector. And

According to a third aspect of the present invention, in the first aspect of the present invention, the mark does not have the same pixel pattern on the same scanning line in the picked-up image, and has a plurality of intersections of line segments in the luminance differential image. It has a detectable pattern.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the mark has a pattern formed by combining areas having different luminances by combining triangles.

According to a fifth aspect of the present invention, in the first aspect of the present invention, the mark has a pattern in which an area having a luminance different from that of the surroundings is formed in an X-shape.

That is, in the distance measuring origin recognizing device according to the present invention, the mark set at the distance measuring origin is picked up by a stereo camera, and a corresponding position is searched for a pair of images picked up by the stereo camera to search for an object. Obtain the pixel shift amount that occurs according to the distance to, and generate a distance image that digitizes the perspective information to the object obtained from this pixel shift amount,
A line segment shape is extracted by differentiating the brightness of the picked-up image of the mark.
Recognize dimensional position and orientation. The line segment shape can be extracted based on the interval and frequency between the peak values of the histogram with the angle of the luminance differential vector as the axis.

As a mark to be set at the origin of the distance measurement, there is no identical pixel pattern that causes an error in distance measurement by the stereo method on the same scanning line on the captured image, and the intersection of line segments in the luminance differential image. Is desirably provided by a pattern in which regions having different luminances are formed by combining triangles, or a pattern in which regions having different luminances from the surroundings are formed in an X-shape.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. The drawings relate to an embodiment of the present invention. FIG. 1 is a basic configuration diagram of a distance measuring origin recognition device, FIG. 2 is a flowchart of a position / azimuth detection routine, and FIG. 3 is a diagram showing landmarks of a simple lattice pattern. FIGS. 4A and 4B are landmark pattern examples 1 in which mismatching does not occur.
FIG. 5 is an explanatory diagram showing a landmark pattern example 2 in which no mismatching occurs, FIG. 6 is an explanatory diagram showing a landmark pattern example 3 in which no mismatching occurs, and FIG. FIG. 8 is an explanatory diagram showing extraction of a small area, FIG. 8 is an explanatory diagram showing an example of a luminance differential vector, and FIG.
FIG. 4 is an explanatory diagram showing an angle histogram of a luminance differential vector.

In FIG. 1, reference numeral 1 denotes a distance measuring origin recognizing device for recognizing a mark set at a distance measuring starting point (origin) and detecting an absolute position and an absolute direction of the origin when recognizing the self-position of the autonomous moving body. As a basic configuration, a stereo camera 10 including a pair of cameras 10a and 10b for capturing the mark, and a pair of images captured by the stereo camera 10 are stereo-matched to generate a distance image. Stereo processing unit 20, this stereo processing unit 2
Distance image memory 3 for storing the distance image generated at 0
0, an original image memory 40 for storing an original image captured by the stereo camera 10, a recognition processing unit 50 for recognizing the mark by using the original image and the distance image of the mark, and detecting a position / direction. I have.

The distance measuring origin recognizing device 1 can be constituted as a part of a system for recognizing a self-position of a moving body by image processing. For example, with an autonomous flying vehicle such as an unmanned helicopter, a set of two stereo cameras for imaging a forward scenery (distant scenery) and a pair of two stereo cameras for imaging a lower scenery (ground surface). The optical flow between time-series images of distant scenery and the optical flow between time-series images of lower scenery are converted into the amount of movement in real space based on each distance image. The speed component is obtained, the rotational speed component caused by the movement of the distant image is removed from the speed component caused by the movement of the lower image, the pure translation speed component is obtained, and then converted to the translation speed component viewed from the distance measurement start point (origin). In a system that determines the navigation trajectory in a three-dimensional space by accumulating data, it is necessary to know the absolute position and azimuth of the origin by recognizing the landmark for detecting the origin installed on the ground when starting ranging. However, in such a case, the stereo camera 10 can be also used as a stereo camera for imaging the ground surface, and the stereo processing unit 20 and the recognition processing unit 50 can be incorporated as a part of a system. .

In order to secure the detection accuracy of the position and orientation, it is desirable that the landmark be a pattern having many detection points and many line segments. The pattern must have no adverse effects.

That is, in the stereo processing unit 20,
For a pair of images captured by a set of two stereo cameras 10, a city block distance is calculated for each small region of each image and a correlation between the city block distances is calculated to specify a corresponding region, and the distance to the object is determined. In order to obtain the three-dimensional image information (distance image) obtained by digitizing the perspective information to the object obtained from the shift amount, the parallax detection direction is obtained. If there are a plurality of regions similar to each other (in the horizontal scanning direction), parallax is erroneously detected (mismatched). The processing of generating a distance image from a captured image of a stereo camera is described in detail in Japanese Patent Application Laid-Open No. H5-114099 by the present applicant.

For example, since the landmark 59 of the pattern as shown in FIG. 3 is a simple grid-like pattern, the edge of the luminance difference extracted as a distance image is detected in the parallax detection direction of AA ′. Including small areas a, b, c, d,
Among the areas e and f, the areas a, c and e and the areas b, d and f are similar small areas, and a mismatch occurs in stereo processing, so that correct distance data cannot be obtained.

Therefore, according to the present invention, there is no landmark of a pattern in which there is no identical pixel pattern on the same scanning line that causes an error in distance measurement by the stereo method and there are a plurality of intersections of line segments, that is, stereo matching. In this case, there is no small area similar to the parallax detection direction, and a landmark of a pattern in which a similar area does not exist regardless of the direction in which parallax detection is performed (however the pattern is tilted in the image). Is adopted.

An example of such a landmark pattern is:
The landmark 60 shown in FIGS.
5A, each of the landmarks 60B shown in FIG. 5 has a pattern in which regions having different luminances are formed by combining triangles, and the small regions a, a with respect to the parallax detection direction AA ′.
There is no similar region among b, c, d, e, f, and g, and no mismatch occurs. Further, the landmark 60C shown in FIG. 6 is a pattern in which regions having different luminance from the surroundings are formed in an X-shape.
There is no similar area among the small areas a, b, c, d, e, and f with respect to A ′, and the resolution can be secured by securing the imaging length of the line segment. .

The above landmarks 60A (60B, 60B)
In response to C), the recognition processing unit 50 executes the position and orientation detection routine of FIG. 2 to detect the position and orientation.

In the position and orientation detection routine, first, in step S101, a search area set in advance to detect a mark is divided into small areas in the original image of the mark, and the luminance of the small area is calculated from the small area. , And a luminance differential vector ΔP. For example, as shown in FIG. 7, the mark search area is divided into small areas of 2 × 2 pixels, and the luminance data P0,
When luminance data P2 and P3 are obtained in the P1 and j directions,
The luminance differential vector ΔP can be given by the following equation (1).

The above-described luminance differential vector ΔP can be expressed by the following equations (2) and (3) when rewritten with the angle α and the length (size) L. α = tan-1 (−ΔPi / ΔPj) (2) L = (ΔPi ² + ΔPj ² ) ^1/2 (3)

Next, the process proceeds to step S102, and it is checked whether or not the length L of the luminance differential vector ΔP exceeds a predetermined threshold value for removing noise. If the length L of the vector does not exceed the threshold, the process jumps to step S104. If the length L exceeds the threshold, the process proceeds to step S103.
The frequency is added to a histogram h [α] (α = −90 to +90 deg) with the vector angle α as an axis, the coordinate data i [n], j [n], and the distance data obtained from the distance image. d [n] and the angle data K [n] are stored in the memory, and the process proceeds to step S104.

In step S104, it is determined whether or not the processing of the search range has been completed. If the processing of the search range has not been completed, the process returns to step S101 to repeat the above-described processing. When the luminance differential vector ΔP is obtained for the entire area, the process proceeds to step S105, where the angle histogram is filtered, and the process proceeds to step S105.
In S106, which group in the histogram is the detection result of which line segment in the pattern is determined from the interval and frequency of the peak value of the angle histogram, the line segment is classified, and the slope and intercept of the line segment are calculated.

For example, in the case of the landmark 60A having the pattern shown in FIG. 4, when the mark is horizontally displayed in the image, an angle histogram as shown in FIG. When the coordinates of the small area are plotted on the image, line segments LA, LB, LC, LD, LE as shown in FIG. 8 are obtained corresponding to the pattern. Line segments LF, LG, LH, and LI are luminance differential vectors with respect to the background.

In this case, in the pattern of FIG.
It is known that a luminance differential vector exists in a combination of g, ± 60 deg, ± 90 deg. From the relationship between the angle of the line segment shown in FIG. 8 and the histogram group in FIG. 9, among the groups H1 to H6, The line segments LB and LE for the group H1, the line segments LF and LH for the group H2, and the line segment L for the group H3.
It can be determined that the line segments LC, LG, and LI correspond to A, LD, and the group H4, respectively.

Then, by averaging the group H1, the inclination θ BE of the line segments LB and LE is obtained.
In order to divide B and the line segment LE, the coordinates belonging to the line segment LB and the line segment L from the angle data K [n] stored in the memory.
The coordinates belonging to E are extracted, and the intercept MB of the line segment LB and the intercept ME of the line segment LE are obtained by Hough transform.

The same processing is performed on the other groups to obtain all the inclinations and intercepts of the line segments LA to LI. Then, the process proceeds from step S106 to step S107, where the coordinates of the intersection of each line segment are calculated.
Calculate the angle to recognize the two-dimensional position and orientation of the mark,
Further, an area is set from the inclination and intercept of each line segment, and determined as a line segment in a three-dimensional space using the extracted coordinates i [n] and j [n] and the distance data d [n]. Thus, the absolute position and the absolute direction in the three-dimensional space are detected.

Thus, the self-position of the moving body in the three-dimensional space can be accurately recognized based on the absolute position and the absolute azimuth, and furthermore, it can be realized by a small and inexpensive device that can be mounted on the moving body.

In the case of the patterns of FIGS. 5 and 6, the combination of angles is 0 deg, ± 45 deg, ± 90 deg.
By performing the same processing, a three-dimensional position and orientation can be recognized.

[0034]

As described above, according to the present invention, a mark placed at the distance measurement origin is picked up by a stereo camera to generate a distance image in which the perspective information is quantified, and a picked-up image of the mark is obtained. Is a small and inexpensive device that can be mounted on a moving body or the like, in order to extract a line segment shape by differentiating the luminance and to recognize a three-dimensional position and orientation with respect to the distance measurement origin based on the line segment shape and the distance image. Thus, an excellent effect is obtained, such that the absolute position and the absolute azimuth of the distance measurement start origin in the three-dimensional space can be accurately detected.

[Brief description of the drawings]

FIG. 1 is a basic configuration diagram of a distance measurement origin recognition device.

FIG. 2 is a flowchart of a position / azimuth detection routine.

FIG. 3 is an explanatory view showing landmarks of a simple lattice pattern.

FIG. 4 is an explanatory diagram showing a landmark pattern example 1 in which mismatching does not occur;

FIG. 5 is an explanatory diagram showing a landmark pattern example 2 in which mismatching does not occur;

FIG. 6 is an explanatory diagram showing a landmark pattern example 3 in which mismatching does not occur.

FIG. 7 is an explanatory diagram showing extraction of a small area for calculating a luminance derivative.

FIG. 8 is an explanatory diagram showing an example of a luminance differential vector.

FIG. 9 is an explanatory diagram showing an angle histogram of a luminance differential vector.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Distance measurement origin recognition apparatus 10 ... Stereo camera 20 ... Stereo processing part 50 ... Recognition processing part 60A, 60B, 60C ... Landmark

Claims (5)

[Claims] 1. A pair of stereo cameras for picking up a mark set at a distance measuring origin, and a pair of images picked up by the stereo camera are searched for a corresponding position, and a corresponding position is searched according to a distance to an object. A stereo processing unit that determines the amount of pixel shift that occurs due to this, and generates a distance image that quantifies the perspective information to the object obtained from the pixel shift amount, and extracts the line segment shape by differentiating the brightness of the captured image of the mark And a recognition processing unit for recognizing a three-dimensional position and orientation with respect to the distance measurement origin based on the shape of the line segment and the distance image. 2. The distance measuring origin recognizing device according to claim 1, wherein the line segment shape is extracted based on an interval between peak values and a frequency of a histogram centering on an angle of the luminance differential vector. 3. The mark according to claim 1, wherein the mark does not have the same pixel pattern on the same scan line in the captured image, and has a pattern capable of detecting a plurality of intersections of the line segment in the luminance differential image. Distance measuring origin recognition device described. 4. The distance measuring origin recognizing device according to claim 1, wherein the mark has a pattern in which regions having different luminances are formed by combining triangles. 5. The distance measuring origin recognizing device according to claim 1, wherein the mark has a pattern in which an area having a luminance different from that of a surrounding area is formed in an X-shape.