JPH0674732A - Track recognizing apparatus - Google Patents

Abstract

PURPOSE:To make highly accurate high speed measurement of a yaw angle possible by determining an angle corresponding to the maximum energy of a two-dimensional power spectrum for each of more than one imaging region and then determining curvature or radius of a lane marker based on the difference. CONSTITUTION:Image data VD in cartesian coordinates (x-y) is converted into image data DD in cartesian coordinates (X-Y) corresponding to imaging in the direction normal to the running pavement thus eliminating the effect of distance. The image data DD is then split into front and rear region image data DDA, DDB in the advancing direction and the data DDA, DDB are subjected to fast Fourier transform. Power spectrum in cartesian coordinates system thus obtained is then converted into polar coordinates system of radius (r) and angle theta and integrated for every predetermined angle within an angle range of 0-180 deg. thus determining angles theta1, theta2 corresponding to the maximum peaks in a unique spectrum distribution corresponding to the trace feature of the lane marker. Radius or curvature of the lane marker can be determined based on the difference ¦theta1-theta2¦ of the angles theta1, theta2.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a road recognizing device for recognizing a deviation from regular running by determining an angle formed by a traveling direction of a running vehicle with respect to a lane marker displayed and displayed on a running road.

[0002]

2. Description of the Related Art When a vehicle deviates from a regular traveling road while traveling on a highway, for example, such a traveling road recognizing device automatically recognizes this and recognizes it on the display device provided in the driver's seat. It is applied to driving assistance devices that warn the driver of abnormalities or suggest safe driving by displaying an image of the deviation from the running path or sounding a buzzer, and so as to follow the regular running path. It has been applied to a wide range of technologies such as self-propelled material transfer robots and autonomous mobile vehicles while recognizing images.

In addition, the track recognition device of the present invention recognizes and displays line markers in advance along a route along which a vehicle should travel (a so-called center line described and displayed on the road surface or a floor such as a factory floor). A video camera for picking up a travel line, etc., and an area discriminating means for discriminating an area including the lane marker from other areas by performing edge detection or the like on the imaged data obtained by this imaging,
Based on the imaged data of the area of the lane marker separated by the area discrimination means, based on the imaged data obtained from the extracting means and the imaged data of the lane marker to be the target of the recognition process. Hough Transform
The angle between the lane marker and the traveling direction of the vehicle by performing signal processing (such as nsform) (hereinafter referred to as the yaw angle)
It is composed of a determining means for determining Then, when the yaw angle is large, it is determined that the vehicle is not traveling along the regular route direction, and the above-mentioned abnormality warning is issued,
It is applied to control such as returning the route to a regular route.
In addition, a region discriminating means for performing these signal processing, an extracting means
The deciding means and the like are realized by a microprocessor or the like having an arithmetic function.

Further, in order to process the traveling of the vehicle in real time, that is, in order to realize high-speed processing, the image data of the lane marker obtained by the extracting means is
By dividing the image data into multiple small areas along the lane marker and performing image processing only on the small parts of the lane markers included in the divided small areas, the amount of data processing is reduced and the above high speed is achieved. I am trying to make it.

Further, when the lane marker draws an arc corresponding to a curved road, the curvature is obtained by approximating the arc with a circle having a specific radius of 1 or 2 or more. Correction processing such as obtaining the yaw angle with respect to the tangential direction of the circle based on the radius and the curvature is performed.

[0006]

In order to determine the yaw angle with high accuracy in such a road recognition device, there are the following problems to be solved. First, since the road image projected on the image receiving surface (hereinafter referred to as the angle of view) of the video camera includes not only lane markers but also background images of houses, trees, etc. Since it is necessary to accurately separate the image of the lane marker and the image of the lane marker, there is a problem that the process is complicated and the process is delayed. Secondly, since the raw image data of the lane marker includes the image of the road surface and the like, the amount of processing is enormous to extract characteristic data showing only the trajectory of the lane marker by eliminating unnecessary road surface images and the like. Therefore, there is a problem that high-speed processing becomes difficult. Thirdly, as one method for solving the first and second problems, the lane marker is divided into imaging data for each of a plurality of small areas along the lane marker, and lanes included in the divided small areas. When the amount of processed data is reduced by performing image processing only on the small part of the marker, the image of the lane marker is divided into small parts, making it difficult to extract the features of the entire lane marker, and the yaw angle determination accuracy Is reduced. That is, in the lane marker portion for each small region, the features such as faintness, damage, and broken portions become prominent, so that the original trajectory feature of the lane marker cannot be accurately extracted.
If the characteristics of the entire lane marker are to be obtained from the respective characteristics obtained from these small areas, another correction means is required, which leads to an increase in the size of the device.

SUMMARY OF THE INVENTION It is an object of the present invention to solve the various problems in the prior art and to provide a track recognition device capable of determining a yaw angle with high speed processing and high accuracy.

[0008]

In order to achieve such an object, the present invention takes an image of a lane marker displayed on a traveling road by an imaging device provided on a traveling vehicle, and displays the lane marker of the imaged lane marker. An image receiving surface that is on a two-dimensional coordinate orthogonal to the optical axis of the image pickup device, which is intended for a track recognition device that determines the radius or curvature of the lane marker and the yaw angle of a traveling vehicle based on image pickup data. Data conversion means for converting the imaged data obtained in 1. into image data projected in two-dimensional coordinates that are parallel and opposed to the traveling road surface; and the imaged data with respect to the traveling direction of the traveling vehicle. Data dividing means for dividing each image data in two or more imaging regions in the front and rear, and two-dimensional power spectrum on polar coordinates defined by radius and angle based on each image data The calculation means and the energy distribution for each angle are calculated based on the respective two-dimensional power spectrums, the angle with respect to each maximum energy is calculated, and the angle difference between the predetermined two angles is predetermined. And a determination means for determining the radius or curvature of the lane marker from the first correlation data, and determining the yaw angle from the predetermined second correlation data with respect to the radius or curvature and the angle with respect to the maximum energy. It was configured.

[0009]

In the track recognizing device of the present invention having such a structure, the two-dimensional power spectrum on the polar coordinates defined by the radius and angle has a high energy distribution along the angle formed by the traveling vehicle and the lane marker. Is used. In addition, this 2
Since the dimensional power spectrum is obtained and the angle with respect to each maximum energy is obtained, and further, the curvature or radius of the lane marker is obtained based on these angle differences, the curvature or radius can be obtained with high accuracy. Since the yaw angle is obtained based on these curvatures or radii, the deviation of the traveling direction of the vehicle with respect to the traveling path can be obtained with high accuracy.

Further, since the image pickup device is installed at a high place of the vehicle and its optical axis is directed to the traveling road side with a depression angle, unnecessary images outside the traveling road can be blocked, so that the yaw angle can be reduced. It can be obtained more accurately.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. First, when installing in a vehicle such as an automobile, as shown in FIG. 1, an electronic image pickup device 4 such as a video camera or an electronic still camera is installed in a high place of the vehicle 2 and an image is picked up by the electronic image pickup device 4. A main body portion 6 of a track recognition device for signal-processing the obtained frame image pickup data, and a display device 8 for displaying the processing result and a display device 8 such as an alarm device for ringing at the time of alarm are installed in the vehicle 2 or the like. There is. Here, as a result of the electronic image pickup device 4 being installed so that the light-receiving axis is directed toward the front of the road surface of the traveling road 10 with a predetermined depression angle, a predetermined road surface range in front of the vehicle 2 (at the angle of view of the electronic image pickup device 4). An image of the set range W is looked down from diagonally above.

Further, the main body portion 6 of the track recognition device is shown in FIG.
As shown in, the image conversion and image division unit 12 that receives the image data VD of the frame image output from the electronic imaging device 4 and performs preprocessing, and the image data obtained by the processing of the image conversion and image division unit 12 are A two-dimensional power spectrum calculation unit 14 that obtains a power spectrum of image data by performing a Fourier transform, and a two-dimensional power spectrum in a rectangular coordinate system obtained by the two-dimensional power spectrum calculation unit 14 in polar coordinates defined by a radius and an angle. An energy distribution calculation unit 16 for obtaining an energy distribution for each angle by converting the power spectrum into a two-dimensional power spectrum of the system and further performing an integral calculation of the power spectrum for each angle, and an angle at which the energy distribution has a maximum peak. Based on the feature data, and And a determination unit 20 a final yaw angle Te. Specifically, these parts are realized by a structure having an arithmetic function such as a microprocessor and a logic circuit.

The function of each unit will be further described in detail. As described above, the electronic image pickup device 4 is installed at a predetermined height h from the road surface and is installed at a predetermined depression angle α with respect to a predetermined range W in front of the traveling road 10. The relationship is shown in FIG. The image in the range W incident through the image pickup optical system 22 installed on the light receiving axis is formed on the image receiving surface 24 of the two-dimensional solid-state image pickup device (CCD) arranged orthogonal to the light receiving axis. To be imaged. As a result, on the image receiving surface 24, the lane marker included in the predetermined range W becomes narrower as it gets farther, and a larger image is formed as it gets closer.

Here, in this embodiment, the image conversion data and the image division are performed by using the image data VD formed of the pixel data group imaged by the pixel group of the two-dimensional solid-state image pickup device as the frame image data of the orthogonal coordinate system (xy). It is output to the unit 12.

The image conversion and image division unit 12 converts the above-mentioned image pickup data VD represented by the Cartesian coordinate system (xy) into a Cartesian coordinate system of a virtual projection plane (parallel to the road surface of the traveling road 10). (X-Y) data conversion to image data DD. That is, in the imaging data VD of the orthogonal coordinate system (xy), as shown in FIG. 4A, for example, the image at the far place is small due to the distance difference from the vehicle, and the image at the near place is smaller than that. As shown in FIG. 4B, the data is converted into the image data DD of the orthogonal coordinate system (XY) corresponding to the case where the image is taken from the vertical direction with respect to the road surface of the traveling road. I try to eliminate the effect of perspective. As a result, the image data DD becomes orthogonal coordinate data in which the traveling direction of the vehicle 2 is the Y axis and the direction orthogonal thereto is the X axis. In this coordinate conversion, the height is h, the depression angle is α, and the focal length of the imaging optical system is f.
Then, it is calculated by the following equations (1) and (2).

[0016]

[Equation 1]

Further, the image data DD thus obtained is subjected to edge enhancement processing to sharpen the image of the lane marker with respect to the image of the road surface, and then the image data of the front area A in the traveling direction. The image data DDB of the DDA and the rear area B are divided and supplied to the two-dimensional power spectrum calculation unit 14.

The two-dimensional power spectrum calculation unit 14 is
Fast Fourier Transform (FFT) for image data DDA
Then, the power spectrum in the Cartesian coordinate system obtained by the Fourier transform is transformed into the power spectrum | F _A (r, θ) | ² in the polar coordinate system defined by the radius r and the angle θ. A distribution as shown in (c) is obtained. Similarly, for the image data DDB, the fast Fourier transform (FFT) is performed, and the power spectrum in the orthogonal coordinate system obtained by the Fourier transform is calculated with the radius r and the angle θ.
Power spectrum of polar coordinate system defined by | F
_By performing coordinate conversion into _B (r, θ) | ² , a distribution as shown in FIG. 4 (d) is obtained. When such arithmetic processing is performed,
In FIGS. 4C and 4D, the reference coordinate L of the angle θ = 0 °
A unique spectrum distribution showing the characteristics of the locus of the lane marker appears along the direction L1 of an angle θ1 with respect to 0 and the direction L2 of the angle θ2.

Next, the energy distribution calculation unit 16 calculates the power spectrum of the polar coordinate system based on the image data DDA.
F _A (r, θ) | ² is integrated for each predetermined angle Δθ within an angle range of 0 ° ≦ θ ≦ 180 ° to obtain energy distribution data FA (θ) as shown in FIG. 4 (e). , And the power spectrum of polar coordinate system based on the image data DDB |
Similarly, for F _B (r, θ) | ² , 0 ° ≦ θ ≦ 18
As shown in FIG. 4, integration is performed for each predetermined angle Δθ in the angle range of 0 °.
Energy distribution data FB (θ) as shown in (f)
Ask for.

When such an integral calculation is performed, a peak of a peculiar spectrum distribution corresponding to the above-mentioned lane marker locus feature appears, and the angle θ corresponding to the maximum peak appears.
1 and θ2 are obtained.

The feature extraction unit 18 uses the angles θ1 and θ2 obtained by the energy distribution calculation unit 16 as the feature data of the lane marker. The characteristic data θ1 accurately represents the angle formed by the direction of the lane marker included in the front area A and the traveling direction of the vehicle 2, and the characteristic data θ2 represents the direction of the lane marker included in the rear area B and the traveling direction of the vehicle 2. Accurately express the angle formed with the direction.

The determination unit 20 incorporates, as a first lookup table, correlation data of the approximate radius r of the lane marker with respect to the absolute value | θ2-θ1 | of the difference between the feature data θ2 and θ1 as shown in FIG. At the same time, the feature data θ1 or θ2 or their average value (θ1 + θ) is used with the approximate radius of the lane marker as a parameter, as shown in FIG.
The correlation data of the yaw angle with respect to 2) / 2 is provided in advance as a second lookup table.

The correlation data shown in FIG. 5 is prepared in advance by the following method and stored in the ROM or the like.
That is, as shown in FIG. 7, first, the direction of the light receiving axis of the electronic image pickup device 4 and the traveling direction of the vehicle 2 coincide with each other, and the (XY)
A Cartesian coordinate system is applied, and a plurality of circles R1, R2, R3 ... Of equal radius r through which a circular arc always passes through the center of the image receiving surface of the electronic image pickup device 4 are virtually set, and further, each circle R1. , R2, R3, ..., Arc portions ΔR1, Δ included in the imaging range W corresponding to the angle of view of the electronic imaging device 4.
The data of only R2, ΔR3 ... Are approximated to the trajectory of the curved lane marker.

Since the center O _R2 of the circle R2 is on the coordinate axis X, the yaw angle of the traveling direction Y of the vehicle 2 with respect to the arc portion ΔR2 is determined to be θ = 0 °. Further, the yaw angle formed by the traveling direction Y of the vehicle 2 with respect to the arc portion ΔR3 of the circle R3 having the center O _R3 on the front side of the vehicle 2 is θ _{+ y} , and conversely, the circle having the center O _R1 on the rear side of the vehicle 2. Arc part of R1 Δ
The yaw angle formed by the traveling direction Y of the vehicle 2 with respect to R1 is θ _−y . That is, the yaw angle defined here is the center O _R1 , of the circles R1, R2, R3 ... With reference to the coordinate axis X.
The angle formed by O _R2 , O _R3 ... And the line connecting the centers of the image receiving surfaces of the electronic image pickup devices 4.

Then, these arc portions ΔR1 and ΔR
For each of 2, ΔR3 ... The energy distribution is obtained by integrating for each angle Δθ, and the absolute value | θ2-θ1 | of the difference between the angles θ2 and θ1 when the energy distribution reaches a peak and the respective arc portions ΔR1, ΔR
Correlation data showing the correlation with the radius r of 2, ΔR3 ... Is created as a first look-up table, and further, the angle θ1 or θ2 or an average value (θ1 + θ2) / 2 of these is obtained.
The correlation data of the yaw angle with respect to is created as a second look-up table. Further, not only the correlation data for one type of radius r but also the correlation data for other types of radius r and the correlation data when a straight line extending in the traveling direction is used as the lane marker are the first and second look. Store in the up table. Further, in the approximation method shown in FIG. 7, a circle having a center in the coordinate area on the left side of the drawing is applied to the traveling direction Y of the traveling vehicle, but Correlation data for the case of applying a plurality of types of circles having a radius r and having a center in the coordinate area on the right side is also created at the same time. That is, the Cartesian coordinates (XY) in the figure
The tangential direction in the area W of the circle having the center on the left side (coordinates in the negative direction) with respect to the origin O of the
Since the tangential direction in the area W of the circle having the center on the right side (coordinates in the plus direction) with respect to the origin O is the right direction with respect to the vehicle traveling direction Y, the direction is left. The correlation data of both the curved line marker and the curved line marker to the right is obtained.

FIG. 5 shows an example of correlation data stored in the first lookup table, and FIG. 6 shows an example of correlation data stored in the second lookup table. Incidentally, FIG.
In the above, the plus or minus sign is attached to the curve radius, for example, regarding the correlation data of the line marker where the minus sign curve radius curves to the left, and the line where the plus sign curve radius curves to the right. It relates to marker correlation data.

Then, the judging section 20 makes the above first judgment based on the characteristic data θ1 and θ2 extracted by the characteristic extracting section 18.
By searching the correlation data of the lookup table (see FIG. 5) for the curve or the curvature of the road on which the vehicle is traveling, and further the correlation data of the second lookup table (see FIG. 6). ) To find the yaw angle of the vehicle.

According to this embodiment, as shown in FIG.
When the line marker drawn along the extending direction of the traveling path is a straight line ((a) and (b) in the figure), the magnitude and direction of the yaw angle with respect to the drawn direction are θ _{+ y} and θ _{−. When} the line marker is curved, obtained from _y ((c) in the same figure ~
In (f), the magnitude and direction of the yaw angle with respect to the tangential direction of the line marker can be obtained from θ _{+ y} and θ _−y . Further, the area W corresponding to the angle of view of the electronic image pickup device 4 is defined as the above-mentioned two areas A and B.
Since the absolute value | θ2-θ1 | of the difference between the feature data θ2 and θ1 for each line marker included in these regions is obtained, the curvature of the curved line marker and the feature in the tangential direction can be accurately obtained. To be

[0029]

As described above, according to the present invention, the two-dimensional power spectrum on the polar coordinates defined by the radius and the angle has a high energy distribution along the angle between the traveling vehicle and the lane marker. This characteristic is utilized. Further, the two-dimensional power spectrum is obtained for each of two or more imaging regions to find the angle with respect to each maximum energy, and the curvature of the lane marker or Since the radius is obtained, the curvature or radius can be obtained accurately. Since the yaw angle is obtained based on these curvatures or radii, the deviation of the traveling direction of the vehicle with respect to the traveling path can be obtained with high accuracy. Further, since the image pickup device is installed at a high place of the vehicle and the light receiving axis thereof is directed to the traveling road side with a depression angle, unnecessary images outside the traveling road can be blocked, so that the yaw angle can be more accurately measured. You can ask.

[Brief description of drawings]

FIG. 1 is an explanatory view showing a state in which a track recognition device of the present invention is installed in an automobile or the like.

FIG. 2 is a block diagram showing a configuration of a track recognition device of the embodiment.

FIG. 3 is an explanatory diagram showing the imaging principle of the embodiment.

FIG. 4 is an explanatory diagram showing a feature extraction principle of the embodiment.

FIG. 5 is an explanatory diagram illustrating a principle for determining a radius of a lane marker based on feature-extracted data.

FIG. 6 is an explanatory diagram illustrating a principle for determining a yaw angle of a vehicle based on feature-extracted data.

FIG. 7 is an explanatory diagram for explaining a principle of creating correlation data created in advance to determine a radius of a lane marker and a yaw angle of a vehicle.

FIG. 8 is an explanatory diagram for explaining the operation of the embodiment.

[Explanation of symbols]

2 ... Vehicle, 4 ... Electronic imaging device, 6 ... Main part of runway recognition device, 8 ... Display device, 10 ... Runway, 12 Image conversion and image division unit, 14 ... Two-dimensional power spectrum calculation unit,
16 ... Energy distribution calculation unit, 18 ... Feature extraction unit, 20
... Judgment part, 22 ... Imaging optical system.

Front page continued (72) Inventor Yukio Maruhashi 1126, Ichinomachi, Hamamatsu, Shizuoka Prefecture 1126 Hamamatsu Photonics Co., Ltd. In-house (72) Inventor Kazuo Kurasawa 8 at 3826 Handamachi, Hamamatsu City, Shizuoka Prefecture

Claims (3)

[Claims] 1. A lane marker displayed on a traveling road is imaged by an imaging device provided in a traveling vehicle, and the shape of the lane marker with respect to the traveling direction of the traveling vehicle is determined based on the imaged data of the imaged lane marker. In a road recognizing device for determining, the image data obtained on an image receiving surface existing on a two-dimensional coordinate orthogonal to an optical axis of the image capturing device is converted into a two-dimensional coordinate that is parallel and opposite to the road surface. Data conversion means for converting the image data into projected image data; data division means for dividing the converted image data into respective image data in two or more image pickup areas in the front and rear with respect to the traveling direction of the traveling vehicle; Calculating means for obtaining a two-dimensional power spectrum on polar coordinates defined by a radius and an angle based on the respective image data of The energy distribution for each angle is obtained based on the tram, the angles for the respective maximum energies are obtained, and the radius of the lane marker is determined from the first correlation data that is predetermined for the angle difference between the two predetermined angles. Or a determining means for obtaining the curvature,
A track recognition device comprising: 2. A yaw angle of the traveling vehicle with respect to the lane marker is determined based on imaged data of the captured lane marker, by capturing an image of the lane marker displayed on the traveling path with an imaging device provided in the traveling vehicle. In the traveling road recognition device, the imaged data obtained on the image receiving surface on the two-dimensional coordinates orthogonal to the optical axis of the imaging device is projected onto the two-dimensional coordinates that are parallel and opposed to the traveling road surface. Data conversion means for converting the converted image data into image data after the conversion, and a data dividing means for dividing the converted image data into respective image data in two or more image pickup areas in the front and rear with respect to the traveling direction of the traveling vehicle. Calculating means for obtaining a two-dimensional power spectrum on polar coordinates defined by a radius and an angle based on each image data; and each of the two-dimensional power spectra Based on the above, the energy distribution for each angle is obtained, and the angles for the respective maximum energies are obtained, and the radius of the lane marker or the radius of the lane marker is determined from the first correlation data that is predetermined for the angle difference between the two predetermined angles. A lane recognition device, comprising: a determining unit that determines a curvature and determines the yaw angle from predetermined second correlation data with respect to the radius or the curvature and the angle with respect to the maximum energy. 3. The roadway recognition according to claim 1 or 2, wherein the image pickup device is installed at a high place of a traveling vehicle, and a light-receiving axis is directed toward a traveling roadside with a depression angle. apparatus.