JP2005157731A - Lane recognition device and lane recognition method - Google Patents

Abstract

An object of the present invention is to prevent erroneous recognition of a white line due to road repair marks on a road.
A road surface luminance calculation window is set in an image captured by a camera that images the front of a host vehicle, and an average value or mode value of luminance in the image in the set road surface luminance calculation window is set. Calculated as road surface brightness. Then, an edge having a luminance larger than the calculated road surface luminance is extracted from the image captured by the camera 14, and a white line is recognized based on the extracted edge.
[Selection] Figure 1

Description

  The present invention relates to a lane recognition device and a lane recognition method that recognize a lane by detecting a white line on a road ahead of the host vehicle.

2. Description of the Related Art Conventionally, a technique for detecting a white line on a road on which the host vehicle travels from an image captured by an electronic camera provided on the host vehicle and recognizing the lane (traveling path) of the host vehicle is known.
For example, in the conventional apparatus disclosed in Patent Document 1 below, a window is set in an image captured by a camera, an edge is detected based on a luminance change in the window, and a white line is detected based on the edge.

JP-A-7-78234

In the above prior art, an edge is detected based on a luminance change in a window set in an image captured by a camera, and a white line is detected based on the edge. For this reason, in addition to the white line to be detected, road repair marks due to construction may show the same luminance change. In such a case, there is a possibility that this road repair trace may be mistakenly recognized as a white line.
An object of this invention is to provide the lane recognition apparatus and lane recognition method which can prevent the misrecognition of the white line by the road repair trace on a road.

  In order to solve the above problems, the present invention relates to an image acquisition unit that captures the front of the host vehicle, a window setting unit that sets a window for calculating road surface luminance in the captured image, and an image in the set window. Based on the extracted edge, a road surface brightness calculating means for calculating an average value or a mode value of the brightness as the road surface brightness, an edge extracting means for extracting an edge having a brightness larger than the calculated road surface brightness from the captured image, and White line recognition means for recognizing a white line.

  According to the present invention, it is possible to prevent erroneous recognition of white lines due to road repair marks on the road.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings described below, components having the same function are denoted by the same reference numerals, and repeated description thereof is omitted.
FIG. 2 is a diagram showing a vehicle equipped with a lane recognition device according to an embodiment of the present invention, and FIG. 1 is a block diagram showing a basic configuration of the lane recognition device according to the present embodiment.
In FIG. 2, 12 is the own vehicle, 10 is the lane recognition device (road white line recognition device) of the present embodiment, 14 is an electronic camera, 16 is a processor constituting the main part of the lane recognition device 10, and 12a is the own vehicle. An interior ceiling of the vehicle 12, 12b is a windshield of the host vehicle 12, 17 is a radar, and R is a road.
As shown in FIG. 2, a camera 14 that acquires a road image in front of the host vehicle 12, a processor 16 that processes an image signal acquired by the camera 14 (hereinafter simply referred to as an image), and a front of the host vehicle 12. A radar 17 that detects a three-dimensional object is installed in the host vehicle 12. The camera 14 is preferably a charge coupled device (CCD) camera or a complementary metal-oxide semiconductor (CMOS) camera capable of high-sensitivity imaging. The camera 14 is installed in the front center portion of the indoor ceiling 12a of the host vehicle 12 so as to face the front lower side, and acquires an image of the road R in front of the host vehicle 12 through the windshield 12b. The processor 16 is installed at an appropriate location of the host vehicle 12 that is not affected by heat or wind and rain, recognizes the white line of the road R from the image of the road R acquired by the camera 14, and forms the road R and the host vehicle with respect to the road R. Control for estimating 12 postures is executed. The radar 17 is installed in front of the host vehicle 12 and is installed as an inter-vehicle distance detection means for measuring the inter-vehicle distance from the preceding vehicle. The radar 17 sweeps laser light forward and receives reflected light from a three-dimensional object in front of the host vehicle 12 to detect the three-dimensional object. In addition, what sweeps electromagnetic waves instead of a laser beam is applicable.

In FIG. 1, 15 is an image memory for storing an image of the road R ahead of the host vehicle 12 acquired by the camera 14 mounted on the host vehicle 12, 101 is a road surface luminance measuring unit, 102 is a window setting unit, and 103 is a white line candidate. Point extraction means, 104 is a white line candidate line calculation means, and 105 is road parameter estimation means.
As shown in FIG. 1, the lane recognition device 10 estimates road parameters based on the image of the road R stored in the image memory 15 and the arrangement of the three-dimensional object ahead of the host vehicle 12 acquired by the radar 17. For example, a processor 16 using a microprocessor is provided.
The processor 16 includes road surface brightness measuring means 101, window setting means 102, white line candidate point extracting means 103, white line candidate line calculating means 104, and road parameter estimating means 105. And these means 101-105 are implement | achieved by program control. Needless to say, these means 101 to 105 indicate control functions executed by the processor 16 according to the following flowcharts (FIGS. 3 and 4).
The road surface brightness measuring unit 101 obtains inter-vehicle distance information with the preceding vehicle from the radar 17 and measures the road surface brightness between the preceding vehicle and the host vehicle 12. The window setting means 102 provides a plurality of windows on the road R image stored in the image memory 15 along the extending direction of the white line so that each window includes the white line of the road R. The white line candidate point extraction means 103 processes the image for each window and extracts white line candidate points. The white line candidate line calculation means 104 calculates the image coordinates and inclination of the white line candidate line with the highest possibility of a white line, based on the white line candidate points extracted for each window. The road parameter estimation means 105 estimates a road parameter including at least a lateral displacement of the host vehicle 12 with respect to the lane using the image coordinates and inclination of the white line candidate line calculated for each of these windows and the road model formula. .
As the road model formula, the following formula 1 is used to estimate the shape of the road R and the posture of the host vehicle 12 with respect to the road R as described above.

式１において、道路パラメータＡ，Ｂ，Ｃ，Ｄ，Ｈは、それぞれ、自車両１２の車線に対する横変位（Ａ）、道路曲率（Ｂ）、自車両１２の車線に対するヨー角（Ｃ）、自車両１２のピッチ角（Ｄ）、および道路Ｒの路面からのカメラ１４の高さ（Ｈ）である。
また、Ｅ_０は車線幅（左右の白線の内側間の距離）を示す定数、ｆはカメラ１４のカメラ透視変換定数、ｊは左右の白線を区別するパラメータとし、左白線のとき、ｊ＝０、右白線のときｊ＝１とする。また、（ｘ，ｙ）は、左白線または右白線内端上の任意の点の画像上の座標であり、画像左上を原点に取り、右方向がｘ軸正方向、下方向がｙ軸正方向とする。
なお、式１は、道路モデルを表す一手法を示したものであり、例えば、Ｅ_０を変数にしたもの、Ｈを固定したもの、車線幅を前述のように左右の白線の内側間の距離ではなく、左白線中央と右白線中央との間の距離とし、（ｘ，ｙ）を白線幅の中央位置の画像座標としても定義することができる。

In Equation 1, road parameters A, B, C, D, and H are respectively the lateral displacement (A) with respect to the lane of the own vehicle 12, the road curvature (B), the yaw angle (C) with respect to the lane of the own vehicle 12, The pitch angle (D) of the vehicle 12 and the height (H) of the camera 14 from the road surface of the road R.
E ₀ is a constant indicating the lane width (the distance between the insides of the left and right white lines), f is a camera perspective conversion constant of the camera 14, and j is a parameter for distinguishing the left and right white lines. When the right white line, j = 1. Also, (x, y) is the coordinates on the image of an arbitrary point on the inner edge of the left white line or right white line, taking the upper left of the image as the origin, the right direction is the x-axis positive direction, and the lower direction is the y-axis positive The direction.
Note that Equation 1, which shows one technique for representing the road model, for example, those of E ₀ to the variable, which is fixed to H, the distance between the inner white line of the right and left lane width as described above Instead, it can be defined as the distance between the center of the left white line and the center of the right white line, and (x, y) can also be defined as the image coordinates of the center position of the white line width.

FIG. 5A is a diagram showing a setting positional relationship of the road surface luminance calculation window according to the present invention with respect to the host vehicle and the preceding vehicle, and FIGS. 5B and 6 are road surface luminance calculation windows set on the screen. It is a figure which shows arrangement | positioning. 5A, 18 is a white line on the road, 19 is a road surface, 23 is a preceding vehicle, 24 is a setting position of a road surface luminance calculation window, 25 in FIG. 5B, 25 is a road repair trace, and 26 is a road surface. In the brightness calculation window, in FIG. 6, reference numeral 27 denotes characters drawn on the road surface 19.
FIG. 8 is a diagram showing an arrangement state of a plurality of windows set on the screen, and 20 is a window for white line recognition.
FIG. 10 is a diagram showing the state of the white line 18 and the road repair trace in the window 20, and 25 is the road repair trace.
FIG. 12 is a diagram showing a plot state of white line candidate points in the window 20, where 20a is the upper side of the window 20, 20b is the lower side of the window 20, and 21 is a white line candidate point.
FIG. 14 is a diagram showing a selection state of white line candidate lines in the window 20, and 22 is a white line candidate line.

3 and 4 are flowcharts of a program for estimating the road parameters by the processor 16. For example, it is assumed that the process is repeated every 30 [msec] while the host vehicle 12 is traveling.
Next, an outline process for each step in FIG. 3 will be described.
First, in step S1, a front image of the host vehicle 12 is taken into the image memory 15 from the camera 14 of FIG.
Next, in step S2, the radar 17 acquires the relative inter-vehicle distance from the preceding vehicle.
Next, in step S3, a road surface brightness calculation window 26 (see FIGS. 5 and 6) is set in the image captured in step S1, based on the relative inter-vehicle distance from the preceding vehicle.
Next, in step S4, the road surface luminance is calculated based on the road surface luminance calculation window 26.

Next, in step S5, a plurality (m) of white line recognition windows 20 (see FIG. 8) are set on the image captured in step S1.
In step S6, the window processing number i is set to an initial value of 1.
Next, in step S7, a white line candidate point 21 (for detecting the luminance change in the horizontal direction and recognizing the position of the end of the white line 18 from the image data of the i-th window 20 (hereinafter referred to as window i). (See FIG. 12).
Next, in step S8, based on the coordinate position of the white line candidate point 21 in the window 20, the white line candidate line 22 (see FIG. 14) having the highest possibility of the white line 18 from these white line candidate points 21 is calculated.
Next, in step S9, the window process number i is incremented by one.
Next, in step S10, it is determined whether i exceeds the number of windows (m).
If it is determined in step S10 that i does not exceed the number of windows (m), the process returns to step S7 to process the next window 20.
If it is determined in step S10 that i exceeds the number of windows (m), the pre-processing for all the windows has been completed. In step S20, road parameter estimation processing is performed.

Hereinafter, the processing of the program will be described in more detail for each step.
(1) Capture forward image (S1)
An image in front of the host vehicle 12 captured by the camera 14 is taken into the image memory 15 every predetermined time for which the processor 16 processes the program, for example, every 30 [msec]. The image is represented as luminance data for each pixel.
(2) Acquisition of relative inter-vehicle distance from the preceding vehicle 23 (S2)
The radar 17 obtains the relative inter-vehicle distance with the preceding vehicle 23 that is output every fixed processing cycle (for example, 30 [msec]).
(3) Setting of road brightness calculation window 26 (S3)
As shown in FIG. 5B, a road luminance calculation window 26 is set in the image taken into the image memory 15. This road surface brightness calculation window 26 acquires the position of the preceding vehicle 23 by the radar 17 in (2) above, and as shown in FIG. It is set so as to be inside the estimated position of the white line 18. The optimum setting position and size of the road surface luminance calculation window 26 are obtained by experimental evaluation.
(4) Calculation of road surface brightness (S4)
The luminance of the road surface 19 is calculated from the road luminance calculation window 26 set in (3) above. As shown in FIG. 6, a case is considered in which disturbance elements for road surface brightness other than the road surface 19 such as road repair marks 25 and characters 27 (such as speed limit and arrow) on the road surface 19 are within a set window 26.
FIG. 7 is a diagram showing an outline of the luminance histogram in the road surface luminance calculation window 26 shown in FIG. 6, with the luminance value on the horizontal axis and the frequency of occurrence of the luminance value on the vertical axis. 71 is a peak due to the road repair trace 25, 72 is a peak due to the road surface 19, 73 is a peak due to the character 27, 74 is an average value of luminance in the image in the road surface luminance calculation window 26, and 74 is in the road luminance calculation window 26. It is a mode value that is a luminance value that appears most frequently in the image.
In this case, the histogram with the luminance value on the horizontal axis and the frequency of occurrence of the luminance value on the vertical axis is as shown in FIG. Here, the brightness of the road surface 19 is represented by an average value 74 or a mode value 75. Thus, the road surface brightness is derived by taking a histogram in the road surface brightness calculation window 26 and calculating the average value 74 or the mode value 75. An average value excluding the maximum value and the minimum value of luminance may be used.

(5) Setting of a plurality of white line recognition windows 20 (S5)
As shown in FIG. 8, a plurality of white line recognition windows 20 are provided along the extending direction of the white line 18 on the road R (in the example of FIG. 5, five for the left white line and five for the right white line). It is done. Based on the parameters of the road model obtained from the previous image processing result, the position of each window 20 is set so that the white line 18 is captured in each window 20, and the white line 18 is detected in these windows 20. In FIG. 5, the vertical direction indicates the image coordinate y-axis direction, and the lower side in the figure is y plus. The horizontal direction indicates the image coordinate x-axis direction, and the right side in the figure is x plus.
Here, the y coordinate of the upper side 20a (FIG. 6) of the nth window (i = n) is yn, and the previous parameter estimation results are {A (-1), B (-1), C (-1 ), D (−1), H (−1)}, the central coordinate x _{est in} the horizontal direction of the current window 20 is expressed by Equation 2 from Equation 1.

ウインドウ２０のｘ方向の幅は固定値としてもよく、また、特開平８−２６１７５６号公報に示されるようにパラメータの分散から合理的に設定することもできる。

The width of the window 20 in the x direction may be a fixed value, or may be set rationally from the dispersion of parameters as disclosed in JP-A-8-261756.

(6) Initialization of window processing number i (S6)
A loop control parameter (i.e., window processing number) i for performing the processing from step S4 to step S7 on all windows 20 is initialized to 1.
(7) Extraction of white line candidate points 21 (S7)
Using the edge detection (brightness change detection) process such as the sobel filter process and the brightness evaluation from the image of the range of each white line recognition window 20 set as described above to the image captured from the camera 14, the white line Candidate point 21 (FIG. 12) is selected.
FIG. 9 is a diagram for explaining the edge detection processing, in which 28 is a pixel (all squares in the figure are pixels), and 29 is a point of interest.
In this edge detection, as shown in FIG. 9, the luminances of the left and right pixels 28 of the point of interest 29 are compared, and the luminance of the pixel 28 on the right (the x value of the image coordinate is large) is left (the x value of the image coordinate is It is assumed that the output of the filter is positive when it is greater than the luminance of the small pixel 28. In the luminance evaluation, the luminance is determined using the road surface luminance obtained by the above (4) calculation of the road surface luminance.
11 (a) and 11 (b) are diagrams for explaining the conditions to be satisfied by the white line 18. In FIG. 11 (a), the luminance value is taken on the vertical axis, and in FIG. 11 (b), the edge intensity value is taken on the vertical axis. ) And (b) both have x values on the horizontal axis.
Now, the left white line 18 as shown in FIG. 10 will be described as an example. The white line conditions are as shown in FIGS.
a) Scanning from the left end, there is an edge point on the right side where the output of the filter exceeds the negative edge discrimination value. Here, the edge discrimination value may be determined uniquely from the characteristics of the camera 14 and the experimental evaluation, or may be set as appropriate depending on the entire image or the average luminance, contrast, etc. for each window.
b) On the left side of the edge point, a pixel having a luminance higher than the road surface luminance (considered as the white line 18) is greater than or equal to a predetermined width. Here, the brightness higher than the road surface brightness is determined using a threshold value obtained by adding the road surface brightness to a value uniquely determined by the characteristics of the camera 14 and experimental evaluation. Here, the predetermined width refers to a range of length that can be seen when the camera 14 captures the minimum to maximum white line width assumed to be on an actual road.
c) At the terminal end (left end) of the pixel having the road surface luminance or higher, there is an edge point at which the output of the filter exceeds the positive edge judgment value.
The white line candidate points 21 are evaluated in order and satisfy all the conditions. This operation is performed for all the image data in the white line recognition window 20. Thereby, white line candidate points 21 as shown in FIG. 12 are obtained. Thus, although it is difficult to remove the road repair trace 25 only by the conditions a) and c), the road repair trace 25 can be easily removed by adding the condition b). In the case of the right white line, it is simply applied in the reverse of the above description.
FIG. 13 is a diagram showing a change in edge strength on a road having a white line 18 and a road road repair trace 25, in which 31 is a plus edge and 32 is a minus edge.
When a black and white image is captured by the camera 14 and the white line 18 is recognized by edge detection, the white line on the left side from the host vehicle 12 is scanned from the left end, for example, as shown in FIG. This is recognized by the presence of the negative edge 32 region. The white line on the right side is the same. For example, when two near-parallel road repair traces 25 are scanned from the left side, an area A ′ surrounded by a plus edge 31 and a minus edge 32 similar to the area A of the white line 18 exists, and therefore, the edge From the image alone, it is difficult to distinguish it from a true white line, and there is a possibility of misrecognition. In the present embodiment, the average value or the mode value of the road surface luminance is obtained, and the erroneous recognition is prevented by determining the white line 18 that is larger than the luminance and has an edge. It is also possible to scan from the center.
In this edge detection, the white line candidate point 21 is extracted from the change in luminance of the pixels adjacent to the left and right, but the white line candidate point 21 is determined from the change in luminance of the pixels adjacent in the vertical direction in the captured image. May be extracted. However, in this case, since the white line candidate point 21 is extracted from the stop line existing at the intersection in addition to the white line 18 present on the side of the road, the left and right adjacent pixels described in the former are extracted. It is preferable to extract the white line candidate point 21 from the luminance change.

(8) Calculation of white line candidate line 22 (S8)
A white line candidate line 22 (FIG. 14) is searched from a set of white line candidate points 21 extracted from the white line recognition window 20. For this search, a Hough transform, a least square method, or the like can be used. In the present embodiment, the Hough transform is used. In the Hough transform straight line approximation, as shown in FIG. 14, among the straight lines passing through the window 20, the white line candidate line 22 that penetrates the white line candidate point 21 most frequently, that is, a white line candidate. The white line candidate line 22 on which the most points 21 are placed is the white line candidate line 22 with the highest possibility of the white line 18.
At this time, a straight line passing through the coordinates (x, y) can be expressed as the following Expression 3 using the parameters a and b. Here, a is the slope (∂x / ∂y) of the white line candidate line 22, and b is its x intercept.

図１５は、白線候補線２２を探索する際のハフ変換による配列表を示す図、図１６は、ハフ変換を配列要素で示す図である。
すなわち、白線候補点２１の座標を（ｘ，ｙ）とすると、ａを決めるとｂが式３より計算でき、これにより図１５に示すハフ変換による配列が得られる。この配列の横方向は、一定刻みのａの値であり、縦方向は一定刻みのｂの値である。ここで、配列の空欄は零を意味する。また、「１」が立っている（ａ，ｂ）の組み合わせの中に真値が含まれる。この配列を白線候補点２１の全てに対してそれぞれ作成し、作成した配列を重ね合わせて、各配列要素毎の数を合計することにより図１６に示す配列が得られる。図１６において、配列要素（ａ_ｒ，ｂ_ｒ）の数ｚｒは、式４が貫く白線候補点２１の数を表している。

FIG. 15 is a diagram showing an array table by Hough transform when searching for the white line candidate line 22, and FIG. 16 is a diagram showing Hough transform by array elements.
That is, assuming that the coordinates of the white line candidate point 21 are (x, y), b can be calculated from Equation 3 when a is determined, thereby obtaining an array by the Hough transform shown in FIG. The horizontal direction of this array is the value of a in constant increments, and the vertical direction is the value of b in constant increments. Here, the blank in the array means zero. In addition, a true value is included in the combination of (a, b) where “1” stands. This array is created for each of the white line candidate points 21, and the created arrays are overlaid and the numbers for each array element are summed to obtain the array shown in FIG. In FIG. 16, the number zr of array elements (a _r , b _r ) represents the number of white line candidate points 21 that Formula 4 penetrates.

したがって、白線候補点２１を最も多く貫く白線候補線２２が最も白線１８の可能性が高い白線候補線２２であり、そのａ，ｂを求めることが白線候補線２２を算出することになる。すなわち、配列要素の値として最大のｚを持つ配列要素（ａ，ｂ）が白線候補線２２を示すことになる。
図１７は、ウインドウ２０の上辺２０ａと白線候補線２２との交点との道路座標を示す図である。
こうして、白線候補線２２が算出されると、次いで、図１７に示すように、ウインドウ処理番号ｎにおける白線候補線２２とウインドウ２０の上辺２０ａとの交点の画像上の座標である道路座標（ｘｎ，ｙｎ）を算出する。ウインドウ２０の上辺２０ａのｙ座標ｙｎは、ウインドウ処理番号ｎに対する固定値である。道路座標のｘ成分ｘｎは、式５で算出される。

Therefore, the white line candidate line 22 that penetrates most of the white line candidate points 21 is the white line candidate line 22 that has the highest possibility of the white line 18, and obtaining the a and b will calculate the white line candidate line 22. That is, the array element (a, b) having the maximum z as the value of the array element indicates the white line candidate line 22.
FIG. 17 is a diagram illustrating road coordinates between the upper side 20 a of the window 20 and the intersection of the white line candidate line 22.
When the white line candidate line 22 is calculated in this way, next, as shown in FIG. 17, road coordinates (xn) which are coordinates on the image of the intersection of the white line candidate line 22 and the upper side 20a of the window 20 in the window processing number n. , Yn). The y coordinate yn of the upper side 20a of the window 20 is a fixed value for the window processing number n. The x component xn of the road coordinates is calculated by Equation 5.

（９）道路パラメータの推定（Ｓ２０）
前記の式１をｙで微分して式６を得る。

(9) Road parameter estimation (S20)
Equation 6 is differentiated by y to obtain Equation 6.

そして、式１と式６からＡを消去すると、式７を得る。

Then, when A is eliminated from Equations 1 and 6, Equation 7 is obtained.

このとき、Ａは自車両１２の車線に対する横位置のパラメータであり、このＡを消去すると同時に、車線幅を表すＥ_０も消去されることになる。つまり、道路幅がいかなるものであれ、式７が成立することになる。
次に、カルマンフィルタを用いて道路パラメータを推定する場合、式８に示す方程式が与えられることにより、パラメータｚを推定することができる。ただし、同式中のパラメータｙｙの値およびｆの構造はそれぞれ既知とし、一般にｙｙはベクトルとなる。

At this time, A is a parameter of the lateral position with respect to the lane of the host vehicle 12, and at the same time as A is deleted, E ₀ indicating the lane width is also deleted. That is, Expression 7 is established regardless of the road width.
Next, when the road parameter is estimated using the Kalman filter, the equation z shown in Expression 8 is given, whereby the parameter z can be estimated. However, the value of the parameter yy and the structure of f in the equation are known, and generally yy is a vector.

ここで、この式８に相当する観測方程式（仮称）は、前記式１が式９として成立する。

Here, the observation equation (tentative name) corresponding to Equation 8 is established as Equation 9 above.

この式９は、ｘ＝ｆｚ（Ａ，Ｂ，Ｃ，Ｄ，Ｈ）…（式９’）と表すこともできる。
式９’は一般に複数列より構成され、これらを線型近似した上で並べることにより次の行列式１０が得られる。

Expression 9 can also be expressed as x = fz (A, B, C, D, H) (Expression 9 ′).
Expression 9 ′ is generally composed of a plurality of columns, and the following determinant 10 is obtained by arranging these after linear approximation.

この行列式１０中、ｍは、１画面中に設定されたウインドウ２０の数である。この場合、ｚｎは、ｚｎ＝ｘｎとなる。

In this determinant 10, m is the number of windows 20 set in one screen. In this case, zn is zn = xn.

Next, with reference to FIG. 4, the subroutine P3 of the road parameter estimation process will be described.
From the road parameter estimation (S20) also shown in FIG. 3, first, the window processing number parameter i is initialized to 1 in step S21.
Next, in step S22, zi in Expression 10 is set to xi, and the right side of Expression 9 ′ is linearly approximated, so that Mi1, Mi2, Mi3, Mi4, and Mi5 are dfz / dA, dfz / dB, dfz / dC, dfz / dD, and dfz / dH.
Next, in step S23, the window process number i is incremented by one.
Next, in step S24, it is determined whether or not the window process number i exceeds the window number m.

If it is determined in step S24 that i does not exceed the number m of windows, the process returns to step S22 to process the next window 20.
If it is determined in step S24 that i exceeds the number of windows m, parameters (A, B, C, D, H) are estimated by Kalman filter processing in step S25, and the process returns to the main routine.

  As described above, the lane recognition device 10 and the lane recognition method according to the present embodiment sets the road luminance calculation window 26 in the image captured by the camera 14 that images the front of the host vehicle 12, and sets the set road luminance. The average value or mode value of the luminance in the image in the calculation window 26 is calculated as the road surface luminance. Then, an edge having luminance greater than the calculated road surface luminance is extracted from the image captured by the camera 14, and the white line 18 is recognized based on the extracted edge. The average value or mode value of the brightness in the set road surface brightness calculation window 26 indicates the road surface brightness before road repair (there is no road repair trace 25). The brightness of the road repair trace 25 is generally It becomes smaller (darker, blacker) than the road surface brightness before road repair. Therefore, the average value or mode value of the road surface luminance is obtained, and the white line 18 that is larger than this luminance and has an edge can be determined as the white line 18, thereby preventing erroneous recognition by the road repair trace 25 on the road. For example, when the present invention is applied to a lane departure prevention device or a lane departure warning device, if the road repair trace 25 is mistakenly recognized as the white line 18, there is a possibility that erroneous steering is performed to prevent the lane departure. Yes, there is a possibility of issuing an alarm at an incorrect timing because the lane width is recognized incorrectly. In the present embodiment, when the luminance between the white line and the edge similar to the road repair mark 25 is close to the calculated road surface luminance, it is determined as erroneous recognition, and the erroneous recognition as described above is performed. In order to avoid lane departure, it is possible to reduce the possibility of performing an incorrect steering or issuing an alarm at an incorrect timing.

In the present embodiment, the road surface brightness is calculated by using the road surface brightness calculation window 26 every predetermined cycle, for example, every 30 [msec]. Therefore, it is a premise that the calculated road surface brightness is updated to the latest value as the host vehicle 12 travels. For example, when the road surface brightness calculation window 26 is set when there is a stop line ahead, such as when stopping at the head of an intersection, or when there are many curve warning lines in front of the curve, The road surface brightness is calculated. Therefore, if the white line 18 is recognized based on the road surface luminance, it is considered that accurate recognition cannot be performed. In such a case, the road surface brightness is not calculated to prevent the abnormal value from being updated. Specifically, for example, the following method is used.
(1) If the average value or the maximum frequency of the calculated road surface brightness is out of the range of the past road surface average value ± predetermined value, the calculated road surface brightness is not used, The road surface brightness value is used.
(2) When it is detected that the vehicle is an intersection or a curve from navigation map information by a GPS (Global Positioning System) system, the reference road surface brightness is not calculated. Alternatively, the interval for calculating road surface brightness is lengthened.
(3) When a stop line or a curve warning line is detected from the image captured by the camera 14 by pattern matching, the reference road surface brightness is not calculated.
As described above, by obtaining the road surface luminance after eliminating the influence of the white line such as the stop line or the curve warning line, erroneous recognition by the road repair trace 25 can be more reliably prevented.
Further, as described above, the position of the preceding vehicle 23 is acquired by the radar 17, and as shown in FIG. 5A, it is in the vicinity of the preceding vehicle 23 and inside the estimated position of the left and right white lines 18 calculated last time. When the white line 18 is detected in the window 26, the road surface brightness calculation window 26 set so as to become is set so as not to include the previously detected white line 18. Thus, by obtaining the road surface brightness after eliminating the influence of the white line 18, it is possible to prevent erroneous recognition by the road repair trace 25 more reliably.
In addition, as shown in FIG. 11 b, in order to extract an edge having a luminance higher than the road surface luminance, an edge having a luminance of a predetermined width or more is extracted. In addition, as shown in c) and a) of FIG. 11, when the luminance changes positively and then changes negatively at the extracted edge, the white line 18 is recognized. By these, the white line 18 can be recognized reliably and the misrecognition by the road repair trace 25 can be prevented more reliably.
In addition, a plurality of white line recognition windows 20 are set in the image captured by the camera 14, and the white line 18 is recognized based on the luminance change of adjacent pixels in the window 20. Further, a plurality of white line candidate points 21 are extracted based on the luminance change of adjacent pixels in the image captured by the camera 14, and the white line 18 is recognized by connecting these white line candidate points 21 with a straight line. With these configurations, the white line 18 can be detected easily and reliably.

The camera 14 in FIGS. 1 and 2 and step S1 in FIG. 3 in the present embodiment correspond to the image acquisition means in the claims. The processor 16 in FIG. 1 and steps S2 and S3 in FIG. 3 in the present embodiment correspond to window setting means. 1 in this embodiment, the road surface brightness measuring means 101 in FIG. 1 and step S4 in FIG. 3 correspond to the road surface brightness calculating means. The processor 16 in FIG. 1, the window setting unit 102 in FIG. 1, the white line candidate point extracting unit 103, and steps S5 to S7 in FIG. 3 correspond to the edge extracting unit. The processor 16 in FIG. 1, the white line candidate line extraction unit 104 in FIG. 1, and step S8 in FIG. 3 in the present embodiment correspond to the white line recognition unit.
Further, the embodiments described above are described for facilitating the understanding of the present invention, and are not described for limiting the present invention. Therefore, each element disclosed in the above embodiment includes all design changes and equivalents belonging to the technical scope of the present invention.

It is a block diagram which shows the basic composition of the lane recognition apparatus of embodiment of this invention. It is a figure which shows the vehicle carrying the lane recognition apparatus of embodiment of this invention. It is a flowchart for performing the image processing program by embodiment of this invention. It is a flowchart which shows the estimation subroutine of the road parameter in the image processing program by embodiment of this invention. (A) is a figure which shows the setting positional relationship of the road surface luminance calculation window with respect to the own vehicle and a preceding vehicle in embodiment of this invention, (b) is a figure which shows arrangement | positioning of the road surface luminance calculation window set to a screen. is there. It is a figure which shows arrangement | positioning of the window surface brightness calculation window set to a screen in embodiment of this invention. FIG. 7 is a schematic diagram of a luminance histogram inside the road surface luminance calculation window of FIG. 6. It is a figure which shows the arrangement | sequence state of the several white line recognition window set to the screen in embodiment of this invention. It is explanatory drawing of the edge detection process in embodiment of this invention. It is a figure which shows the state of the white line in the road surface luminance calculation window set in embodiment of this invention, and a road repair trace. It is a figure explaining the conditions which the white line should satisfy in embodiment of this invention. It is a figure which shows the plot state of the white line candidate point in the window set in embodiment of this invention. It is a figure which shows the change of the edge strength in the road which has a white line and road road repair trace in embodiment of this invention. It is a figure which shows the selection state of the white line candidate line in the window set in embodiment of this invention. It is a figure which shows the arrangement | sequence table by the Hough transform at the time of searching for the white line candidate line performed in embodiment of this invention. It is a figure which shows the Hough transformation performed in embodiment of this invention by an array element. It is a figure which shows the road coordinate of the intersection of the upper side of the window set in embodiment of this invention, and a white line candidate line.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Lane recognition apparatus 12 ... Own vehicle 12a ... Indoor ceiling 12b ... Windshield 14 ... Camera 16 ... Processor 17 ... Radar R ... Road 18 ... White line 19 ... Road surface 20 ... Window 20a ... Upper side 20b ... Lower side 21 ... White line candidate point 22 ... white line candidate line 23 ... preceding vehicle 24 ... setting position of road brightness calculation window 25 ... road repair trace 26 ... road brightness calculation window 27 ... character 28 ... pixel 29 ... attention point 31 ... plus edge 32 ... minus edge 101 ... Road surface brightness measuring means 102 ... Window setting means 103 ... White line candidate point extracting means 104 ... White line candidate line calculating means 105 ... Road parameter estimating means

Claims (8)

Image acquisition means for imaging the front of the host vehicle;
A window setting means for setting a window in the image captured by the image acquisition means;
Road surface brightness calculating means for calculating an average value or mode value of brightness in the image in the window set by the window setting means as road surface brightness;
Edge extraction means for extracting an edge having a brightness larger than the road surface brightness calculated by the road surface brightness calculation means from the image captured by the image acquisition means;
A lane recognition device comprising: white line recognition means for recognizing a white line based on the edge extracted by the edge extraction means.   The lane recognition device according to claim 1, wherein the window setting means sets the window so as not to include a white line.   3. The lane recognition device according to claim 1, wherein when the window includes a white line, the road surface brightness calculation unit does not calculate the road surface brightness. 4.   4. The edge extraction unit according to claim 1, wherein, in order to extract the edge having a luminance higher than the road surface luminance, an edge having a luminance of the edge equal to or larger than a predetermined width is extracted. Lane recognition device.   5. The white line recognizing unit recognizes the white line when the luminance extracted from the edge extracted by the edge extracting unit changes positively and then changes negatively. The lane recognition device described. The window setting means sets a plurality of white line recognition windows in the image captured by the image acquisition means,
6. The lane recognition device according to claim 1, wherein the white line recognition unit recognizes the white line based on a luminance change of an adjacent pixel in the window.   The white line recognizing unit extracts a plurality of white line candidate points based on a luminance change of adjacent pixels in the image captured by the image acquisition unit, and recognizes the white line by connecting the white line candidate points with a straight line. A lane recognition device according to any one of claims 1 to 6, wherein: An image acquisition step of imaging the front of the host vehicle;
A window setting step for setting a window in the image captured by the image acquisition step;
A road surface brightness calculating step of calculating an average value or mode value of brightness in the image in the window set by the window setting step as road surface brightness;
An edge extraction step of extracting an edge having a luminance greater than the road surface luminance calculated by the road surface luminance calculation step from the image captured by the image acquisition step;
And a white line recognition step of recognizing a white line based on the edge extracted by the edge extraction step.

Images