JP4956099B2 - Wall detector - Google Patents

Description

  The present invention relates to a wall detection device, and more particularly to a wall detection device capable of detecting a roadside wall from a captured image obtained by imaging a traveling path of a host vehicle including a road.

  In recent years, in order to improve driving safety of automobiles, automatic control of vehicles, etc., image processing is performed on images taken with in-vehicle stereo cameras and video cameras, etc., and overtaking prohibited lines, roadside belts and roadways are Various devices have been proposed for detecting the position and speed of obstacles such as lanes, preceding vehicles, pedestrians, guardrails, etc., which are marked on a road surface such as a lane marking that divides the road (for example, Patent Documents 1 to 4). Etc.).

These technologies, for example, automatically control the driving of the vehicle so as to follow the lane and deviate from the lane, and do not collide with detected pedestrians or obstacles. Therefore, it is necessary to be able to detect lanes, obstacles, and the like with high accuracy, because this leads to a technique for automatically controlling the host vehicle so as to avoid and follow the preceding vehicle.
JP-A-10-283461 JP 2001-92970 A JP 2003-204546 A JP 2001-48036 A

  In these devices, for example, in the lane departure avoidance control, when there is no obstacle such as an oncoming vehicle or a preceding vehicle in the adjacent lane on the lane side where the own vehicle is likely to deviate, the own vehicle steps on the lane. In some cases, it is permitted to control the host vehicle so that the vehicle travels slightly outside the lane.

  However, the roadside walls, that is, the walls of houses and buildings as well as block fences, guardrails, hedges, etc., cannot be handled in the same way as the lane as described above, and touch or collide with the walls. The situation must be reliably avoided. In the present invention, three-dimensional objects by the road such as walls of houses and buildings, block fences, guardrails, hedges, etc. are collectively referred to as walls.

  Especially on roads where guardrails are arranged or hedges are planted, there are cases where lanes are not marked in the vicinity of guardrails, hedges, etc., and it is not sufficient to perform automatic control based on detected lanes. In addition, it is necessary to reliably and accurately detect the position of the wall beside the road. There is also a need for a detection device that can reliably and accurately detect the position of the wall beside the road.

  The present invention has been made in view of such circumstances, and wall detection capable of reliably and accurately detecting a wall such as a guardrail beside a road from a captured image obtained by imaging a traveling path of the host vehicle including the road. An object is to provide an apparatus.

In order to solve the above problem, the wall detection device of the first invention is:
Imaging means for imaging a traveling path of the host vehicle including a road and outputting a pair of images;
Image processing means for calculating a distance in real space for each setting region of at least one image based on the pair of captured images;
Extraction means for extracting the setting region that can be considered to exist above the road surface based on the information of the distance;
Based on the extracted setting area, a straight line representing a wall on the left and right of the own vehicle is detected in an area close to the own vehicle, and a far straight or curved wall shape is detected based on the straight line representing the wall. And a wall detecting means for detecting the position of the wall by combining the straight line representing the wall and the shape of the wall.

  According to a second aspect of the present invention, in the wall detection device according to the first aspect, the wall detection means invalidates a detection result when the detected wall is not continuously detected for a predetermined distance or more. And

  According to a third aspect of the present invention, in the wall detection device of the first or second aspect, the wall detection means invalidates the detection result when the detected wall has a moving speed equal to or higher than a predetermined speed. Features.

  According to a fourth aspect of the present invention, in the wall detection device according to any one of the first to third aspects, the wall detection unit converts the extracted setting region into a point on the real space, and the point on the real space. Based on the above, at least a straight line representing the wall is detected.

  According to a fifth aspect of the present invention, in the wall detection device according to any one of the first to third aspects, the wall detecting means is a setting area that can be regarded as an image of the same three-dimensional object among the extracted setting areas. A straight line representing at least the wall is detected based on a position on the one image of the group having a predetermined inclination with respect to the traveling direction of the host vehicle or a position on the real space.

According to a sixth aspect of the present invention, in the wall detection device according to any one of the first to fifth aspects, the wall detection means converts the extracted set area into a point on a real space, and the point is detected by the own vehicle. Of the wall obtained by pattern-matching the number of plots of each block on the real space in the left-right direction of the host vehicle. A straight line representing the wall or a shape of the wall is detected based on a position.

According to the first invention, the parallax is calculated by the stereo matching process for each setting area set in one of the pair of images captured by the stereo camera, and the parallax is captured based on the calculated parallax. The distance to the three-dimensional object is accurately calculated. Based on the distance information, the straight line representing the wall in the area close to the host vehicle is reliably detected, and the shape of the farther wall is detected based on the detected straight line. Detects the position of walls such as guardrails on the left and right side of the road.

  Therefore, it is possible to reliably and accurately detect a wall such as a guardrail beside the road, and to perform automatic control of the own vehicle based on the position information of the wall output by the wall detection device according to the present invention. If a control system is constructed, it becomes possible to appropriately and accurately control the host vehicle based on information on walls such as roadside guardrails that are reliably and accurately detected.

  According to the second aspect of the invention, for example, when a side surface of a vehicle traveling in a traveling lane adjacent to the traveling lane in which the host vehicle is traveling is detected as a wall, the wall is continuously detected for a predetermined distance or more. If the detection result is invalidated, the erroneous detection of the wall can be avoided reliably, and the effect of the first aspect of the invention can be exhibited more accurately.

  According to the third aspect of the invention, the detected wall has a speed equal to or higher than a predetermined speed, for example, when the side surface of the vehicle traveling in the traveling lane adjacent to the traveling lane in which the host vehicle is traveling is detected as a wall. If the detection result is invalidated, erroneous detection of the wall can be avoided reliably, and the effects of the inventions can be exhibited more accurately.

  According to the fourth invention, the setting area corresponding to the three-dimensional object existing above the road surface among the setting areas having significant distance information is converted to a point on the real space, and the converted real space By detecting a straight line representing a wall by straight line approximation such as Hough transform based on the point, it becomes possible to detect a straight line representing the wall reliably and accurately in an area close to the host vehicle, and the effects of the respective inventions can be obtained. It becomes possible to demonstrate more accurately.

  According to the fifth invention, the setting area corresponding to the three-dimensional object existing above the road surface among the setting areas having significant distance information is converted into points on the real space, The points corresponding to the setting area that can be regarded as having captured the same three-dimensional object are grouped and approximated by a straight line, and the straight line on the left and right sides of the vehicle closest to the own vehicle that is a group that is almost parallel to the traveling direction of the own vehicle By detecting it as a straight line, it is possible to detect a straight line that represents the wall reliably and accurately in an area close to the host vehicle, and the effects of the above-described inventions can be exhibited more accurately.

  According to the sixth invention, the setting area corresponding to the three-dimensional object existing above the road surface among the setting areas having significant distance information is converted into points on the real space, and these points are converted into the own vehicle. Plot each block in real space divided into a grid in front, and detect the wall position based on the position of the wall obtained by pattern matching the number of plots of each block in the left and right direction of the host vehicle For example, even when the converted points in the real space are not arranged in the real space and are plotted so as to be scattered in the real space, the positions of the left and right walls of the host vehicle are accurately detected. It becomes possible, and it becomes possible to exhibit the effect of each said invention more correctly.

  Embodiments of a wall detection apparatus according to the present invention will be described below with reference to the drawings.

[First Embodiment]
As shown in FIG. 1, the wall detection apparatus 1 according to the first embodiment mainly includes an imaging unit 2, a conversion unit 3, an image processing unit 6, and a detection unit 9.

  The imaging means 2 is for imaging the periphery of the vehicle, and is configured to capture a landscape including a road ahead of the vehicle at a predetermined sampling period and output a pair of images. In the present embodiment, a stereo camera including a pair of main camera 2a and sub-camera 2b each incorporating a synchronized image sensor such as a CCD or CMOS sensor is used. In the present embodiment, CCD cameras are used for the main camera 2a and the sub camera 2b.

  The main camera 2a and the sub camera 2b are attached, for example, in the vicinity of a rearview mirror with a predetermined interval in the vehicle width direction. Of the pair of stereo cameras, a camera closer to the driver, as will be described later, a main camera 2a that captures an image serving as a basis for detecting distances and detecting lanes for each pixel, and a camera farther from the driver Is a sub-camera 2b that captures an image to be compared in order to obtain the distance or the like.

  A / D converters 3a and 3b as conversion means 3 are connected to the main camera 2a and the sub camera 2b, respectively. In the A / D converters 3a and 3b, each of the pair of analog images output from the main camera 2a and the sub camera 2b has a luminance value of a predetermined luminance gradation such as a gray scale of 256 gradations for each pixel. It is configured to be converted into an image.

  From the A / D converter 3a, a digital image converted from an image which is captured by the main camera 2a, the distance is calculated for each pixel described above, and the lane is detected is output as a reference image, and the A / D converter 3b is output. The digital image captured and converted by the sub camera 3b is output as a comparison image.

  An image correction unit 4 is connected to the A / D converters 3a and 3b. In the image correction unit 4, the main camera 2a and the comparison image are output from the A / D converters 3a and 3b. Image correction such as correction of a luminance value including deviation due to an error in the mounting position of the sub camera 2b, noise removal, and the like is performed using affine transformation or the like.

  For example, the reference image T is output from the image correction unit 4 as image data having a luminance value at each pixel as shown in FIG. 2, and the comparison image is also output as image data having a luminance value at each pixel. It is configured.

  An image data memory 5 is connected to the image correction unit 4, and the respective image data of the reference image T and the comparison image are stored in the image data memory 5 and simultaneously transmitted to the detection means 9. ing.

  An image processing means 6 is connected to the image correction unit 4. The image processing means 6 is mainly composed of an image processor 7 and a distance data memory 8.

  In the image processor 7, each setting area composed of each pixel or a block composed of a plurality of pixels of the reference image T based on the digital data of the reference image T and the comparison image output from the image correction unit 4 by the stereo matching process and the filtering process. The parallax dp for calculating the distance in the real space is calculated.

  The calculation of the parallax dp is described in detail in Japanese Patent Application Laid-Open No. 5-1114099 previously filed by the applicant of the present application.

  The image processor 7 divides the reference image T into setting areas, for example, 4 × 4 pixel areas, and calculates one parallax dp for each setting area by stereo matching processing. Specifically, for example, when the reference image T is an image having 512 pixels in the horizontal direction and 200 pixels in the vertical direction, the image processor 7 converts the reference image T into 128 × 50 pixels by 4 × 4 pixels in a grid pattern. The setting area is formed by dividing.

  As described above, the luminance value p1ij of 0 to 255 is assigned to each of the 16 pixels constituting one setting area, and the luminance value p1ij of the 16 pixels forms a luminance value characteristic unique to the setting area. .

  The subscripts i and j of the luminance value p1ij represent the i and j coordinates of a pixel when the lower left corner of the image plane of the reference image T is the origin, the horizontal direction is the i coordinate axis, and the vertical direction is the j coordinate axis. As described above, when the reference image T is 512 × 200 pixels, 0 ≦ i ≦ 511 and 0 ≦ j ≦ 199. For the comparison image, the i-coordinate and the j-coordinate are similarly taken with the pixel previously associated with the origin of the reference image T as the origin.

The image processor 7 divides the comparison image into four-pixel-wide horizontal lines extending in the horizontal direction, takes out one setting area of the reference image T, and horizontally in the horizontal direction of the corresponding comparison image one pixel at a time. That is, while shifting in the i direction, the absolute values of the differences between the luminance values p1ij of the 16 pixels in the setting region of the reference image T and the luminance values p2ij of the 16 pixels in the corresponding comparison image are totaled as follows ( The setting area on the horizontal line where the city block distance CB calculated by the equation (1) is minimum, that is, the setting area on the comparative image having the luminance value characteristic closest to the setting area of the reference image T is searched. .
CB = Σ | p1ij−p2ij | (1)

  The image processor 7 calculates the amount of deviation between the setting region on the comparison image specified in this way and the setting region on the reference image T, and sets the amount of deviation as the parallax dp to the setting region on the reference image T. It is supposed to be associated.

  The parallax dp is a relative displacement amount in the horizontal direction with respect to the mapping position of the same object in the reference image T and the comparison image derived from the main camera 2a and the sub camera 2b separated by a certain distance. The distance from the center position of the camera 2b to the object and the parallax dp can be associated based on the principle of triangulation.

Specifically, in real space, a point on the road surface directly below the center of the main camera 2a and the sub camera 2b is set as the origin, and the vehicle width direction of the own vehicle, that is, the X axis in the left-right direction, the Y axis in the vehicle height direction, When the Z axis is taken in the vehicle length direction, that is, the distance direction, the coordinate conversion from the point (i, j) on the reference image to which the parallax dp is assigned to the point (X, Y, Z) in the real space is as follows ( This is performed based on the equations (2) to (4).
X = CD / 2 + Z * PW * (i-IV) (2)
Y = CH + Z × PW × (j−JV) (3)
Z = CD / (PW × (dp−DP)) (4)

  The reference image T in which the parallax dp or the distance Lij described later is assigned for each setting area in this way is referred to as a distance image. In the equations (2) to (4), CD is the distance between the main camera 2a and the sub camera 2b, PW is the viewing angle per pixel, CH is the mounting height of the main camera 2a and the sub camera 2b, IV And JV represent i-coordinate and j-coordinate on the distance image of the infinity point in front of the host vehicle, and DP represents the vanishing point parallax.

  That is, the center position of the main camera 2a and the sub camera 2b, more precisely the distance Lij and the parallax dp from the point on the road surface just below the center to the object can be obtained by setting Z in the equation (4) as the distance Lij. Uniquely associated. Further, by substituting Z obtained from the parallax dp based on the equation (4) into the equations (2) and (3), it can be obtained in association with the i coordinate and the j coordinate on the distance image.

  Further, for the purpose of improving the reliability of the parallax dp, the image processor 7 performs a filtering process on the parallax dp thus obtained, and outputs only the valid parallax dp.

  That is, for example, even if a 4 × 4 pixel setting region having only features of a roadway image is scanned on a horizontal line having a width of 4 pixels of the comparison image, all the portions of the comparison image where the roadway is imaged are correlated. Even if the corresponding setting area is specified and the parallax dp is calculated, the reliability of the parallax dp is low. Therefore, such a parallax dp is invalidated in the filtering process, and 0 is output as the value of the parallax dp.

  Therefore, the distance Lij of each pixel of the reference image T output from the image processor 7, that is, the parallax dp for calculating the distance in the real space for each setting region of the reference image T is usually in the left-right direction of the reference image T. The data has an effective value only for a so-called edge portion in which the difference in luminance value p1ij is large between adjacent pixels.

  The parallax dp of each setting area of the reference image T calculated by the image processor 7 is converted to Z, that is, a distance Lij in the real space based on the equation (4), and this distance Lij corresponds to the setting area on the reference image T. In addition, it is stored in the distance data memory 8 of the image processing means 6 as a distance image.

  Note that the coordinates (i, j) on the distance image correspond to the coordinates (i, j) on the reference image T. Further, in the processing in the detection means 9 described below, when one setting area on the distance image is handled as 4 × 4 pixels, it is assumed that each of the 16 pixels belonging to one setting area has a distance Lij. When the setting area is handled as one processing target, the setting area is treated as having a distance Lij.

  The detection means 9 is composed of a microcomputer in which a CPU, ROM, RAM, input / output interface and the like (not shown) are connected to a bus. The detection means 9 is connected to sensors A such as a vehicle speed sensor and a steering angle sensor for measuring the steering angle of the steering wheel.

  As shown in FIG. 3, the detection unit 9 includes a lane detection unit 91, a lane model formation unit 92, an extraction unit 93, a wall detection unit 94, and a memory 95. Necessary data is input from the sensors A via the I / O interface 96 to each means of the detection means 9.

  The lane detection unit 91 detects a lane candidate point on the reference image T based on the luminance value p1ij of each pixel of the reference image T and the distance Lij of each pixel in the real space, and the host vehicle based on the detected lane candidate point The left and right lane positions are detected.

  The lane detection unit 91 may be any device that can detect the lane position from the reference image T, and the method of detecting the lane position is not limited to a specific method. In the present embodiment, the lane detecting unit 91 and a lane model forming unit 92 described later are configured based on the lane recognition device described in Patent Document 2. A detailed description of the configuration is left to the same publication, but the configuration will be briefly described below.

  The lane detection unit 91 reads information on the luminance value p1ij of each pixel of the reference image T from the image data memory 5, and reads information on the distance Lij in the real space of each pixel of the distance image from the distance data memory 8. Then, a search is performed while offsetting one pixel in the left-right direction on the horizontal line j having a width of one pixel on the reference image T, and each pixel is determined based on the luminance value p1ij of each pixel of the reference image T as shown in FIG. Are detected as lane candidate points (Ij, Jj), respectively.

  At that time, based on the information on the distance Lij of each pixel assigned to the distance image corresponding to the reference image T, the detected pixel is excluded when it is not on the road surface and is not detected as a lane candidate point. It has become.

  The lane detecting means 91 sequentially detects lane candidate points while offsetting the horizontal line j to be searched upward by one pixel width from the lower side of the reference image T, and as shown in FIG. Lane candidate points are detected in the area A and the left area B, respectively. Then, by discarding lane candidate points that cannot be consistent among them and connecting the remaining lane candidate points, the right lane position LR on the right side of the host vehicle on the reference image T as shown in FIG. The left lane position LL is detected on the left side. The left and right lane positions LR and LL are also converted into lane positions LR and LL in real space, respectively.

  In the present embodiment, the lane detection unit 91 detects the left and right lane positions LR and LL by approximating the coordinates of the detected lane candidate points with a straight line or a curve. Also, for regions far from the host vehicle where no lane candidate points are detected, based on the approximated straight line or curve, or according to the rate of change in the slope of the straight line connecting the detected lane candidate points, etc. The detected left and right lane positions LR and LL are appropriately extended and estimated.

  The search for lane candidate points is performed by setting a search area on the reference image T. That is, in this detection process, a search area is set in a certain range on the reference image T including the lane position based on the lane position detected in the previous detection process. In this detection, when the pixels are sequentially detected while the horizontal line j is offset by one pixel width upward from the lower side of the reference image T, no pixel satisfying the above condition is detected in a certain horizontal line j. In this case, the search area is expanded on the next horizontal line to search for lane candidate points.

  The lane detecting means 91 stores information on the left and right lane positions LR, LL and lane candidate points detected in this way in the memory 95 and outputs them to the outside.

  The lane model forming means 92 is configured to form a lane model in a three-dimensional manner based on information on the left and right lane positions LR and LL detected by the lane detecting means 91 and lane candidate points. In this embodiment, as shown in FIGS. 7A and 7B, the lane model forming unit 92 approximates the left and right lanes of the host vehicle with a three-dimensional linear formula for each predetermined section, and forms them in a polygonal line shape. A lane model expressed by connecting to the lane is formed. 7A shows a road model on the ZX plane, that is, a horizontal shape model, and FIG. 7B shows a road model on the ZY plane, that is, a road height model.

Specifically, the lane model forming unit 92 divides the real space ahead of the host vehicle into each section up to the distance Z7 from the position of the host vehicle, and the lane candidate point detected by the lane detecting unit 91 on the real space. Based on the position (X, Y, Z), the lane candidate points in each section are linearly approximated by the least square method, and parameters a _R , b _R , A lane model is formed by calculating a _L , b _L , c _R , d _R , c _L , and d _L.

[Horizontal shape model]
Right lane X = a _R · Z + b _R (5)
Right lane X = a _L · Z + b _L (6)
[Road height model]
Right lane Y = c _R · Z + d _R (7)
Right lane Y = c _L · Z + d _L (8)

As described above, the lane model forming means 92 can detect the height of the road surface in the real space by forming the road height model based on the information of the distance Lij. Corresponds to means. The lane model forming unit 92 stores the road model thus formed, that is, the calculated parameters a _{R to} d _L of each section in the memory 95 and outputs them to the outside.

  The origin and the X, Y, and Z axes are taken in the same manner as described above. The lane model formed by the lane model forming unit 92 is determined based on whether or not the pixel detected in the next sampling cycle is on the road surface when the lane detection unit 91 detects the lane candidate point. And is used for processing by each means to be described later.

  The extracting means 93 is predetermined from the road surface from among the reference images in which the distance Lij is assigned for each setting area based on the information on the distance Lij and the lane model formed in the lane model forming means 92, that is, in each setting area of the distance image. A setting area that is considered to correspond to a three-dimensional object that exists above the height is extracted.

  Specifically, the extracting means 93 substitutes the distance Lij of the setting area into Z in the above equation (3) for each setting area to which a non-zero significant distance Lij is assigned among the setting areas of the distance image. Then, Y is calculated, the model height at the distance Lij is calculated from the road height model of the lane model, and the height from the road surface of the three-dimensional object corresponding to the set region is calculated from the difference.

  And the extraction means 93 extracts only the setting area | region whose difference is more than the predetermined height set beforehand. The extracting means 93 stores the setting area extracted in this way in the memory 95.

  The wall detecting means 94 detects a straight line representing a wall on the left and right sides of the own vehicle, that is, a wall straight line in an area close to the own vehicle based on the setting area extracted by the extracting means 93, and moreover based on the detected wall straight line. A distant linear or curved wall line, that is, a wall line is detected, and the wall straight line and the wall line are combined to detect walls on the left and right of the host vehicle.

  In the present embodiment, the wall detection unit 94 converts the set area extracted by the extraction unit 93 into a point on the real space and detects the straight line on the vehicle by linear approximation based on the point on the real space. Wall straight lines are detected on the left and right sides of the vehicle in a close area.

  This wall straight line detection can be performed for all the extracted setting areas, but some of the extracted setting areas include those with low reliability of the distance Lij and those unnecessary for wall detection. Therefore, in this embodiment, a wall straight line is detected by the following method.

  That is, the wall detecting means 94 first divides the distance image read from the distance data memory 8 into strip-shaped sections extending in the vertical direction with a predetermined pixel width such as 4 pixels. Then, a histogram as shown in FIG. 8 showing the distribution of the total distance Lij of the extracted setting region belonging to one strip-shaped section is created, and the distance to the section with the maximum frequency is determined as the strip-shaped section. The representative distance Ln of the three-dimensional object at. Do this for all strips.

  The wall detecting means 94 converts the calculated representative distance Ln of each strip-shaped section into a point in real space. That is, the representative distance Ln is set as a Z coordinate, and the X coordinate is calculated by substituting the Z coordinate into the equation (2). Then, when the points for which the X and Z coordinates are calculated in this way are plotted on the real space represented by the XZ plane, as shown in FIG. Are plotted so as to line up with the part facing the host vehicle MC.

  The wall detecting means 94 performs Hough transform for each point in the real space calculated in this way. The Hough transform may be performed on all points in the calculated real space, and each strip-shaped section is divided into a right half and a left half of the distance image, and each section is configured to be performed on each. It is also possible. Moreover, a well-known method is used as Hough transform, for example, it is performed by the method shown below.

Specifically, for example, the coordinates of each point converted from the representative distance Ln of each strip-shaped section is (Xn, Zn), and each point is a straight line on the reference image T X = aZ + b (9)
Assuming that it exists above, Xn and Zn are
Xn = aZn + b (10)
Meet.

When the equation (10) is transformed,
b = −Zn × a + Xn (11)
It becomes. As can be seen from the equation (11), when each point (Xn, Zn) is calculated, one straight line is formed on the Hough plane represented by the ab plane with −Zn and Xn as inclinations and b intercepts, respectively. Can be drawn.

  The Hough plane is divided into cells of a predetermined size in advance, and each point (Xn, Zn) is calculated and the count value of the cells passing the straight line is increased by 1 each time the straight line represented by the equation (11) is drawn. Let Then, for each strip-shaped section, the representative distance Ln is converted to a point in the real space to perform a Hough transform. Finally, a straight line is calculated for all the sections and the addition of the Hough plane cell is finished. A predetermined number of a set of a and b giving the maximum value for the count value of each square in the plane, that is, a straight line obtained by substituting them into the equation (9) is extracted.

For example, when a Hough transform is performed on each point shown in FIG. 9, a plurality of straight lines are obtained. The wall detecting means 94 includes, for example, a plurality of straight lines obtained, for example,
(A) A straight line passes through the inside of the own vehicle MC or passes very close to the own vehicle MC. (B) A straight line that satisfies the condition that the parallelism between the straight line and the trajectory estimated from the behavior of the own vehicle MC is low. Is rejected as not suitable as a wall, and straight lines are selected until the straight lines are narrowed to the left and right of the host vehicle MC one by one.

  If the extracted straight line does not satisfy the above condition, or if the count value of each square on the Hough plane is equal to or less than the threshold value, the maximum value may not be satisfied. A straight line may not be selected on the right side, the left side, or both of the vehicle MC.

  In this way, when the straight lines r1 and l1 are selected one by one on the left and right of the host vehicle MC, for example, as shown in FIG. The distance in the real space from the straight lines r1 and l1 is calculated, and the first point at which the distance from the straight lines r1 and l1 is a predetermined value or less is extracted as a wall candidate point. Then, the wall detecting unit 94 sequentially searches each point in the direction away from the host vehicle MC, and the distance from the straight lines r1 and l1 is equal to or less than a predetermined value, and the nearest wall candidate point on the host vehicle MC side is Each point whose distance is equal to or less than a predetermined distance is extracted as a wall candidate point.

  In this embodiment, a straight line representing a wall in a region close to the host vehicle is searched by sequentially extracting the wall candidate points that are close to the straight lines r1 and l1 from the side closer to the host vehicle MC. The straight line is reliably detected at a position close to the own vehicle. When wall candidate points are extracted in this way, for example, wall candidate points indicated by white circles in FIG. 11 are extracted from the points shown in FIG.

  The wall detection means 94 extracts the wall candidate points for the straight lines r1 and l1 as described above, and the wall candidate farthest from the wall candidate point closest to the host vehicle MC, as shown in FIG. 12, of the straight lines r1 and l1. Up to a point is detected as a straight line representing a wall in a region close to the host vehicle MC, that is, wall straight lines r1 and l1.

  Subsequently, the wall detecting means 94 detects a farther straight or curved wall line with reference to the wall straight lines r1 and l1 thus detected.

  In this case, for example, when each point in the real space converted from the representative distance Ln to the three-dimensional object is arranged in an orderly manner for each strip-shaped section of the distance image as shown by black circles or white circles in FIG. Each point is tracked sequentially from the end farthest from the host vehicle MC of the wall straight lines r1 and l1 shown in FIG. 12, and a farther straight or curved wall line can be detected.

  Specifically, each point far from the wall candidate point farthest from the host vehicle belonging to the wall straight lines r1 and l1 is sequentially searched from the side closer to the host vehicle. First, it is determined whether or not the displacement in the X-axis direction and the Z-axis direction in the real space between the wall candidate point farthest from the host vehicle belonging to the wall straight lines r1 and l1 and the next point is within a specified value. If it is determined that the value is within the specified value, the point is recorded as a wall candidate point.

  Thereafter, similarly, as shown in FIG. 13 (A), when the next point is detected in the direction away from the host vehicle, the displacement in the X-axis direction and Z-axis direction with the wall candidate point a detected immediately before is detected. It is determined whether or not the value is within the specified value. If it is determined that the value is within the specified value, the point detected this time is recorded as the wall candidate point b.

  Further, even when the displacement in the X-axis direction and the Z-axis direction between the next point and the wall candidate point detected immediately before is not within the specified value, a straight line connecting the wall candidate points detected up to the previous time, that is, FIG. In B), if the displacement in the X-axis direction from the straight line connecting the wall candidate point a and the wall candidate point b is within a specified value, the currently detected point is recorded as the wall candidate points c, d,.

  In this way, it is possible to track each point sequentially from the end farthest from the host vehicle MC of the wall straight lines r1 and l1, and to detect a farther straight or curved wall line. is there.

  However, in actuality, when the representative distance Ln to the three-dimensional object is converted into each point in the real space for each strip-shaped section of the distance image, the converted points are not neatly arranged as shown in FIG. In some cases, the plots are scattered so as to be scattered in the real space represented by the XZ plane. Therefore, in this embodiment, the wall detection means 94 is comprised so that a distant wall line may be detected with the following methods.

  That is, the wall detecting means 94 first has a plurality of lattice-shaped grids having a predetermined size in the distance direction of the host vehicle, that is, the Z-axis direction and the left-right direction, that is, the X-axis direction, as shown in FIG. Divide into blocks. Then, in the area far from the point farthest from the vehicle MC of the wall straight lines r1 and l1 detected by the detection of the wall straight line, each strip-shaped section of the distance image is converted from the representative distance Ln to the three-dimensional object. Each point on the space is plotted in the corresponding block, and the number of plots in that block is increased by 1 each time each point is applied.

After plotting all points in the real space in each block, for example, the distribution in the X-axis direction of the number of plots p at the points indicated by arrows shown in FIG. 14 is as shown in FIG. The wall detection means 94 applies a pattern of the number of points w shown in FIG. 16 to the distribution of the number of plots p, for example, calculates the coincidence Q according to the following equation (12), and matches the pattern while moving the pattern in the X-axis direction. Pattern matching is performed by sequentially calculating the degree Q and searching for the maximum value.
Q = Σw · p (12)

  The wall detecting means 94 matches the pattern of the point w in FIG. 16 to the position of the origin of the pattern when the degree of coincidence Q is maximum in the pattern matching in the X-axis direction, that is, for example, the distribution of the number of plots p in FIG. In this case, a coordinate Xr in the X-axis direction shown in FIG. 17 is extracted as the X coordinate of the wall candidate point.

  The Z coordinate of the wall candidate point may be the center position of the block in the real space to which the wall candidate point belongs, or may be the position closest to the host vehicle MC of the block. Further, when it is desired to obtain the left wall candidate point of the host vehicle MC, for example, it can be extracted by performing the same pattern matching using the pattern of the number of points w shown in FIG.

  The wall detecting means 94 detects the polygonal wall line by connecting the wall candidate points extracted in this way. The wall detecting means 94 detects the positions WR and WL of the left and right walls of the host vehicle MC as shown in FIG. 19 by combining the wall straight lines r1 and l1 detected as described above and the left and right wall lines. It has become. Further, the wall detection means 94 stores the information of the walls WR and WL detected in this way in the memory 95, respectively.

  In addition, it is also possible to connect the wall candidate points extracted in this way into a curved line to form a wall line. Further, when the extracted wall candidate point is not within a reasonable range from the position of another wall candidate point, the wall candidate point is deleted. Further, in the present embodiment, a case has been described in which each point on the real space converted from the representative distance Ln of each section of the distance image is plotted on a block on the real space. It is also possible to convert the setting region into a point in the real space and plot it in each block.

  On the other hand, in the present embodiment, the wall detection means 94 further invalidates the detection result when the detected walls WR and WL are not continuously detected over a predetermined distance in the real space. Yes. As described above, in this embodiment, for example, a wall is continuously detected for a predetermined distance or more as when a side surface of a vehicle traveling in a traveling lane adjacent to a traveling lane in which the host vehicle is traveling is detected as a wall. If the detection result is invalid, the false detection of the wall is avoided by invalidating the detection result.

  Specifically, the wall detecting unit 94 calculates the distance in the real space between the extracted wall candidate points when the above-described wall straight lines r1, l1 and the wall line are detected, and the distance is set. When the wall candidate points that are within the threshold are connected, the distance in real space between the wall candidate point closest to the host vehicle MC and the farthest wall candidate point does not reach a predetermined distance set to 10 m, for example. In this case, the walls detected based on these wall candidate points are invalidated.

  For example, in the wall WL detected on the left side of the host vehicle MC in FIG. 19, the distance in real space between the wall candidate point closest to the host vehicle MC belonging to the wall WL and the farthest wall candidate point must reach a predetermined distance. In this case, the detection result of the wall WL is invalidated.

  Moreover, even if the distance in real space between the wall candidate point closest to the host vehicle MC and the farthest wall candidate point is a predetermined distance or more as in the wall WR of FIG. If the distance in the real space between the wall candidate point closest to the host vehicle MC and the wall candidate point on the host vehicle side of the point does not reach the predetermined distance, there is a wall. The detection result of WR is invalidated. That is, even if the detected wall is not less than the predetermined distance, the detection result is invalid if the detected wall is not continuously longer than the predetermined distance. Whether or not the detected walls are continuous is determined based on whether or not the setting regions of the images corresponding to the nearest wall candidate point and the farthest wall candidate point to the host vehicle are separated from each other by a predetermined interval or more. It is also possible to determine by

  In the embodiment shown in FIG. 2, as shown in FIG. 10, the wall candidate points belonging to the wall WR on the right side of the host vehicle have no points where the distance between adjacent wall candidate points is larger than the set threshold value, as shown in FIG. The wall WR is effective because it is continuously longer than a predetermined distance, and the wall WL on the left side of the host vehicle is invalid because it does not reach the predetermined distance. Therefore, in the wall detection based on the reference image T shown in FIG. 2, only the guard rail on the right side of the host vehicle is detected as the wall WR and output to the outside of the apparatus.

  Further, in the present embodiment, the wall detection means 94 invalidates the detection result when the detected wall has a moving speed equal to or higher than a predetermined speed. Thus, in this embodiment, for example, the detected wall has a speed equal to or higher than a predetermined speed as when the side surface of the vehicle traveling in the traveling lane adjacent to the traveling lane in which the host vehicle is traveling is detected as a wall. If it has, the detection result is invalidated to avoid erroneous wall detection.

  As a method of calculating the moving speed of the wall, for example, it is detected as a wall from information on the relative speed by a measuring means such as a radar that measures the relative speed between the host vehicle and the three-dimensional object and information on the traveling speed of the host vehicle from the vehicle speed sensor It is possible to calculate the moving speed of the three-dimensional object. Further, the three-dimensional object detected as the vehicle and the wall from the difference between the previous detection position and the current detection position of each point converted into the real space from the representative distance Ln of each section of the distance image shown in FIG. It is also possible to calculate the moving speed of the three-dimensional object detected as a wall from the information on the traveling speed of the host vehicle. In addition, a method capable of calculating the moving speed of the three-dimensional object detected as a wall can be appropriately employed.

  The invalid detection result may be configured to be deleted from the memory 95 or may be configured to be stored in the memory 95 while being labeled as invalid. It is also possible to configure so that the detection result invalidated is labeled invalid and then output to the outside.

  As described above, according to the wall detection device 1 according to the present embodiment, the parallax dp is calculated for each setting region of the reference image T by the stereo matching process, and the three-dimensional object imaged in the setting region is calculated based on the parallax dp. The distance Lij is calculated with high accuracy.

  Then, based on the distance image formed by assigning the distance Lij to the reference image T or each setting area of the reference image T, the straight line representing the wall in the area close to the host vehicle is reliably detected and detected. The position of a far wall is detected based on the straight line, and the position of a wall such as a guard rail existing on the left and right road sides of the host vehicle is detected.

  In particular, as in the present embodiment, the setting area corresponding to the three-dimensional object existing above the road surface among the setting areas having the information of the significant distance Lij is converted into a point on the real space, Based on this, by detecting a straight line representing a wall by linear approximation such as Hough transform, it becomes possible to detect a straight line representing the wall reliably and accurately in an area close to the host vehicle.

  Therefore, it is possible to reliably and accurately detect a wall such as a guardrail beside the road, and to perform automatic control of the host vehicle based on the position information of the wall output by the wall detection device 1 according to the present embodiment. If the automatic control system to perform is constructed, it becomes possible to appropriately and accurately control the own vehicle based on the information of the walls such as the roadside guardrails reliably and accurately detected.

  In the present embodiment, the pattern matching method using the blocks in the real space shown in FIG. 14 or the like is used only for detection of a wall line farther after the wall straight lines WR and WL are detected. Although described, it is also possible to configure to detect wall straight lines in an area close to the host vehicle using this method.

[Second Embodiment]
The wall detection apparatus according to the second embodiment has the same configuration as that of the wall detection apparatus 1 according to the first embodiment shown in FIG. 1, and the configuration of the wall detection means in the detection means 9 is different. Hereinafter, the same components as those in the first embodiment will be described with the same reference numerals.

  The wall detection unit according to the present embodiment detects wall straight lines on the left and right sides of the host vehicle in a region close to the host vehicle based on the setting region extracted by the extraction unit 93, and is further distant from the detected wall straight line as a reference. The straight line or curved wall line is detected, and the wall straight line and the wall line are combined to detect the walls on the left and right sides of the host vehicle. Different.

  The wall detection means divides the distance image read from the distance data memory 8 into strip-shaped sections extending in the vertical direction with a predetermined pixel width in the detection of wall straight lines, as in the first embodiment. A histogram is created for the total distance Lij of the extracted setting region belonging to the shape section, the representative distance Ln of the three-dimensional object in the strip-shaped section is calculated, and each calculated representative distance Ln is a point in the real space. These points are plotted in real space as shown in FIG.

  Subsequently, the wall detection unit groups each point that can be regarded as the same three-dimensional object based on the distance and directionality of each adjacent point as shown in FIG. 20 with respect to each point in the real space shown in FIG. Grouped into G1 to G7. Then, as shown in FIG. 21, a group in which each point is arranged substantially parallel to the X-axis direction perpendicular to the traveling direction of the host vehicle is labeled front O1-O3, and each point is the traveling of the host vehicle. The groups arranged substantially parallel to the Z-axis direction, which is the direction, are labeled as side walls W1 to W4, and the groups corresponding to the three-dimensional object are detected as “front” and “side walls”, respectively.

  The wall detecting means linearly approximates each point belonging to each of the groups G1 to G7 detected as “front” or “sidewall” using a method such as Hough transform or least square method, and is closest to the host vehicle MC. The straight lines representing the left and right side walls W1 and the straight lines representing the side walls W4 are detected as wall straight lines W1 and W4, respectively.

In this embodiment, as described above, when detecting a wall straight line,
(A) A straight line passes through the inside of the own vehicle MC or passes very close to the own vehicle MC. (B) A straight line that satisfies the condition that the parallelism between the straight line and the trajectory estimated from the behavior of the own vehicle MC is low. Is rejected as a straight line that is not suitable as a straight line representing a wall.

  On the other hand, detection of wall lines farther from the wall straight lines W1 and W4 can be configured to be processed using a pattern matching method using blocks in the real space, as in the first embodiment. It is. Further, as shown in FIG. 21, when a farther side wall W3 continuous with the side wall W4 is detected, the side wall W3 detected by the method of the present embodiment is detected as a wall line. It is also possible to configure.

  In the present embodiment, the wall detecting means thus aligns the wall straight line W1 with the left wall position WL of the host vehicle MC, and the wall straight line W4 and the wall line W3 together with the right wall position WR of the host vehicle MC. It is supposed to detect as. Further, the wall detection means stores the information of the left and right walls thus detected in the memory 95.

  In the present embodiment as well, invalidation processing of detection results based on the detected wall length, continuity, and moving speed is performed. Therefore, also in the present embodiment, the wall WL detected on the left side of the host vehicle MC is invalidated. As a result, only the guard rail on the right side of the host vehicle is detected as the wall WR by the wall detection based on the reference image T shown in FIG. And output to the outside of the device.

  As described above, according to the wall detection device according to the present embodiment, the parallax dp is calculated for each setting region of the reference image T by the stereo matching process, similarly to the wall detection device 1 according to the first embodiment. Based on this, the distance Lij to the three-dimensional object imaged in the setting area is accurately calculated.

  Then, based on the distance image formed by assigning the distance Lij to the reference image T or each setting area of the reference image T, the straight line representing the wall in the area close to the host vehicle is reliably detected and detected. The position of a far wall is detected based on the straight line, and the position of a wall such as a guard rail existing on the left and right road sides of the host vehicle is detected.

  In particular, as in the present embodiment, a setting area corresponding to a three-dimensional object existing above the road surface is converted into a point on the real space from among the setting areas having significant distance Lij information. The points corresponding to the setting areas that can be regarded as having captured the same three-dimensional object are grouped and approximated by a straight line, and the straight line on the left and right sides of the own vehicle that is the group that is almost parallel to the traveling direction of the own vehicle and that is closest to the own vehicle. By detecting it as a wall straight line, it is possible to detect a straight line representing the wall reliably and accurately in an area close to the host vehicle.

  Therefore, it is possible to reliably and accurately detect a wall such as a guardrail beside the road, and automatically control the host vehicle based on the position information of the wall output by the wall detection device according to the present embodiment. If an automatic control system is constructed, it becomes possible to control the host vehicle appropriately and accurately based on information on walls such as roadside guardrails reliably and accurately detected.

  Note that the grouping process shown in the present embodiment may be configured to be performed on the reference image T or the distance image.

It is a block diagram which shows the structure of the wall detection apparatus which concerns on 1st Embodiment. It is a figure which shows an example of a reference | standard image. It is a block diagram which shows the structure of the detection means which concerns on this embodiment. It is a figure explaining the lane candidate point detected on the horizontal line j. It is a figure which shows the detected lane candidate point. It is a figure which shows the detected right lane position and left lane position. It is a figure which illustrates (A) horizontal shape model of a lane model, and (B) road height model. It is a figure explaining the histogram which shows distribution of the total distance of a setting area | region. It is the figure which plotted each point converted from the setting area | region on real space. It is a figure which shows the straight line selected by the wall detection means on either side of the own vehicle. It is a figure explaining the wall candidate point extracted from each point shown in FIG. It is a figure which shows the detected wall straight line. It is a figure explaining the conditions which record the point on a real area as a wall candidate point. It is a figure explaining the real space divided | segmented into the some block. It is a graph which shows distribution of the X-axis direction of the number of plots. It is a figure which shows the pattern of the number of points used for pattern matching. It is a figure which shows the X coordinate of the wall candidate point extracted as a result of pattern matching. It is a figure which shows the pattern of the number of points for extracting the wall candidate point on the opposite side to FIG. It is a figure which shows the position of the detected wall. It is a figure which shows each group formed by grouping each point of FIG. 9 in 2nd Embodiment. It is a figure which shows the position of the front surface and side wall which were grouped, and the detected wall.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Wall detection apparatus 2 Imaging means 6 Image processing means 93 Extraction means 94 Wall detection means T Reference | standard image (one image)
Lij Distance WR, WL Wall position MC Vehicle r1, l1 Straight line G1 to G7 representing the wall Group p Number of plots

Claims (6)

Imaging means for imaging a traveling path of the host vehicle including a road and outputting a pair of images;
Image processing means for calculating a distance in real space for each setting region of at least one image based on the pair of captured images;
Extraction means for extracting the setting region that can be considered to exist above the road surface based on the information of the distance;
Based on the extracted setting area, a straight line representing a wall on the left and right of the own vehicle is detected in an area close to the own vehicle, and a far straight or curved wall shape is detected based on the straight line representing the wall. And a wall detecting means for detecting a position of the wall by combining a straight line representing the wall and the shape of the wall.   The wall detection device according to claim 1, wherein the wall detection unit invalidates a detection result when the detected wall is not continuously detected for a predetermined distance or more.   The wall detection apparatus according to claim 1, wherein the wall detection unit invalidates a detection result when the detected wall has a moving speed equal to or higher than a predetermined speed.   The said wall detection means converts the extracted setting area into a point on the real space, and detects a straight line representing at least the wall based on the point on the real space. Item 4. The wall detection device according to any one of items 3 to 3.   The wall detection means groups the setting areas that can be regarded as images of the same three-dimensional object among the extracted setting areas, and the one image of the group having a predetermined inclination with respect to the traveling direction of the host vehicle. The wall detection apparatus according to any one of claims 1 to 3, wherein a straight line representing at least the wall is detected based on an upper position or a position in real space. The wall detecting means converts the extracted setting region into a point on the real space, and plots the point on each block on the real space divided into a grid in the distance direction and the left-right direction of the host vehicle. And detecting the straight line representing the wall or the shape of the wall based on the position of the wall obtained by pattern matching the plot number of each block in the real space in the left-right direction of the host vehicle. The wall detection apparatus as described in any one of Claims 1-5.

Images