JP4575031B2 - In-vehicle road marking detection device - Google Patents

Description

  The present invention relates to an on-vehicle road marking detection device that detects markings on a road surface such as a white line or a pedestrian crossing line that divides a traveling lane.

  As a conventional road marking detection device that detects a white line and a pedestrian crossing from an image taken by an in-vehicle camera, for example, there is one disclosed in Patent Document 1 (Japanese Patent Laid-Open No. 2003-252148). In this road marking detection device, image data captured by a vehicle-mounted camera is compared with pattern matching reference data in a database, and pattern recognition of the image captured by the vehicle-mounted camera is performed. At this time, white line and pedestrian crossing pattern data are stored in the database as reference data for pattern matching, thereby enabling detection of white lines and pedestrian crossings.

JP 2003-252148 A

  Conventional road marking detection devices are configured as described above, and white lines and pedestrian crossings are detected by comparing input images with a database prepared in advance. A lot of reference data is necessary to deal with a complex image, and all the reference data and features (luminance values, etc.) must be compared and verified for detection, which takes a lot of processing time. was there.

  The present invention has been made to solve the above-described problems. For detecting white lines and pedestrian crossing lines having different structures, line segment features are extracted, and white lines and crossings are extracted based on the structure of the line segment features. The purpose of the present invention is to realize a road marking detection apparatus that does not require a database and can detect at high speed because it detects a sidewalk line.

An in-vehicle road marking detection apparatus according to the present invention includes an image input unit that inputs image data in front of a traveling vehicle, an edge extraction unit that extracts an edge point from an image captured by the image input unit, and the image Image dividing means for dividing the image of the input means into a plurality of partial images, line segment extracting means for extracting line segments passing through edge points existing in each partial image, and line segments obtained by the line segment extracting means of a feature extraction means for the slope and end positions extracted as feature present in a predetermined interval endpoint positions in the horizontal direction of the obtained line segments by the feature extraction means, and they line segment one vanishing point When formed, road marking detection means for determining that a pedestrian crossing exists ahead is provided.

  The present invention provides a feature extraction means for extracting features of white line candidates constituting white lines and pedestrian crossing lines, and pays attention to the difference in structure between the white lines and the pedestrian crossing lines based on the features extracted by the feature extraction means. Since the road markings of the white line and the pedestrian crossing line are detected, a database is unnecessary and the detection can be performed at high speed.

Embodiment 1.
Embodiments of a road marking detection apparatus according to the present invention will be described below with reference to the drawings.
FIG. 1 is a configuration diagram showing a first embodiment of the present invention. The first embodiment is an image input means 1 composed of an in-vehicle camera or the like for inputting image data in front of the traveling vehicle, and the image input. Edge extracting means 2 for extracting edge points from the image captured by means 1, image dividing means 3 for dividing the image of the image input means into a plurality of partial images (blocks), and edges existing in each block Line segment extracting means 4 for detecting a plurality of line segments passing through many points, and the line segments having a difference between the line segment inclination and the end point position within a predetermined range as one set, It comprises a feature extraction means 5 for extracting the end point position as a feature, a road marking detection means 6 for detecting road marking based on the feature obtained by the feature extraction means, and a control means 7 for controlling the respective means. .

  FIG. 2 is a flowchart showing the flow of processing according to the first embodiment, and the operation will be described with reference to FIG.

First, the white line detection process will be described.
In step S11 of FIG. 2, the control means 7 instructs the image input means 1 to obtain image information. Here, it is assumed that the image of FIG. 3 is obtained.

  In step S12, the control unit 7 sends the input image obtained by the image input unit 1 to the edge extraction unit 2. The edge extraction means 2 performs edge extraction from the obtained image. The edge extraction process can be performed by using a SOBEL operator which is a general method.

  FIG. 4 is an example of an edge image generated by the edge extraction unit 2 from the input image of FIG. In the edge extraction process, in addition to the outline of the white line, a shadow change part or a curb included in the background is extracted as an edge.

  In step S13, the control unit 7 sends the edge image obtained by the edge extracting unit 2 to the image dividing unit 3. The image dividing unit 3 divides the input edge image into partial images.

  FIG. 5 shows an example in which the edge image of FIG. 4 is divided into partial images by the image dividing means 3, which is divided into two parts in the vertical direction and four parts in the horizontal direction, and is divided into a total of eight partial images. One white line is approximated by a line segment divided in the horizontal direction.

Next, proceeding to step S14, the control means 7 performs a process of extracting features of white line candidates by the feature extraction means 5 for all the partial images divided in the vertical direction shown in FIGS. 6 (a) and 6 (b). It is determined whether or not. If feature extraction processing has been performed on all partial images, the process proceeds to step S18. If not, the process proceeds to step S15. Here, since the feature extraction process is not performed, the process proceeds to step S15.
First, a process of extracting white line candidate features by the feature extraction unit 5 using the partial image shown in FIG.

  In step S15, the control means 7 sends the partial images (blocks 1 to 4) to the line segment extraction means 4, and the line segment extraction means 4 extracts the line segments. In the line segment extraction process, two arbitrary edge points existing in the block are selected, and the number of other edge points located on the line segment connecting them is counted. This is performed for all combinations, and the top few line segments with a large count are extracted.

  FIG. 7 shows an example in which the line segment extraction unit 4 extracts line segments from the partial image of FIG. In FIG. 7, k1 to k13 mean line segments in each block, and each line segment has end points (s1 to s13, e1 to e13) corresponding to the line segment numbers.

Next, in step S16, the control means 7 instructs the feature extraction means 5 to generate white line candidates from the line segments obtained by the line segment extraction means 4. The generation of the white line candidates is performed by grouping the line segments based on the slopes of the line segments obtained by the line segment extracting unit 4 and the position coordinates of the end points. Specifically, when the inclination of the line segment k and a _k, the end point s _k, and e _k, performs grouping is determined that the line segment constituted those satisfying the formula 1, 2 from the same straight line , White line candidates.

  Note that when determining grouping from other than the adjacent line segment, the coordinate value at the end point position to be compared is calculated from the slope of the line segment.

  FIG. 8 is an image example in which white line candidates are generated from FIG. In FIG. 8, L1 to L7 indicate white line candidates, and even if the white line is a broken line, a distant line segment can be recognized as one white line candidate without being affected by the discontinuous portion.

  In step S17, the control unit 7 instructs the feature extraction unit 5 to perform processing. The feature extraction unit 5 determines the inclination of the white line candidates obtained in step S16 and the cameras of the white line candidates (image input unit). 1) The end point position close to the side (the lower side of the image) is extracted as a feature.

  FIG. 9 is an example showing inclinations p1 to p7 of white line candidates, which are features of the white line candidates L1 to L7 in FIG. 8, and end point positions t1 to t7 close to the camera (image input means 1) side.

  Next, returning to step S14, the processing from step S15 to step S17 is performed on the partial image shown in FIG. FIG. 10 shows an example in which the white line candidate inclinations p8 to p11 and the end point positions t8 to t11 close to the camera (image input means 1) are extracted from FIG. 6B. As described above, since the process of extracting white line candidate features has been performed on all the partial images divided in the vertical direction, the process proceeds to step S18.

  In step S18, the control means 7 sends the features obtained by the feature extraction means 5 to the road marking detection means 6, and the road marking detection means 6 detects white lines and pedestrian crossings.

  FIG. 11 is a flowchart of white line and pedestrian crossing detection by the road marking detection means 6, and its operation will be described with reference to FIG.

First, in step S601, the control means 7 determines whether or not processing for detecting a white line has been performed on all partial images divided in the vertical direction. If white line detection processing is performed for all partial images, the process proceeds to step S603. If not, the process proceeds to step S602. Here, since the white line detection process is not performed on all the partial images, the process proceeds to step S602.
First, processing for detecting a white line using the characteristics of the white line candidate shown in FIG. 9 will be described.

  In step S602, the control unit 7 instructs the white line detection unit of the road marking detection unit 6 to perform processing, and the white line detection unit detects a white line. The white line detection unit sets the following conditions and detects a white line.

Conditions for detecting white lines (Condition 1)
When the coordinates of the end point t _i close to the camera (image input unit 1) side of the white line candidate obtained by the feature extraction unit 5 is (x _i , y _i ), t _i is located closer to the center of the image Select as white line. For example, it can be performed by using the evaluation function shown in Expression 3.

(Condition 2)
Since the input image data changes smoothly, the end point position of the white line obtained by the previous white line detection and the end point position of the current white line candidate are substantially equal. Specifically, the current time is m, the endpoint coordinates of the current white line candidate are (x _i (m), y _i (m)), and the endpoint coordinates of the previous detection result are (x _i (m-1), When y _i (m−1)), a line that satisfies Equation 4 is selected as a white line.

(Condition 3)
A threshold relating to the width of the white line is provided, and a white line whose endpoint coordinate distance between white line candidates satisfies a predetermined threshold is selected. Specifically, the one that satisfies Equation 5 is selected as a white line.

FIG. 12 shows an example in which a white line is detected from the features of the white line candidate in FIG. 9 using the above conditions. FIG. 13 shows an example in which a white line is detected from the characteristics of the white line candidate in FIG.
As described above, since the white line detection process has been performed on all the partial images divided in the vertical direction, the process proceeds to step S603.
In this embodiment, the partial image divided into two in the vertical direction is used. This is performed in order to suppress the number of combinations of white line candidates when the white line is detected by the white line detection unit. Even when the image is not divided in the direction, white line detection can be performed.

  In step S603, the control unit 7 causes the road marking detection unit 6 to intersect the white line candidates (L1 to L11) obtained by the feature extraction unit 5 with a preset reference line as shown in FIG. A and B) are requested. Next, the road marking detection means 6 determines whether or not the distance between the intersections satisfies a certain ratio. If the certain ratio is satisfied, the process proceeds to step S604, and if not, the detection process is terminated. Here, the pedestrian crossing is not included in the image, the distance between the intersections does not satisfy a certain ratio with the distance between the other intersections, the detection process is terminated, and it is determined that the white line is detected.

Next, pedestrian crossing detection processing will be described.
In step S11 of FIG. 2, the control means 7 instructs the image input means 1 to obtain an image. Here, it is assumed that an image having a pedestrian crossing in the image shown in FIG. 15 is obtained. The acquired image is subjected to the processing from step S12 to step S17 in the same manner as the white line detection processing by the image input means 1, and the characteristics of the white line candidate are extracted.

  FIG. 16 shows an example in which the features of white line candidates are extracted from FIG. 15. The feature extraction unit 5 uses the inclination of the white line candidates indicated by L12 to L21 in the figure and the camera (image input unit 1) indicated by the mark ●. ) The end points t12 to t21 close to the side are extracted as features.

  Next, it progresses to step S18, and the control means 7 instruct | indicates a process to the road marking detection means 6, and the road marking detection means 6 detects a white line and a pedestrian crossing. This detection process will be described with reference to FIG. Here, the processing of the white line detection unit in the road marking detection unit 6 in step S602 is the same as the processing described in the white line detection processing described above, and thus the description thereof is omitted.

  Next, the control means 7 proceeds to step S603, as shown in FIG. 17, the intersection (A to H) between the white line candidates (L12 to L21) obtained by the feature extraction means 5 and the preset reference line. Is instructed to the road marking detection means 6. Next, the road marking detection means 6 determines whether or not the distance between the intersections satisfies a certain ratio. If the certain ratio is satisfied, the process proceeds to step S604, and if not, the detection process is terminated. Here, since the pedestrian crossing is included in the image and the distance between the intersections satisfies a certain ratio, the process proceeds to step S604.

Next, in step S604, the control means 7 instructs the road marking detection means 6 to calculate the intersection of white line candidates that satisfy the certain ratio obtained in step S603. The intersection point (x _vp , y _vp ) can be calculated by Equations 6 and 7, where α _i and q _i are the slope and intercept of the white line candidate. Since the white lines constituting the pedestrian crossing are three-dimensionally parallel, the image data obtained by projecting the two-dimensionally crosses one point at the vanishing point. _Whether or not this vanishing point is formed is determined by providing a threshold value in a range that the intersection (x _vp , y _vp ) can take. If the above condition is satisfied, the process proceeds to step S605. If no vanishing point is formed, the detection process is terminated. Here, as shown in FIG. 18, since the white line candidate forms a vanishing point, the process proceeds to step S605.

  Next, in step S605, the control means 7 detects the upper and lower lines of the pedestrian crossing (white lines located before and behind the pedestrian crossing that is drawn across the white line parallel to the vehicle traveling direction). In addition, the road marking detection means 6 is instructed to determine the approximate positions of the upper and lower ends of the pedestrian crossing. For determining the position, as shown in FIG. 19, first, a scanning line is generated in the direction of the extension line of the white line candidate, centering on the midpoint of each white line candidate line. Next, the luminance value variance w (j) is calculated in the horizontal direction along the scanning line. Since the variance value is large in the white line portion and small in the road surface portion, the change point is set as the upper and lower end points of the pedestrian crossing.

  FIG. 20 shows an example of upper and lower end points of the pedestrian crossing obtained in step S605.

  Next, in step S606, the control means 7 issues an instruction to the road marking detection means 6 for determining the search area for obtaining the vertical line of the pedestrian crossing based on the upper and lower end points obtained in step S605. In the search area determination process, for example, first, a straight line is calculated for each of the upper and lower end points obtained in step S605 using the least square estimation method. Next, when the calculated straight line is y = ax + b, the region satisfying Expression 8 can be set as a search region. However, v and ν in Equation 8 are threshold values for setting the processing range.

  FIG. 21 is an example in which the search areas of the pedestrian crossing vertical line obtained in step S606 are set as search area 1 and search area 2, respectively. Next, in step S607, the vertical line of the pedestrian crossing is obtained.

  In step S <b> 607, the control unit 7 instructs the road marking detection unit 6 to detect the straight line that passes through the most edge points in the search area as the upper and lower lines of the pedestrian crossing. In the process of detecting the vertical line of the pedestrian crossing, two arbitrary edge points in each search area are selected, and the number of other edge points located on the line segment connecting them is counted. This is done in all combinations, and the line segment with the largest count is taken as the vertical line of the pedestrian crossing.

  FIG. 22 is an example showing the vertical lines of the pedestrian crossing obtained in step S607.

  The first embodiment has been described above. In the present embodiment, the road marking detection unit 6 determines the presence or absence of a pedestrian crossing by using the processing of step S603 and step S604. As another example of the road marking detection unit 6, FIG. As shown, it may be changed to one having the pedestrian crossing presence / absence determination processing function of step S701 instead of the processing of step S603 and step S604. In the present embodiment, the pedestrian crossing presence / absence determination processing step S701 of the road surface marking detection means 6 determines whether or not there is a pedestrian crossing by the following procedure. As shown in FIG. 24A, the extension line direction of the white line candidate generated in step S16 in FIG. 2 is set as a scanning line, and an edge distribution located on the scanning line is obtained. FIG. 24B shows an example of the distribution of edges located on the scanning line. Since the edges constituting the pedestrian crossing are distributed at specific positions on the scanning lines of the white line candidates, the dispersion relating to the edge positions on the scanning lines increases. By using this dispersion value as an index, it is possible to determine a pedestrian crossing.

  In the present embodiment, the road marking detection means 6 determines whether or not there is a pedestrian crossing by using the processing of step S603 and step S604. However, the white line width and the road white line constituting the pedestrian crossing are different. By providing a threshold value regarding the white line width, it may be changed or added to the process for determining whether or not a pedestrian crossing exists.

Further, in the present embodiment, the road marking detection unit 6 determines the presence or absence of a pedestrian crossing by using the processing of step S603 and step S604. As still another example of the road marking detection unit 6, FIG. It may have a function of adding a crosswalk presence / absence determination process in step S702 as shown in the flowchart of FIG. In step S702, the presence / absence of a pedestrian crossing is determined based on whether the number of white line candidates generated in step S16 is a predetermined number or more. Since a pedestrian crossing is composed of a predetermined number or more of white line candidates, it can be performed by providing a threshold value regarding the number of white line candidates.
If such a road marking detection means 6 is used, the accuracy of the determination process of the presence or absence of a pedestrian crossing is further improved.

  In the present embodiment, the edge extraction processing of the input image in the edge extraction means 2 may be performed by a method other than the step SOBEL operator, for example, using an edge extraction filter such as a Canny operator.

  Further, in the present embodiment, the line segment extraction processing in the line segment extraction unit 4 may be performed using a technique such as Hough transform or least square method, which is one of the line detection algorithms.

  As described above, according to the present embodiment, the feature extraction unit 5 that extracts the features of the white line candidates constituting the white line and the pedestrian crossing line is provided, and the white line and the white line based on the feature extracted by the feature extraction unit 5 are provided. Since the detection is made by paying attention to the difference in the structure of the pedestrian crossing, a database is unnecessary and the detection can be performed at high speed.

Embodiment 2.
The second embodiment of the present invention will be described below with reference to FIGS. The second embodiment is the same as the first embodiment shown in FIG. 1 except for the configuration and function of the road marking detection means 6, and the schematic configuration is the same as that of the first embodiment shown in FIG. This is the same as the flowchart shown in FIG. FIG. 26 is a flowchart of the road marking detection means 6 in the second embodiment.
In the figure, the same or corresponding parts as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

The operation of the second embodiment will be described with reference to the flowchart shown in FIG.
The control means 7 first instructs the image input means 1 and obtains an image in the same procedure as in the first embodiment. Here, it is assumed that the image of FIG. 27 is obtained. As in the first embodiment, the obtained image is subjected to the processing from step S12 to step S17 to extract the features of white line candidates.

  FIG. 28 is an example in which the features of white line candidates are extracted from the obtained image of FIG. 27, and the feature extraction means 5 uses the camera (image) indicated by the inclinations of the white line candidates indicated by L22 to L29 in the figure and the ● marks. The end points t22 to t29 close to the input means 1) side are extracted as features.

  Next, the control means 7 instructs the road marking detection means 6 to perform processing, and the road marking detection means 6 detects white lines and pedestrian crossings. The flow of operations for detecting white lines and pedestrian crossings will be described with reference to the flowchart of FIG. Note that the white line detection in steps S601 and S602 in the road marking detection means 6 is the same as that in the first embodiment, and a description thereof will be omitted.

  In step S603, the control means 7, as shown in FIG. 29, intersects (A, B, C, D) between the white line candidates (L22 to L29) obtained by the feature extraction means 5 and a preset reference line. Is instructed to the road marking detection means 6. It is determined whether or not the distance between the intersections satisfies a certain ratio. If the certain ratio is satisfied, the process proceeds to step S604, and if not, the process proceeds to step S703. Here, the white lines that make up the pedestrian crossing were not partially detected by blurring, so the ratio of the distances between the intersections did not become a constant ratio. The process proceeds to step S703.

  Next, in step S703, the road surface marking detection means 6 estimates an intersection that has not been detected by fading or the like only from the detected intersection, and performs a crosswalk presence / absence determination process. Specifically, the distance between each intersection is calculated from the detected intersections (A, B, C, D), and using the calculated distances AB, BC, CD, whether the expressions 9 and 10 are satisfied? Can be done with no. When the conditions of Expressions 9 and 10 are not satisfied, the detection process is terminated. Here, the condition is satisfied, and the process proceeds to step S604.

  Next, in step S604, the control means 7 gives an instruction for determining whether or not the white line candidate obtained in step S703 forms a vanishing point to the road marking detection means 6. Here, since the white line candidate forms a vanishing point, the process proceeds to step S605. If no vanishing point is formed, the detection process ends.

  Next, in step S605, the control means 7 instructs the road marking detection means 6 to determine the approximate positions of the upper and lower pedestrian crossings in order to detect the upper and lower lines of the pedestrian crossing, and the road marking detection means 6 Extracts the top and bottom points of the pedestrian crossing.

  Next, in step S606, the control means 7 issues an instruction to the road marking detection means 6 to determine the search area for obtaining the vertical line of the pedestrian crossing based on the upper and lower end points obtained in step S605. The detection means 6 determines a search area.

  In step S607, the control means 7 instructs the road marking detection means 6 to detect the straight line passing through the most edge points in the search area as the vertical line of the pedestrian crossing, and the road marking detection means 6 displays the vertical line of the pedestrian crossing. To detect.

  As described above, according to the present embodiment, in addition to the effect of the first embodiment, the road marking detection means 6 is provided with a process for re-determining the presence or absence of a pedestrian crossing. Detection can be performed even when the part is faint or concealed.

  Since the present invention can detect the road markings of the white line and the pedestrian crossing line at high speed without using a database, it is applied to a navigation apparatus and used together with a detection signal of the vehicle position from a GPS (Global Positioning System). Thus, the detection accuracy of the position of the own vehicle can be increased.

It is a block diagram which shows Embodiment 1 of this invention. 4 is a flowchart showing a flow of processing in the first embodiment. It is a figure which shows the image information acquired by the image input means. It is a figure which shows the edge image produced | generated from the input image. It is a figure which shows the partial image divided | segmented by the image division means. It is a figure which shows the partial image divided | segmented to the orthogonal | vertical direction separately to (a) (b). FIG. 6A is a diagram showing a line segment extracted image extracted from the partial image. It is a figure which shows the example of an image of the white line candidate produced | generated from the line segment extraction image. It is a figure which shows the inclination of a white line candidate, and the end point position near the camera (image input means) side. FIG. 6B is a diagram showing the inclination of the white line candidate generated from the partial image and the end point position close to the camera (image input means) side. It is a flowchart which shows the flow of a process of a road marking detection means. It is a figure which shows the example which detected the white line from the characteristic of the white line candidate of FIG. It is a figure which shows the example which detected the white line from the characteristic of the white line candidate of FIG. It is a figure which shows the intersection of a white line candidate and a reference line. It is a figure which shows the example of the input image in which a pedestrian crossing exists. It is a figure which shows the example which extracted the feature of the white line candidate from the input image in which a pedestrian crossing exists. It is a figure which shows the intersection of the white line candidate from the input image in which a crosswalk exists, and the preset reference line. It is a figure for demonstrating the vanishing point of a white line candidate. It is explanatory drawing for the upper-lower end detection of a pedestrian crossing. It is a figure which shows the example of the upper-lower-end point of a pedestrian crossing. It is a figure which shows the search area | region of a pedestrian crossing vertical line. It is a figure which shows the up-down line of a pedestrian crossing. It is a flowchart which shows the flow of the process by the other Example of a road marking detection means. It is explanatory drawing for discrimination | determination of the presence or absence of a pedestrian crossing by the other Example of a road marking detection means. It is a flowchart which shows the flow of the process by the further another Example of a road marking detection means. 10 is a flowchart showing a flow of processing of road marking detection means in the second embodiment. FIG. 11 is a diagram showing an input image in the second embodiment. FIG. 10 is a diagram illustrating a feature extraction example of white line candidates in the second embodiment. It is a figure which shows the intersection of the white line candidate in Embodiment 2, and a reference line.

Explanation of symbols

  1: image input means, 2: edge extraction means, 3: image segmentation means, 4: line segment extraction means, 5: feature extraction means, 6: road marking detection means, 7: control means.

Claims (3)

  An image input means for inputting image data in front of the traveling vehicle, an edge extraction means for extracting edge points from the image captured by the image input means, and an image of the image input means is divided into a plurality of partial images. Image segmentation means, line segment extraction means for extracting line segments passing through edge points existing in each partial image, and feature for extracting the slope and end point position of the line segment obtained by the line segment extraction means as features When the end points of the line segment obtained by the extraction means and the feature extraction means exist at regular intervals in the horizontal direction and the line segments form one vanishing point, it is determined that there is a pedestrian crossing ahead. An on-vehicle road marking detection device comprising road marking detection means for performing the above-mentioned.   An image input means for inputting image data in front of the traveling vehicle, an edge extraction means for extracting edge points from the image captured by the image input means, and an image of the image input means is divided into a plurality of partial images. Image segmentation means, line segment extraction means for extracting line segments passing through edge points present in each partial image, and the inclination and end point position of the line segment obtained by the line segment extraction means are detected, And a feature extraction means for extracting the slopes and end point positions of the line segments as a set of line segments in which the difference between the inclination and the end point position is within a predetermined range, and a set in the feature extraction means. An in-vehicle road marking detection device comprising road marking detection means for determining that a pedestrian crossing exists ahead when a predetermined number or more of line segments are obtained. The road marking detection means determines a rough position of the upper and lower ends of a plurality of line segments determined to be a pedestrian crossing or a plurality of line segments in a set, and any one of the divided partial images is used as a search area. 3. The configuration according to claim 1, wherein the line is determined and detected as a straight line that passes through the edge points in the search region most frequently as a vertical line before and after the white line that forms the pedestrian crossing . In-vehicle road marking detection device.

Images