KR101362595B1 - Method and system for lane detection - Google Patents

Abstract

The present invention relates to a lane detection method and a lane detection system capable of detecting each lane in a color video including a plurality of color lanes such as white, yellow and blue lanes. The lane detection method is a lane detection method for detecting lanes in a color video including a plurality of color lanes, comprising the steps of: generating a black-and-white video by making an input color video black-and-white; extracting edges from the black-and-white video; sampling specific pixels from the color video based on the edges; detecting the color of a lane via the sampled pixels; determining a coefficient corresponding to the detected color of the lane; generating the emphasis video of the color video or the black-and-white video using the determined coefficient; and detecting the lane from the emphasis video. [Reference numerals] (S101) Make a color video black-and-white; (S103) Sample specific color pixels; (S104) Detect the color of a lane; (S11) Generate an emphasis video by applying a predetermined weight to the detected color of the lane; (S12,S102) Extract edges; (S13) Generate information on the lane

Description

Lane Recognition Method and System {METHOD AND SYSTEM FOR LANE DETECTION}

The present invention relates to a lane recognition method and system, and more particularly, to a lane recognition method and system that can recognize each lane well in a color image containing a plurality of color lanes, such as white, yellow, blue.

The lane recognition method is a technology for recognizing a lane in an image photographed from a front camera of a vehicle. Such a lane recognition method or system may be mounted on a vehicle safety driving assistance device such as a lane departure warning device or a lane keeping assistance device or a driver assistance system to recognize a lane.

An example of a lane recognition method is disclosed in Korean Laid-Open Patent Publication No. 10-2004-0017644. This publication binarizes the input image, scans the binarized image at regular intervals, extracts the edges, compares the data on the points on the extracted edges, distinguishes the white and blue lanes, After the road modeling using the equation, the lane detection method to confirm the final lane through the road verification is disclosed.

As described above, most conventional lane recognition methods generate a grayscale image by an average of each channel of a color input image, extract edges from the grayscale image, and recognize the lane using the extracted edges.

However, when lane recognition is performed using a grayscale image, it is difficult to distinguish each lane separately from a color input image including lanes of different colors. This is because the edge of the white lane is detected well in the grayscale image, but the edge of the yellow lane or the blue lane is not detected well. In particular, in a situation in which the lane is obscured or contaminated, there is a disadvantage in that the difference in brightness of the road surface is not large, so that the edge extraction of the yellow lane and the blue lane fails to be extracted or the recognition distance is shortened because the recognized edge length is short.

Republic of Korea Patent Publication No. 10-2004-0017644 (2004.02.27)

The present invention has been derived in view of the above-described problems of the prior art, and an object of the present invention is a lane recognition method and system capable of effectively recognizing each lane in a color image containing multiple color lanes such as white, yellow, and blue. To provide.

It is also an object of the present invention to provide a lane recognition method and system that can be efficiently used for lane detection in a lane departure warning device or lane keeping assistance device that can precisely assist in lane keeping or an alarm upon lane departure. will be.

It is also an object of the present invention to provide a computer readable medium recording a program for performing a lane recognition method for effectively recognizing each lane in a color image including a plurality of lanes of white, yellow, and blue. .

In order to solve the above technical problem, a lane recognition method according to an aspect of the present invention is a lane recognition method for recognizing a lane in a color image containing a plurality of color lanes, and generating a black and white image by blackening the input color image. step; Extracting edges from the black and white image; Sampling a particular pixel of the color image based on the edge; Recognizing lane colors from the sampled pixels; Determining a coefficient matching the color of the recognized lane; Generating an emphasis image of a color image or a black and white image using the determined coefficients; And recognizing a lane in the highlighted image.

Preferably, the lane recognition method may further include removing noise from the black-and-white image before extracting the edges.

Preferably, the step of recognizing the color of the lane may include a Hue Satruation Value (HSV), a Hue Satruation Lightness (HSL), and a Hue Satruation Brightness (HSB) representing the sampled color pixels as points of a red green and blue (RGB) color model. The method may include determining a color of the lane based on at least one color model or a color space of HSI (Hue Satruation Intensity).

The generating of the emphasis image may include generating the emphasis image by multiplying a weight with respect to each color pixel value of the color image input through the plurality of channels.

Preferably, the step of generating an emphasis image comprises generating the emphasis image by multiplying different weights or tuning parameters for each color pixel (R, G, B) of the color image when the color image is RGB. Can be.

According to another aspect of the present invention, there is provided a method of recognizing a lane, comprising: generating a first emphasis image of an input color image; Extracting an edge from the first emphasis image; Estimating a lane using an edge of the first emphasis image; Recognizing the color of the estimated lane; Generating a second emphasis image for the color image or the first emphasis image by applying only a weight of the recognized color; And outputting lane information of the recognized lane based on the second emphasis image.

Preferably, the generating of the first emphasis image may include generating a first emphasis image by multiplying a preset weight for each color pixel value of the color image input through the plurality of channels.

Preferably, the generating of the second emphasis image may include generating a second emphasis image by multiplying a preset weight only with respect to the color pixel value determined as the color of the lane among the color pixel values of the color image or the first emphasis image. Can be.

The lane recognition system according to an aspect of the present invention is a lane recognition system for recognizing a lane in a color image from a front camera of a vehicle, and recognizing a lane primarily from a color image and detecting a lane color from the primarily recognized lane. Recognizing preprocessor; An emphasis image generator for generating a weight image by applying a preset weight to the recognized lane color; An edge extractor for extracting edges in the highlighted image; And a lane information generation unit generating lane information using edges.

Preferably, the pre-processing unit, the monochrome image generating unit for generating a monochrome image by black and white color image; An edge extractor for extracting edges from the black and white image; A pixel sampling unit for sampling a specific color pixel in a color image based on location information of a lane estimated based on an edge; And a lane color recognition unit recognizing a color of the lane based on the sampled pixels.

Preferably, the preprocessing unit comprises: a first emphasis image generating unit generating a first emphasis image by multiplying a predetermined weight for each color pixel value of the color image; An edge extractor extracting an edge from the first emphasis image; A lane estimator for estimating a lane using an edge; And a lane color recognition unit recognizing the estimated lane color. In this case, the emphasis image generator corresponds to the second emphasis image generator that generates the second emphasis image, and the color determined as the lane color among the color pixels of the color image or the first emphasis image when the second emphasis image is generated. The second emphasis image may be generated by multiplying a predetermined weight only with respect to the pixel value.

A lane recognition system according to an aspect of the present invention includes a lane recognition system provided in a driving assistance system or a driver assistance system including a lane departure warning device and a lane keeping assistance device, including: a memory; And a processor that executes a program stored in the memory. Here, the processor first recognizes the lane from the color image input by the program, recognizes the lane color from the primarily recognized lane, generates a weighted image by applying a preset weight to the recognized lane color, and In addition, the image processing of the highlighted image is performed to finally recognize the lane.

A computer readable medium according to an aspect of the present invention includes a computer readable medium having recorded thereon a program for performing the lane recognition method of any one of the above-described preferred examples.

According to the present invention, it is possible to provide a lane recognition method and system capable of effectively recognizing each lane in a color image including a plurality of color lanes such as white, yellow, and blue.

According to the present invention, there is provided a lane recognition method and system that can be efficiently used for lane detection in a lane departure warning device or a lane keeping assistance device that can precisely assist lane keeping or an alarm upon lane departure. Can be.

In addition, according to the present invention, a computer-readable medium having recorded thereon a program for performing a lane recognition method capable of recognizing each lane well in a color image containing a plurality of color lanes such as white, yellow, and blue can be provided. have.

1 is a schematic flowchart of a lane recognition method according to an embodiment of the present invention.
2 to 4 are photographs for explaining the operation and effect of the lane recognition method of FIG.
5 is a schematic flowchart of a lane recognizing method according to another exemplary embodiment of the present invention.
6 is a schematic block diagram of a lane recognition system according to an exemplary embodiment.
FIG. 7 is a schematic block diagram of an embodiment of a preprocessor that may be employed in the lane recognition system of FIG. 6.
FIG. 8 is a schematic block diagram of another exemplary embodiment of the preprocessor that may be employed in the lane recognition system of FIG. 6.
FIG. 9 is a schematic block diagram illustrating a lane departure warning apparatus that may employ the lane recognition system of FIG. 6.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic flowchart of a lane recognition method according to an embodiment of the present invention.

Referring to FIG. 1, in the lane recognizing method according to the present exemplary embodiment, an emphasis image for transforming an input color image into a gray scale image for extracting edges is applied so that an edge of a lane having color is well represented. And use these highlighted images to effectively recognize lanes.

To this end, the lane recognition method first generates a black and white image by black and white the color image input from the front camera of the vehicle (S101). As a technique for converting a color image to generate a black and white image, a binarization method or an averaging method may be used. The averaging method refers to a method of generating a black and white image as an average of each channel of a color image of a predetermined format, such as RGB (Red Green Blue) and YUV. Here, the YUV format is a color representation format in which a color is represented by three pieces of information: a luminance signal Y, a difference signal between a luminance signal and a blue component, and a difference signal V between a luminance signal and a red component.

Noise may be removed from the image before or after generating the black and white image or before extracting the edges below.

Next, an edge is extracted from the black and white image (S102). The pixel (color pixel) of a specific color of the color image is sampled based on the edge (S103).

Next, the lane color is recognized from the sampled color pixels (S104). The process of recognizing the color of the lane includes the Hue Satruation Value (HSV), Hue Satruation Lightness (HSL), Hue Satruation Brightness (HSB), and HSI, which represent the sampled color pixels as points in a red green and blue (RGB) color model. Hue Satruation Intensity) may be performed to determine the color of the lane based on at least one color model or color space. For example, the color of the lane may be recognized by applying a reference value tuned to the color characteristics of the HSV.

In the present embodiment, the above-described steps S101 to S104 may correspond to a preprocessing step S10 for recognizing lane colors prior to lane recognition.

Next, an emphasis image is generated by applying a preset weight to the recognized lane color (S11). The generation of the weighted image may be performed by multiplying a weight wi for each color pixel value Vi of a color image input through i (i is one or more natural numbers) channels. This can be expressed by the following equation (1).

Here, emG is an emphasis image and wi is a parameter used for tuning as a weight.

For example, when the color image is RGB, an emphasis image may be generated by multiplying different weights or tuning parameters w1, w2, and w3 with respect to each color pixel R, G, and B of the color image. Here, the tuning parameters w1, w2, w3 may be 0.1, 0.4, 0.5 in the order described. This is represented by the following equation (2).

Next, an edge is extracted from the highlighted image (S12). Edge extraction may be performed using any one of various existing image processing techniques for extracting a region of interest from an image. However, even though the conventional edge extraction technique is used, the emphasis image according to the present embodiment is processed to facilitate edge extraction, and thus more points and farther points can be extracted than the conventional edge extraction results.

Next, the lane is recognized from the highlighted image using the edge and the recognized lane information is output (S13).

2 to 4 are photographs for explaining the operation and effect of the lane recognition method of FIG.

FIG. 2 illustrates a color input image 20 photographed by a front camera of a vehicle and input to a lane recognition system. FIG. 3 illustrates a result of edge extraction from an emphasis image employed in the lane recognition method of the present embodiment. As a comparative example, edge extraction results from grayscale images employed in a conventional general lane recognition method are shown.

3 and 4, it can be seen that the edge on the yellow lane in the edge extraction result image 21 according to the present embodiment is much higher than the edge extraction result image 22 of the comparative example. In addition, in the edge extraction result image 21 according to the present embodiment, it may be confirmed that the edge extraction of the lane that is farther than that of the edge extraction result image 22 of the comparative example is detected. As described above, when comparing the edge extraction result of the present embodiment with the edge extraction result of the comparative example, it can be seen that the lane recognition accuracy performance of the present embodiment is improved and the lane recognition distance performance is increased compared to the conventional method.

According to the present embodiment, edge detection is easily performed by modifying an input color image into a grayscale image for edge extraction to create an emphasis image for representing an edge of a lane having a color such as yellow or blue in addition to white. can do.

5 is a schematic flowchart of a lane recognizing method according to another exemplary embodiment of the present invention.

Referring to FIG. 5, the lane recognition method according to the present embodiment generates a first emphasis image by modifying a color input image, recognizes a lane color based on the first emphasis image, and the edge of the lane of a specific color The second highlighted image is emphasized by emphasizing a specific color pixel or edge in the image corresponding to the previously recognized lane color so as to be well represented, and effectively recognizing the lane using the second highlighted image.

To this end, the lane recognition method first converts a color image input from a front camera of the vehicle to generate a first emphasis image (S501). The generation of the first emphasis image may be performed by multiplying a preset weight for each color pixel value of the color image input through the plurality of channels.

Next, an edge is extracted from the first emphasis image (S502). The lane is estimated using the edge of the first emphasis image (S503). The estimation of the lane refers to primarily detecting the lane using an edge extracted from the first emphasis image.

Next, the color of the estimated lane is recognized (S504). The color recognition may be performed by sampling pixels in the estimated lane area and applying a tuned reference value to a specific color characteristic (HSV, etc.).

In the present exemplary embodiment, the above-described steps S501 to S504 may correspond to a preprocessing step S50 for recognizing lane colors prior to lane recognition.

Next, a second emphasis image for the color image or the first emphasis image is generated by applying only the weight of the recognized color (S51). The weight is preferably a tuning parameter preset for each of the different colors. For example, the tuning parameters may be set as shown in Table 1 below.

Colorpixel value weight 00000001 0.0039 00000010 0.0078 00000011 0.0156 : : 11111110 0.9961 11111111 1.0000

In the present embodiment, the weight may be applied in the form of applying only the weight of the color previously recognized in Equation 1 or Equation 2.

Next, an edge is extracted from the second emphasis image (S52). Then, the lane is recognized using the edge and lane information on the recognition lane is generated (S53).

6 is a schematic block diagram of a lane recognition system according to an exemplary embodiment.

Referring to FIG. 6, the lane recognition system 60 according to the present embodiment converts a color image input from a front camera of a vehicle into a black and white image, extracts an edge from the black and white image, and extracts lane and lane colors using the extracted edge. After generating the highlighted image with different weights according to the recognized and recognized lane colors, the processed image is processed to generate lane information, or the lane is primarily recognized from the color image input from the front camera of the vehicle. After recognizing the lane color from the lane recognized as a pre-processing unit 61 to generate a highlighted image by applying a predetermined weight to the recognized lane color, and to process the image to highlight the final image to output the lane information, the preprocessor 61 And an emphasis image generator 62, an edge extractor 63, and a lane information generator 64.

The preprocessor 61 is primarily a means for recognizing the lane and confirming the lane color, or a component that performs a function corresponding to the lane. The structure and the function which can be employ | adopted for the preprocessing part 61 are demonstrated in detail below.

The emphasis image generator 62 is a means for generating an emphasis image weighted to a specific lane according to the lane color recognized by the preprocessor 61 or a component that performs a function corresponding to the means. The emphasis image generating unit 62 may generate a second emphasis image by converting a color image, a black and white image, or a first emphasis image based on Equation 1 or Equation 2 described above.

The edge extractor 63 extracts an edge from the highlighted image, and an existing edge extraction technique may be used.

The lane information generation unit 64 is a means for recognizing a lane using the edge extracted by the edge extraction unit 63 and generating or outputting information on the recognized lane, or a component that performs a function corresponding to the lane. to be. Similar to the edge extractor 63, the lane information generation unit 64 may use an existing technique of recognizing a lane using an edge.

FIG. 7 is a schematic block diagram of an embodiment of a preprocessor that may be employed in the lane recognition system of FIG. 6.

Referring to FIG. 7, the preprocessor 61 according to the present embodiment extracts an edge from a black and white image and recognizes a lane using the extracted edge, and then recognizes a color of the recognized lane. 611, an edge extractor 612, a pixel sampling unit 613, and a lane color recognition unit 614.

The black and white image generator 611 converts the input color image to generate a black and white image. The edge extractor 612 extracts an edge from the black and white image. The edge extractor 612 may be configured by the same means as the edge extractor that extracts an edge from the highlighted image or a component that performs a function corresponding to the means.

The pixel sampling unit 613 samples the pixel on the edge extracted from the black and white image to determine the color of the lane.

The lane color recognizer 614 recognizes a color by applying a reference value preset for a specific color characteristic (HSV, etc.) to a point (pixel) sampled by the pixel sampler 614. The value of the sampled point (color pixel value) recognized by the preset reference value may correspond to a specific parameter or weight tuned for each color. The color recognized through the above-described process is used to generate an emphasis image from a color image or a black and white image.

FIG. 8 is a schematic block diagram of another exemplary embodiment of the preprocessor that may be employed in the lane recognition system of FIG. 6.

Referring to FIG. 8, the preprocessor 61 according to the present exemplary embodiment first recognizes a lane in a color image input from a front camera of a vehicle and recognizes a lane color from a primarily recognized lane. A weighted image generator 615, an edge extractor 616, a lane estimator 617, and a lane color recognizer 618 are provided.

The first emphasis image generator 615 generates a first emphasis image by applying a weight to an input color image according to equation (1). The edge extractor 616 extracts an edge from the first emphasis image. The lane estimator 617 primarily recognizes a lane using an edge extracted by the edge extractor 616. The lane color recognizer 618 samples pixels in the lane estimated by the lane estimator 617, and recognizes colors by applying a reference value tuned to the color characteristics of the sampled pixels. The color recognized through the above-described process is used to generate the second emphasis image from the color image or the first emphasis image.

According to the above-described embodiments, the edge of the white lane is well detected when using a general grayscale image, but the edge detection of the yellow lane or the blue lane is not well performed, that is, the problem of lowering the recognition rate of the yellow or blue lane compared to the white lane. Can be surely improved. In other words, if the lane color is detected as white, yellow, blue, etc., additional performance can be achieved by creating and using a second-deformed highlight image that is specialized for the color recognized in the preprocessing so that the edges of all detected lanes can be detected well. It can lead to improvement.

FIG. 9 is a schematic block diagram illustrating a lane departure warning apparatus that may employ the lane recognition system of FIG. 6.

Referring to FIG. 9, a lane departure warning system (LDWS) 100 including a lane recognition system 60 according to the present exemplary embodiment may be obtained through a road image acquisition means such as a camera 102. Recognizing the lane through the lane recognition system 60 in the image and determining lane departure or lane keeping of the vehicle based on the recognized lane, and then warns the driver through the alarm device 104 or controls the steering of the vehicle in anticipation of lane departure. A device (not shown) is operated to prevent lane departure of the vehicle.

The lane departure warning device 100 may be implemented with at least one electronic control module mounted in a vehicle, and may be connected to the camera 102 or the camera module and the alarm device 104 through a controller area network (CAN) communication. .

The lane recognizing system 60 corresponds to the lane recognizing system implementing the lane recognizing method of the present embodiment described above with reference to FIGS. 1 to 5.

The lane recognition system 60 may be mounted in an electronic control module constituting a driving assistance system or a driver assistance system such as a lane departure warning device and a lane keeping assistance device. For example, the lane recognition system 60 may include a memory (not shown) and a processor (not shown) that executes a program stored in the memory. In this case, the processor primarily recognizes the lane in the color image input from the front camera of the vehicle by a program stored in the memory and similarly detects the lane color from the lane that is primarily recognized, similarly to the lane recognition method of the present embodiment. Recognizing and generating a weighted image by applying a predetermined weight to the recognized lane color, and processing the image of the highlighted image to finally output the lane information.

According to the present exemplary embodiment, the lane departure warning apparatus displays a color image including at least two color lanes among a plurality of color lanes such as white, yellow, and blue through the lane recognition system 60 implementing the lane detection method described above. Robust lane edge detection can be implemented, thereby accurately recognizing lanes even in a bad situation where lanes are covered by shadows or dust, and effectively performing preset scenarios such as lane departure warning output and steering control. Can be. That is, the reliability of the vehicle safety driving assistance device and the driver assistance system can be improved. In addition, the vehicle can provide a lane departure warning device or lane keeping assistance system that can be effectively used in driverless cars, such as a car recognizes the lane and drives the car by itself.

Meanwhile, the present invention including the above-described lane recognition method may be implemented as a computer-readable medium including program instructions for performing operations implemented in various types of computer devices. Such media may be implemented alone or in combination with program instructions, data files, data structures, and the like. The medium may also be a hard disk, a floppy disk, a magnetic tape, a hard disk, an optical recording medium such as a CD or a DVD, a magnetic-optical medium and hardware configured to store program instructions such as ROM, Device. ≪ / RTI > Further, the medium may be embodied as a transmission medium such as an optical fiber, a metal wire, and a waveguide for transmitting a signal designating a program command, a data structure, and the like. Program instructions include machine language code such as those generated by the compiler, high-level language code that can be executed by a computer using an interpreter, and the like.

It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims . It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

Claims (13)

delete delete delete delete Generating a first emphasis image of the input color image;
Extracting an edge from the first emphasis image;
Estimating a lane using an edge of the first emphasis image;
Recognizing a color of the estimated lane;
Generating a second emphasis image on the color image by applying only a weight of the recognized color; And
Outputting lane information of the recognized lane based on the second emphasis image
, ≪ / RTI &
The generating of the first emphasis image may include generating a first emphasis image by multiplying a predetermined weight value for each color pixel value of the color image input through a plurality of channels.
The generating of the second emphasis image may include generating a second emphasis image by multiplying a predetermined weight only with respect to the color pixel value determined as the color of the lane among the color pixel values of the color image or the first emphasis image. Lane recognition method comprising a.
delete delete In a lane recognition system for recognizing a lane in a color image from a front camera of a vehicle,
A preprocessing unit that first recognizes a lane from a color image and recognizes a lane color from the first recognized lane;
An emphasis image generator configured to generate a weight image by applying a preset weight to the recognized lane color;
An edge extracting unit extracting an edge to the highlighted image; And
A lane information generation unit for generating lane information using the edge,
The pre-
A first emphasis image generator configured to generate a first emphasis image by multiplying a predetermined weight for each color pixel value of the color image;
An edge extractor extracting an edge from the first emphasis image;
A lane estimator estimating a lane using the edge; And
A lane color recognition unit recognizing a color of the estimated lane;
The emphasis image generating unit corresponds to a second emphasis image generating unit generating a second emphasis image, and the color pixel determined as the color of the lane among the color pixels of the color image or the first emphasis image when the second emphasis image is generated. And a second weighted image is generated by multiplying a predetermined weight only with respect to the value.
delete delete delete delete A computer-readable medium having recorded thereon a program for performing the lane detection method of claim 5.

Images