Disclosure of Invention
In view of the above, the present invention provides a method for assisting vehicle navigation and positioning based on road marking line visual identification, which uses monocular visual information to extract any point on a road line and a deflection angle of the road line, and can be used for assisting vehicle navigation and positioning.
The invention provides a method for assisting vehicle navigation and positioning based on road marking line visual identification, which comprises the following steps:
s1, acquiring a color image of a road in which a vehicle is running in real time, and processing the color image to obtain a gray image;
further, the specific method of step S1 is as follows:
s11, obtaining a color image of the road on which the vehicle is running by utilizing the camera;
s12, the color image is processed into a gray scale image, and the gray scale value of an arbitrary point P (x, y) in the pixel array of m rows and n columns of the image captured by the camera is represented as g (x, y).
S2, carrying out binarization processing on the gray level image to obtain a binarized image;
further, the specific method of step S2 is as follows:
s21, solving according to the inner and outer orientation elements of the camera to obtain a projection point of a parallel line parallel to the main longitudinal line direction in the image, namely a main point of coincidence;
s22, determining the lower half part of the image by taking the main joint obtained in the step S21 as a reference;
s23, regarding any point P (x, y) in the lower half of the image determined in step S22, the gray scale value of the point is recorded as g (x, y), and when the gray scale value of the point satisfies the following condition:
in the formula
L is the projection width, i.e. the set of the width of the road mark line projected on the photo, and 0.025 times of the number of image columns is taken as L
maxE is an additional normal number, preferably e-5, D is the contrast threshold obtained by the lighting conditions,
the gray scale value of the point P (x, y) is represented by 255, and if the gray scale value is not satisfied, the gray scale value is 0, so that the image formed in the way is a binary image;
and S24, determining the part of the front hood in the image according to the relative position of the front hood and the camera for the binary image obtained in the step S23, and rejecting the part of the image.
S3, denoising the binary image to obtain a road line image, wherein each road line corresponds to a rectangle;
further, the specific method of step S3 is as follows:
s31, detecting each image block of the binary image obtained in the step S2 and extracting the boundary of each image block;
s32, calculating the area of the image blocks according to the extracted boundary, wherein the image blocks smaller than the critical value are regarded as noise and filtered, and the critical value S ismin0.00015 m · n, wherein m is the number of pixel rows corresponding to the road color image, and n is the number of pixel columns corresponding to the road color image;
s33, regarding the adjacent white pixel points as points on the same image block, wrapping the image block by a minimum rectangle, and processing by replacing the image block with the rectangle;
s34, regarding a straight line passing through the middle point of the rectangle and along the long side direction as a projection of the center line of the road, the algorithm identifies as the road line the segment replaced by the rectangle satisfying the following two conditions:
(1) the inclination angle of the rectangular central line is between 10 and 70 degrees (or between 70 and 10 degrees) or vertical and is positioned at the center of the image;
(2) the rectangular aspect ratio is greater than 2.
S4, extracting the center line of the rectangle in the step S3 to obtain a road marking line;
further, the calculation formula of the rectangular center line in step S4 is:
y=tanα(x-x0)+y0
where α is the angle between the long side of the rectangle and the x-axis, x0、y0Is the center point of the rectangle.
And S5, determining the lane where the vehicle is located according to the road marking line extracted in the step S4 by combining the vehicle position information provided by the vehicle-mounted GNSS and the road information in the corresponding map database.
Further, the specific method of step S5 is as follows:
s51, extracting road information of corresponding positions from a map database through vehicle coordinate information provided by a vehicle-mounted GNSS, wherein the road information comprises the number of lanes of a road and the quasi-driving direction information of each lane;
and S52, comparing the road information in the step S51 according to the road marking line extracted in the step S4, determining the specific lane information of the vehicle, and assisting the navigation and positioning of the vehicle.
The technical scheme provided by the invention has the beneficial effects that: the method utilizes the consumption-level automobile data recorder image to carry out the algorithm of road route extraction in real time, has simple model, easy realization and high operation efficiency, and is suitable for assisting the vehicle to carry out real-time navigation positioning data processing; the algorithm adopts a dynamic threshold value to perform image processing, so that the method has a good road route extraction effect even in an environment with poor illumination conditions, and finally realizes the lane-level navigation and positioning service for the running vehicle.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be further described with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the invention selects the pictures shot by the consumption-level automobile data recorder in the process of driving the vehicle on the highway, the flow of the invention is shown in figure 1, and the invention comprises the following steps:
s1, acquiring a color image of a road in which a vehicle is running in real time by using a consumer-grade automobile data recorder, and processing the color image to obtain a gray image;
this is because the gray scale map can simultaneously recognize images under different illumination conditions in the day and night, and the gray scale value of the processed gray scale image is represented as g (x, y) for any point P (x, y) in the pixel array of n rows and m columns of the image captured by the drive recorder as shown in fig. 2.
S2, carrying out binarization processing on the gray level image to obtain a binarized image;
and solving to obtain projection joint points of parallel lines parallel to the main longitudinal line direction in the image, namely main joint points according to the inner and outer orientation elements of the camera. Determining the lower half part of the image by taking the main joint as a reference;
the reason for this is that, for an image perpendicular to the ground plane, the principal point of convergence is the image principal point, and the projections of all the points on the ground should be below this point, so it can be seen that the projections of all the road lines on the ground are in the lower half of the image.
The lane where the vehicle is traveling is usually dark (small gray scale value) and the road line on the road surface is usually white or yellow (large gray scale value), so that any line satisfying the condition of darker sides and lighter middle may be a road line. The threshold condition for binarization is thus determined: the grayscale value of any point P (x, y) in the lower half of the image is denoted as g (x, y), and when the grayscale value of the point satisfies the following condition:
in the formula
L is the projection width, i.e. the set of the width of the road marking line projected on the photo, and 0.025 times of the number of image columns is taken as L after test
maxWhere ε is an additional normal number, preferably tested as 5, D is the contrast threshold obtained by the light conditions,
it should be noted that, 5 is adopted as the minimum limit of D, which aims to ensure that a region with poor lighting conditions does not generate a large amount of noise;
the gradation value of the point is represented by 255, otherwise the gradation value thereof is represented by 0, thereby outputting a binarized image.
It should be noted that, when a single image is processed, it is found that the reflection occurs at the bonnet, and since the reflection brightness is too high and is related to the weather at that time, it is difficult to remove the reflection effectively by the denoising means. Therefore, the binary image is processed by determining the position of the hood in the image according to the relative position of the hood and the camera, and skipping the image processing of the position, the effect is shown in fig. 3.
S3, denoising the binary image to obtain a road line image, wherein each road line corresponds to a rectangle;
for the binarized image obtained in step S2, detecting each block of image block and extracting its boundary;
usually, the noise exists in the form of small spots, and the road path is a long and regular pattern and is approximately rectangular. Calculating the area of the image blocks according to the extracted boundary, wherein the image blocks smaller than a critical value are regarded as noise and filtered, and the critical value SminM is 0.00015. m.n, m is the number of pixel rows corresponding to the road color image, and n is the number of columns of pixels corresponding to the road color image;
the road surface is shaded by trees or railings beside the road, and the road line is also shaded by the shading, so that black dots exist in white blocks of the binary image of the road line, and the direction interpretation of the road line is not facilitated. Therefore, it is necessary to extract the boundary nodes of the image blocks, regard the adjacent white pixel points as the points on the same image block, wrap the image block with a minimum rectangle, and use this rectangle to replace the image block for processing, as shown in fig. 4, the white image block is the binarized image block, the white rectangle is the minimum rectangle that can contain the whole image block, and the rectangle is used to replace the image block, so that a plurality of geometric parameters of the image block can be accurately and conveniently obtained;
taking a straight line passing through the middle point of the rectangle and along the long side direction as a projection of the center line of the road route, it can be known from the basic principle of center projection that the projection of the road route is not horizontal, and only the road line at the center of the image is vertical, as shown in fig. 5. Therefore, a line having an inclination angle between 10 and 70 degrees (or between 70 and 10 degrees) and a line perpendicular and at the center of the image are identified as the lane lines;
it should be noted that the length of the image block is limited, and the road line has a slender characteristic and can be distinguished from the noise such as puddles, road signs and the like, so the aspect ratio of the rectangle representing the road line should be larger than 2;
the image after the denoising process is the road line image extracted in this embodiment, and as shown in fig. 6, the comparison between the road extraction image and the original gray scale value image can find that all the road lines in the driver's sight line range have been accurately extracted.
S4, extracting the center line of the rectangle in the step S3 to obtain a road marking line;
because the precision demand of vehicle navigation location does not reach centimetre level, consequently can ignore the width of lane, regard it as the straight line that does not have the width with it, can directly calculate the rectangle central line as road line projection image expression:
y=tanα(x-x0)+y0
where α is the angle between the long side of the rectangle and the x-axis, x0、y0Is the center point of the rectangle, as shown in fig. 7, the relationship between the mathematical expression of the road line and the corresponding rectangle is shown schematically.
In the embodiment, videos recorded by the automobile data recorders under different road conditions and different illumination conditions are used as research data for testing, the different resolutions of the videos recorded by the automobile data recorders of different brands are considered, and in order to ensure that data extracted from different videos have comparability during testing, the resolutions of the videos are unified to 320 × 480 and the number of frames per second of the videos is 29.97.
In this embodiment, three common adverse conditions in daily driving are selected and the road marking line extraction test is performed by using the above steps:
(1) the image of the road route extraction process during driving at night is shown in fig. 8, the current driving lane is a 4-lane road, the driving direction is two lanes, 3 lanes of the road route exist, the road condition is good, and a large amount of vehicles do not block the road route. In the figure, the upper left corner is a gray scale image obtained after a video picture shot by a driving recorder is processed, the lower left corner is a binary image, the lower right corner is road line extraction information after processing is finished, and the upper right corner is comparison between an extracted road line and an original image. It can be seen that 3 road routes are extracted by the algorithm, all the road routes within the shooting range of the current camera are extracted, the equation parameters of the extracted road routes are shown in fig. 9, and the directions of the road routes are represented by the central points and the inclination angles of the road routes;
(2) an extracted image of a certain overhead driving time road route is shown in fig. 10, the current intersection is wide, and the ground marking line is dark in color due to abrasion. It can be seen that the algorithm extracts the road lines, but the road lines of the far lanes cannot be identified, and the identification of the road lines is difficult for the areas with darker colors or more serious wear due to the limitation of the algorithm binarization method;
(3) the road route extraction image when a road is driving is shown in fig. 11, and the current road route is clear but has vehicle interference. It can be seen that the algorithm extracts 3 road routes on the right side, and the vehicle is not mistakenly judged as a road route due to vehicle interference.
And S5, determining the lane where the vehicle is located according to the road marking line extracted in the step S4 by combining the vehicle position information provided by the vehicle-mounted GNSS and the road information in the corresponding map database.
Specifically, according to vehicle coordinate information provided by the vehicle-mounted GNSS, road information of corresponding positions is extracted from a map database, wherein the road information comprises the number of lanes of a road and quasi-driving direction information of each lane. According to the road marking lines extracted in step S4, the road information is compared, as shown in fig. 12, an exemplary diagram of specific lane information where the vehicle is located is determined during night driving, and finally the determined specific lane information is used to assist vehicle navigation and positioning.
In the embodiment, the road surface road line identification under different conditions is tested, so that the method can adapt to the illumination conditions in the day and at night, has higher resistance to the interference of factors such as vehicles and the like, and the road line extraction of the method is interfered when the road line is shallow or characters with the same color as the road line are printed on the ground.
The embodiment provides the algorithm for extracting the road lines by utilizing the monocular video of the automobile data recorder in the driving process of the vehicle, the algorithm is quick and effective, the real-time resolving capability is realized, and the quantitative information of the road marking lines extracted by the algorithm can be applied to data fusion positioning resolving with GNSS observation signals in subsequent work.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention.