CN112435293B - Method and device for determining structural parameter representation of lane line - Google Patents
Method and device for determining structural parameter representation of lane line Download PDFInfo
- Publication number
- CN112435293B CN112435293B CN201910787011.1A CN201910787011A CN112435293B CN 112435293 B CN112435293 B CN 112435293B CN 201910787011 A CN201910787011 A CN 201910787011A CN 112435293 B CN112435293 B CN 112435293B
- Authority
- CN
- China
- Prior art keywords
- lane line
- parameter representation
- determining
- structural parameter
- representation
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Processing Or Creating Images (AREA)
Abstract
Disclosed are a method, an apparatus, a computer-readable storage medium, and an electronic device for determining a structured parameter representation of a lane line, the method comprising: acquiring semantic information carried by pixel points in at least one frame of image; determining pixel coordinates of a reference point of the lane line corresponding to the semantic information; acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image; acquiring an initial structural parameter representation of the lane line according to the first space coordinate set; and determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image and the semantic information. The method and the device have the advantages that the plurality of sampling points corresponding to the lane lines in the high-precision map are replaced by the structural parameter representation of the lane lines, so that the storage pressure and the access pressure of the high-precision map are effectively reduced.
Description
Technical Field
The disclosure relates to the technical field of image analysis, and more particularly, to a method and a device for determining structural parameter representation of lane lines.
Background
The lane lines are important components in the road scene, are indispensable elements in the high-precision map, and accurate lane line representation in the high-precision map is a precondition for realizing automatic driving.
At present, a laser radar is often used for scanning a lane line, a point cloud of the lane line is obtained, then sampling points of the lane line are extracted from the point cloud, the sampling points are used for representing the lane line in a high-precision map, and a large number of sampling points can cause larger storage pressure and access pressure when the high-precision map is used, so that it is important to determine a lightweight lane line representation method.
Disclosure of Invention
The present disclosure has been made in order to solve the above technical problems. Embodiments of the present disclosure provide a method, an apparatus, a computer-readable storage medium, and an electronic device for determining a structured parameter representation of a lane line, which replace a plurality of sampling points corresponding to the lane line in a high-precision map with the structured parameter representation of the lane line, so as to effectively reduce storage pressure and access pressure using the high-precision map.
According to a first aspect of the present disclosure, there is provided a method for determining a structured parametric representation of a lane line, comprising:
acquiring semantic information carried by pixel points in at least one frame of image;
determining pixel coordinates of a reference point of the lane line corresponding to the semantic information;
Acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image;
acquiring an initial structural parameter representation of the lane line according to the first space coordinate set;
And determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image and the semantic information.
According to a second aspect of the present disclosure, there is provided a structured parametric representation determination apparatus of lane lines, comprising:
The semantic information acquisition module is used for acquiring semantic information carried by pixel points in at least one frame of image;
And the pixel coordinate determining module is used for determining the pixel coordinate of the reference point of the lane line corresponding to the semantic information.
The space coordinate acquisition module is used for acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image;
The first parameter representation module is used for acquiring initial structural parameter representation of the lane line according to the first space coordinate set;
and the second parameter representation module is used for determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image and the semantic information.
According to a third aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the above-described structured parameter representation of lane lines determination method.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising:
A processor;
A memory for storing the processor-executable instructions;
The processor is used for reading the executable instructions from the memory and executing the instructions to realize the method for determining the structured parameter representation of the lane line.
Compared with the prior art, the method and the device for determining the structural parameter representation of the lane line, the computer-readable storage medium and the electronic equipment have the following beneficial effects:
On the one hand, the embodiment considers that the fact that the high-precision map is used for representing the lane lines by utilizing a plurality of sampling points leads to larger storage pressure and access pressure when the high-precision map is used, so that the structural parameter representation of the lane lines is further obtained according to the space coordinate set by determining that semantic information in the image is the space coordinate set corresponding to the reference points of the lane lines, and the obtained structural parameter representation is optimized to determine the structural parameter representation capable of accurately representing the lane lines in the high-precision map, so that the structural parameter representation is utilized to replace the plurality of sampling points, the lightweight representation of the lane lines in the high-precision map is realized, and the storage pressure and the access pressure when the high-precision map is used are effectively reduced.
On the other hand, the embodiment utilizes the vision sensor to collect the image, so as to obtain the structural parameter representation of the lane line, avoid utilizing the laser radar with high price, and effectively save the cost.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing embodiments thereof in more detail with reference to the accompanying drawings. The accompanying drawings are included to provide a further understanding of embodiments of the disclosure, and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure, without limitation to the disclosure. In the drawings, like reference numerals generally refer to like parts or steps.
FIG. 1 is a flow chart of a method for determining a structured parametric representation of lane lines provided by an exemplary embodiment of the present disclosure;
FIG. 2 is a scene graph of a method of determining a structured parametric representation of lane lines provided by an exemplary embodiment of the present disclosure;
FIG. 3 is a flow chart illustrating step 30 of a method for determining a structured parametric representation of lane lines according to an exemplary embodiment of the present disclosure;
FIG. 4 is a flow chart of step 40 of a method for determining a structured parametric representation of lane lines according to an exemplary embodiment of the present disclosure;
FIG. 5 is a flow chart of step 50 of a method for determining a structured parametric representation of lane lines according to an exemplary embodiment of the present disclosure;
FIG. 6 is a flow chart illustrating a step 503 of a method for determining a structured parameter representation of a lane line according to an exemplary embodiment of the present disclosure;
FIG. 7 is a flow chart of steps 40 and 50 of a method for determining a structured parametric representation of lane lines according to an exemplary embodiment of the present disclosure;
FIG. 8 is a flowchart illustrating a method for determining a structural parameter representation of a lane line according to an exemplary embodiment of the present disclosure, wherein the method further includes step 503;
FIG. 9 is a schematic structural view of a lane line structured parameter representation determining apparatus provided in a first exemplary embodiment of the present disclosure;
FIG. 10 is a schematic structural view of a lane line structured parameter representation determining apparatus provided in a second exemplary embodiment of the present disclosure;
FIG. 11 is a schematic diagram of a first second parameter representation unit in a lane line structured parameter representation determining apparatus according to an exemplary embodiment of the present disclosure;
FIG. 12 is a schematic structural view of a first parameter representation module in a lane line structured parameter representation determining apparatus according to an exemplary embodiment of the present disclosure;
FIG. 13 is a schematic diagram of a second type of second parameter representation unit in a lane line structured parameter representation determining apparatus according to an exemplary embodiment of the present disclosure;
FIG. 14 is a schematic diagram of a third second parameter representation unit in a lane line structured parameter representation determining apparatus according to an exemplary embodiment of the present disclosure;
fig. 15 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some of the embodiments of the present disclosure and not all of the embodiments of the present disclosure, and that the present disclosure is not limited by the example embodiments described herein.
Summary of the application
The lane line is important information for ensuring safe running of the vehicle, is an important component part in a road scene, and is an accurate lane line representation as a precondition when automatic driving is realized by using a high-precision map. At present, a laser radar scanning mode is adopted to obtain point clouds of lane lines, a large amount of redundant data exists in the obtained point cloud data, sampling points capable of representing accurate positions of the lane lines are required to be extracted from the point cloud data, the extracted sampling points are utilized to represent the lane lines in a high-precision map, the high-precision map contains a large number of lane lines, and therefore a large number of sampling points exist in the high-precision map, and accordingly large storage pressure and access pressure exist when the high-precision map is used.
According to the method for determining the structured parameter representation of the lane line, the semantic information in the image is determined to be the space coordinate set corresponding to the reference point of the lane line, the structured parameter representation of the lane line is further obtained according to the space coordinate set, the obtained structured parameter representation is optimized to determine the structured parameter representation capable of accurately representing the lane line in the high-precision map, the structured parameter representation can represent the lane line in the high-precision map by using a small amount of data, so that a plurality of sampling points in the high-precision map are replaced, the lightweight representation of the lane line in the high-precision map is realized, and the storage pressure and the access pressure of the high-precision map are effectively reduced. Moreover, the embodiment utilizes the vision sensor to collect the image, so that the structural parameter representation of the lane line is obtained, the utilization of the laser radar with high price is avoided, and the cost is effectively saved.
Exemplary method
Fig. 1 is a flow chart of a method for determining a structured parameter representation of a lane line according to an exemplary embodiment of the present disclosure.
The embodiment can be applied to electronic equipment, and particularly can be applied to a server or a general computer. As shown in fig. 1, the method for determining the structural parameter representation of the lane line according to an exemplary embodiment of the present disclosure at least includes the following steps:
And step 10, acquiring semantic information carried by pixel points in at least one frame of image.
When the vision sensor mounted on the vehicle is used for collecting images, a series of images, namely at least one frame of images, are obtained, and after the images are obtained, the images are subjected to semantic segmentation, namely different objects in the images are segmented from the angle of pixels according to the content of the images, so that semantic information carried by pixel points in the images is determined.
When the vehicle a is running on the road surface as shown in fig. 2, the vision sensor mounted on the vehicle collects images in real time, so in one possible implementation manner, the image in at least one frame of image may correspond to the current frame of image collected by the vision sensor at the current moment, that is, when the vision sensor collects one frame of current frame of image, the semantic information carried by the pixels in the current frame of image is obtained, and as the vision sensor continuously collects images, the current frame of image continuously changes, so that the semantic information carried by the pixels in at least one frame of image may also be obtained. There is of course another possible implementation manner, that is, a series of images have been acquired in advance, so that semantic information carried by pixels in one or more frames of images may be acquired at a time.
And step 20, determining pixel coordinates of the reference point of the lane line corresponding to the semantic information.
In the running process of the vehicle, the visual field of the visual sensor not only comprises a lane line, but also other objects (such as pedestrians, vehicles, sky and the like) exist in the visual field of the visual sensor, so that various objects also exist in the image acquired by the visual sensor, the embodiment needs to determine the structural parameter representation of the lane line, so that the semantic information is required to be determined to be a reference point of the lane line, and the pixel coordinates of the reference point are required to be determined, thereby not only ensuring the accuracy of the structural parameter representation of the lane line obtained in the subsequent process, but also avoiding bringing all the pixel points into the subsequent operation and improving the working efficiency of the method.
And step 30, acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to at least one frame of image.
The pixel coordinates of the reference points correspond to the position information of the lane lines in the image, so that after the pixel coordinates corresponding to the reference points are obtained, the first space coordinates corresponding to the pixel coordinates are determined according to the camera pose corresponding to each frame of image, and the first space coordinates are formed into a first space coordinate set by each first space coordinate. Specifically, the camera pose corresponding to each frame of image can be provided by a positioning module, such as a satellite positioning system, an inertial measurement unit and the like.
And step 40, acquiring an initial structural parameter representation of the lane line according to the first space coordinate set.
The first space coordinates in the first space coordinate set represent the position information of the lane line in the three-dimensional world, and the lane line in the three-dimensional world is parameterized according to each first space coordinate in the first space coordinate set, namely, the geometric shape of the lane line is defined by a small amount of parameters, so that the initial structured parameter representation of the lane line is obtained.
And 50, determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose and semantic information corresponding to at least one frame of image.
Because the initial structural parameter representation may not accurately represent the lane line, the initial structural parameter representation is optimized by using the camera pose corresponding to each frame image and the semantic information carried by the pixel points in each frame image, so as to determine the optimized structural parameter representation of the lane line.
The method for determining the structural parameter representation of the lane line has the advantages that:
On the one hand, in the embodiment, the fact that the lane lines are represented by a plurality of sampling points in the high-precision map causes larger storage pressure and access pressure when the high-precision map is used, so that the initial structural parameter representation of the lane lines is further obtained according to a first space coordinate set corresponding to the reference points of the lane lines by determining the semantic information in the image, the obtained initial structural parameter representation is optimized to determine the optimized structural parameter representation capable of accurately representing the lane lines in the high-precision map, the optimized structural parameter representation can accurately represent the lane lines in the high-precision map by using a small amount of data, and accordingly the plurality of sampling points in the high-precision map are replaced, the lightweight representation of the lane lines in the high-precision map is realized, and the storage pressure and the access pressure when the high-precision map is used are effectively reduced.
On the other hand, the embodiment utilizes the vision sensor to collect the image, so as to obtain the structural parameter representation of the lane line, avoid utilizing the laser radar with high price, and effectively save the cost.
Fig. 3 is a schematic flow chart of acquiring a first spatial coordinate set corresponding to pixel coordinates according to the pixel coordinates and the camera pose corresponding to at least one frame of image in the embodiment shown in fig. 1.
As shown in fig. 3, in an exemplary embodiment of the present application based on the embodiment shown in fig. 1, the step 30 of obtaining the first spatial coordinate set corresponding to the pixel coordinate may specifically include the following steps:
In step 301, inverse perspective transformation is performed on the pixel coordinates, so as to obtain sixth spatial coordinates corresponding to the pixel coordinates.
The inverse perspective transformation is a technology for transforming a two-dimensional plane image into a three-dimensional space, and a sixth space coordinate corresponding to a pixel coordinate in the two-dimensional plane image in the three-dimensional space can be obtained through the inverse perspective transformation, wherein the sixth space coordinate indicates position information of a reference point of a lane line in the three-dimensional space, and is basically the same as a first space coordinate in a first space coordinate set, and the sixth space coordinate is used only for convenience of distinguishing.
Step 302, determining a tracking code of the lane line corresponding to the sixth space coordinate in at least one frame of image.
When the vehicle is traveling on the road surface, as shown in fig. 2, there is often more than one lane in the visual field of the vision sensor, and in order to accurately determine various corresponding structural parameter representations of the lane lines, the sixth space coordinates need to be divided according to the lane lines. When each lane line appears in the image for the first time, a tracking code (track id) with a unique identification is allocated to the lane line, and when the lane line appears in the image for the second time, the tracking code is unchanged, so that when the pixel coordinates corresponding to the reference points of the lane line are determined, the tracking code corresponding to the lane line can be obtained, and information transmission is carried out on the tracking code in the process of carrying out inverse perspective transformation on the pixel coordinates to obtain sixth space coordinates, so that each sixth space coordinate corresponds to the tracking code of the lane line.
In step 303, the corresponding sixth spatial coordinates with the same tracking code are formed into the first spatial coordinate set.
And carrying out cluster fitting on the sixth space coordinates according to the tracking codes, forming a first space coordinate set by using the sixth space coordinates with the same tracking codes, and adding the sixth space coordinates with different tracking codes into different first space coordinate sets, namely, each lane line corresponds to the first space coordinate set.
In this embodiment, the sixth space coordinate corresponding to the pixel coordinate is obtained through inverse perspective transformation, and the first space coordinate set corresponding to each lane line is determined according to the tracking code corresponding to the sixth space coordinate, so that the first space coordinate in the first space coordinate set is guaranteed to belong to the same lane line, and further the accuracy of the structural parameter representation of each lane line obtained subsequently is guaranteed.
FIG. 4 is a flow chart illustrating the acquisition of an initial structural parametric representation of a lane line according to a first set of spatial coordinates in the embodiment shown in FIG. 1.
As shown in fig. 4, in an exemplary embodiment of the present application based on the embodiment shown in fig. 1, the step 40 of obtaining the initial structural parameter representation of the lane line may specifically include the following steps:
step 4011, randomly selecting at least four fifth spatial coordinates from the first set of spatial coordinates;
When the initial structural parameter representation of the lane line is obtained, because the first space coordinates in the first space coordinate set are more, each first space coordinate cannot be utilized to obtain the initial structural parameter representation of the lane line, and at least four fifth space coordinates need to be randomly selected from the first space coordinate set, wherein the fifth space coordinates and the first space coordinates are basically consistent only for distinguishing and conveniently using the fifth space coordinates.
Step 4012, according to the fifth space coordinate, an initial Bezier curve corresponding to the lane line is obtained.
And acquiring an initial Bezier curve corresponding to the lane line according to the acquired at least four fifth space coordinates. Specifically, if the initial bezier curve is a second-order bezier curve, the parameters of the initial bezier curve are expressed as follows: b (t) = (1-t)/(2×p0+2×t (1-t) ×p1+t×p2, where t is 0 to 1, P0, P2 is a starting point of the bezier curve determined based on the fifth spatial coordinate, and P1 corresponds to a control point of the bezier curve, and is used for controlling the shape of the bezier curve.
In this embodiment, at least four fifth space coordinates are selected from a larger number of first space coordinates, and a second-order bezier curve is used to represent the lane line according to the at least four fifth space coordinates, where the shape of the curve corresponding to the second-order bezier curve is close to the shape of the real lane line, so that the lane line can be represented more accurately by using the second-order bezier curve.
FIG. 5 shows a flow diagram of determining an optimized structural parametric representation of a lane line based on an initial structural parametric representation, camera pose and semantic information corresponding to each of at least one frame of images, in the embodiment shown in FIG. 1.
As shown in fig. 5, in an exemplary embodiment of the present application based on the embodiment shown in fig. 1, the determining the optimized structural parameter representation of the lane line shown in step 50 may specifically include the following steps:
Step 501, determining a second space coordinate corresponding to the lane line according to the initial structural parameter representation.
The initialization structure parameter representation corresponds to a lane line in a three-dimensional space, points are selected on the initialization structure parameter representation to determine second space coordinates corresponding to the lane line, and the accuracy of the initialization structure parameter representation is verified by using the second space coordinates. Specifically, the parameter represented by the initial structural parameter is represented by B (t) = (1-t)/(2×p0+2×t (1-t) ×p1+t×p2, and different values of t are selected to obtain different second spatial coordinates, such as 0, 0.1, 0.2, …,1 of t at equal intervals, so as to obtain points on the initial structural parameter representation, i.e. the second spatial coordinates.
Step 502, according to the second spatial coordinates and the camera pose corresponding to each of the at least one frame of image, the corresponding projection points of the second spatial coordinates in the at least one frame of image are obtained.
In order to verify the accuracy of the initial structural parameter representation by using the second space coordinates, the second space coordinates need to be projected into the images according to the respective corresponding camera pose of each frame of image so as to determine the projection points of the second space coordinates in each frame of image. Specifically, { p_sample } = K x Tcw { p_sample }, where { p_sample } represents the projection point, K represents the reference matrix of the camera, tcw represents the inverse matrix of the camera pose Twc, twc= [ R, t ] is provided by the positioning module, and { p_sample } represents the second spatial coordinate.
Step 503, determining the optimized structural parameter representation of the lane line according to the projection points and the semantic information.
And determining the optimized structural parameter representation of the lane line according to the projection points of the second space coordinates in each frame of image and the semantic information carried by the pixel points in each frame of image.
In this embodiment, since the initial structural parameter representation may not be able to better represent the lane line, optimization is required for the initial structural parameter, the second spatial coordinates are determined by selecting points in the initial structural parameter representation, and the initial structural parameter representation is optimized according to the projection points of the second spatial coordinates in each frame of image and the semantic information carried by the pixel points in each frame of image, so as to determine the optimized structural parameter representation, so that the obtained optimized structural parameter representation has higher accuracy, and can represent the lane line more accurately.
FIG. 6 shows a flow diagram of an optimized structured parametric representation for determining lane lines based on proxels and semantic information in the embodiment shown in FIG. 5.
As shown in fig. 6, in an exemplary embodiment of the present application based on the embodiment shown in fig. 5, the determining the optimized structural parameter representation of the lane line shown in step 503 may specifically include the following steps:
Step 5031, determining semantic information corresponding to the projection points according to the semantic information carried by the pixel points in at least one frame of image.
After the projection of the second space coordinate in each frame of image is performed to determine the corresponding projection point of the second space coordinate in each frame of image, different pixel points in the image carry different semantic information, so that the semantic information corresponding to the projection point needs to be determined first.
In step 5032, if the semantic information corresponding to the projection point is a lane line, the projection point is marked.
Different second space coordinates correspond to different projection points, and different projection points may correspond to different semantic information, but the initial structural parameter representation is used for representing the lane line, that is, the semantic information of the projection points corresponding to the second space coordinates in the image should all correspond to the lane line in theory, however, due to the accuracy problem of the initial structural parameter, the semantic information corresponding to some projection points is not the lane line, if the semantic information corresponding to the projection points is the lane line, the projection points are marked, so that the accuracy degree of the initial structural parameter representation is judged by using the marked projection points later.
Specifically, when the semantic information corresponding to the projection point is a lane line, determining that the projection point corresponds to a forward vote. Because there are two lane lines that are closer together, such as a double-yellow solid line, in one possible implementation, after determining that the semantic information corresponding to the projection point is a lane line, determining the tracking code of the lane line corresponding to the projection point, where the projection point corresponds to a positive vote if the tracking code of the lane line is consistent with the tracking code of the first set of spatial coordinates corresponding to the initial structural parameter representation.
Step 5033 determines a sum of the corresponding marked projection points of the initial structured parameter representation.
The statistical semantic information is the sum of projection points of the lane lines, the accuracy of the initial structural parameter representation is judged by the sum of the projection points, and the larger the sum of the projection points is, the more accurate the initial structural parameter representation can represent the lane lines.
In step 5034, if the sum of the marked projection points satisfies a first preset condition, the initial structural parameter representation is determined as an optimized structural parameter representation of the lane line.
Setting a first preset condition to be larger than a preset threshold, wherein the preset threshold is a threshold of the sum of projection points, and determining the initial structural parameter representation at the moment as the optimized structural parameter representation of the lane line only when the sum of the projection points is larger than the preset threshold.
In this embodiment, by marking the projection points of the lane lines corresponding to the semantic information, and counting the sum of the marked projection points, the sum of the projection points can be used to determine the accuracy of the initial structural parameter representation, and the greater the sum of the projection points, the more accurate the initial structural parameter representation is, and in order to select the accurate initial structural parameter representation, the first preset condition is set to be greater than the preset threshold, so that the initial structural parameter representation conforming to the first preset condition is determined as the optimized structural parameter representation, and the accuracy of the obtained optimized structural parameter representation is higher.
FIG. 7 is a flow chart illustrating the acquisition of an initial structural parametric representation of a lane line according to a first spatial coordinate in the embodiment shown in FIG. 6.
As shown in fig. 7, in an exemplary embodiment of the present application based on the embodiment shown in fig. 6, the step 40 of obtaining the initial structural parameter representation of the lane line may specifically include the following steps:
Step 4021, selecting a third set of spatial coordinates from the first set of spatial coordinates.
A third set of spatial coordinates is selected from the first set of spatial coordinates, the third set of spatial coordinates being part of the first set of spatial coordinates.
Step 4022, obtaining an initial structural parameter representation of the lane line according to the third set of spatial coordinates.
And acquiring an initial structural parameter representation of the lane line according to the selected third space coordinate set.
Based on the steps 4021 and 4022, as shown in fig. 7, after determining the sum of the corresponding marked projection points of the initial structural parameter representation in step 5033, the method specifically may further include the following steps:
in step 5035, if the sum of the marked projection points does not meet the first preset condition, a fourth set of spatial coordinates is selected from the first set of spatial coordinates.
The method includes that the sum of the projection points corresponding to the initial structural parameter representation is not met with a first preset condition, namely the sum of the projection points is smaller than or equal to a preset threshold value, and the initial structural parameter representation cannot accurately represent the lane line, so that a fourth space coordinate set needs to be selected again from the first space coordinate set, the fourth space coordinate set is a new third space coordinate set, and the corresponding space coordinates in the fourth space coordinate set and the third space coordinate set are different.
Step 5036 updates the initial structured parameter representation according to the fourth set of spatial coordinates.
And updating the initial structural parameter representation according to the re-selected fourth space coordinate set, namely re-determining a new initial structural parameter representation according to the fourth space coordinate in the fourth space coordinate set.
In this embodiment, since the initial structural parameter representation is determined by the third spatial coordinate set, when the sum of the corresponding marked projection points of the initial structural parameter representation does not meet the first preset condition, the accuracy of the initial structural parameter representation at this time is low, that is, the lane line cannot be accurately represented, so that the first spatial coordinate set is selected again, the fourth spatial coordinate set is determined to update the initial structural parameter representation, and step 50 is executed again according to the updated initial structural parameter representation, so that it is known that step 50 is repeatedly executed continuously to determine the optimized structural parameter representation, and the accuracy of the optimized structural parameter representation of the finally obtained lane line is ensured to be high.
Fig. 8 shows a schematic flow chart further included after selecting the fourth set of spatial coordinates from the first set of spatial coordinates in the embodiment shown in fig. 7.
As shown in fig. 8, in an exemplary embodiment of the present application based on the embodiment shown in fig. 7, after the fourth space coordinate set is selected from the first space coordinate set in step 5035, the method specifically may further include the following steps:
In step 5037, the number of selections corresponding to the fourth set of spatial coordinates is determined.
When the sum of the marked projection points does not meet the first preset condition, a fourth space coordinate set is selected from the first space coordinate set, the selection times of the fourth space coordinate set are recorded, the fourth space coordinate set is selected again, and the selection times are increased by one.
In step 5038, if the number of choices satisfies the second preset condition, determining the initial structural parameter representation that the sum of the marked projection points satisfies the third preset condition as the optimized structural parameter representation of the lane line.
Setting a second preset condition to reach the maximum selection times, wherein each time the fourth space coordinate set is selected, the maximum selection times are the maximum iteration times when the iteration process is performed, a plurality of iterations exist, but each time the sum of the corresponding marked projection points of the new initial structural parameter representation obtained according to the fourth space coordinate set still cannot meet the first preset condition, setting a second preset condition to prevent the method from falling into endless iteration, determining the initial structural parameter with the maximum sum of the projection points in all the initial structural parameter representations as the optimal structural parameter representation of the lane line after the maximum selection times are reached, setting a third preset condition to determine the maximum sum of the projection points, and the optimal structural parameter representation at the moment can relatively accurately represent the lane line.
In this embodiment, considering that there may be a case where the sum of projection points corresponding to the initial structural parameter representations obtained after the multiple iterations still does not satisfy the first preset condition, by setting the second preset condition, after the number of times of selection of the fourth space coordinate set reaches the maximum number of times of selection, determining the initial structural parameter representation with the maximum sum of projection points in all the initial structural parameter representations, and determining the initial structural parameter representation as the optimized structural parameter representation, where the obtained optimized structural parameter representation can relatively accurately represent the lane line, thereby avoiding the method from being involved in endless iterations.
Exemplary apparatus
Based on the same conception as the embodiment of the method, the embodiment of the application also provides a device for determining the structural parameter representation of the lane line.
Fig. 9 shows a schematic structural diagram of a lane line structured parameter representation determining apparatus according to an exemplary embodiment of the present application.
As shown in fig. 9, a device for determining a structural parameter representation of a lane line according to an exemplary embodiment of the present application includes:
the semantic information obtaining module 91 is configured to obtain semantic information carried by a pixel point in at least one frame of image;
the pixel coordinate determining module 92 is configured to determine pixel coordinates of a reference point of the lane line corresponding to the semantic information.
A space coordinate obtaining module 93, configured to obtain a first space coordinate set corresponding to the pixel coordinate according to the pixel coordinate and a pose of a camera corresponding to the at least one frame of image;
a first parameter representation module 94, configured to obtain an initial structural parameter representation of the lane line according to the first set of spatial coordinates;
a second parameter representation module 95, configured to determine an optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image, and the semantic information.
As shown in fig. 10, in one exemplary embodiment, the spatial coordinate acquisition module 93 includes: an inverse perspective transforming unit 931, a tracking code determining unit 932, a first obtaining unit 933;
An inverse perspective transformation unit 931, configured to perform inverse perspective transformation on the pixel coordinates, and obtain sixth spatial coordinates corresponding to the pixel coordinates;
a tracking code determining unit 932, configured to determine a tracking code of a lane line corresponding to the sixth spatial coordinate in at least one frame of image;
The first obtaining unit 933 is configured to compose a first set of spatial coordinates with the same corresponding tracking code.
As shown in fig. 10, in one exemplary embodiment, the first parameter representation module 94 includes: a first selection unit 9411 and a first parameter representation unit 9412;
A first selecting unit 9411, configured to randomly select at least four fifth spatial coordinates from the first set of spatial coordinates;
the first parameter representation unit 9412 is configured to obtain an initial bezier curve corresponding to the lane line according to the fifth spatial coordinate.
As shown in fig. 10, in one exemplary embodiment, the second parameter representation module 95 includes: a second acquisition unit 951, a projection point determination unit 952, and a second parameter representation unit 953;
A second obtaining unit 951, configured to determine a second spatial coordinate corresponding to the lane line according to the initial structural parameter representation;
the projection point determining unit 952 is configured to obtain, according to the second spatial coordinates and the respective camera pose corresponding to the at least one frame of image, the respective projection points of the second spatial coordinates in the at least one frame of image;
A second parameter representation unit 953 for determining an optimized structured parameter representation of the lane lines based on the projection points and the semantic information.
As shown in fig. 11, in one exemplary embodiment, the second parameter representation unit 953 includes: a first determination subunit 9531, a projected point marker subunit 9532, a second determination subunit 9533, a first parameter representation subunit 9534;
A first determining subunit 9531, configured to determine semantic information corresponding to the projection point according to semantic information carried by the pixel point in at least one frame of image;
a projection point marking subunit 9532, configured to mark a projection point if the semantic information corresponding to the projection point is a lane line;
a second determining subunit 9533, configured to determine a sum of the marked projection points corresponding to the initial structural parameter representation;
a first parameter representation subunit 9534, configured to determine the initial structured parameter representation as an optimized structured parameter representation of the lane line if the sum of the marked projection points satisfies a first preset condition.
As shown in fig. 12, in one exemplary embodiment, the first parameter representation module 94 includes: a second selection unit 9421 and a third parameter representation unit 9422;
a second selecting unit 9421, configured to select a third set of spatial coordinates from the first set of spatial coordinates;
a third parameter representation unit 9422, configured to obtain an initial structural parameter representation of the lane line according to the third set of spatial coordinates;
When the first parameter representation module 94 includes the second selection unit 9421 and the third parameter representation unit 9422, as shown in fig. 13, in an exemplary embodiment, the second parameter representation unit 953 further includes: a select subunit 9535, the parameter representing an update subunit 9536;
A selecting subunit 9535, configured to select a fourth set of spatial coordinates from the first set of spatial coordinates if the sum of the marked projection points does not meet the first preset condition;
A parameter representation updating subunit 9536, configured to update the initial structured parameter representation according to the fourth set of spatial coordinates.
As shown in fig. 14, in an exemplary embodiment, the second parameter representation unit 953 further includes: a third determination subunit 9537 and a second parameter representation subunit 9538;
a third determining subunit 9537, configured to determine a number of selections corresponding to the fourth set of spatial coordinates;
The second parameter representation subunit 9538 is configured to determine, as the optimized structural parameter representation of the lane line, an initial structural parameter representation in which the sum of the marked projection points satisfies the third preset condition if the number of choices satisfies the second preset condition.
Exemplary electronic device
Fig. 15 illustrates a block diagram of an electronic device according to an embodiment of the application.
As shown in fig. 15, the electronic device 100 includes one or more processors 101 and memory 102.
The processor 101 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities and may control other components in the electronic device 100 to perform desired functions.
Memory 102 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. Volatile memory can include, for example, random Access Memory (RAM) and/or cache memory (cache) and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on a computer readable storage medium and the processor 101 may execute the program instructions to implement the above method of determining a structured parametric representation of lane lines and/or other desired functions of various embodiments of the present application.
In one example, the electronic device 100 may further include: an input device 103 and an output device 104, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).
Of course, only some of the components of the electronic device 100 relevant to the present application are shown in fig. 15 for simplicity, components such as buses, input/output interfaces, and the like being omitted. In addition, the electronic device 100 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer readable storage Medium
In addition to the methods and apparatus described above, embodiments of the application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform the steps in a method of determining a structured parameter representation of a lane line according to various embodiments of the application described in the "exemplary methods" section of this specification.
The computer program product may write program code for performing operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the steps in a method for determining a structured parametric representation of a lane line according to various embodiments of the present application described in the above-mentioned "exemplary method" section of the present specification.
The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The basic principles of the present application have been described above in connection with specific embodiments, but it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not intended to be limiting, and these advantages, benefits, effects, etc. are not to be construed as necessarily possessed by the various embodiments of the application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not necessarily limited to practice with the above described specific details.
The block diagrams of the devices, apparatuses, devices, systems referred to in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.
It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent aspects of the present application.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.
Claims (10)
1. A method of structural parametric representation determination of lane lines, comprising:
acquiring semantic information carried by pixel points in at least one frame of image;
determining pixel coordinates of a reference point of the lane line corresponding to the semantic information;
Acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image;
Acquiring an initial structural parameter representation of the lane line according to the first space coordinate set, wherein the initial structural parameter representation of the lane line is used for defining the geometric shape of the lane line by parameters;
And determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image and the semantic information.
2. The method of claim 1, wherein the determining the optimized structural parametric representation of the lane line from the initial structural parametric representation, the camera pose to which the at least one frame of image corresponds, and the semantic information comprises:
Determining a second space coordinate corresponding to the lane line according to the initial structural parameter representation;
according to the second space coordinates and the camera pose corresponding to each of the at least one frame of image, obtaining the corresponding projection points of the second space coordinates in each of the at least one frame of image;
and determining the optimized structural parameter representation of the lane line according to the projection points and the semantic information.
3. The method of claim 2, wherein the determining the optimized structured parametric representation of the lane line from the projection points and the semantic information comprises:
determining semantic information corresponding to the projection points according to the semantic information carried by the pixel points in the at least one frame of image;
if the semantic information corresponding to the projection points is lane lines, marking the projection points;
determining a sum of the corresponding marked projection points of the initial structural parameter representation;
And if the sum of the marked projection points meets a first preset condition, determining the initial structural parameter representation as the optimized structural parameter representation of the lane line.
4. A method according to claim 3, wherein said obtaining an initial structured parametric representation of the lane line from the first spatial coordinates comprises:
selecting a third set of spatial coordinates from the first set of spatial coordinates;
Acquiring an initial structural parameter representation of the lane line according to the third space coordinate set;
then, after determining the sum of the corresponding marked projection points of the initial structural parameter representation, the method further includes:
if the sum of the marked projection points does not meet the first preset condition, selecting a fourth space coordinate set from the first space coordinate set;
and updating the initial structural parameter representation according to the fourth space coordinate set.
5. The method of claim 4, wherein after selecting the fourth set of spatial coordinates from the first set of spatial coordinates, further comprising:
determining the selection times corresponding to the fourth space coordinate set;
and if the selected times meet a second preset condition, determining the initial structural parameter representation of which the sum of the marked projection points meets a third preset condition as the optimized structural parameter representation of the lane line.
6. The method of claim 1, wherein the obtaining an initial structured parametric representation of the lane line from the first set of spatial coordinates comprises:
Randomly selecting at least four fifth space coordinates from the first space coordinate set;
And acquiring an initial Bezier curve corresponding to the lane line according to the fifth space coordinate.
7. The method according to any one of claims 1-6, wherein the obtaining a first set of spatial coordinates corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image includes:
performing inverse perspective transformation on the pixel coordinates to obtain sixth space coordinates corresponding to the pixel coordinates;
Determining the tracking code of the lane line corresponding to the sixth space coordinate in the at least one frame of image;
and forming the sixth space coordinates with the same corresponding tracking codes into a first space coordinate set.
8. A structured parametric representation determination apparatus for lane lines, comprising:
The semantic information acquisition module is used for acquiring semantic information carried by pixel points in at least one frame of image;
The pixel coordinate determining module is used for determining pixel coordinates of a reference point of the lane line corresponding to the semantic information;
the space coordinate acquisition module is used for acquiring a first space coordinate set corresponding to the pixel coordinates according to the pixel coordinates and the camera pose corresponding to the at least one frame of image;
The first parameter representation module is used for acquiring an initial structural parameter representation of the lane line according to the first space coordinate set, wherein the initial structural parameter representation of the lane line is used for defining the geometric shape of the lane line by parameters;
and the second parameter representation module is used for determining the optimized structural parameter representation of the lane line according to the initial structural parameter representation, the camera pose corresponding to each of the at least one frame of image and the semantic information.
9. A computer readable storage medium storing a computer program for executing the structured parametric representation of lane lines determination method according to any one of the preceding claims 1-7.
10. An electronic device, the electronic device comprising:
A processor;
A memory for storing the processor-executable instructions;
The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for determining the structured parametric representation of lane lines according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910787011.1A CN112435293B (en) | 2019-08-24 | 2019-08-24 | Method and device for determining structural parameter representation of lane line |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910787011.1A CN112435293B (en) | 2019-08-24 | 2019-08-24 | Method and device for determining structural parameter representation of lane line |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112435293A CN112435293A (en) | 2021-03-02 |
| CN112435293B true CN112435293B (en) | 2024-04-19 |
Family
ID=74690023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910787011.1A Active CN112435293B (en) | 2019-08-24 | 2019-08-24 | Method and device for determining structural parameter representation of lane line |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112435293B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090064946A (en) * | 2007-12-17 | 2009-06-22 | 한국전자통신연구원 | Method and apparatus for creating a virtual lane for image based navigation |
| CN105667518A (en) * | 2016-02-25 | 2016-06-15 | 福州华鹰重工机械有限公司 | Lane detection method and device |
| JP2018169947A (en) * | 2017-03-30 | 2018-11-01 | 株式会社日立情報通信エンジニアリング | Lane recognition apparatus and lane recognition program |
| CN109543520A (en) * | 2018-10-17 | 2019-03-29 | 天津大学 | A kind of lane line parametric method of Semantic-Oriented segmentation result |
| CN109948470A (en) * | 2019-03-01 | 2019-06-28 | 武汉光庭科技有限公司 | 'STOP' line ahead detection method and system based on Hough transformation |
| CN110084095A (en) * | 2019-03-12 | 2019-08-02 | 浙江大华技术股份有限公司 | Method for detecting lane lines, lane detection device and computer storage medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11022982B2 (en) * | 2014-03-18 | 2021-06-01 | Transforation Ip Holdings, Llc | Optical route examination system and method |
| JP5926080B2 (en) * | 2012-03-19 | 2016-05-25 | 株式会社日本自動車部品総合研究所 | Traveling lane marking recognition device and program |
| CN105260699B (en) * | 2015-09-10 | 2018-06-26 | 百度在线网络技术（北京）有限公司 | A kind of processing method and processing device of lane line data |
| US10373002B2 (en) * | 2017-03-31 | 2019-08-06 | Here Global B.V. | Method, apparatus, and system for a parametric representation of lane lines |
-
2019
- 2019-08-24 CN CN201910787011.1A patent/CN112435293B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20090064946A (en) * | 2007-12-17 | 2009-06-22 | 한국전자통신연구원 | Method and apparatus for creating a virtual lane for image based navigation |
| CN105667518A (en) * | 2016-02-25 | 2016-06-15 | 福州华鹰重工机械有限公司 | Lane detection method and device |
| JP2018169947A (en) * | 2017-03-30 | 2018-11-01 | 株式会社日立情報通信エンジニアリング | Lane recognition apparatus and lane recognition program |
| CN109543520A (en) * | 2018-10-17 | 2019-03-29 | 天津大学 | A kind of lane line parametric method of Semantic-Oriented segmentation result |
| CN109948470A (en) * | 2019-03-01 | 2019-06-28 | 武汉光庭科技有限公司 | 'STOP' line ahead detection method and system based on Hough transformation |
| CN110084095A (en) * | 2019-03-12 | 2019-08-02 | 浙江大华技术股份有限公司 | Method for detecting lane lines, lane detection device and computer storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112435293A (en) | 2021-03-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11422261B2 (en) | Robot relocalization method and apparatus and robot using the same | |
| CN110163930B (en) | Lane line generation method, device, equipment, system and readable storage medium | |
| CN108871353B (en) | Road network map generation method, system, equipment and storage medium | |
| CN110009718B (en) | Three-dimensional high-precision map generation method and device | |
| CN112154445B (en) | Method and device for determining lane lines in high-precision maps | |
| CN108279670B (en) | Method, apparatus and computer readable medium for adjusting point cloud data acquisition trajectory | |
| JP2021119507A (en) | Traffic lane determination method, traffic lane positioning accuracy evaluation method, traffic lane determination apparatus, traffic lane positioning accuracy evaluation apparatus, electronic device, computer readable storage medium, and program | |
| CN108734780B (en) | Method, device and equipment for generating map | |
| CN113761999A (en) | Target detection method and device, electronic equipment and storage medium | |
| CN110853085B (en) | Semantic SLAM-based mapping method and device and electronic equipment | |
| JP2020057387A (en) | Vehicle positioning method, vehicle positioning device, electronic apparatus, and computer-readable storage medium | |
| CN111539484A (en) | Method and device for training neural network | |
| CN112700486B (en) | Method and device for estimating depth of road surface lane line in image | |
| CN112330815A (en) | Three-dimensional point cloud data processing method, device and equipment based on obstacle fusion | |
| CN116597096B (en) | Scene reconstruction method, device, storage medium and electronic device | |
| CN110068824B (en) | Sensor pose determining method and device | |
| CN110660103A (en) | Unmanned vehicle positioning method and device | |
| CN112150529A (en) | Method and device for determining depth information of image feature points | |
| CN116883504B (en) | Method and device for calibrating vehicle orientation, computer equipment and storage medium | |
| CN112435293B (en) | Method and device for determining structural parameter representation of lane line | |
| CN112212873B (en) | Construction method and device of high-precision map | |
| CN110827337B (en) | Method and device for determining posture of vehicle-mounted camera and electronic equipment | |
| CN114489341B (en) | Gesture determination method and device, electronic equipment and storage medium | |
| CN112348874B (en) | Method and device for determining structural parameter representation of lane line | |
| CN112417924B (en) | Space coordinate acquisition method and device for marker post |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |