CN114545359B - Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media. - Google Patents
Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media.Info
- Publication number
- CN114545359B CN114545359B CN202210157512.3A CN202210157512A CN114545359B CN 114545359 B CN114545359 B CN 114545359B CN 202210157512 A CN202210157512 A CN 202210157512A CN 114545359 B CN114545359 B CN 114545359B
- Authority
- CN
- China
- Prior art keywords
- tractor
- trailer
- target
- current
- relative angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/26—Measuring arrangements characterised by the use of optical techniques for measuring angles or tapers; for testing the alignment of axes
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C1/00—Measuring angles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Optical Radar Systems And Details Thereof (AREA)
Abstract
The application discloses a vehicle relative angle detection method and device, electronic equipment and a storage medium, wherein the method comprises the steps of obtaining point cloud data of two markers arranged on the left side and the right side of a tractor and on the left side and the right side of a trailer, determining the marker with larger data quantity of the point cloud data of the two markers as a target marker, determining current target angle information based on the point cloud data of the target marker, wherein the current target angle information is target angle information related to the current relative angle of the tractor and the trailer, and calculating the current relative angle of the tractor and the trailer based on the current target angle information.
Description
Technical Field
The present application relates to the field of automatic driving technologies, and in particular, to a method and apparatus for detecting a relative angle of a vehicle, an electronic device, and a storage medium.
Background
For a partially larger truck, which is composed of a tractor and a trailer connected by a hinge, a certain relative angle between the tractor and the trailer occurs when the tractor turns. The method has important significance for accurately detecting the relative angle of the tractor and the trailer and for auxiliary driving and automatic driving of the vehicle.
The existing detection of the relative angle of the tractor and the trailer mainly comprises the steps of installing a camera on the tractor, shooting image information of a fixed marker on the trailer through the camera, and further determining the relative angle between the tractor and the trailer based on the acquired image information.
However, because the camera is easily affected by external factors such as light and weather, the camera is easy to obtain the image information of the fixed marker in a poor light condition or in a rainy and snowy day, and therefore the accurate relative angle between the tractor and the trailer cannot be effectively ensured.
Disclosure of Invention
Based on the defects of the prior art, the application provides a method and a device for detecting the relative angle of a vehicle, electronic equipment and a storage medium, so as to solve the problem that the existing method cannot effectively ensure that the relative angle between a tractor and a trailer can be accurately detected.
In order to achieve the above object, the present application provides the following technical solutions:
the first aspect of the present application provides a method for detecting a relative angle of a vehicle, including:
Acquiring point cloud data of two markers on the left side and the right side of a currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor;
determining the marker with larger data volume of the point cloud data in the two markers as a target marker;
Determining current target angle information based on the point cloud data of the target marker, wherein the current target angle information is target angle information related to the relative angle of the tractor and the trailer;
and calculating the current relative angle of the tractor and the trailer based on the current target angle information.
Optionally, in the above method for detecting a relative angle of a vehicle, the target angle information is an angle between a line connecting a detection point of interest on the target marker to a center of articulation of the trailer and a target horizontal line, wherein the target horizontal line is a horizontal line passing through the detection point of interest on the target marker and perpendicular to a symmetry axis of the tractor;
wherein said calculating a current relative angle of said tractor and said trailer based on said current target angle information comprises:
And calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer, wherein the calibrated target angle information corresponding to the target marker is the target angle information when the relative angle between the tractor and the trailer is zero.
Optionally, in the above method for detecting a relative angle of a vehicle, the acquiring the point cloud data of the two markers installed on the left and right sides of the tractor and the two markers on the left and right sides of the trailer currently acquired includes:
acquiring point cloud data currently acquired by two three-dimensional laser radars arranged on the left side and the right side of the tractor;
and removing the pre-determined point cloud data of the tractor from the current point cloud data of interest to obtain the point cloud data of the two markers, wherein the current point cloud data of interest is the point cloud data in a preset region of interest in the currently acquired point cloud data.
Optionally, in the method for detecting a relative angle of a vehicle, the determining current target angle information based on the point cloud data of the target marker includes:
Calculating the distance between each point in the point cloud data of the target marker and the three-dimensional laser radar on the same side of the target marker;
Determining a point at which a distance of the three-dimensional lidar on the same side as the target marker is maximum as a detection point of interest on the target marker;
and calculating the angle between the straight line connecting the interesting detection point on the target marker to the hinging center of the trailer and the target horizontal line based on the coordinates of the interesting detection point on the target marker and the coordinates of the hinging center of the trailer, and obtaining the current target angle information.
Optionally, in the method for detecting a vehicle relative angle, after calculating a current relative angle between the tractor and the trailer based on the current target information, the method further includes:
And filtering the current relative angle of the tractor and the trailer by using a Kalman filtering algorithm to obtain the current optimized relative angle of the tractor and the trailer.
Optionally, in the method for detecting a relative angle of a vehicle, the filtering processing is performed on the current relative angle of the tractor and the trailer by using a kalman filtering algorithm to obtain the current optimized relative angle of the tractor and the trailer, including:
Calculating the current theoretical relative angle of the tractor and the trailer;
And based on the Kalman filtering algorithm, comprehensively calculating the theoretical relative angle of the tractor and the trailer and the current relative angle of the tractor and the trailer to obtain the optimal relative angle of the tractor and the trailer.
Optionally, in the method for detecting a vehicle relative angle, the performing a comprehensive operation on the theoretical relative angle between the tractor and the trailer and the current relative angle between the tractor and the trailer based on using a kalman filtering algorithm to obtain the optimized relative angle between the tractor and the trailer includes:
Calculating to obtain a current error variance predicted value by utilizing the error variance and the process noise variance of the last moment;
calculating a current Kalman gain by using the current error variance predicted value, the measured noise variance and the measured coefficient;
calculating the product of the theoretical relative angle of the tractor and the trailer and the measurement coefficient to obtain an angle difference value;
And adding the product of the angle difference value and the current Kalman gain to the theoretical relative angle of the tractor and the trailer to obtain the optimized relative angle of the tractor and the trailer.
A second aspect of the present application provides a device for detecting a relative angle of a vehicle, including:
the acquisition unit is used for acquiring point cloud data of two markers on the left side and the right side of the currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor;
a determining unit configured to determine, as a target marker, the marker whose data amount of the point cloud data is larger among the two markers;
The target information calculating unit is used for determining current target angle information based on the point cloud data of the target marker, wherein the current target angle information is target angle information related to the relative angle of the tractor and the trailer at present;
and the relative angle calculating unit is used for calculating the current relative angle of the tractor and the trailer based on the current target angle information.
Optionally, in the above vehicle relative angle detection device, the target angle information is an angle between a line connecting the detection point of interest on the target marker to the articulation center of the trailer and a target horizontal line, wherein the target horizontal line is a horizontal line passing through the detection point of interest on the target marker and perpendicular to the symmetry axis of the tractor;
wherein the relative angle calculation unit includes:
And the relative angle calculating subunit is used for calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer, wherein the calibrated target angle information corresponding to the target marker is the target angle information when the relative angle between the tractor and the trailer is zero.
Optionally, in the above vehicle relative angle detection device, the acquisition unit includes:
The acquisition subunit is used for acquiring the point cloud data currently acquired by the two three-dimensional laser radars arranged on the left side and the right side of the tractor;
the eliminating unit is used for eliminating the pre-determined point cloud data of the tractor from the current point cloud data of interest to obtain the point cloud data of the two markers, wherein the current point cloud data of interest is the point cloud data in a preset region of interest in the currently acquired point cloud data.
Optionally, in the above vehicle relative angle detection device, the target angle calculation unit includes:
a distance calculation unit for calculating the distance between each point in the point cloud data of the target marker and the three-dimensional laser radar on the same side of the target marker;
a detection point determining unit configured to determine a point at which a distance from the three-dimensional lidar on the same side as the target marker is largest as a detection point of interest on the target marker;
and the target angle calculating subunit is used for calculating the angle between the straight line connecting the interesting detection point on the target marker to the hinging center of the trailer and the target horizontal line based on the coordinates of the interesting detection point on the target marker and the coordinates of the hinging center of the trailer, so as to obtain the current target angle information.
Optionally, the above device for detecting a relative angle of a vehicle further includes:
And the optimizing unit is used for carrying out filtering processing on the current relative angle between the tractor and the trailer by using a Kalman filtering algorithm to obtain the current optimized relative angle between the tractor and the trailer.
Optionally, in the above device for detecting a relative angle of a vehicle, the optimizing unit includes:
an angle estimation unit for calculating the theoretical relative angle between the tractor and the trailer;
And the comprehensive operation unit is used for comprehensively operating the theoretical relative angle between the tractor and the trailer at present and the relative angle between the tractor and the trailer at present based on a Kalman filtering algorithm to obtain the optimized relative angle between the tractor and the trailer at present.
Optionally, in the above vehicle relative angle detection device, the comprehensive operation unit includes:
The error prediction unit is used for calculating and obtaining a current error variance predicted value by utilizing the error variance and the process noise variance of the last moment;
a gain calculation unit for calculating a current kalman gain using the current error variance prediction value, the measurement noise variance, and the measurement coefficient;
the difference value calculation unit is used for calculating the product of the theoretical relative angle of the tractor and the trailer and the measurement coefficient to obtain an angle difference value;
And the optimized angle calculation unit is used for adding the product of the angle difference value and the current Kalman gain to the current theoretical relative angle of the tractor and the trailer to obtain the current optimized relative angle of the tractor and the trailer.
A third aspect of the present application provides an electronic device, comprising:
A memory and a processor;
Wherein the memory is used for storing programs;
The processor is configured to execute the program, and when the program is executed, the program is specifically configured to implement the method for detecting a relative angle of a vehicle according to any one of the foregoing.
A fourth aspect of the present application provides a computer storage medium storing a computer program which, when executed, is adapted to carry out a method of detecting a relative angle of a vehicle as claimed in any one of the preceding claims.
According to the vehicle detection method for the relative angle of the vehicle, the cameras are not adopted any more through the point cloud data of the two markers on the left side and the right side of the trailer, which are acquired by the two three-dimensional laser radars arranged on the left side and the right side of the tractor. And then determining the marker with larger data quantity of point cloud data in the two markers as a target marker, and determining current target angle information based on the point cloud data of the target marker. And finally, calculating the relative angle of the current tractor and the trailer based on the current target angle information. Therefore, based on the point cloud data acquired by the three-dimensional laser radar, the detection of the relative angle between the tractor and the trailer is realized, and the three-dimensional laser radar is not easily affected by external factors such as light and weather, so that the accuracy of a detection result can be effectively ensured.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present application, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for detecting relative angles of vehicles according to an embodiment of the present application;
FIG. 2 is a schematic illustration of the position of a tractor and trailer provided by an embodiment of the present application;
FIG. 3 is a flowchart of a method for obtaining point cloud data of two markers according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a region of interest according to an embodiment of the present application;
FIG. 5 is a schematic illustration of the position of a tractor and trailer when a vehicle turns, according to an embodiment of the present application;
FIG. 6 is a flowchart of a method for calculating a current target angle according to an embodiment of the present application;
FIG. 7 is a flowchart of a method for filtering relative angles according to an embodiment of the present application;
FIG. 8 is a flowchart of a method for performing a comprehensive operation on a theoretical relative angle and a relative angle based on using a Kalman filtering algorithm according to an embodiment of the present application;
FIG. 9 is a schematic diagram of a device for detecting a relative angle of a vehicle according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
In the present application, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
The embodiment of the application provides a method for detecting the relative angle of a vehicle, which is shown in fig. 1 and comprises the following steps:
S101, acquiring point cloud data of two markers on the left side and the right side of a currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor.
In the embodiment of the application, a three-dimensional laser radar is respectively installed on the left side and the right side of the tractor of the truck. As shown in fig. 2, a three-dimensional lidar is mounted on the leftmost side and the rightmost side of the front of the tractor, respectively, and two three-dimensional lidars may be disposed on the same horizontal plane in general. The two three-dimensional laser radars are respectively used for scanning markers on the left side and the right side of the trailer, so that point cloud data of the two markers are obtained.
Because the scanning range of the three-dimensional laser radar is 360 degrees, the marker can be effectively scanned, the laser radar is not easily affected by light and weather, the accuracy of measured data can be ensured, and the robustness of a detection result can be further ensured.
Alternatively, the three-dimensional lidar in the embodiment of the present application may employ a mechanical lidar. If the unmanned truck itself has three-dimensional lidar on the left and right sides of the tractor, no additional installation is necessary.
Referring also to fig. 2, the markers are also on the left and right sides of the trailer, and are typically on the same horizontal line. Alternatively, the marker may be an anti-collision block or a side barrier or the like on the trailer.
The left three-dimensional laser radar can scan the left marker and the right three-dimensional laser radar can scan the right marker when the vehicle is traveling straight or near straight. However, when the vehicle turns left, only the left three-dimensional laser radar can scan the left marker, and the obtained point cloud data of the two markers is empty. Similarly, when the vehicle turns right, only the right three-dimensional laser radar can scan the right marker, and the point cloud data of the left marker is empty in the point cloud data of the two markers acquired at the moment.
Optionally, in another embodiment of the present application, a specific implementation manner of step S102, as shown in fig. 3, includes:
s301, acquiring point cloud data currently acquired by two three-dimensional laser radars arranged on the left side and the right side of the tractor.
It should be noted that, the three-dimensional lidar scans the point cloud data of the object within 360 degrees, that is, or scans the object within 360 degrees, but only the point cloud data on the marker is needed later, so after step S301 is executed, step S302 needs to be executed.
S302, removing the pre-determined point cloud data of the tractor from the current point cloud data of interest to obtain the point cloud data of the two markers.
The current point cloud data of interest is cloud data in a preset region of interest in the currently acquired point cloud data. In other words, the point cloud data on the two markers at the same time on the left and right sides of the trailer and the preset region of interest in the acquired point cloud data are determined to be the point cloud data of the two markers.
It should be noted that, in the embodiment of the present application, the point cloud data of the marker is determined by setting the region of interest (region of interest, ROI). Wherein the ROI may be a rectangular frame, a circular ring shape, etc. As shown in fig. 4, the ROI is in a circular shape, and the point cloud data of the marker located in the ROI in the currently acquired point cloud data is extracted as the current point cloud data of interest. Wherein the radius R and the radius R of the two circles of the ROI can be calibrated by measuring the vehicle. As can be seen from fig. 4, for the region of interest, the three-dimensional laser radar can collect the point cloud data of the tractor and the point cloud data of the two markers, but the three-dimensional laser radar is fixedly arranged on the tractor because the markers are shielded, so that the collected point cloud data of the tractor is fixed and can be calibrated in advance. Since the relative position between the tag and the three-dimensional lidar varies with the relative position of the tractor and the trailer, the processing in step S302 is required.
S102, determining the marker with larger data quantity of point cloud data in the two markers as a target marker.
When the vehicle is not in a complete straight line, that is, the tractor and the trailer are not in a straight line, the relative angle exists between the tractor and the trailer, so that the data size of the point cloud data of the markers collected by the two three-dimensional laser radars is different, even when the turning angle of the vehicle is large, that is, the relative angle between the tractor and the trailer is large, the data size of the point cloud data of the markers collected by the three-dimensional laser radars on the side opposite to the turning direction is zero, and therefore, whether the vehicle turns left or right can be determined by comparing the data sizes of the point cloud data of the two markers, and the relative angle between the tractor and the trailer can be determined in a targeted manner.
For example, as shown in fig. 5, when the vehicle turns right, it is obvious that the three-dimensional laser radar on the right side cannot scan the marker on the right side of the trailer, so the data amount of the point cloud data of the marker on the right side is zero, and therefore the marker on the left side is determined as the current target marker, and further the relative angle between the current tractor and the trailer when the vehicle turns left can be calculated.
If the data amount of the point cloud data in the two markers is as large, the vehicle is illustrated to run in a straight line, and the tractor and the trailer are in a straight line, so that the relative angle of the tractor and the trailer is zero at this time.
S103, determining current target angle information based on the point cloud data of the target marker.
The current target angle information is target angle information related to the relative angle of the current tractor and the trailer. The target angle information refers to information related to a relative angle between the tractor and the trailer, and is not necessarily an angle. The target angle information changes along with the change of the relative angle of the tractor and the trailer, and the relative angle of the tractor and the trailer can be calculated correspondingly according to the target angle information of each time. Step S104 can be performed after step S103 is performed.
S104, calculating the relative angle of the current tractor and the trailer based on the current target angle information.
Alternatively, the correlation expression between the relative angle of the tractor and the trailer and the target angle information can be determined in advance according to the correlation between the relative angle of the tractor and the trailer and the target angle information, so that after the current target angle information is obtained, the current target angle information is substituted into the correlation expression for calculation, and the relative angle of the current tractor and the trailer is obtained.
Optionally, in another embodiment of the present application, the target angle information is an angle between a straight line connecting the articulation center of the trailer and the target horizontal line at the detection point of interest on the target marker, so in the embodiment of the present application, the current target angle information is the current target angle. The target horizon is a horizon that passes through a point of interest on the target marker and is perpendicular to the symmetry axis of the tractor.
It should be noted that, the angle referred to in the embodiment of the present application refers to an angle on a two-dimensional plane, that is, an angle on a top view.
Wherein the detection point of interest on the target marker refers to a fixed point on the marker for detection. For example, as shown in fig. 5, when the vehicle turns left, the target marker is the marker on the left side, so the detection point of interest on the target marker is the point where the marker on the left side is furthest from the three-dimensional lidar on the left side. And the angle of the straight line connecting the articulation center of the trailer and the target horizontal line at the detection point of interest on the current target marker is the current target angle, i.e., angle γ in fig. 5 is the current target angle.
It should be noted that, since the relative positions of the marker and the tractor are continuously changed, the relative positions of the detection point of interest on the target marker and the tractor are also continuously changed, and thus the position of the target horizontal line is also correspondingly changed.
Alternatively, as shown in fig. 6, an embodiment of step S103 includes:
s601, calculating the distance between each point in the point cloud data of the target marker and the three-dimensional laser radar on the same side of the target marker.
The point cloud data of the target marker includes coordinates of a plurality of points on the scanned target marker, and the three-dimensional laser radar is fixed, that is, the coordinates of the three-dimensional laser radar are also fixed, so that the distance between each point and the three-dimensional laser radar can be calculated based on the coordinates of each point and the coordinates of the three-dimensional laser radar.
S602, determining a point with the largest distance with the three-dimensional laser radar on the same side of the target marker as a detection point of interest on the target marker.
Since the point of the three-dimensional lidar on the same side as the target marker at which the distance is the largest is fixed, in the embodiment of the present application, the point of the three-dimensional lidar on the same side as the target marker at which the distance is the largest is determined as the detection point of interest on the target marker. Of course, other ways of determining the detection point of interest on the target marker may be used, for example, fitting the point cloud data of the target detection area may be performed, and the detection point of interest may be identified in combination with the nature of the target marker, etc. However, step S602 determines that the calculation amount of the detection point of interest is relatively small.
And S603, calculating the angle between the straight line of the interested detection point on the current target marker, which is connected with the hinging center of the trailer, and the target horizontal line based on the coordinates of the interested detection point on the target marker and the coordinates of the hinging center of the trailer, and obtaining the current target angle information.
Specifically, as shown in fig. 5, since the hinge center is fixed, that is, the coordinates of the hinge center are determined, the distance between the two is calculated based on the coordinates of the detection point of interest on the target marker and the coordinates of the hinge center of the trailer, and the distance between the detection point of interest on the target marker and the perpendicular line of the target horizontal line can be determined according to the coordinates of the detection point of interest on the target marker, so that finally, the current target angle can be calculated by using the trigonometric function.
In the embodiment of the present application, the specific implementation manner of step S104 includes:
And calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer.
The calibration target angle information corresponding to the target marker is a numerical value of target angle information when the relative angle of the tractor and the trailer is zero, namely the calibration target angle information is a calibration target angle. In the embodiment of the application, the target angle information is the angle between the straight line connecting the articulation center of the trailer and the target horizontal line of the detection point of interest on the target marker, so that the calibration target angle corresponding to the target marker is zero relative angle between the tractor and the trailer, namely the angle between the straight line connecting the articulation center of the trailer and the target horizontal line of the detection point of interest on the target marker when the vehicle is in a righted state.
Alternatively, the angles of the straight line connecting the articulation center of the trailer and the target horizontal line of the interested detection points on the two markers can be calculated in advance in the vehicle alignment state, so as to obtain the calibration target angle information corresponding to the two markers. For example, angle α and angle β as shown in fig. 5.
As shown in fig. 5, the relative angle between the tractor and the trailer is the following, which is obtained according to the principle that the sum of the internal angles of the triangles is 180 degrees:
α+δ+θ+90°=γ+θ+δ+θ+90°
Therefore, the relative angle of the tractor and the trailer is:
θ=α-γ
Therefore, the relative angle of the current tractor and the trailer can be obtained by calculating the difference value of the current target angle and the calibrated target angle corresponding to the target marker. Specifically, the difference between the current target angle and the calibrated target angle corresponding to the target marker is calculated, namely, the variation of the target angle is calculated, and because the calibrated target angle corresponding to the target marker is the angle between the straight line connecting the articulation center of the trailer and the target horizontal line of the detection point of interest on the target marker when the relative angle between the tractor and the trailer is zero, the variation of the target angle is the relative angle between the current tractor and the trailer.
In the above mode, the hinge center is used as a reference to calculate, alternatively, the same principle can be adopted, and three-dimensional laser radar coordinates are directly used as calculation to calculate the relative angle of the current tractor and the trailer.
Alternatively, the target angle information may be the geometrical distance between the detection point of interest on the target marker and the three-dimensional lidar on the same side. Since the distance is correlated with the relative angle of the tractor and the trailer, an expression of the correlation of the distance and the relative angle of the tractor and the trailer may be fitted in advance, then the geometric distance between the detection point of interest on the current target marker and the three-dimensional laser radar on the same side is calculated through step S103, and when step S104 is performed, the calculated distance is substituted into the expression of the correlation, thereby obtaining the relative angle of the tractor and the trailer.
The marker is arranged on the trailer, the three-dimensional laser radar is arranged on the tractor, so that the relative positions of the marker and the tractor are changed, more factors are considered in the calculation process, the error is larger than that of a mode of calculating by taking the hinge center as a reference, and the accuracy is relatively poorer because the comparison depends on the accuracy of fitting based on the correlation expression of the fitting distance and the relative angle of the tractor and the trailer. And the highest accuracy was also known by testing in the manner corresponding to fig. 5. Of course, the above modes are all alternative modes, and other modes can be adopted.
Optionally, in order to provide robustness of the final output result, in view of possible errors in the calculation process, in another embodiment of the present application, after performing step S104, further steps are performed:
and filtering the relative angle of the current tractor and the trailer by using a Kalman filtering algorithm to obtain the optimized relative angle of the current tractor and the trailer.
Specifically, the relative angle between the current tractor and the trailer is filtered by using a kalman filtering algorithm to obtain an optimized relative angle between the current tractor and the trailer, as shown in fig. 7, which includes:
S701, calculating the theoretical relative angle of the current tractor and the trailer.
Alternatively, the relative angle of the current tractor to the trailer at the next time, i.e. the theoretical relative angle of the current tractor to the trailer, may be estimated theoretically, depending on the relative angle of the current tractor to the trailer at the previous time and the dynamics of the vehicle.
S702, based on the Kalman filtering algorithm, the theoretical relative angle between the current tractor and the trailer and the relative angle between the current tractor and the trailer are comprehensively calculated, and the optimized relative angle between the current tractor and the trailer is obtained.
Because the estimated value is not necessarily completely accurate and has errors, the theoretical relative angle of the current tractor and the trailer and the relative angle of the current tractor and the trailer need to be comprehensively considered, and therefore, in the embodiment of the application, the optimal relative angle of the current tractor and the trailer needs to be obtained by comprehensively calculating the theoretical relative angle of the current tractor and the trailer and the relative angle of the current tractor and the trailer based on the Kalman filtering algorithm.
Alternatively, a specific embodiment of step S702, as shown in fig. 8, includes:
s801, calculating to obtain a current error variance predicted value by using the error variance and the process noise variance of the previous moment.
From the kalman filtering algorithm, it can be known that in the embodiment of the present application, the kalman filtering prediction process can be expressed as:
θi+1,pre＝A·f(vi,θi)
Ppre＝A·P·AT+Q
Wherein, θ i+1,pre is the theoretical relative angle of the tractor and the trailer at the moment i+1, θ i is the relative angle of the tractor and the trailer at the moment i, v i is the speed of the vehicle at the moment i, P represents the error variance predicted value at the moment i, the initial value of the error variance predicted value can be set according to the actual working condition, P pre represents the error variance predicted value at the moment i+1, namely the current error variance predicted value, A is the transfer coefficient, and particularly 1;Q is the process noise variance.
The theoretical relative angle of the tractor to the trailer, the theoretical relative angle of the current tractor to the trailer, and the current error variance prediction can be obtained according to the formula in the above.
S802, calculating the current Kalman gain by using the current error variance predicted value, the measured noise variance and the measurement coefficient.
According to the Kalman filtering algorithm, in the Kalman filtering updating process, the calculation formula of the Kalman gain can be expressed as follows:
K=Ppre·HT·(H·Ppre·HT+R)-1
wherein, the measurement coefficient H is 1, and R marks the measurement noise variance, namely the variance of the measurement noise of the laser radar.
Therefore, the current error variance predicted value, the measured noise variance and the measured coefficient are input into the above formula, and the current Kalman gain can be calculated.
S803, calculating the relative angle of the current tractor and the trailer, and subtracting the product of the theoretical relative angle of the current tractor and the trailer and the measurement coefficient to obtain an angle difference value.
According to the Kalman filtering algorithm, in the Kalman filtering updating process, the optimized relative angle of the tractor and the trailer at the moment i+1 can be expressed as follows:
θi+1,filter＝θi+1,pre+K(θi+1,det-H·θi+1,pre)
The optimal relative angle of the tractor and the trailer at the moment i+1 is represented by theta i+1,filter, the relative angle of the tractor and the trailer at the moment i+1 is represented by theta i+1,det, namely the current relative angle calculated in the step S104, the theoretical relative angle of the tractor and the trailer at the moment i+1 is represented by theta i+1,pre, the Kalman gain is represented by K, and the measurement coefficient is represented by H.
Therefore, the angle difference is obtained by subtracting the product of the theoretical relative angle of the tractor and the trailer and the measurement coefficient from the relative angle of the tractor and the trailer, and then step S804 is performed, so that the optimized relative angle of the tractor and the trailer at the current moment can be obtained.
S804, adding the product of the angle difference value and the current Kalman gain to the theoretical relative angle of the current tractor and the trailer to obtain the optimized relative angle of the current tractor and the trailer.
It should be noted that, when the optimal relative angle between the tractor and the trailer is obtained, the error variance needs to be updated for subsequent calculation. The specific update formula may be:
P=(1-K·H)Ppre
According to the vehicle detection method for the relative angle of the vehicle, provided by the embodiment of the application, the cameras are not adopted any more through the point cloud data of the two markers on the left side and the right side of the trailer, which are acquired by the two three-dimensional laser radars arranged on the left side and the right side of the tractor. And then determining the marker with larger data quantity of point cloud data in the two markers as a target marker, and determining current target angle information based on the point cloud data of the target marker. And finally, calculating the relative angle of the current tractor and the trailer based on the current target angle information. Therefore, based on the point cloud data acquired by the three-dimensional laser radar, the detection of the relative angle between the tractor and the trailer is realized, and the three-dimensional laser radar is not easily affected by external factors such as light and weather, so that the accuracy of a detection result can be effectively ensured.
Another embodiment of the present application provides a device for detecting a relative angle of a vehicle, as shown in fig. 9, including:
the acquisition unit 901 is used for acquiring point cloud data of two markers on the left side and the right side of a currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor.
A determining unit 902, configured to determine, as the target marker, a marker whose data amount of point cloud data is larger in the two markers.
The target information calculating unit 903 is configured to determine current target angle information based on the point cloud data of the target marker.
The current target angle information is target angle information related to the relative angle of the current tractor and the trailer.
A relative angle calculating unit 904 for calculating a current relative angle of the tractor and the trailer based on the current target angle information.
Optionally, in the vehicle relative angle detection device provided in another embodiment of the present application, the target angle information is an angle between a straight line connecting a center of articulation of the trailer and a target horizontal line at a detection point of interest on the target marker. A target horizon is a horizon that passes through a detection point of interest on the target marker and is perpendicular to the axis of symmetry of the tractor;
the relative angle calculating unit in the embodiment of the application comprises:
and the relative angle calculating subunit is used for calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer.
The calibration target angle information corresponding to the target marker is a numerical value of target angle information when the relative angle of the tractor and the trailer is zero.
Optionally, in the apparatus for detecting a relative angle of a vehicle provided in another embodiment of the present application, the acquiring unit includes:
and the acquisition subunit is used for acquiring the point cloud data currently acquired by the two three-dimensional laser radars arranged on the left side and the right side of the tractor.
And the rejecting unit is used for rejecting the pre-determined point cloud data of the tractor from the current point cloud data of interest to obtain the point cloud data of the two markers.
The current point cloud data of interest is point cloud data in a preset region of interest in the currently acquired point cloud data.
Optionally, in the apparatus for detecting a relative angle of a vehicle provided in another embodiment of the present application, the target angle calculating unit includes:
and the distance calculation unit is used for calculating the distance between each point in the point cloud data of the target marker and the three-dimensional laser radar on the same side of the target marker.
And the detection point determining unit is used for determining the point with the largest distance with the three-dimensional laser radar on the same side of the target marker as the detection point of interest on the target marker.
And the target angle calculating subunit is used for calculating the angle between the straight line of the interested detection point on the current target marker, which is connected with the hinging center of the trailer, and the target horizontal line based on the coordinates of the interested detection point on the target marker and the coordinates of the hinging center of the trailer, so as to obtain the current target angle information.
Optionally, in the apparatus for detecting a relative angle of a vehicle according to another embodiment of the present application, the apparatus further includes:
and the optimizing unit is used for carrying out filtering treatment on the relative angle between the current tractor and the trailer by using a Kalman filtering algorithm to obtain the optimized relative angle between the current tractor and the trailer.
Optionally, in the apparatus for detecting a relative angle of a vehicle provided in another embodiment of the present application, the optimizing unit includes:
and the angle estimation unit is used for calculating the theoretical relative angle of the current tractor and the trailer.
And the comprehensive operation unit is used for comprehensively operating the theoretical relative angle of the current tractor and the trailer and the relative angle of the current tractor and the trailer based on the Kalman filtering algorithm to obtain the optimized relative angle of the current tractor and the trailer.
Optionally, in the apparatus for detecting a relative angle of a vehicle provided in another embodiment of the present application, the comprehensive operation unit includes:
the error prediction unit is used for calculating and obtaining a current error variance predicted value by utilizing the error variance and the process noise variance of the last moment.
And the gain calculation unit is used for calculating the current Kalman gain by using the current error variance predicted value, the measured noise variance and the measurement coefficient.
And the difference value calculation unit is used for calculating the product of the theoretical relative angle of the current tractor and the trailer and the measurement coefficient subtracted from the relative angle of the current tractor and the trailer to obtain an angle difference value.
And the optimized angle calculation unit is used for adding the product of the angle difference value and the current Kalman gain to the theoretical relative angle of the current tractor and the trailer to obtain the optimized relative angle of the current tractor and the trailer.
It should be noted that, for the specific working process of each unit provided in the above embodiment of the present application, reference may be made to corresponding steps in the above method embodiment accordingly, which is not described herein again.
Another embodiment of the present application provides an electronic device, as shown in fig. 10, including:
A memory 1001 and a processor 1002.
Wherein the memory 1001 is used for storing programs.
The processor 1002 is configured to execute a program, and when the program is executed, the program is specifically configured to implement the method for detecting a relative angle of a vehicle provided in any one of the embodiments described above.
Another embodiment of the present application provides a computer storage medium storing a computer program for implementing the method for detecting a relative angle of a vehicle according to any one of the above embodiments when the computer program is executed.
Computer storage media, including both non-transitory and non-transitory, removable and non-removable media, may be implemented in any method or technology for storage of information. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, read only compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A method for detecting a relative angle of a vehicle, comprising:
Acquiring point cloud data of two markers on the left side and the right side of a currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor;
determining the marker with larger data volume of the point cloud data in the two markers as a target marker;
Determining current target angle information based on the point cloud data of the target marker, wherein the current target angle information is target angle information related to the relative angle of the tractor and the trailer, and the target angle information is the angle between a straight line connecting a detection point of interest on the target marker with the hinging center of the trailer and a target horizontal line, wherein the target horizontal line is a horizontal line passing through the detection point of interest on the target marker and perpendicular to the symmetry axis of the tractor;
calculating the current relative angle of the tractor and the trailer based on the current target angle information;
wherein said calculating a current relative angle of said tractor and said trailer based on said current target angle information comprises:
And calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer, wherein the calibrated target angle information corresponding to the target marker is the target angle information when the relative angle between the tractor and the trailer is zero.
2. The method of claim 1, wherein the acquiring the point cloud data of the two markers mounted on the left and right sides of the tractor, the two markers currently acquired on the left and right sides of the trailer, comprises:
acquiring point cloud data currently acquired by two three-dimensional laser radars arranged on the left side and the right side of the tractor;
and removing the pre-determined point cloud data of the tractor from the current point cloud data of interest to obtain the point cloud data of the two markers, wherein the current point cloud data of interest is the point cloud data in a preset region of interest in the currently acquired point cloud data.
3. The method of claim 1, wherein the determining current target angle information based on the point cloud data of the target marker comprises:
Calculating the distance between each point in the point cloud data of the target marker and the three-dimensional laser radar on the same side of the target marker;
Determining a point at which a distance of the three-dimensional lidar on the same side as the target marker is maximum as a detection point of interest on the target marker;
and calculating the angle between the straight line connecting the interesting detection point on the target marker to the hinging center of the trailer and the target horizontal line based on the coordinates of the interesting detection point on the target marker and the coordinates of the hinging center of the trailer, and obtaining the current target angle information.
4. The method of claim 1, wherein after calculating a current relative angle of the tractor and the trailer based on the current target angle information, further comprising:
And filtering the current relative angle of the tractor and the trailer by using a Kalman filtering algorithm to obtain the current optimized relative angle of the tractor and the trailer.
5. The method of claim 4, wherein filtering the current relative angle of the tractor and the trailer using a kalman filter algorithm to obtain the current optimal relative angle of the tractor and the trailer comprises:
Calculating the current theoretical relative angle of the tractor and the trailer;
And based on the Kalman filtering algorithm, comprehensively calculating the theoretical relative angle of the tractor and the trailer and the current relative angle of the tractor and the trailer to obtain the optimal relative angle of the tractor and the trailer.
6. The method of claim 5, wherein the performing the comprehensive operation on the theoretical relative angle of the tractor and the trailer and the relative angle of the tractor and the trailer based on the kalman filter algorithm to obtain the optimized relative angle of the tractor and the trailer comprises:
Calculating to obtain a current error variance predicted value by utilizing the error variance and the process noise variance of the last moment;
calculating a current Kalman gain by using the current error variance predicted value, the measured noise variance and the measured coefficient;
calculating the product of the theoretical relative angle of the tractor and the trailer and the measurement coefficient to obtain an angle difference value;
And adding the product of the angle difference value and the current Kalman gain to the theoretical relative angle of the tractor and the trailer to obtain the optimized relative angle of the tractor and the trailer.
7. A device for detecting a relative angle of a vehicle, comprising:
the acquisition unit is used for acquiring point cloud data of two markers on the left side and the right side of the currently acquired trailer, wherein the two three-dimensional laser radars are arranged on the left side and the right side of the tractor;
a determining unit configured to determine, as a target marker, the marker whose data amount of the point cloud data is larger among the two markers;
The system comprises a target information calculation unit, a target information calculation unit and a target information processing unit, wherein the target information calculation unit is used for determining current target angle information based on point cloud data of a target marker, the current target angle information is target angle information related to the relative angle of a tractor and a trailer at present, the target angle information is the angle between a straight line connecting a detection point of interest on the target marker with the hinging center of the trailer and a target horizontal line, and the target horizontal line is a horizontal line which passes through the detection point of interest on the target marker and is perpendicular to the symmetry axis of the tractor;
A relative angle calculation unit for calculating a current relative angle of the tractor and the trailer based on the current target angle information;
wherein the relative angle calculation unit includes:
And the relative angle calculating subunit is used for calculating the difference value between the calibrated target angle information corresponding to the target marker and the current target angle information to obtain the current relative angle between the tractor and the trailer, wherein the calibrated target angle information corresponding to the target marker is the target angle information when the relative angle between the tractor and the trailer is zero.
8. An electronic device, comprising:
A memory and a processor;
Wherein the memory is used for storing programs;
the processor is configured to execute the program, and the program is specifically configured to implement the method for detecting a relative angle of a vehicle according to any one of claims 1 to 6 when executed.
9. A computer storage medium storing a computer program which, when executed, is adapted to carry out the method of detecting a relative angle of a vehicle as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210157512.3A CN114545359B (en) | 2022-02-21 | 2022-02-21 | Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media. |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210157512.3A CN114545359B (en) | 2022-02-21 | 2022-02-21 | Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media. |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114545359A CN114545359A (en) | 2022-05-27 |
| CN114545359B true CN114545359B (en) | 2025-11-14 |
Family
ID=81675935
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210157512.3A Active CN114545359B (en) | 2022-02-21 | 2022-02-21 | Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media. |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114545359B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115018917B (en) * | 2022-06-29 | 2025-11-04 | 杭州海康汽车软件有限公司 | Methods, devices, electronic equipment and storage media for generating panoramic vehicle images |
| CN116543045A (en) * | 2023-04-11 | 2023-08-04 | 深圳海星智驾科技有限公司 | Hanging position and orientation detection method and device, target trailer and storage medium |
| CN119068444B (en) * | 2024-11-05 | 2025-01-03 | 厦门中科星晨科技有限公司 | Traction angle detection method for tractor, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111366938A (en) * | 2018-12-10 | 2020-07-03 | 北京图森智途科技有限公司 | Method and device for measuring included angle of trailer and vehicle |
| CN112362055A (en) * | 2020-12-01 | 2021-02-12 | 苏州挚途科技有限公司 | Attitude estimation method and device and electronic equipment |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102582626B (en) * | 2012-02-16 | 2015-06-10 | 吉林大学 | Method for estimating heavy semitrailer status |
| US9446713B2 (en) * | 2012-09-26 | 2016-09-20 | Magna Electronics Inc. | Trailer angle detection system |
| CN110850431A (en) * | 2019-11-25 | 2020-02-28 | 盟识(上海)科技有限公司 | System and method for measuring trailer deflection angle |
| CN113063396B (en) * | 2021-03-19 | 2022-09-27 | 北京斯年智驾科技有限公司 | Method, device and equipment for detecting included angle of semitrailer and storage medium |
-
2022
- 2022-02-21 CN CN202210157512.3A patent/CN114545359B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111366938A (en) * | 2018-12-10 | 2020-07-03 | 北京图森智途科技有限公司 | Method and device for measuring included angle of trailer and vehicle |
| CN112362055A (en) * | 2020-12-01 | 2021-02-12 | 苏州挚途科技有限公司 | Attitude estimation method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114545359A (en) | 2022-05-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114545359B (en) | Methods and devices for detecting relative vehicle angles, electronic equipment, and storage media. | |
| US11620837B2 (en) | Systems and methods for augmenting upright object detection | |
| CN108028023B (en) | Information processing apparatus, information processing method, and computer-readable storage medium | |
| US11677931B2 (en) | Automated real-time calibration | |
| CN106991389B (en) | Apparatus and method for determining road edge | |
| JP6450294B2 (en) | Object detection apparatus, object detection method, and program | |
| US20230065727A1 (en) | Vehicle and vehicle control method | |
| CN110379178A (en) | Pilotless automobile intelligent parking method based on millimetre-wave radar imaging | |
| US11169244B2 (en) | Method of calibrating alignment model for sensors and electronic device performing the method | |
| CN114114241A (en) | Data fusion method and device | |
| RU2767949C2 (en) | Method (options) and system for calibrating several lidar sensors | |
| CN112801024B (en) | Detection information processing method and device | |
| CN113239459A (en) | Target screening method, screening system, electronic device, and storage medium | |
| US20190257936A1 (en) | Apparatus and method for determining a speed of a vehicle | |
| US12518424B2 (en) | Method and apparatus with calibration | |
| US20250109947A1 (en) | Navigation system with parking space and obstruction detection mechanism and method of operation thereof | |
| JP6704307B2 (en) | Moving amount calculating device and moving amount calculating method | |
| CN114248782A (en) | Unmanned vehicle pitch angle determination method and device and computer readable storage medium | |
| US12394093B2 (en) | Method and device for calibrating a camera mounted on a vehicle | |
| CN110542902B (en) | A simple height measurement method suitable for vehicle-mounted millimeter wave radar | |
| WO2019151109A1 (en) | Road surface information acquisition method | |
| JP7174131B2 (en) | Determination device, determination method and determination program | |
| CN117719515A (en) | A lane departure warning method and its system | |
| WO2020021596A1 (en) | Vehicle position estimation device and vehicle position estimation method | |
| CN117687041A (en) | Detection methods, systems, storage media and vehicles for suspended obstacles in parking lots |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |