CN109961476A - Underground parking lot positioning method based on vision - Google Patents

Abstract

The visual-based positioning method of underground parking lot includes: S1, collect parking lot environmental information; S2, perform feature extraction and tracking on parking lot environmental information; S3, perform feature matching on the extracted and tracked information; S4, select the key frame; S5, calculate the pose of the vehicle in the parking lot; S6, calculate the global pose of the vehicle in the world coordinate system. Abundant underground parking lot information can be obtained through the camera without interference from external signals and other factors; in addition, the monocular camera has many advantages such as simple structure, flexible movement, easy calibration, low cost, and easy procurement and installation.

Description

A Vision-Based Localization Method for Underground Parking Lot

technical field

The invention belongs to the technical field of parking detection, in particular to a positioning method of an underground parking lot based on vision.

Background technique

In recent years, with the continuous advancement of technology and the improvement of people's living standards, more and more people choose to buy private cars. With the increase of cars, the requirements for parking lots are also getting higher and higher, especially some parking lots are even more than one floor. In order to avoid the car owner getting lost in the process of finding the vehicle, it is necessary to know the exact location of the car in the parking lot. The current positioning methods mainly include: 1) GPS-based positioning technology; 2) Bluetooth 4.0-based positioning technology; 3) Wireless network positioning technology; 4) Ranging positioning technology based on infrared, ultrasonic laser and other sensors.

However, the above positioning technology has its own limitations when applied to the underground parking lot.

1) GPS-based positioning technology; the key to precise positioning of the GPS system lies in the accurate calculation of the distance between the satellite and the receiver. According to a fixed pattern: distance = speed × time, after the time is determined, the speed is determined by the propagation speed of electromagnetic waves. As we all know, electromagnetic waves travel very fast in a vacuum, but the atmosphere is not in a vacuum state, and the signal is heavily interfered by the ionosphere and the troposphere. The GPS system can only perform average calculation on this, so there must be errors in some specific areas, which are embodied in the following aspects: the accuracy of the GPS system is meter-level, the error is large, and the accuracy is not high; the signal is weak. It will fail in some remote places, or near overpasses and high-rise buildings. 3 It is easily affected by the weather. In cloudy and rainy days, no stars can be found. Especially in the environment of indoor and underground parking lot, there is no GPS signal at all, and the positioning is invalid.

2) Bluetooth-based positioning technology. Bluetooth technology locates by measuring signal strength. This is a short-range, low-power wireless transmission technology. Install an appropriate Bluetooth LAN access point indoors, configure the network to be a multi-user basic network connection mode, and ensure that the Bluetooth LAN access point is always the master of this device to obtain the user's location information. Bluetooth technology is mainly used for small-range positioning. The biggest advantage of Bluetooth indoor positioning technology is that the device is small in size and easy to integrate in PDAs, PCs and mobile phones, so it is easy to popularize. Theoretically, for a user who holds a mobile terminal device with integrated Bluetooth function, as long as the Bluetooth function of the device is turned on, the Bluetooth indoor positioning system can determine its location. When using this technology for indoor short-distance positioning, it is easy to find the device and the signal transmission is not affected by the line-of-sight. The disadvantage is that the price of Bluetooth devices and equipment is relatively expensive, and for complex space environments, the stability of the Bluetooth system is slightly poor, and it is greatly interfered by noise signals. Especially in the plane range of 100*100 meters, the accuracy of Bluetooth wireless positioning is about 5 meters, and the error is large.

3) Based on the wireless network positioning technology; the wireless local area network can realize complex large-scale positioning, monitoring and tracking tasks, and the positioning of the network node itself is the basis and premise of most applications. The currently popular Wi-Fi positioning is a positioning solution of the IEEE802.11 series of wireless local area network standards. However, the system uses a combination of empirical testing and signal propagation models, requires base station installation, and needs to use the same underlying wireless network structure, and is expensive.

4) Based on infrared positioning technology; the principle of infrared positioning technology positioning is: infrared rays emit modulated infrared rays, which are received by optical sensors for positioning. Although infrared rays have relatively high positioning accuracy, because light cannot pass through obstacles, infrared rays can only travel at the line-of-sight. The two main shortcomings of the line-of-sight distance and the short transmission distance make the indoor positioning effect very poor. When the logo is placed in a pocket or there are walls and other obstructions, it will not work properly, and a receiving antenna needs to be installed in each space, which is costly. Therefore, infrared rays are only suitable for short-distance propagation, and are easily disturbed by fluorescent lamps or lights in the room, and have limitations in precise positioning.

5) Ultrasonic positioning technology; ultrasonic ranging mainly adopts the reflective ranging method, and determines the position of the object through algorithms such as triangular positioning, that is, transmits ultrasonic waves and receives echoes generated by the measured object, and calculates according to the time difference between the echoes and the transmitted waves. out the distance to be measured. Some use the one-way ranging method. The ultrasonic positioning system can be composed of several transponders and a main range finder. The main range finder is placed on the object to be measured, and is sent to the fixed transponder under the action of the microcomputer command signal. The radio signal of the same frequency is transmitted, and the transponder transmits an ultrasonic signal to the main range finder at the same time after receiving the radio signal to obtain the distance between the main range finder and each transponder. When three or more transponders that are not on the same line respond at the same time, the position of the measured object in the two-dimensional coordinate system can be determined according to relevant calculations. The overall positioning accuracy of ultrasonic positioning is high and the structure is simple. However, ultrasonic waves are greatly affected by multi-path effects and non-line-of-sight propagation. At the same time, a large amount of investment in underlying hardware facilities is required, and the cost is too high.

In order to prevent the car owner from getting lost in the process of looking for the vehicle, in the underground parking lot environment without GPS signal, it is necessary to know the exact position of the car in the parking lot. one of the concerns.

SUMMARY OF THE INVENTION

In view of the above-mentioned shortcomings and deficiencies in the prior art, the present invention provides a visual-based positioning method for underground parking lots, which can obtain abundant information on underground parking lots through cameras without being disturbed by external signals and other factors; The eye camera has many advantages, such as simple structure, flexible movement, easy calibration, low cost, easy procurement and installation.

To achieve the above object, the present invention provides a vision-based positioning method for an underground parking lot, comprising:

S1, collect parking lot environmental information;

S2, feature extraction and tracking of parking lot environmental information;

S3, perform feature matching on the extracted and tracked information;

S4, select a key frame;

S5, calculate the pose of the vehicle in the parking lot;

S6, calculate the global pose of the vehicle in the world coordinate system.

Further, the specific method in step S1 is as follows: a monocular camera is erected at the front end of the vehicle, so that the optical axis of the camera is parallel to the vehicle body.

Further, in step S2, the specific method for feature extraction of the parking lot environment information is as follows:

S21: When acquiring the image sequence, uniform size transformation is performed, and the image sequence is sampled;

S22: Use K-level image pyramid to extract FAST corners,

S23: Divide each pyramid into grids, and extract at least L corner points in each grid;

S24: If the number of corner points <L, increase the threshold and perform extraction again;

S25 : According to the extracted FAST corners, use the Brief algorithm to calculate the orientation and ORB feature descriptors.

Further, the calculation direction using the BRIEF algorithm is as follows:

Step 1. Make a circle O with the key point P as the center and d as the radius;

Step 2. Select N point pairs in a certain pattern in circle O;

Step 3. Define Operation T

Among them, I _A represents the gray value of A, and I _B represents the gray value of B;

Step 4. Perform T operations on the selected point pairs respectively, and combine the obtained results.

Further, in step S3, the feature matching of the extracted and tracked information is specifically:

S31. Initialize interior points, and randomly select N pairs of matching points from a given pair of matching points;

S32. Calculate the fundamental matrix F through the interior points;

S33. For the remaining matching point pairs in the matching point pairs, calculate the distance between them and the basic matrix, and if the result is less than a certain threshold, determine that it is an asymmetric matching point, and remove it;

S34. Repeat the previous step until the nearest neighbor matching point is obtained as the final matching point of the feature.

Further, selecting a key frame in step S4 is specifically: when the number of inner points is greater than a certain number, determining the frame as a key frame.

As a further step, calculating the pose of the vehicle in the parking lot in step S5 is as follows:

First normalize all feature points, and then solve the fundamental matrix and essential matrix according to the key frame;

The base matrix is:

x'Fx=0

in, is any pair of matching points of the two images; when enough matching points are given, this formula is used to calculate the unknown fundamental matrix F;

The normalized essential matrix is:

E=t×R=[t] _x ·R

where E represents the essential matrix, t represents the translation vector, [t] _x represents the antisymmetric matrix of t, and the rotation matrix is R.

As a further step, use SH to represent the score of the essential matrix, and SF to represent the score of the fundamental matrix. The judgment is made according to the following judgment model, if:

If the ratio is greater than 0.45, select the result obtained by the essential matrix; if the ratio is less than or equal to 0.45, select the result obtained by the fundamental matrix.

As a further step, S6 calculates the global pose of the vehicle in the world coordinate system as follows:

According to the matched three-dimensional point pair, fundamental matrix and essential matrix, solve the point in the three-dimensional world corresponding to the feature point, that is, the global pose of the vehicle in the world coordinate system;

The obtained motion parameters of each frame are accumulated to obtain the global pose of the vehicle motion in the world coordinate system, that is, the real-time position and _corner information in the parking lot. The camera position is denoted as C _k-1 , where C _k =C _k-1 T _k,k-1 . At this time, the trajectory of the vehicle in the underground parking lot is reconstructed, and the next frame of image input is waited for, and then start from step 1 Repeat the steps in a loop.

By adopting the above technical solutions, the present invention can achieve the following technical effects: when acquiring an image sequence, uniform size transformation is performed, and the image sequence is uniformly sized and subjected to grayscale processing to reduce the amount of calculation and improve the speed; Pyramid, extracts FAST corner points, among which, the pyramid is divided into grids to ensure a single mapping distribution, so that the vehicle can detect feature points even if there is vibration during the movement process, which improves the effectiveness of feature extraction.

Abundant underground parking lot information can be obtained through the camera without interference from external signals and other factors; in addition, the monocular camera has many advantages such as simple structure, flexible movement, easy calibration, low cost, and easy procurement and installation.

Description of drawings

The present invention has 1 accompanying drawing:

Fig. 1 is the flow chart of the localization method of the underground parking lot based on vision;

Detailed ways

The technical solutions of the present invention will be further described in detail below through examples and in conjunction with the accompanying drawings.

This embodiment provides a method for locating an underground parking lot based on vision, including:

S1, collect parking lot environmental information: a monocular camera is set up at the front of the vehicle, so that the optical axis of the camera is parallel to the vehicle body; therefore, the vehicle can capture the parking lot environmental information from a wide angle of view when the vehicle is traveling in the underground parking lot . At the same time, the sampling frame rate is set to 30fps, the image recorded at the nth time is in, and the image at the n-1 time is in-1;

S2, perform feature extraction and tracking on the parking lot environmental information, specifically:

S21: Regardless of whether the standard video is used when collecting the video of the parking lot, when acquiring the image sequence, a uniform size transformation is performed to sample the image sequence; Under the condition of ensuring the accuracy, the amount of ORB extraction can be greatly reduced, and the system acquisition speed can be improved;

S22: The 8-level image pyramid is used to extract the FAST corner points. After a lot of experiments, it is found that the extraction effect is the best when the scale is 1.25, so the scale used in this application is 1.25;

S23: In order to ensure a single mapping distribution, each layer of the pyramid is divided into grids, and the optimal threshold value of each layer grid is 50 through a large number of experiments, and at least 5 corner points are extracted in each grid;

S24: If the number of corner points is less than 5, increase the threshold and perform extraction again;

S25: According to the extracted FAST corner points, use the BRIEF algorithm to calculate the direction and ORB feature descriptor;

S3, perform feature matching on the extracted and tracked information, specifically:

S31. Initialize interior points, and randomly select 4 pairs of matching points from a given pair of matching points;

S32. Calculate the fundamental matrix F through the interior points;

S33. For the remaining matching point pairs in the matching point pairs, calculate the distance between them and the basic matrix. If the result is less than a certain threshold, this paper sets the threshold to 0.7, then it is determined to be asymmetric matching points, and it is eliminated. ;

S34. Repeat the previous step until the nearest neighbor matching point is obtained as the final matching point of the feature;

S4, select key frames; in order to reduce the subsequent calculation amount and improve the calculation processing speed of the positioning system, it is necessary to eliminate invalid frames, and only perform subsequent image processing on representative key frames. When it is greater than a certain number, such as >70%, the frame is determined as a key frame;

S5, calculate the pose of the vehicle in the parking lot:

First normalize all feature points, and then solve the fundamental matrix and essential matrix according to the key frame;

The base matrix is:

x'Fx=0

in, is any pair of matching points of the two images; when enough matching points are given, such as greater than or equal to 7 pairs, use this formula to calculate the unknown fundamental matrix F;

Optionally, x=(x,y,1) ^T and x'=(x',y',1) ^T , then each set of matching points provides a linear equation for the unknown element of F, whose coefficients can be very It is easily represented by the coordinates of the known points x and x'. Specifically, the equation corresponding to a pair of points x=(x,y,1) ^T and x'=(x',y',1) ^T is xx'f ₁₁ +yx'f ₁₂ +x'f ₁₃ +xy'f ₂₁ +yy'f ₂₂ +y'f ₂₃ +xf ₃₁ +yf ₃₂ +f ₃₃ =0 If the vector f is used to represent a 9-dimensional vector composed of the elements of F and arranged in row order, you can use The formula says.

(xx',yx',x',xy',yy',y',x,y,1)f=0

From the geometry of n sets of points matching, we can get the following linear equations,

From this linear equation system, if the rank of matrix A is 8, there is a unique solution, which can ensure that the obtained fundamental matrix remains unchanged.

The essential matrix is a special form of the fundamental matrix in normalized image coordinates, so the normalized essential matrix is:

E=t×R=[t] _x ·R

where E represents the essential matrix, t represents the translation vector, [t] _x represents the antisymmetric matrix of t, and the rotation matrix is R.

Optionally, the essential matrix can be solved by the following equation

beg. in and respectively represent the homogeneous coordinates of the same set of feature points in two adjacent frames of images.

Use SH to represent the score of the essential matrix, and SF to represent the score of the fundamental matrix. The judgment is made according to the following judgment model, if:

If the ratio is greater than 0.45, select the result obtained by the essential matrix; if the ratio is less than or equal to 0.45, select the result obtained by the fundamental matrix.

Example 2

As a supplement to Embodiment 1, the above method also includes: S6, calculating the global pose of the vehicle in the world coordinate system is: according to the matched three-dimensional point pair, fundamental matrix and essential matrix, solve the point in the three-dimensional world corresponding to the feature point , that is, the global pose of the vehicle in the world coordinate system;

Let U=W(U,V,1) ^T represents a point in a homogeneous coordinate system, and W is a scale factor.

Let X=(x,y,z,1) ^T represents the three-dimensional point in the world corresponding to point U, and P _i ^T is the i-th row element of the transformation matrix [R _1,2 |t _1,2 ].

Obtain four sets of linear equations about point X, the representation is AX=0, where A is a 4*4 matrix, and then perform singular value decomposition calculation to obtain four sets of solutions for X: (R1, t1), (R1, t2), (R2, t1), (R2, t2).

Tn and Rn constitute Tn, Rn-1.

in It represents the transformation matrix of the position of the monocular camera at time n and time n-1.

Accumulate the obtained motion parameters of each frame to obtain the global pose of the vehicle in the world coordinate system, that is, the real-time position and corner information in the parking lot, and denote the position of the monocular camera at time n as C _k , k-1 The position of the monocular camera at the moment is C _k-1 , where C _k =C _k-1 T _k,k-1 . At this time, the trajectory of the vehicle in the underground parking lot is reconstructed, and the next frame of image input is waited for, and then from Step S1 starts to repeat the steps cyclically.

Example 3

As a supplement to Embodiment 1 or 2, step S25: using the BRIEF algorithm to calculate the direction, specifically:

Step 1. Make a circle O with the key point P as the center and d as the radius;

Step 2. Select N point pairs in a certain pattern in the circle O; here, for the convenience of description, N=4, and in the application of parking lot positioning, N is 512 in this application; it is assumed that the currently selected 4 points are respectively marked as: P1( _A ,B), P2( _A ,B), _P3 (A,B), P4( _A ,B);

Step 3. Define Operation T

Among them, I _A represents the gray value of A, and I _B represents the gray value of B;

Step 4. Perform T operations on the selected point pairs respectively, and combine the obtained results.

Suppose: T(P ₁ (A,B))=1, T(P ₂ (A,B))=0, T(P ₃ (A,B))=1, T(P ₄ (A,B) )=1, the final descriptor is: 1011

Optionally, for example, the obtained descriptors of feature points A and B are as follows:

A: 1010, B: 1011

We set the threshold to 75%. When the similarity of the descriptors of A and B is greater than 75%, we judge that A and B are the same feature points, that is, the two points are successfully matched.

This application is based on the method of computer vision to realize the positioning in the environment without GPS signal in the underground parking lot, which can be divided into the following three parts: firstly, during the driving process, the video information of the parking lot is collected by the vehicle-mounted monocular camera; Obtain the video record of the vehicle in the process of moving in the parking lot; after collecting the video information, perform ORB feature extraction and matching; then obtain the key frame, then track the key frame and solve the essential matrix and the fundamental matrix, so as to calculate the rotation matrix and translation vector. Finally, according to the accumulated parameters, the vehicle attitude is estimated, and the global vehicle pose in the world coordinate system is obtained.

When acquiring the image sequence, a uniform size transformation is carried out, and the image sequence is unified into a size of 640*480 and grayscale processing is performed to reduce the amount of calculation and improve the speed; an 8-level image pyramid is used to extract the FAST corner points, among which, the The pyramid is divided into grids to ensure a single mapping distribution, so that the vehicle can detect feature points even if there is vibration during the movement process, which improves the effectiveness of feature extraction.

The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited to this. The equivalent replacement or change of the inventive concept thereof shall be included within the protection scope of the present invention.

Claims (9)

1. the positioning method of the underground parking lot based on vision, is characterized in that, comprises: S1, collect parking lot environmental information; S2, feature extraction and tracking of parking lot environmental information; S3, perform feature matching on the extracted and tracked information; S4, select a key frame; S5, calculate the pose of the vehicle in the parking lot; S6, calculate the global pose of the vehicle in the world coordinate system. 2 . The positioning method of the vision-based underground parking lot according to claim 1 , wherein the specific method in step S1 is: erecting a monocular camera at the front end of the vehicle, so that the optical axis of the camera is parallel to the vehicle body. 3 . 3. the positioning method of the underground parking lot based on vision according to claim 1, is characterized in that, in step S2, the parking lot environmental information is carried out feature extraction The concrete method is: S21: When acquiring the image sequence, uniform size transformation is performed, and the image sequence is sampled; S22: Use K-level image pyramid to extract FAST corner points; S23: Divide each layer of pyramids into grids, and extract at least L corner points from each grid; S24: If the number of corner points <L, increase the threshold and perform extraction again; S25 : According to the extracted FAST corners, use the Brief algorithm to calculate the orientation and ORB feature descriptors. 4. the positioning method of the described underground parking lot based on vision according to claim 3, is characterized in that, adopting Brief algorithm to calculate direction is specially: Step 1. Make a circle O with the key point P as the center and d as the radius; Step 2. Select N point pairs in a certain pattern in circle O; Step 3. Define Operation T Among them, I _A represents the gray value of A, and I _B represents the gray value of B; Step 4. Perform T operations on the selected point pairs respectively, and combine the obtained results. 5. the positioning method of the underground parking lot based on vision according to claim 4, is characterized in that, in step S3, the information after extraction and tracking is carried out feature matching specifically: adopt the nearest neighbor matching point method to obtain the final matching of this feature point. 6 . The positioning method of the vision-based underground parking lot according to claim 1 , wherein selecting a key frame in step S4 is specifically: when the number of inner points is greater than a certain number, determining the frame as a key frame. 7 . 7. the positioning method of the underground parking lot based on vision according to claim 1 is characterized in that, in step S5, the pose of calculating the vehicle in the parking lot is specifically: First normalize all feature points, and then solve the fundamental matrix and essential matrix according to the key frame; The base matrix is: x'Fx=0 in, is any pair of matching points of the two images; when enough matching points are given, this formula is used to calculate the unknown fundamental matrix F; The normalized essential matrix is: E=t×R=[t] _x ·R where E represents the essential matrix, t represents the translation vector, [t] _x represents the antisymmetric matrix of t, and the rotation matrix is R. 8. the positioning method of the described underground parking lot based on vision according to claim 7, it is characterized in that, represent the score of essential matrix with SH, SF represents the score of basic matrix, judge according to following judgment model, if: If the ratio is greater than 0.45, select the result obtained by the essential matrix; if the ratio is less than or equal to 0.45, select the result obtained by the fundamental matrix. 9. the positioning method of the underground parking lot based on vision according to claim 1, is characterized in that, the global pose of vehicle under S6 calculation world coordinate system is specifically: according to matching three-dimensional point pair, basic matrix and essential matrix, solve The points in the three-dimensional world corresponding to the feature points are accumulated with the pose parameters of each frame to obtain the global pose of the vehicle motion in the world coordinate system, that is, the real-time position and corner information in the parking lot.