CN112184818B - Vision-based vehicle positioning method and parking lot management system applying same - Google Patents

Abstract

The invention relates to a vision-based vehicle positioning method and a parking lot management system applying the method, and belongs to the field of informatization. The method comprises the following steps: (1) position calibration: establishing a three-dimensional world coordinate system and a parking space plane electronic map of the parking lot, carrying out position calibration on a unique identifier in the parking lot, and storing a three-dimensional world coordinate and a two-dimensional image plane coordinate of each parking space number; (2) image acquisition: acquiring environmental information around a vehicle body by using a vehicle-mounted camera; (3) and (3) identifying the number: analyzing and processing the acquired image to complete the identification of the parking space number; (4) position calculation: and reading corresponding two-dimensional image coordinates and three-dimensional world coordinates according to the identified garage number, and obtaining the position of the camera by utilizing an algorithm used by the camera to realize positioning. The invention can realize the parking space management and empty parking space searching functions and solve the problem of slow recovery of satellite signals when vehicles leave a garage. The experience of parking and getting the car of user can be better promoted.

Description

Vision-based vehicle localization method and parking lot management system using the same

technical field

The invention belongs to the field of informatization, and relates to a vision-based vehicle positioning method and a parking lot management system applying the method.

Background technique

In the underground parking lot without GPS signal coverage, most car owners will lose their direction and find a parking space in circles. The existing parking lot often only provides the number of free parking spaces, and it is not intuitive and convenient for car owners to find empty parking spaces, so they need to be indoors. The vehicle positioning method in the environment is used to realize the parking space navigation function in the underground parking lot.

Regarding the car positioning and navigation methods of underground parking lots, it can be divided into two categories, one is the method of transforming the parking lot or the car: adding base stations or deploying location tags, such as based on UWB, Ibeacon, WIFI, millimeter wave radar, RFID, etc. The device for indoor positioning technology, and CN201710095994.3 propose to deploy several guides in the parking lot to dynamically detect parking spaces and cars, and to guide cars to find parking spaces; there is also a two-dimensional code method: CN201610601248.2 proposes to mark with two-dimensional code Parking spaces are found by scanning devices. In addition to using two-dimensional codes, CN201710397595.2 is also equipped with a fish-eye camera to identify garage information. The SLAM technology is used to construct a garage plane distribution map, and the map results are used to achieve real-time high-speed autonomous vehicles. Accuracy of decimeter-level positioning; CN201910549396.8 is to install a two-dimensional code on the vehicle, and use the camera in the parking lot to identify the two-dimensional code on the vehicle body for continuous positioning. This kind of method has a small amount of calculation for accurate positioning, but needs to bear the cost of modification and equipment.

Another method is to build a model, for example: CN2016105562339 proposes to build a fingerprint map, build a fingerprint map based on the RSSI (Received Signal Strength Indication) value of the reference point coordinates and the RSSI value of the car to determine the position; CN201710379483.4 proposes signal strength and scene The positioning method of image double matching, first do signal strength matching, and then use scene images to obtain more accurate positions; CN201510708839.5 Pre-map road characteristics map: estimate T according to T _i-1 state (geographical position, driving speed and angular velocity) _i position, in addition, the vehicle's trajectory characteristics and map road characteristics are matched and positioned, and the two positions are fused to obtain the optimal position; CN201910838826.8 fuses external sensor data, GPS data, and high-precision map data for positioning. These methods involve calculations at the software and algorithm levels, and have problems such as higher system performance requirements and higher power consumption.

In addition, the CN201710847934.2 garage vehicle positioning and navigation method based on image feature recognition, which is similar to the positioning method of the present invention, locates and navigates by identifying the parking space number. The number approximately understands the lane where the car is located in its vicinity. However, if the current view of the car can only see the number of the parking space in front, this method will approximately think that the car is at the number of the parking space in front, resulting in a positioning error, so this method does not locate. Accurate; the second is that it does not introduce the identification method of the parking space number. It mentions the method of referring to the license plate recognition method to identify. This method will have the problem of inability to recognize the parking space number or the effect is poor. Different from the license plate, the parking space of the parking lot The numbering has the problem of uncertain location and no unified format requirements for identification, size, and direction. It is lack of consideration to directly apply the method of license plate recognition, and the characteristics of parking space numbering are not considered, so the numbering recognition method is insufficient.

In conclusion, it is necessary to provide a positioning method with low reconstruction cost and accurate positioning, and a reliable and efficient parking lot navigation management system.

SUMMARY OF THE INVENTION

In view of this, the purpose of the present invention is to provide a vision-based vehicle positioning method and a parking lot management system using the method.

For achieving the above object, the present invention provides the following technical solutions:

Vision-based vehicle localization method, the method includes the following steps:

S1: position calibration;

S2: image acquisition;

S3: number identification;

S4: Position solution.

Optionally, the S1 is specifically:

Establish the 3D world coordinate system, polar coordinate system, electronic map of parking spaces and 3D images of parking spaces, and perform location calibration for the signs in the parking lot, and store the 3D world coordinates, polar coordinates, and 2D images of each parking space number. Plane coordinates and 3D image coordinates.

Optionally, the S2 is specifically:

Use the vehicle-mounted camera to obtain the environmental information around the vehicle body; the environmental information includes: luminosity information, humidity information, obstacle information and communication signal information, the front pillar or the parking space number mark on the wall obtained by the vehicle-mounted recorder, and the data obtained by the cameras on both sides. The parking space number information on the ground.

Optionally, the S3 is specifically:

Analyze and process the collected images, complete the identification of parking space numbers, and select four valid image segmentation and identification of parking space numbers;

According to the identified garage number, read the corresponding two-dimensional image coordinates, three-dimensional world coordinates and polar coordinates, and use the vehicle positioning algorithm to obtain the position of the camera to achieve positioning;

The garage numbering method includes the following steps:

S41: Image preprocessing: image denoising, binarization and morphological operation processing; use Gaussian blur to reduce noise, open operation and weighting to enhance contrast, binarization and Canny edge detection to find object contours, use first close The back open operation finds the rectangular position of the entire block;

S42: Effective area positioning: comprehensively detect the parking space number position in the picture based on color features, geometric features and texture features, several areas that meet the characteristics are selected as candidate areas, and the area with the largest outline size is selected as the effective position area, that is, the closest parking space number position , and then separate it from the image;

Using the feature extraction method, the candidate area is obtained by detecting the color, geometry and texture features of the parking space number:

Mark the collected parking space number pictures and use a program called LabelImg to create a data set, and train the network, and then use the trained network to identify the collected pictures, get an image that outlines the candidate area, and then filter the candidate area to select the effective area. , here 4 areas are set as effective areas to facilitate the subsequent position calculation, and the selected 4 effective areas are divided from the picture;

S43: character segmentation and recognition; character segmentation is performed by synchronous processing of multiple valid areas, and a single character image is obtained by segmentation;

Character segmentation uses a combination of vertical projection and horizontal projection, and uses the peak and trough analysis of the histogram of the pixel distribution of the binarized image to find the boundary points of adjacent characters for segmentation;

Or select the connected domain segmentation method to find blocks with continuous text. If the length is greater than a certain threshold, it is considered that the block consists of two characters and needs to be divided;

S44: Character Recognition: Recognize the obtained single character character image, combine the results to realize the task of parking space number recognition; choose a template matching algorithm, binarize the segmented character and scale its size to the size of the template in the character database , then match all templates, and select the best match as the result;

Or choose the classification algorithm of support vector machine SVM, and train two SVM classifiers, one SVM is used to recognize uppercase letters, and the other SVM is used to recognize numbers;

Or choose a method based on a deep neural network, input the image into the network, and the network automatically realizes feature extraction until the result is recognized;

The vehicle positioning method uses a vehicle positioning algorithm to obtain the position of the camera, and realizes specific positioning: according to the obtained three-dimensional world coordinates, polar coordinates and two-dimensional image coordinates of a plurality of points, the camera is obtained in the three-dimensional world coordinate system and the polar coordinate system. Position coordinates, the vehicle positioning algorithm is specifically:

(1) Calculate the three-dimensional coordinates and polar coordinates of multiple points in the current camera coordinate system according to the properties of the triangle and the internal parameters of the camera;

(2) According to the three-dimensional coordinates in the world coordinate system and the three-dimensional coordinates and polar coordinates in the current camera coordinate system, the iterative closest point ICP algorithm is used to obtain the camera rotation matrix R and the translation vector t, and the camera pose is obtained.

Optionally, the S4 is specifically:

Pass the identified garage number to the memory, read the corresponding two-dimensional map coordinates and three-dimensional world coordinates, and obtain the position of the camera, that is, the position of the car according to the geometric projection algorithm, so as to realize the positioning; the geometric projection algorithm is based on the obtained multiple points. The algorithm for obtaining the three-dimensional position coordinates of the camera from the three-dimensional world coordinates and two-dimensional image coordinates of , the specific implementation process is:

S41: Read two-dimensional map coordinates of multiple parking space numbers, denoted as point set R={r ₁ , _{r 2} ,..., rn } and three-dimensional world coordinates, denoted as point set P _w ={p _w1 , p _w2 ,...,p _wn };

S42: Calculate the spatial distance between any two points according to the spatial coordinates of the multiple points in the point set Pw by using the distance formula, and calculate the distance between any two points and the camera optical center according to the corresponding two-dimensional coordinate _r and the cosine theorem and the triangle similarity theorem cosine of the included angle;

S43: Construct a constraint equation according to the pinhole imaging model and the cosine theorem, and calculate the three-dimensional coordinates of multiple points in the current camera coordinate system by solving the parameters, which are recorded as point set Q _c ={q _c1 ,q _c2 ,...,q _cn };

S44: According to the three-dimensional coordinate point P _w in the multi-point world coordinate system and the three-dimensional coordinate Q _c in the current camera coordinate system, use the SVD algorithm to solve the following formulas to obtain the camera rotation matrix R and the translation vector t;

t ^* = μ _q − R μ _p where

Obtain the coordinate position of the camera in world coordinates according to the following formula, denoted as O _w ;

O _w = -R ^-1 t.

A parking lot management system applying the method, including a parking lot cloud service center and a vehicle terminal;

The cloud service center includes a data storage module, a communication module, a connection management module, a storage management module, a storage management module, a parking space management module, a positioning module, a navigation module and an empty parking space search module;

The vehicle-mounted terminal includes a communication module, a map processing and display module and an image acquisition and identification module.

Optionally, the data storage and calculation are cloud-based cloud computing and cloud storage, which supports multi-terminal cloud interactive operations; summarizes all electronic cloud maps of parking lots in the area, and the interface supports on-board maps or Baidu AutoNavi maps. Access;

The communication module is connected to the terminal for communication, including a map interface, a vehicle-mounted mobile terminal interface and a mobile phone terminal, and the communication mode is WIFI, bluetooth or mobile network data communication mode according to the needs of different occasions;

The connection management module is a module that processes various requests sent by external interfaces and terminals, and transfers them to corresponding functions after security verification;

The positioning module includes the use and idle state of the parking space, realizes the positioning of the vehicle according to the positioning method proposed above, and provides it to the navigation module;

The navigation module is to process the received positioning of each frame, and in combination with the electronic map of the parking lot, trace and mark the current position on the map to realize car tracking;

The storage management module is responsible for the task scheduling of cars entering the parking lot, and realizes storage tasks by calling other functional modules;

The outgoing management module is responsible for the task scheduling of cars leaving the parking lot, and realizes outgoing tasks by calling other functional modules;

The parking space management module records and updates the use and idle state of the parking space;

The empty parking space search module is responsible for searching for empty parking spaces;

The map processing and display module processes and displays the received map data;

The image acquisition and recognition module calls the camera to acquire image information around the vehicle body, and then recognizes the parking space number.

The beneficial effects of the present invention are as follows: the present invention proposes two solutions, one is a positioning method, which uses the image information collected by the camera to detect the parking space number, and uses a multi-point geometric positioning algorithm to obtain accurate vehicle position and attitude according to multiple valid positions. , thereby realizing the positioning of the indoor parking lot, realizing that the coordinates of the parameter points can be detected without adding a base station or equipment, and reducing the equipment cost and renovation cost of the car and the parking lot. On the other hand, the parking lot management system based on the positioning method uses cloud computing, cloud storage and high-bandwidth communication technology to construct an efficient navigation management system, which realizes the navigation functions of vehicles entering and leaving the warehouse and mobile phone search, and parking space management. and empty parking space search function, and solve the problem of slow recovery of satellite signals when vehicles leave the warehouse. It can better improve the user's parking and pickup experience.

Other advantages, objects, and features of the present invention will be set forth in the description that follows, and will be apparent to those skilled in the art based on a study of the following, to the extent that is taught in the practice of the present invention. The objectives and other advantages of the present invention may be realized and attained by the following description.

Description of drawings

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be preferably described in detail below with reference to the accompanying drawings, wherein:

1 is a flow chart of a method for locating a vehicle in an indoor parking lot according to the present invention;

Fig. 2 is the flow chart of the identification parking lot garage numbering method of the present invention;

Fig. 3 is the flow chart of the position solving algorithm of the camera of the present invention;

Fig. 4 is the system function module diagram of the parking lot management system provided by the present invention;

Fig. 5 is the flow chart that the present invention realizes the task of navigating and warehousing the vehicle into the parking lot;

FIG. 6 is a flow chart of the present invention for realizing the task of navigating out of the parking lot and out of the warehouse.

Detailed ways

The embodiments of the present invention are described below through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following embodiments are only used to illustrate the basic idea of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

Among them, the accompanying drawings are only used for exemplary description, and represent only schematic diagrams, not physical drawings, and should not be construed as limitations of the present invention; in order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings will be omitted, The enlargement or reduction does not represent the size of the actual product; it is understandable to those skilled in the art that some well-known structures and their descriptions in the accompanying drawings may be omitted.

The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms “upper”, “lower”, “left” and “right” , "front", "rear" and other indicated orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must be It has a specific orientation, is constructed and operated in a specific orientation, so the terms describing the positional relationship in the accompanying drawings are only used for exemplary illustration, and should not be construed as a limitation of the present invention. situation to understand the specific meaning of the above terms.

FIG. 1 is a flow chart of a method for locating a vehicle in an indoor parking lot according to the present invention. The method includes:

(1) Position calibration: establish a three-dimensional world coordinate system of the parking lot, perform position calibration on the garage number in the parking lot, and store the three-dimensional world coordinates of each parking space number, marked as capital letters plus w subscript, such as X _w ; Make an electronic map of the parking lot, mark the two-dimensional coordinates of the parking space number in the two-dimensional map coordinate system, and mark it as a lowercase letter, such as x.

(2) Image collection: Use the on-board camera on the car to take pictures of the surrounding garage. It can be the car recorder in front to obtain the parking space number mark on the front pillar or the wall, or the cameras on both sides can obtain the parking space number information on the ground.

(3) Number recognition: process the collected images, and select four valid parking space number images for segmentation and recognition.

The process flow of the number recognition method is shown in Figure 2, which includes the following steps:

① Image preprocessing: image denoising, binarization and morphological operations. In a specific embodiment, Gaussian blur can be used to reduce noise, open operation and weighting can be used to enhance contrast, binarization and Canny edge detection can be used to find the outline of an object, and a first-close-then-open operation can be used to find the rectangular position of the entire block;

②Effective area positioning: Based on color features, geometric features and texture features, the parking space number position in the picture is comprehensively detected, and several areas that meet the characteristics are selected as candidate areas, and the ones with larger outline size are selected as the effective location areas, that is, the closest ones. Several parking space number locations and then separate them from the image. In a specific embodiment, a traditional feature extraction method can be used to detect and obtain a candidate area according to the color, geometry and texture features of the parking space number: the color feature can be two combinations of light-colored characters on a dark background and dark-colored characters on a light-colored background, The geometric features can be rectangles or quadrilaterals, and the texture features can be unique texture features with a regular arrangement of one letter and three digits; you can also use the deep network method to calibrate the data set and train the target detection network (Yolo or other) to achieve automatic extraction. The feature obtains the candidate area, considering that the deep learning method has better robustness. Here, the method of training the deep network is selected to obtain the candidate area. The obtained candidate area is further analyzed and judged, and the outline and aspect ratio of the candidate area are screened to obtain an effective area, and several areas with larger areas are selected for identification. The specific process is: first mark the collected parking space number pictures, use a program named LabelImg to create a data set, and train the network, and then use the trained network to identify the collected pictures, get the images that frame the candidate area, and then analyze the candidate areas. The area is selected as the effective area for screening. Here, 4 areas are set as the effective area to facilitate the subsequent position calculation. Finally, the selected 4 effective areas are divided from the picture;

③Character segmentation: Perform character segmentation on multiple valid areas synchronously to obtain a single character image. The specific implementation is that the character segmentation can use a combination of vertical projection and horizontal projection, and use the peak and trough analysis of the histogram of the pixel distribution of the binarized image to find the boundary points of adjacent characters for segmentation, or choose to connect. The domain segmentation method looks for blocks with continuous text. If the length is greater than a certain threshold, the block is considered to be composed of two characters and needs to be divided. Taking the connected domain segmentation method as an example, the specific implementation process is to first perform a grayscale operation, then use the Canny operator to detect and dilate the edges, and then use the method of marking connected areas in the skimage.measure module to achieve character segmentation, and finally extract a single Character images for recognition;

④Character recognition: recognize the obtained single character character image, and combine the results to realize the task of parking space number recognition. The specific implementation method is that based on the template matching algorithm, firstly, the segmented characters are binarized and their size is scaled to the size of the template in the character database, and then all templates are matched, and the best match is selected as the result. ; You can also choose a classification algorithm based on a machine learning algorithm - Support Vector Machine (SVM), and train two SVM classifiers, one SVM is used to recognize capital letters (such as B), and the other SVM is used to recognize numbers; You can also choose a method based on a deep neural network, input the image into the network, and the network automatically realizes feature extraction until the result is recognized. Taking the method of convolutional neural network (CNN) as an example, the specific implementation process is to first train a CNN with a data set of numbers and letters and save the weight data, and then use the network to identify each character image after segmentation, and store the result. The complete number is obtained by outputting in the array.

(4) Position calculation: transfer the identified garage number to the memory, read the corresponding two-dimensional map coordinates and three-dimensional world coordinates, and obtain the position of the camera, that is, the position of the car according to the geometric projection algorithm, so as to realize the positioning and position solution. The flow of the calculation algorithm is shown in Figure 3. The geometric projection algorithm is an algorithm for obtaining the three-dimensional position coordinates of the camera according to the obtained three-dimensional world coordinates and two-dimensional image coordinates of multiple points. The specific implementation process is as follows:

①Read two-dimensional map coordinates of multiple parking space numbers (denoted as point set R={r ₁ , _{r 2} ,...,rn }) and three-dimensional world coordinates (denoted as point set P _w ={p _w1 , p _w2 ,...,p _wn });

_②According to the spatial coordinates of multiple points in the point set Pw, the spatial distance between any two points is calculated by the distance formula, and the distance between any two points and the camera optical center is calculated according to the corresponding two-dimensional coordinate r and the cosine theorem and the triangle similarity theorem. cosine of the angle;

③Construct the constraint equation according to the pinhole imaging model and the cosine theorem, and solve the parameters to calculate the three-dimensional coordinates of multiple points in the current camera coordinate system (denoted as point set Q _c ={q _c1 ,q _c2 ,...,q _cn ) });

④ According to the three-dimensional coordinate point (P _w ) in the multi-point world coordinate system and the three-dimensional coordinate (Q _c ) in the current camera coordinate system, use the SVD algorithm to solve the following formulas to obtain the camera rotation matrix R and translation vector t;

t ^* = μ _q − R μ _p where

⑤ According to the following formula, the coordinate position of the camera in the world coordinate (denoted as O _w ) is obtained.

O _w = -R ^-1 t

FIG. 4 is a functional block diagram of a parking lot management system proposed by the present invention. The system includes a parking lot cloud service center and a vehicle terminal. It is characterized in that: the cloud service center includes a data storage module, a communication module, a connection management module, a storage management module, a storage management module, a parking space management module, a positioning module, a navigation module, and an empty parking space search module; the vehicle terminal includes a communication module. module, map processing display module, image acquisition and recognition module. Among them, data storage and computing are cloud-based cloud computing and cloud storage, which can support multi-terminal cloud interactive operations, summarize all parking lot electronic cloud maps in a certain area (urban area), and provide interfaces to support car maps or Baidu high-speed Access to German maps; the communication module is a communication module with other interfaces, such as the communication module of the service center to communicate with the map interface, vehicle mobile terminal, mobile terminal program, etc., and the vehicle communication module to exchange information with the service center, mobile phones, etc.; The connection management module is used to process various requests sent by external systems to ensure the security of the system, such as connecting parking lot map requests, car warehousing requests, warehousing requests, etc., to perform security authentication on the connection, and then transfer to Corresponding functional modules; the positioning module is to realize the positioning of the vehicle according to the positioning method proposed above, and provide the navigation module for navigation; the navigation module is to receive the positioning processing of each frame, and combine the electronic map of the parking lot to draw dot marks on the map The current position is used to realize car tracking; the out/in storage management module is responsible for the task scheduling of cars driving out/in to the parking lot, and the out/in storage tasks are realized by calling other functional modules; the parking space management module is to record the use and idle state of the parking space, Specifically, the corresponding parking space is indicated to be parked according to the parking request for entering the warehouse, and the corresponding parking space is indicated to be free according to the parking request; , in which a request for finding an empty parking space is sent at the warehousing and vehicle end and is called back; the map processing and display module of the vehicle terminal is responsible for processing the received map data and displaying it on the display screen of the car; the image acquisition and recognition module is to call the camera to collect The image information around the car body is then uploaded to the service center according to the identification method to identify the surrounding parking space numbers.

The parking lot management system can realize the positioning and navigation functions of the vehicle in the indoor environment, as well as the navigation map switching and quick location recovery functions of the vehicle entering and leaving the parking lot. The server sends a car search request, finds the corresponding parking space number according to the car matched by the mobile phone, and calls the mobile phone camera for positioning and navigation, so as to find the parking space.

In order to better apply the present invention, the basic work required is: first, make a digital map for the parking lot in the current area, establish a three-dimensional world coordinate system of the parking lot, perform position calibration for the garage number in the parking lot, and store each parking space. The numbered three-dimensional world coordinates are recorded as point set P _w ={p _w1 ,p _w2 ,...,p _wn }; make an electronic map of the parking lot, mark and store the two-dimensional parking space number in the two-dimensional map coordinate system Coordinates, denoted as point set R={r ₁ , _{r 2} ,...,rn }. And the map interface of the parking lot is provided to the service center, so that the service center can call the parking lot data.

When the passenger car with the license plate number Yu A12345 needs parking, it will send a parking request to the service center, search for vacancies in nearby parking lots and select the nearest parking lot with vacant parking spaces; the car arrives at the entrance of the parking lot, and the vehicle is navigated into the warehouse The task, the process is shown in Figure 5:

(1) The car sends a parking request to the service center, including the vehicle identity (foreign suspended vehicle), current location and storage request;

(2) The communication module of the parking lot service center receives the request of the car user in step (1) for connection management, and after confirmation, forwards the vehicle identity and location information to the warehousing management module, and transmits the electronic map of the parking lot to the car end and provides it Recommend parking spaces, load the electronic map of the parking lot on the car end, process it through the map processing and display module, and then display it to the driver on the electronic screen;

(3) The warehousing management module transmits the vehicle information to the data storage module, and sends a positioning request to the vehicle end. The vehicle end calls the image acquisition and recognition module to collect the image of the environment, and recognizes the parking space number according to the recognition algorithm. : The specific steps are to first use the detection network to obtain the image of the candidate area, and then select 4 effective areas to divide them; then synchronously process the 4 effective areas, perform grayscale operation, use Canny operator edge detection and expansion, and then use skimage The method of marking connected regions in the .measure module realizes character segmentation and extracts a single character image for recognition; finally, the trained cnn is used to recognize characters, and the result is stored in an array to output the complete parking space number;

(4) The vehicle-end image acquisition and recognition module repeatedly reads the image sequence until four valid parking space numbers are identified;

(5) Pass the numbering information to the positioning module of the service center, which reads the corresponding 3D world coordinates and 2D image coordinates to calculate the current position according to multiple parking space numbers; the concrete steps are to pass the identified four garage numbers to the memory, Read the corresponding two-dimensional map coordinates (marked as point set {r ₁ , r ₂ r ₃ , r ₄ }) and three-dimensional world coordinates (marked as point set {p _w1 , p _w2 , p _w3 , p _w4 } ), According to the spatial coordinates of multiple points in the point set, the spatial distance between any two points is calculated by the distance formula, and the cosine value of the angle between any two points and the camera optical center is calculated according to the corresponding two-dimensional coordinate r and the cosine theorem and the triangle similarity theorem ;Construct the constraint equation according to the pinhole imaging model and the cosine theorem, and calculate the three-dimensional coordinates of multiple points in the current camera coordinate system (referred to as point set {q _c1 ,q _c2 ,q _c3 ,q _c4 }); The three-dimensional coordinate point (P _w ) under the multi-point world coordinate system and the three-dimensional coordinate (Q _c ) under the current camera coordinate system are solved by using the SVD algorithm to obtain the camera rotation matrix R and the translation vector t; then according to the formula O _w =- R ^-1 t obtains the coordinate position of the camera in the world coordinates (denoted as O _w ), that is, obtains the current position coordinates of the car;

(6) The positioning module sends the current position to the navigation module, which forms a path plan according to the current position and the position of the recommended parking space, transmits the position and the path to the car, and displays it on the electronic map;

(7) Repeat positioning and navigation until the car reaches the target position;

(8) If the parking space at the target location is free, then end the navigation task; otherwise, the vehicle end sends a request for finding a free parking space, and the service center calls the empty parking space search module to find the nearest free parking space and send the number and location to the navigation module;

(9) Repeat (8) to generate a navigation path, and combine the positioning function to navigate and drive the car until a free parking space is found;

(10) The vehicle terminal sends a confirmation parking request, and transmits the parking space information (license plate number and parking space number) to the service center. The parking space management module updates the corresponding parking space status as occupied, and the corresponding license plate number is Yu A12345, and the warehousing task Finish.

So it was easy to find a parking space for the car. Then the car owner can also find his car through the mobile phone. The specific process is similar to the navigation of the vehicle. The camera of the mobile phone is called, and the number information of the current environment is identified by the above parking space number identification method. Four valid numbers are found and sent to the service center to settle the location. And call the navigation module to direct the owner to find the car. Then the car is about to leave the parking lot. Figure 6 shows the flow chart of the task of navigating and leaving the warehouse when the vehicle leaves the parking lot:

(1) The car end sends a warehouse request to the service center, the service center processes the request, and calls the warehouse management module;

(2) The outbound management module calls the parking space management module to change the corresponding parking space status to idle, and at the same time calls the communication module to transmit the electronic map of the parking lot, the exit location and the navigation path to the car;

(3) The car terminal displays the received information on the vehicle display screen, and calls the image acquisition and processing module, uses the recognition algorithm to identify the parking space number information in the environment, and repeatedly reads the image sequence until four valid parking space numbers are obtained. ;

(4) The car end sends the parking space number to the service center, and calls the positioning and navigation module to perform position calculation and positioning and position path information navigation until the car reaches the exit of the parking lot;

(5) When arriving at the exit, the service center marks the exit position of the parking lot on the car navigation map, and uses it to integrate with the satellite positioning data to quickly restore the navigation function, and the outbound task ends.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be Modifications or equivalent replacements, without departing from the spirit and scope of the technical solution, should all be included in the scope of the claims of the present invention.

Claims (4)

1. The vision-based vehicle positioning method is characterized by comprising the following steps: the method comprises the following steps:

s1: position calibration;

s2: collecting an image;

s3: identifying a serial number;

s4: position resolving;

the S1 specifically includes:

establishing a three-dimensional world coordinate system, a polar coordinate system, a parking space plane electronic map and a three-dimensional image of a parking space of the parking lot, calibrating the position of an identifier in the parking lot, and storing the three-dimensional world coordinate, the polar coordinate, a two-dimensional image plane coordinate and a three-dimensional image coordinate of each parking space number;

the S2 specifically includes:

acquiring environmental information around a vehicle body by using a vehicle-mounted camera; the environment information includes: luminosity information, humidity information, barrier information and communication signal information, a vehicle-mounted recorder acquired front pillar or a vehicle position mark on a wall, and vehicle position information on the ground acquired by cameras on two sides;

the S3 specifically includes:

analyzing and processing the collected images, completing the identification of the parking space number, and selecting effective four parking space number images for segmentation and identification;

reading corresponding two-dimensional image coordinates, three-dimensional world coordinates and polar coordinates according to the identified parking space number, and obtaining the position of the camera by using a vehicle positioning algorithm to realize positioning;

the parking space numbering method comprises the following steps:

s31: image preprocessing: denoising, binaryzation and morphological operation processing are carried out on the image; reducing noise by Gaussian blur, strengthening contrast by opening operation and weighting, finding an object contour by binarization and Canny edge detection, and finding a whole rectangular position by closing before opening operation;

s32: and (3) positioning an effective area: comprehensively detecting the parking space number position in the picture based on the color feature, the geometric feature and the texture feature, taking a plurality of regions conforming to the feature as candidate regions, selecting the region with the largest outline size as an effective position region, namely the position of the parking space number closest to the effective position region, and then separating the parking space number position from the image;

detecting according to the color, the geometry and the texture characteristics of the parking space number by using a characteristic extraction method to obtain a candidate area:

marking the acquired parking space number picture, making a data set by using a program named LabelImg, training a network, identifying the acquired picture by using the trained network to obtain an image for framing a candidate area, screening the candidate area to select an effective area, setting 4 areas as the effective areas to facilitate subsequent position resolution, and segmenting the selected 4 effective areas from the picture;

s33: character segmentation and recognition; synchronously processing a plurality of effective areas to carry out character segmentation, and obtaining a single character image by segmentation;

the character segmentation uses a vertical projection and horizontal projection combined method, and utilizes the peak and trough analysis of a pixel distribution histogram of a binary image to find out the boundary points of adjacent characters for segmentation;

or selecting a connected domain segmentation method, searching a block with continuous characters, if the length of the block is greater than a certain threshold value, determining that the block consists of two characters and needs to be segmented;

s34: character recognition: identifying the obtained single character image, combining the results and realizing the parking space number identification task; selecting a template matching algorithm, binarizing the segmented characters, scaling the binary characters to the size of a template in a character database, matching all the templates, and selecting the best matching as a result;

or selecting a classification algorithm of a Support Vector Machine (SVM), and training two SVM classifiers, wherein one SVM is used for recognizing capital letters, and the other SVM is used for recognizing numbers;

or selecting a method based on a deep neural network, inputting the image into the network, and automatically realizing feature extraction by the network until a result is identified;

the vehicle positioning method utilizes a vehicle positioning algorithm to obtain the position of the camera, and realizes specific positioning: obtaining the position coordinates of the camera under a three-dimensional world coordinate system and a polar coordinate system according to the obtained three-dimensional world coordinates, polar coordinates and two-dimensional image coordinates of the plurality of points, wherein the vehicle positioning algorithm specifically comprises the following steps:

(1) according to the properties of the triangle and the internal parameters of the camera, three-dimensional coordinates and polar coordinates of a plurality of points under the current camera coordinate system are obtained;

(2) and according to the three-dimensional coordinates in the world coordinate system and the three-dimensional coordinates and polar coordinates in the current camera coordinate system, solving a camera rotation matrix R and a translation vector t by using an iterative closest point ICP (inductively coupled plasma) algorithm, and solving the camera pose.

2. The vision-based vehicle localization method of claim 1, wherein: the S4 specifically includes:

the recognized parking space number is transmitted to a memory, corresponding two-dimensional map coordinates and three-dimensional world coordinates are read, and the position of a camera, namely the position of an automobile, is obtained according to a geometric projection algorithm, so that positioning is realized; the geometric projection algorithm is an algorithm for obtaining the three-dimensional position coordinate of the camera according to the obtained three-dimensional world coordinate and the two-dimensional image coordinate of a plurality of points, and the specific implementation flow is as follows:

s41: two-dimensional map coordinates for reading a plurality of parking space numbers are recorded as a point set R ═ R₁,r₂,...,r_nAnd three-dimensional world coordinates, denoted as a set of points P_w＝{p_w1,p_w2,...,p_wn}；

S42: according to point set P_wCalculating the space coordinates of the middle points and the multiple points by using a distance formula to obtain the space distance between any two points, and calculating the cosine value of the included angle between any two points and the optical center of the camera according to the corresponding two-dimensional coordinates r and the cosine theorem and the triangle similarity theorem;

s43: constructing a constraint equation according to a pinhole imaging model and a cosine theorem, solving parameters and calculating to obtain three-dimensional coordinates of a plurality of points in a current camera coordinate system, and recording as a point set Q_c＝{q_c1,q_c2,...,q_cn}；

S44: according to three-dimensional coordinate point P under the multi-point world coordinate system_wAnd three-dimensional coordinate Q under current camera coordinate system_cSolving the following formula by using an SVD algorithm to obtain a camera rotation matrix R and a translational vector t;

wherein

The coordinate position of the camera under the world coordinate is obtained according to the following formula and is marked as O_w；

O_w＝-R^-1t。

3. A parking lot management system to which the method according to any one of claims 1 to 2 is applied, characterized in that: the system comprises a parking lot cloud service center and a vehicle-mounted terminal;

the cloud service center comprises a data storage module, a communication module, a connection management module, a warehousing management module, a ex-warehouse management module, a parking space management module, a positioning module, a navigation module and an empty parking space searching module;

the vehicle-mounted terminal comprises a communication module, a map processing and displaying module and an image acquisition and identification module.

4. The parking lot management system according to claim 3, characterized in that:

the data storage and the computation are cloud computing and cloud storage based on cloud, and cloud interactive operation of multiple terminals is supported; summarizing all parking lot electronic cloud maps in the area, wherein an interface supports access of a vehicle-mounted map or a Baidu high-grade map;

the communication module is in communication connection with the terminal and comprises a map interface, a vehicle-mounted mobile terminal interface and a mobile phone terminal, and the communication mode selects a WIFI, Bluetooth or mobile network data communication mode according to different occasion requirements;

the connection management module is a module for processing various requests sent by external interfaces and terminals, and transferring the requests to corresponding functions after security verification;

the positioning module comprises the use and idle states of the parking spaces, realizes the positioning of the vehicle according to the positioning method and provides the positioning method for the navigation module;

the navigation module processes the positioning of each frame of received image, and points are drawn on the map to mark the current position by combining the electronic map of the parking lot, so as to realize automobile tracking;

the warehousing management module is responsible for task scheduling of driving automobiles into the parking lot and realizes warehousing tasks by calling other functional modules;

the ex-warehouse management module is responsible for task scheduling of automobiles exiting the parking lot and realizes ex-warehouse tasks by calling other function modules;

the parking space management module records and updates the use and idle states of the parking spaces;

the empty parking space searching module is responsible for searching empty parking spaces;

the map processing and displaying module is used for processing and displaying the received map data;

the image acquisition and identification module is used for acquiring image information around the vehicle body by calling a camera and then identifying the parking space number.

Images