CN102254345A - Method for registering natural characteristic based on cloud computation - Google Patents

Abstract

The invention discloses a cloud computing-based natural feature registration method. The method first extracts the natural feature point set containing the object to establish a coordinate system; then extracts the natural feature point from the key frame captured by the camera; then matches the natural feature point based on the cloud; finally modifies the space according to the matched natural feature point coordinates, register the virtual object, and render and display it. This method uses commercialized cloud computing to improve the matching speed of natural features, enabling augmented reality marker-free registration to meet real-time and accurate requirements.

Description

Natural Feature Registration Method Based on Cloud Computing

technical field

The invention relates to an augmented reality registration technology, in particular to a cloud computing-based natural feature registration method.

Background technique

As people's requirements for interactive experience continue to increase, augmented reality (Augmented Reality, AR) applications have developed rapidly. Registration is one of the important technologies of augmented reality. Registration is the consistency of the positions of virtual objects and real scenes in three-dimensional space, that is, the integration in space. Tracking and registration is a continuous dynamic process. When the user moves the point of view, the virtual object must be registered with the real object position that the user sees. Therefore, the system must know the exact position and direction of the virtual object, and large registration errors caused by untimely tracking will lead to inaccurate display results. With the change of application requirements, the augmented reality application environment has developed from an indoor marked application environment to an outdoor and unmarked natural environment. Using natural features to accurately register virtual objects in real time has become one of the important research directions.

For augmented reality registration research, such as the Chinese patent: its name is "A New Outdoor Augmented Reality Unmarked Tracking Registration Algorithm", application number 201010523833.8, this invention uses two kinds of features, texture and contour, to express the scene, and proposes Registration based on mixed feature tracking, the algorithm of this invention is complicated, and the real-time effect is not ideal. Such as the Chinese patent: its name is "a three-dimensional registration method and system for augmented reality based on marker points", application number 200710118266.6, this invention projects the generated invisible light marker points onto the bearing surface of the real environment, and adopts an invisible light filter The light film camera shoots the invisible light mark points projected on the bearing surface, and obtains the two-dimensional coordinate data of the invisible light mark points on the screen to register. This invention has high requirements on the camera hardware and has little practical value. . Such as the Chinese patent: its name is "multi-planar dynamic augmented reality registration method based on homography matrix", application number 201010535231.4, according to the special attributes of the features in the scene, it automatically recognizes the known positions in the real environment to realize three-dimensional Registration, using color and shape information for multi-plane dynamic augmented reality registration, but this invention is greatly affected by the external environment.

The discovery of the natural features of objects is still an important prerequisite in the current registration algorithms based on natural features, and the real-time and accuracy requirements of these methods cannot meet the application requirements. Mainly due to insufficient computer performance and inflexible allocation.

Contents of the invention

In view of the problems and deficiencies in the prior art described above, the object of the present invention is to provide a natural feature registration method based on cloud computing. The method first extracts a set of natural feature points from the object containing the object, and establishes a coordinate system; Extract the natural feature points from the captured key frames; then match the natural feature points based on the cloud; finally modify the spatial coordinates according to the matched natural feature points, register the virtual object, and render and display.

In order to achieve the above object, the present invention adopts the following technical idea: according to the principle of photography, computer vision technology, optical technology, cloud computing technology, etc., the speed of natural feature point matching is improved through distributed computing to meet the real-time requirements.

The present invention adopts the following technical solutions to realize the above-mentioned natural feature registration method based on cloud computing, and the steps are as follows:

1. Extract natural feature points and establish a coordinate system: use the camera to shoot the front view of the object to be registered, and use it as a reference image to extract natural feature points from it. And it can be divided according to the area that needs to be registered, and the natural feature points outside the area can be removed. By obtaining natural feature points, a three-dimensional space coordinate system is established, and the projection matrix of the camera is obtained at the same time.

2. Extraction of natural feature points in key frames: the real environment is captured by the camera, and the key frames are extracted. Natural feature extraction is performed on key frames, and the extracted natural feature points are used as points to be matched.

3. Cloud-based matching: use distributed computing, one of the key technologies in cloud computing, to match natural feature points.

1), group the natural feature points obtained in step 2 according to the number of matching nodes. The number of natural feature points is used as the original data, that is, <key, -1>, where key is the number of each natural feature point; -1 is the initial value of the similarity of natural feature points, and -1 represents a mismatch.

2) Send the original data <key, -1> of each group to the matching node for similarity calculation. By calculating the similarity with the natural feature points extracted from the reference image, the data <key, similarity> is obtained, and similarity is the similarity value between the number of the natural feature points in the reference image and the response.

3) Send the data <key, similarity> with the same key to the same judgment node, and divide the same natural feature point with the highest similarity value and the second highest similarity value with the natural feature point in the reference image Calculated, if it is less than the specified threshold, it is considered as a matching feature point. For example, using the distance formula as the similarity calculation, it can be expressed as:

Closest distance/second close distance<threshold

Thus, the preliminary matching result <key, key_match> between the natural feature points of the image to be matched and the natural feature points of the reference image is obtained, where key_match is the number of the natural feature point of the reference image matched with the natural feature point key. According to these preliminary matching results, correct matching results are obtained by eliminating mismatches.

4. Modify the spatial coordinates, register and render the virtual object: According to the matching relationship between the natural feature points and the natural feature points in the reference image, modify the three-dimensional space coordinate system, and calculate the virtual object in this coordinate system, so as to obtain registration and make it In the correct three-dimensional space position, and render the display. After the end, go to step 2 for a new cycle.

Compared with the prior art, the present invention has the following obvious outstanding features and significant advantages: the present invention uses distributed computing in cloud computing to perform matching calculations on natural feature points, is not limited by insufficient computer performance, and is flexible in allocation, greatly improving the computing power. The speed meets the real-time requirements. Different from grid computing, cloud computing has been commercialized, so that users can get the computing power of supercomputers at any time and anywhere as long as they are connected to the network, making mobile augmented reality technology possible. And the present invention has good versatility, for example, existing mature algorithms such as SIFT and FAST algorithms can be used in the natural feature point extraction algorithm. If there is a better natural feature point extraction algorithm in the future, you only need to replace this module, and you can easily upgrade it.

Description of drawings

Fig. 1 is a program block diagram of the natural feature registration method based on cloud computing in the present invention;

Fig. 2 is a schematic diagram of cloud-based matching in the present invention.

Detailed ways

An embodiment of the present invention will be described in detail below in conjunction with the accompanying drawings.

A preferred embodiment of the present invention, as shown in Figure 1, this natural feature registration method based on cloud computing includes steps as follows:

①. Extract the natural feature point set and establish the coordinate system;

②. Extraction of natural feature points in key frames;

③. Cloud-based matching;

④. Modify the space coordinates, register and render the virtual object.

As described in the above step ①, the natural feature point set is extracted and the coordinate system is established. The specific steps are as follows:

(1) Take a picture of the front view of the object to be registered through the camera as a reference picture;

(2) Use the SIFT algorithm to extract natural feature points;

(3) Divide the areas that need to be registered;

(4) Remove SIFT feature points outside the registration area;

(5) Obtain the projection matrix of the camera;

(6) Establish a three-dimensional space coordinate system according to the obtained SIFT feature points.

As described in the above step ②, the extraction of natural feature points in the key frame, the specific steps are as follows:

(7) The camera shoots the real environment and extracts the key frames;

(8) SIFT feature point extraction is performed on the key frame as the point to be matched.

As described in the above step ③, cloud-based matching, a schematic diagram of cloud-based matching with N matching nodes and N judging nodes, as shown in Figure 2 . The specific steps are as follows:

(9), according to the number of matching nodes, group the SIFT feature points obtained in step ②, for example, the number of matching nodes in Figure 2 is 3, that is, all SIFT features are divided into 3 groups of original data;

(10) Record the serial number of the SIFT feature point as key to form the original data <key, -1> and send it to the matching node;

(11) In the matching node, the Euclidean distance calculation is performed between each input SIFT feature point and the feature point in SIFT in the reference image;

(12), record the number of SIFT feature points in the reference graph of Euclidean distance and response as similarity;

(13), sort the data <key, similarity> output from the matching node according to the key, and send the data with the same key, that is, the data of the same SIFT feature point, to the same judgment node;

(14) In the judgment node, sort the input data <key, similarity> according to the size of the Euclidean distance;

(15) Determine whether the shortest distance divided by the next shortest distance is less than the preset threshold. If the condition is met, the SIFT feature point matches the SIFT feature point of the reference image corresponding to the shortest distance, which is recorded as key_match, and then go to step (16 ); if the condition is not met, go to step (14) to judge another SIFT feature point;

(16) For the matched SIFT feature point key_match, the mismatch is eliminated by calculating RANSAC, and the correct matching result is obtained.

As described in the above step ④, modify the spatial coordinates, register and render the virtual object, and the specific steps are as follows:

(17), according to the correct matching relationship between the SIFT feature points obtained in step (16) and the SIFT feature points in the reference image, the three-dimensional space coordinate system is corrected;

(18) Calculate the coordinates of the virtual object in the corrected coordinate system;

(19). Render the virtual object and output it on the display device, go to step (7).

Claims (5)

1. A natural feature registration method based on cloud computing, characterized in that the operation steps are as follows: 1. extract the natural feature point set, set up a coordinate system; 2. extract the natural feature point from the key frame captured by the camera; Feature point matching; ④ modify the space coordinates, register and render the virtual object. 2. according to the described natural feature registration method based on cloud computing of claim 1, it is characterized in that said step 1. extracts the natural feature point set, and the method for establishing the coordinate system is: the front view of the object that wants to be registered is photographed by a camera , as a reference image, extract natural feature points from it, and establish a three-dimensional space coordinate system through these natural feature points. 3. according to the described natural feature registration method based on cloud computing of claim 1, it is characterized in that described step ② extracts the method for natural feature point from the key frame that camera shoots is: from the actual image that camera shoots, extract key Frame image, extract natural feature points in the image as points to be matched. 4. according to the described natural feature registration method based on cloud computing of claim 1, it is characterized in that said step 3. the method for matching by cloud-based natural feature points is: by grouping natural feature points, sending each group of feature points into The matching node performs similarity calculation, and then sends the output result to the judgment node for calculation, and finally obtains the preliminary matching result of the natural feature points of the image to be matched and the natural feature points of the reference image; according to the preliminary matching result, by eliminating the mismatch, the correct matching results. 5. According to the natural feature registration method based on cloud computing according to claim 1, it is characterized in that said step 4) modifies the spatial coordinates, and the method for registering, rendering and displaying the virtual object is: through the natural feature points and the natural feature points in the reference map Matching relationship, modify the three-dimensional space coordinates, register the virtual object, make it in the correct three-dimensional space position, and render and display.

Images