CN101807244B - Machine Recognition and Reconstruction Methods - Google Patents

Abstract

The invention discloses a machine recognition and reconstruction method which is used for recognizing and reconstructing an object. The method comprises the following steps of: acquiring an image of the object to be recognized; extracting image characteristics; calling an object model from an object model knowledge base, and extracting the characteristics of the model; comparing the model characteristics with the image characteristics, if matching, recognizing the object and calling relevant information in the object model knowledge base to reconstruct the object to be recognized; and if not, continuing to calling the next model and repeating the operation above until searching the matched model. By utilizing the method, the robot can easily recognize and reconstruct the object.

Description

Machine Recognition and Reconstruction Methods

technical field

The invention relates to the field of machine vision. More specifically, the present invention relates to a machine recognition and reconstruction method.

Background technique

Object recognition and reconstruction are extremely critical technical issues in the field of computer vision and robot recognition. For example, in the field of industrial manufacturing, at the robot operation site, the robot needs to recognize the objects on the scene, reconstruct the recognized objects on the basis of recognition, and then make corresponding actions for different objects according to the recognition results. How to realize the correct recognition and reconstruction of objects is a technical problem that has always plagued the development of machine vision.

In the prior art, there are mainly two recognition methods: one is a recognition method based on object structure description; the other is an image-based recognition algorithm. The recognition algorithm based on object structure description mainly applies Marr's visual computing theory. This method regards object recognition as a multi-level recognition process. It mainly recognizes local simple features first, and then gradually recognizes complex three-dimensional objects. Since the recognition starts from the local features first, the recognition results of the overall features of the three-dimensional complex objects are not good by using this recognition method, which is often very different from the real structure. The image-based recognition algorithm is mainly a cognitive strategy for human beings to the overall shape of the object, and then corrects the shortcomings in the recognition method based on the object structure description.

Reconstructing objects based on recognition has always been a research hotspot in this field. The existing object reconstruction methods are mainly reconstruction methods based on projection images and object reconstruction methods based on geometric projection features. Due to the complexity of the three-dimensional structure of the object and the complexity of the topological relationship, no object reconstruction method that can reach a practical level has been proposed in the prior art.

Contents of the invention

Therefore, the purpose of the present invention is to provide a machine recognition method for object recognition and reconstruction, so as to realize the robot's recognition of the object and build on the basis of the recognition.

To this end, the present invention provides a machine recognition method for object recognition and/or reconstruction, comprising:

Collect images of objects to be identified;

Feature extraction is performed on the collected object images, that is, image features are extracted;

Provide an object model knowledge base, the object model knowledge base includes N object models, where N≥1; call the first object model in the object model knowledge base;

Perform feature extraction on the extracted object model, that is, extract model features;

Compare image features with model features;

If the matching rate between the image feature and the model feature is not less than the set critical value, record the extracted object model as an alternative model;

If the boundary matching rate between the image feature and the model feature is less than the set critical value, a second object model different from the first object model is retrieved from the object model knowledge base, and the steps of extracting model features and feature comparison are repeated to traverse the object model The third and fourth object models in the knowledge base...the Nth object model, until an object model matching the collected image features and the extracted model features is found.

Compared with the prior art, the present invention has established the object model knowledge base, therefore, when performing machine vision, it only needs to perform feature extraction on the image to be recognized, and compare the extracted features with the features of the model in the object model knowledge base , if the matching rate of the two reaches a certain threshold, the object to be recognized matches the called object model, that is, the recognition of the object is realized, and at the same time, the information about the model stored in the object model knowledge base is called, the realization of To identify the construction of the object. This identification method can be applied in various technical fields, such as factory automation, space exploration, modern medical treatment, etc. For example, in the field of factory automation, robots are often required to identify limited tools or operating objects on site, such as selecting a certain tool from the tool rack, and selecting an appropriate operating object to perform corresponding operations. According to the method of the present invention, if the Establish the models of these limited objects and store them in the object model knowledge base, then the robot can use the recognition and construction method of the present invention to recognize the objects on the scene and make further actions based on them.

These and other features and advantages of the present invention will become apparent from the following description of specific embodiments.

Description of drawings

The invention is described below with reference to the accompanying drawings, which show preferred embodiments of the invention by way of example only. In the picture:

Fig. 1 is a flowchart of an exemplary embodiment of the machine identification method according to the present invention;

Fig. 2 is a flow chart of another exemplary embodiment of the machine identification method according to the present invention.

Detailed ways

The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

Referring to FIG. 1 , it shows a flowchart of an exemplary embodiment of a machine identification method according to the present invention. As shown in the figure, the machine recognition method for object recognition and reconstruction includes:

S101. Collect an image of an object to be identified. in this step. Acquisition To identify object images, various image capture devices (such as cameras, video cameras, etc.) can be used to acquire object images in the scene. In one embodiment, for example, in the field of remote control, the remote robot uses a camera to capture photos of the scene, and sends the photos to the console through a wireless network, and the staff can select the image of the object to be operated by the robot through a graphical interface. In another embodiment, for example, in the field of factory automation, the machining robot selects the image of the object to be recognized from the shot scene according to the program control of the manufacturing process.

S102. Preprocessing the image. In this step, operations such as filtering, denoising, and distortion correction need to be performed on the selected object image to remove various noises of the object image and facilitate feature extraction. In one embodiment, this step can be omitted.

S103. Perform feature extraction on the collected object image, that is, extract image features. In this step, feature extraction is a method of using various features commonly used in the prior art, such as Canny algorithm and its improved algorithm for feature extraction, and sift and its algorithm for structural feature extraction. For simplicity, no detailed description is given.

S104. Provide an object model knowledge base, where the object model knowledge base includes N object models, where N≥1. In this step, multiple object models may be preset in the object model knowledge base. In one embodiment, for example, in factory automation, in the robot operation site, the tools or objects that the robot needs to contact or operate are relatively limited, so the models of the limited tools or objects can be established and stored in object model knowledge base. In addition, object models can also be added as required during the recognition process.

S105. Call the first object model in the object model knowledge base.

S106. Perform feature extraction on the extracted object model, that is, extract model features. The feature extraction of the model is realized by using methods in the prior art, for example, using the Canny algorithm. Extracting the features of the model can extract the structural features, shape features, projection features, boundary features, etc. of the model, for example, the method used in the background art can be used to extract model features.

S107. Comparing the image features with the model features. The image features are compared with the model features to judge the similarity between the image features and the model features. For the convenience of explanation, the matching rate is used to describe this meaning, and the matching rate is used to describe the similarity between two features. For example, the higher the matching rate, the more similar the two are, and a matching rate of 100% means that the two are exactly the same. In the process of machine vision, a critical value (or threshold) of a certain matching rate can be set, such as a matching rate of 70%, 80%, 90, 95%, 99%, etc., which can speed up the process of matching judgment without having to The correct conclusion can be made by completely matching all the features, which can save time and improve efficiency.

Judge the results of the comparison, and perform different steps according to different judgment structures:

S108. If the matching rate between the image feature and the model feature is not less than the set threshold, the object to be identified is identified as the extracted object model or the recorded object model as a candidate model.

S109. If the boundary matching rate between the image feature and the model feature is less than the set critical value, call a second object model different from the first object model from the object model knowledge base, and first judge whether to call the second object model before calling the second object model Whether the first object model is the last model in the object model knowledge base, if not, then execute step S110 to call the next model; and repeat the steps of S106 extracting model features and S107 feature comparison to traverse the object model knowledge base The third and fourth object models...the Nth object model, until an object model matching the collected image features and the extracted model features is found.

In one embodiment, the threshold is adjustable. For example, it can be changed using software settings.

In another embodiment, when the threshold value is set to be small, such as 85%, it is possible to search out a plurality of models related to the image features of the object to be recognized from the model knowledge base, and then select the one with the highest matching rate. An object model is extracted as an object to be recognized.

In another embodiment, when the critical value is set larger, such as 99.999%, the object model may not be searched from the knowledge base. At this time, the critical value can be adjusted to a smaller value, such as 80%. Execute the method above.

Referring to FIG. 2 , FIG. 2 shows a flow chart of another exemplary embodiment of the machine identification method of the present invention.

The difference between the embodiment shown in Fig. 2 and the embodiment in Fig. 1 mainly lies in the feature extraction of the object model, which extracts the projection feature, and performs the following steps:

Step S206, projecting in a selected direction; S207, extracting projection features, for example, using the method described above; S208, performing feature extraction for projection, and comparing the extracted model features with the features of the image, that is, judging features Does it match.

Then, based on the results of the feature comparison, a judgment is made:

If the feature matching rate of the projected feature and the object image reaches the set critical value, the object to be recognized is recognized as an object model;

If the matching rate of the projected features and the features of the object image is less than the set threshold value, step S211 is performed to transform the viewing direction of the model into a second direction.

Then, repeat the above process, if the feature matching rate of the projected feature and the object image reaches the set critical value, stop changing the direction, otherwise change the viewing direction to the third, fourth... Mth direction, M is According to the value set by a certain search strategy, until the matching rate is not less than the set critical value, otherwise continue to execute the above process for the next object model

Since in the space coordinate system, for example, the observation target (ie, the retrieved object model) is taken as the coordinate center, different projections can be observed from different viewing angles. In order to improve the search efficiency of projections, the present invention also provides a preferred search method.

First establish the space coordinate system,

The viewing direction of the model is then changed incrementally. For example, in an angle incremental manner, for example, the angle size can be any value between 1-10°, more preferably 3-5°, or other angle values.

In the process of changing the observation direction of the model, judge the feature matching rate of the projected feature in a certain direction and the feature image of the object and the feature matching rate of the projected feature in the next incremental direction of the direction and the feature image of the object image. If the matching rate becomes larger, then change the observation direction to the next incremental direction of the incremental direction, if the matching rate becomes smaller, then move the observation direction to the opposite direction to the increment, and compare the size of the matching rate, repeat this process, Until the direction with a large matching rate is found; if the matching rate of the projected features in the direction with a large matching rate found is not less than the set critical value, the object to be recognized matches the model, and the loop process ends, otherwise the direction will be observed Turn to determine other coordinate directions of the space coordinate system, and execute the above process until a value with a large matching rate in the set viewing direction is found and the value with a large matching rate is not less than the set critical value. If traversing all directions is If found, it is determined that the model does not match the object to be recognized, and the above loop ends.

It should be understood that projections can be obtained using various methods, and the above methods are only exemplary, and the present invention is not limited thereto. For example, you can take any point in space, take the line passing through the point to the model (ie the viewing direction) as the moving axis in a certain space plane, move at any angle, and continuously obtain projections, and then compare the characteristics of the projections with those to be identified. The matching rate of the characteristics of the object, after moving 360° in this plane, rotate the viewing direction by a certain angle, move in another coordinate plane until the entire coordinate space is traversed, and then find the maximum matching rate, and Compare the maximum matching rate with the set critical value to make a judgment. In addition, there are many other search algorithms to improve the efficiency of the search.

In one embodiment, the projection features of each model in the object model knowledge base in different directions are stored in the object model knowledge base in a pre-extracted manner. For example, the model can be projected in the space coordinate system according to a certain angular increment to transform the viewing direction, and the features of these projections can be extracted, and then these features can be stored in the object model knowledge base. When performing feature matching, the features in the object model knowledge base are compared in sequence, and then a judgment is made. The advantage of this is that it saves the time for projection and extraction, but takes up a lot of storage space. In another embodiment, real-time projection, real-time feature extraction, and then comparison can be performed. The present invention does not specifically limit it.

When the object to be recognized is recognized as the extracted object model, it may further include calling model information in the object model knowledge base, so as to reconstruct the object to be recognized. Model information includes geometry, color, material, physicochemical properties, compositional properties, and motion information.

When all the models in the object model knowledge base have been traversed and no object model matching the collected image features and the extracted model features is found, it is determined that the object to be recognized is a new object, and the user will correspond to the new object The model information of the object model is added to the object model knowledge base.

The present invention has been described above by way of examples, and it should be understood that various modifications and changes can be made to the present invention without departing from the spirit and scope of the present invention.

Claims (8)

1. a machine identification method is used for the identification and/or the reconstruct of object, it is characterized in that, comprising:

Gather and want the recognition object image;

Object image to gathering carries out feature extraction, promptly extracts image feature;

The object model knowledge base is provided, and this object model knowledge base comprises N object model, wherein, and N >=1;

Transfer first object model in the object model knowledge base;

Object model to this extraction carries out feature extraction, i.e. the extraction model characteristic;

The image feature and the aspect of model are compared;

If image feature and aspect of model matching rate are not less than the setting critical value, the object model that then will extract carries out record, with as alternative model;

If image feature and aspect of model matching rate are less than setting critical value; Then from the object model knowledge base, transfer second object model different with first object model; Repeat the step of said extraction model characteristic, the contrast of said characteristic; The the 3rd, the 4th object model in the traversal object model knowledge base ... till the object model that the object model that N object model, the matching rate of the aspect of model that extracts until searching out and the image feature of collection are not less than critical value promptly is complementary with the object that will discern;

Wherein, the object model that extracts is carried out in the step of feature extraction,, carry out projection, carry out feature extraction respectively for the projection on each direction in different directions through the direction of observation of transformation model;

The step of the direction of observation of transformation model comprises:

The setting space coordinate system;

The mode of going forward one by one with angle step changes the direction of observation of model, and it is specially:

Relatively in the size of the characteristic matching rate of the characteristic of the projection of next angle step direction of the characteristic matching rate of the characteristic of a certain direction projection and object image and this direction and object image; If it is big that matching rate becomes; Then direction of observation is changed into the next angle step direction of this angle step direction,, then direction of observation is moved this angle step in the opposite direction if matching rate diminishes; And the size of comparison match rate; This process repeatedly is not less than the direction of the critical value of setting up to the matching rate of the characteristic of characteristic that finds projection and object image, and object and the model that indicate to discern this moment are complementary; Otherwise direction of observation is turned to other coordinate directions of confirming space coordinates, and carry out said process, all do not find, then judge the object that this model does not match and will discern if travel through all directions.

2. machine identification method according to claim 1, wherein said critical value is adjustable.

3. machine identification method according to claim 2; Wherein, When said critical value setting hour, and when from the model knowledge base, searching out the alternative model that a plurality of and the image feature of wanting recognition object be complementary, then choose the object that the highest object model conduct of matching rate identifies.

4. machine identification method according to claim 1; Wherein, The step of the object model that extracts being carried out feature extraction further comprises: the characteristic of the projection model on each direction of being extracted is compared with the characteristic of object image respectively; If the matching rate of the characteristic of the characteristic of projection model and object image is not less than the setting critical value on this direction; The object identification that then will discern is the object model that is extracted; If the matching rate of the characteristic of the characteristic of projection model and object image is then changed direction the direction of observation of institute's extraction model less than setting critical value on this direction, and repeat the step of projection, feature extraction, characteristic contrast; The characteristic matching rate of the characteristic of the projection model on the direction that transforms to and object image is not less than to be set critical value or is converted into the M direction, and M is the numerical value of setting according to certain search strategy.

5. machine identification method according to claim 4; Wherein, The characteristic of each model projection in different directions in the object model knowledge base is stored in the object model knowledge base with the mode of extracting in advance; Or during machine recognition, the characteristic of said each model of extract real-time projection in different directions.

6. machine identification method according to claim 1, wherein, if this object that will discern is identified as the object model of this extraction, then this method further comprises the model information of transferring in the object model knowledge base.

7. machine identification method according to claim 1; Wherein, If traveled through all models in the object model knowledge base; The object model that the object image characteristic that does not search out and gather is complementary judges that then the said object that will discern is new object, and the user will be added into the object model knowledge base corresponding to the model information of this new object.

8. machine identification method according to claim 6, wherein, said model information comprises one or more in geometric configuration, color, physicochemical characteristics and the action message.

Images