CN110111388A - Three-dimension object pose parameter estimation method and visual apparatus - Google Patents

Abstract

The present application relates to a method for estimating pose parameters of a three-dimensional object and a visual device, wherein the visual device can execute the method, and the method includes: acquiring pose parameters determined based on the previous frame image of the three-dimensional object as a method for determining the corresponding position of the current frame image The initial pose parameters of the attitude parameters; based on the initial pose parameters, the three-dimensional space straight line segment of the three-dimensional object is projected onto the two-dimensional image plane, and the projected straight line segment of the three-dimensional space straight line segment on the two-dimensional image plane is obtained; the projected straight line segment is determined The closest image straight segment on the two-dimensional image plane, and determine the distance error between the projected straight segment and the closest image straight segment; judge whether the distance error satisfies the preset condition; if not, determine a new one based on the distance error Pose parameters, and use the new pose parameters as the initial pose parameters; if satisfied, use the initial pose parameters as the pose parameters of the current frame image. This application realizes the estimation of pose parameters for objects with less texture.

Description

3D Object Pose Parameters Estimation Method and Vision Equipment

technical field

The present application relates to the field of visual equipment, in particular to a method for estimating pose parameters of a three-dimensional object and a visual equipment.

Background technique

For the pose estimation of 3D objects with sparse textures, sparse point features affect the accuracy of pose estimation, and relatively stable straight line features are beneficial to pose estimation. The core problem of using straight line features for pose estimation is the matching of 2D and 3D straight line features. Since there is too little description information available for 3D straight lines, it is difficult to directly complete the matching of 2D and 3D straight line features. Generally, the reference frame of the known two-dimensional and three-dimensional straight line correspondence is used to simplify the feature matching of two-dimensional and two-dimensional image straight lines.

At present, line feature matching mainly focuses on the line feature matching of 2D and 2D images, and has been applied to 3D reconstruction, motion recovery structure, real-time positioning and map construction, and has achieved certain results. Line feature matching can be divided into feature description method, point-line invariance method and line-line combination method. The feature description method uses the gray information of the line feature neighborhood to construct a feature vector to characterize the line, and judges whether it is a matching line by comparing the similarity of the feature vectors; the point-line invariance method relies on matching points, and uses the invariance of coplanar point lines Constraints are used to judge whether it is a matching straight line; the line-line combination method uses the constraint relationship between different straight lines to judge whether it is a matching straight line.

Using the two-dimensional and two-dimensional linear feature matching of the image and the reference frame, the two-dimensional and three-dimensional linear feature matching is indirectly realized. One is that the number of reference frames that need to mark the corresponding relationship between 2D and 3D offline is large and cumbersome; the other is that the linear features of the image need to be accurately extracted, but noise, blur and other factors will affect the integrity of the linear features.

From the above analysis, it can be seen that the indirect matching method realizes the pose estimation of texture-sparse objects, relies on historical images to label reference frames, and has a large workload for offline labeling.

Contents of the invention

In order to solve the above technical problems or at least partly solve the above technical problems, the present application provides a method for estimating pose parameters of a three-dimensional object and a visual device.

In the first aspect, the present application provides a method for estimating pose parameters of a three-dimensional object, including: obtaining the pose parameters of the above-mentioned three-dimensional object determined based on the previous frame image of the three-dimensional object, as the corresponding position of the current frame image of the above-mentioned three-dimensional object The initial pose parameters of the pose parameters; based on the above-mentioned initial pose parameters, project the three-dimensional space straight line segment of the above-mentioned three-dimensional object onto the two-dimensional image plane of the above-mentioned current frame image, and obtain the above-mentioned three-dimensional space straight line segment on the above-mentioned two-dimensional image plane The projected straight line segment on the above-mentioned projected straight line segment; determine the closest image straight line segment of the above-mentioned projected straight line segment on the above-mentioned two-dimensional image plane, and determine the distance error between the above-mentioned projected straight line segment and the above-mentioned closest image straight line segment; judge whether the above-mentioned distance error is Satisfy the preset conditions; if the above preset conditions are not met, determine new pose parameters based on the above distance error, and use the above new pose parameters as the above initial pose parameters; if the above preset conditions are met, set the above initial pose parameters The pose parameter is used as the pose parameter of the above-mentioned current frame image of the above-mentioned three-dimensional object.

In some embodiments, based on the above initial pose parameters, the 3D straight line segment of the 3D object is projected onto the 2D image plane of the current frame image to obtain the 3D straight line segment on the 2D image plane Before projecting the straight segment, it also includes: determining the angle between the line of sight of the current frame image and the plane where the straight segment in the three-dimensional space is located, judging the projection visibility of the straight segment in the three-dimensional space based on the angle, and removing the blocked three-dimensional space straight line.

In some embodiments, determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and determining the distance error between the projected straight segment and the closest image straight segment, includes: A plurality of control points are obtained by sampling the straight line segment; the vertical direction of the above-mentioned projected straight line segment is determined; the preset search range at each control point among the above-mentioned multiple control points is searched bidirectionally along the above-mentioned vertical direction, and the pixel of the maximum likelihood ratio is point as a candidate corresponding point to obtain a plurality of candidate corresponding points; determine the distance error between the plurality of candidate corresponding points and the projected straight line segment.

In some embodiments, determining new pose parameters based on the above-mentioned distance error includes: using Lie algebraic space to characterize the above-mentioned initial pose parameters; determining a pose increment according to the above-mentioned distance error; determining a new pose parameter based on the above-mentioned pose increment Attitude parameters.

In some embodiments, determining the pose increment according to the distance error includes: using robust estimation to determine the pose increment according to the distance error. Determine the new pose parameters based on the above pose increments, including: determine the rotation increment and translation increment in Euclidean space through exponential mapping, and determine the new rotation matrix and new translation based on the above rotation increments and the above translation increments vector.

In a second aspect, the present application provides a method for estimating pose parameters of a three-dimensional object, including: acquiring an initial frame image of the above-mentioned three-dimensional object; extracting image geometric features of the three-dimensional object in the above initial frame image; combining the extracted image geometric features with Matching the reference feature library of the above-mentioned three-dimensional object to obtain the three-dimensional space coordinates corresponding to the extracted image geometric features, wherein the above-mentioned reference feature library includes image geometric features and three-dimensional space coordinates extracted from images under multiple viewing angles of the above-mentioned three-dimensional object The corresponding relationship between; based on the three-dimensional model of the above-mentioned three-dimensional object and the two-dimensional image coordinates and three-dimensional space coordinates corresponding to the above-mentioned extracted image geometric features, use the N-point perspective method to determine the initial pose parameters of the above-mentioned three-dimensional object; based on the above-mentioned initial position Attitude parameters project the three-dimensional space straight line segment of the above-mentioned three-dimensional object onto the two-dimensional image plane of the above-mentioned initial frame image, and obtain the projected straight line segment of the above-mentioned three-dimensional space straight line segment on the above-mentioned two-dimensional image plane; determine the above-mentioned projected straight line segment in The closest image straight line segment on the above-mentioned two-dimensional image plane, and determine the distance error between the above-mentioned projection straight line segment and the above-mentioned nearest image straight line segment; judge whether the above-mentioned distance error satisfies the preset condition; if the above-mentioned preset condition is not satisfied , determining a new pose parameter based on the above-mentioned distance error, and using the above-mentioned new pose parameter as the above-mentioned initial pose parameter; if the above-mentioned preset condition is satisfied, using the above-mentioned initial pose parameter as the pose parameter of the above-mentioned three-dimensional object.

In some embodiments, the image geometric features are key corner features and/or straight line features of the three-dimensional object.

In some embodiments, extracting the image geometric features of the three-dimensional object in the initial frame image includes: extracting key corner points and/or straight line segments of the above-mentioned three-dimensional object on the initial frame image; based on the extracted key corner points and/or The local grayscale of the straight line segment determines the above extracted key corner points and/or feature vectors of the straight line segment.

In some embodiments, based on the initial pose parameters, the 3D straight line segment of the 3D object is projected onto the 2D image plane of the initial frame image to obtain the 3D straight line segment on the 2D image plane Before projecting the straight segment, it also includes: determining the angle between the line of sight of the initial frame image and the plane where the straight segment in the three-dimensional space is located, judging the projection visibility of the straight segment in the three-dimensional space based on the angle, and removing the blocked three-dimensional space straight line.

In some embodiments, determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and determining the distance error between the projected straight segment and the closest image straight segment, includes: A plurality of control points are obtained by sampling the straight line segment; the vertical direction of the above-mentioned projected straight line segment is determined; the preset search range at each control point among the above-mentioned multiple control points is searched bidirectionally along the above-mentioned vertical direction, and the pixel of the maximum likelihood ratio is point as a candidate corresponding point to obtain a plurality of candidate corresponding points; determine the distance error between the plurality of candidate corresponding points and the projected straight line segment.

In some embodiments, determining new pose parameters based on the above-mentioned distance error includes: using Lie algebraic space to characterize the above-mentioned initial pose parameters; determining a pose increment according to the above-mentioned distance error; determining a new pose parameter based on the above-mentioned pose increment Attitude parameters.

In some embodiments, determining the pose increment according to the distance error includes: using robust estimation to determine the pose increment according to the distance error. Determine the new pose parameters based on the above pose increments, including: determine the rotation increment and translation increment in Euclidean space through exponential mapping, and determine the new rotation matrix and new translation based on the above rotation increments and the above translation increments vector.

In a third aspect, the present application provides a visual device, which includes: a memory, a processor, and a computer program stored on the memory and operable on the processor; when the computer program is executed by the processor, the Steps in any method for estimating pose parameters of a three-dimensional object in the present application.

In a fourth aspect, the present application provides a computer-readable storage medium. The above-mentioned computer-readable storage medium stores a three-dimensional object pose parameter estimation program. When the above-mentioned three-dimensional object pose parameter estimation program is executed by a processor, any The steps of the three-dimensional object pose parameter estimation method.

Compared with the prior art, the above-mentioned technical solutions provided by the embodiments of the present application have the following advantages:

The method provided by the embodiment of the present application determines the pose parameters of a three-dimensional object through iterative projection, avoids relying on a large number of historical image annotations, and efficiently and accurately realizes pose parameter estimation of a three-dimensional object with sparse texture. Feature matching and pose parameter estimation are performed alternately and iteratively, which can remove the influence of outliers and obtain matching results and pose parameter estimation results at the same time.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description serve to explain the principles of the invention.

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can also be obtained from these drawings without paying creative labor.

FIG. 1 is a schematic diagram of the hardware structure of an implementation of a visual device provided in the embodiment of the present application;

FIG. 2 is a flow chart of an implementation of a method for estimating pose parameters of a three-dimensional object provided in an embodiment of the present application;

FIG. 3 is a flow chart of an embodiment of the distance error determination process provided by the embodiment of the present application;

FIG. 4 is a flow chart of an embodiment of determining new pose parameters based on distance errors provided by the embodiment of the present application;

FIG. 5 is a flow chart of another implementation of the method for estimating the pose parameters of a three-dimensional object provided in the embodiment of the present application;

FIG. 6 is a flow chart of an implementation of a method for establishing a reference feature library provided in an embodiment of the present application;

FIG. 7 is a schematic diagram of projection visibility judgment of a straight line segment in a three-dimensional space provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of a local search for a projected straight line segment provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of the distance between candidate corresponding points and projected straight line segments provided by the embodiment of the present application;

FIG. 10 is a reference frame image used for initialization of the cube provided by the embodiment of the present application;

FIG. 11 is a schematic diagram of initialization matching provided by the embodiment of the present application;

Figure 12 is the non-initial frame image matching and pose estimation results provided by the embodiment of the present application; and

Fig. 13 is a comparison curve between the estimated value of the pose parameter and the marked reference value provided by the embodiment of the present application.

Detailed ways

It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

In the following description, use of suffixes such as 'module', 'part' or 'unit' for denoting elements is only for facilitating description of the present invention and has no specific meaning by itself. Therefore, 'module', 'part' or 'unit' may be used in combination.

The vision devices provided in the embodiments of the present invention include but are not limited to industrial automation equipment, intelligent robots and other equipment or user terminals, which can identify and capture target objects, provide real-time image information and real-time spatial pose information of target objects. The visual device provided in the embodiment of the present invention may include: an RF (Radio Frequency, radio frequency) unit, a WiFi module, an audio output unit, an A/V (audio/video) input unit, a sensor, an interface unit, a memory, a processor, and power supply and other components.

In the subsequent description, the vision device will be taken as an example. Please refer to FIG. 1, which is a schematic diagram of the hardware structure of a vision device implementing various embodiments of the present invention. The vision device 100 may include: one or more optical image sensors 101 , memory 102, processor 103, and power supply 104 and other components. Those skilled in the art can understand that the visual device structure shown in Figure 1 does not constitute a limitation to the visual device, and the visual device may include more or fewer components than shown in the illustration, or combine some components, or different components layout.

In one embodiment, the optical image sensor 101 of the vision device 100 is one or more cameras. By turning on the cameras, images can be captured, and functions such as taking pictures and videos can be realized. The positions of the cameras can be set as required.

Vision device 100 may also include at least one sensor, such as a light sensor, a motion sensor, and other sensors. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is stationary.

The memory 102 can be used to store software programs as well as various data. The memory 102 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a program required by at least one function, and the like. In addition, the memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

The processor 103 is the control center of the visual device, and uses various interfaces and lines to connect various parts of the entire visual device, by running or executing software programs and/or modules stored in the memory 102, and calling data stored in the memory 102 , to perform various functions of the vision equipment and process data, so as to monitor the vision equipment as a whole. Processor 103 may include one or more processing units.

This embodiment provides a method for estimating pose parameters of a three-dimensional object. Referring to FIG. 2 , the method for estimating pose parameters for a three-dimensional object includes steps S201 to S206.

Step S201, acquiring the pose parameters of the above-mentioned three-dimensional object determined based on the previous frame image of the three-dimensional object as initial pose parameters for determining the corresponding pose parameters of the current frame image of the above-mentioned three-dimensional object.

Step S202, based on the initial pose parameters, project the 3D straight line segment of the 3D object onto the 2D image plane of the current frame image to obtain the projected straight line segment of the 3D straight line segment on the 2D image plane .

In this embodiment, the projected straight line segment is represented by the two-dimensional image coordinates of the image.

Step S203, determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and calculating a distance error between the projected straight segment and the closest image straight segment.

In this embodiment, the closest straight line segment of the image on the two-dimensional image of the image may be regarded as a projected straight line segment of the straight line segment in the actual three-dimensional space. The distance error can be characterized based on the initial pose parameters determined

Step S204, judging whether the above-mentioned distance error satisfies the preset condition; if the above-mentioned preset condition is not satisfied, proceed to step S205; if the above-mentioned preset condition is satisfied, proceed to step S206.

In some embodiments, in step S204, the preset condition may be that the distance error is less than the preset distance error. When the distance error is greater than the preset distance error, it is determined that the preset condition is not met. When the distance error is less than or equal to the preset When there is a distance error, it is determined that the preset condition is met. But this embodiment is not limited thereto.

Step S205, determining a new pose parameter based on the above-mentioned distance error, and using the above-mentioned new pose parameter as the initial pose parameter of the above-mentioned step S202.

Step S206, using the above-mentioned initial pose parameters as pose parameters of the above-mentioned current frame image of the above-mentioned three-dimensional object.

Through the method for estimating pose parameters of three-dimensional objects, the pose parameters of three-dimensional objects are determined by iterative projection, which avoids relying on a large number of historical image annotations, and efficiently and accurately realizes pose parameter estimation of three-dimensional objects with sparse textures. Feature matching and pose parameter estimation are performed alternately and iteratively, which can remove the influence of outliers and obtain matching results and pose parameter estimation results at the same time.

In some embodiments, step S205 also accumulates the accumulated times of determining new pose parameters in step S205, and the accumulated times increase by 1 each time a new pose parameter is determined. In step S204, if the above-mentioned preset condition is not satisfied, it is judged whether the accumulated number is greater than the preset value, and if it is smaller than the preset value, go to step S205. Optionally, if it is greater than the preset value, the above initial pose parameter is used as the pose parameter of the above current frame image of the above three-dimensional object.

In some embodiments, some three-dimensional straight line segments are invisible due to self-occlusion and other reasons, and the invisible straight line segments can be removed according to the visibility test before projection, and only the visible straight line segments are retained. For this reason, in some embodiments, before the above step S202, it also includes: determining the angle between the line of sight of the current frame image and the plane where the three-dimensional space straight segment is located, and judging the projection of the above three-dimensional space straight line segment based on the above angle Visibility and removal of occluded straight segments in 3D space.

In some embodiments, as shown in FIG. 7, the occluded three-dimensional straight line segment is obtained by the following method:

Compute the gaze vector and the normal vector of the plane where the three-dimensional line segment is located The quantitative product of , according to which the projective visibility of the straight line segment in 3D space is judged:

Wherein, V(L _i ) is 1, indicating that the straight line segment in the three-dimensional space is visible, and is 0, indicating that the straight line segment in the three-dimensional space is invisible.

In some embodiments, if the straight line segment in the three-dimensional space is visible, the projected straight line segment is uniformly sampled at equal intervals, and the sampling points are called control points.

In some embodiments, the above step S203 is to determine the closest image straight segment of the projected straight segment on the two-dimensional image plane, and calculate the distance error between the projected straight segment and the closest image straight segment, refer to Fig. 3 includes step S301 to step S304.

Step S301 , obtaining a plurality of control points by sampling on the projected straight line segment of the straight line segment in the three-dimensional space on the two-dimensional image plane.

Step S302, determining the vertical direction of the projected straight line segment.

In step S303, the preset search range at each control point among the plurality of control points is bidirectionally searched along the above vertical direction, and the pixel point with the maximum likelihood ratio is used as a candidate corresponding point to obtain a plurality of candidate corresponding points.

In this embodiment, the multiple candidate corresponding points may be equivalent to straight line segments.

Step S304, determining distance errors between the plurality of candidate corresponding points and the projected straight line segment.

The distance error between the projected straight line segment and the closest image straight line segment in step S203 may be represented by the distance error between the multiple candidate corresponding points and the projected straight line segment.

Through this method, under the constraints of the initial pose parameters, the optimal corresponding point can be searched in the preset search range, without the need to search the image features in the whole image, which improves the computational efficiency and reduces the computational complexity.

In some embodiments, in the above step S301, multiple control points may be uniformly sampled to improve search accuracy.

In some embodiments, as shown in FIG. 8 , for each control point, a one-dimensional search is performed along the direction perpendicular to the projected straight line segment, and the search range is {Q _i , j∈[-R, R]}. Calculate the likelihood ratio of each pixel in the search range, and take the pixel with the largest likelihood ratio (referred to as the maximum likelihood ratio point in this embodiment) as a candidate corresponding point Satisfy:

ζ _j is the likelihood ratio, which represents the absolute value of the sum of the convolution values at p _t and Q _j respectively. M _δ is a predefined gradient mask in the δ direction. v(·) represents the adjacent pixels of the location.

In some embodiments, as shown in FIG. 9 , the distance between the candidate corresponding point x′ _i and the corresponding projected straight line segment l _s (r) is:

in, and is the image coordinate of X′ _i .

In some embodiments, the above step S205 is to determine new pose parameters based on the above distance error, referring to FIG. 4 , including steps S401 and S403.

Step S401, using the Lie algebraic space to characterize the above initial pose parameters.

Step S402, determining the pose increment according to the distance error.

Step S403, determining new pose parameters based on the above pose increments.

In some embodiments, the above step S402 determining the pose increment according to the distance error includes: determining the pose increment by using robust estimation according to the above distance error. Robust estimation refers to the use of Tukey's influence function to adaptively assign weights to distance errors to reduce the influence of false candidate corresponding points.

In some embodiments, the above-mentioned step S403 determines new pose parameters based on the above-mentioned pose increments, including: determining the rotation increments and translation increments in Euclidean space through exponential mapping, based on the above-mentioned rotation increments and the above-mentioned translation increments Determine a new rotation matrix and a new translation vector.

In some embodiments, the pose parameters are represented in a Lie algebra se(3) space, v is a translation vector, and ω is a rotation vector. The pose increment calculation formula is as follows:

in, for Moore-Penrose pseudo-inverse. D=diag(ω ₁ , . . . , ω _k ) is a diagonal matrix composed of weight values calculated by the Tukey influence function. is the Jacobian matrix of the distance error vector e _i (r) between the candidate corresponding point and the corresponding projected straight line segment:

In formula (6)

The rotation matrix R and translation vector T corresponding to the Euclidean space can be determined by exponential mapping, expressed as:

The pose parameters after pose incremental update are expressed as:

This embodiment also provides a method for estimating pose parameters of a three-dimensional object, which is used for initializing pose parameters of a three-dimensional object. Referring to FIG. 5 , the method for estimating pose parameters of a three-dimensional object includes steps S501 to S509.

Step S501, acquiring an initial frame image of a three-dimensional object.

Step S502, extracting image geometric features of the three-dimensional object in the initial frame image.

In some embodiments, the features are obtained by a feature detection and description algorithm, and the specific information includes their coordinates in the image coordinate system and feature vectors calculated from adjacent gray levels.

Step S503, matching the extracted geometric features of the image with the reference feature database of the above-mentioned three-dimensional object to obtain the three-dimensional space coordinates corresponding to the extracted geometric features of the image.

Wherein, the above-mentioned reference feature library includes the corresponding relationship between image geometric features extracted from images of the above-mentioned three-dimensional object under multiple viewing angles and three-dimensional space coordinates.

Step S504, based on the 3D model of the 3D object and the 2D image coordinates and 3D space coordinates corresponding to the extracted geometric features of the image, use the N-point perspective method to determine the initial pose parameters of the 3D object.

In this embodiment, the N-point perspective method is a method for optimally solving the pose parameters of the three-dimensional object relative to the camera under the condition that the image point coordinates and the corresponding three-dimensional space point coordinates are known, and the internal camera parameters of the visual device are known.

Step S505, based on the above initial pose parameters, project the 3D straight line segment of the 3D object onto the 2D image plane of the initial frame image to obtain the projected straight line segment of the 3D straight line segment on the 2D image plane .

Step S506, determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and calculating a distance error between the projected straight segment and the closest image straight segment.

Step S507, judging whether the above-mentioned distance error satisfies the preset condition, if not, proceed to step S508; if satisfy the above-mentioned preset condition, proceed to step S509.

Step S508, determine a new pose parameter based on the above-mentioned distance error, and use the above-mentioned new pose parameter as the initial pose parameter of the above-mentioned step S505, and proceed to step S505.

Step S509, using the above-mentioned initial pose parameters as pose parameters of the above-mentioned three-dimensional object.

In some embodiments, the image geometric features are key corner features and/or straight line features of the three-dimensional object.

In some embodiments, the above-mentioned step S502 extracting the image geometric features of the above-mentioned initial frame image includes: extracting the key corner points and/or straight line segments of the three-dimensional object on the above-mentioned initial frame image; based on the extracted key corner points and/or The local grayscale of the straight line segment determines the above extracted key corner points and/or feature vectors of the straight line segment.

In some embodiments, the above step S504, based on the above initial pose parameters, projects the 3D space straight line segment of the above 3D object onto the 2D image plane of the above initial frame image to obtain the above 3D space straight line segment in the above 2D Before projecting the straight line segment on the image plane, it also includes: determining the angle between the line of sight of the initial frame image and the plane where the straight line segment in three-dimensional space is located, judging the projection visibility of the straight line segment in three-dimensional space based on the angle, and removing the Blocked straight line segments in 3D space.

In some embodiments, the above-mentioned step S506, referring to FIG. 3 , may include: obtaining a plurality of control points by sampling on the above-mentioned projected straight line segment; determining the vertical direction of the above-mentioned projected straight line segment; The preset search range at is searched bidirectionally along the above-mentioned vertical direction, using the pixel point with the maximum likelihood ratio as a candidate corresponding point to obtain multiple candidate corresponding points; and determining the distance error between the above-mentioned multiple candidate corresponding points and the projected straight line segment.

In some embodiments, the step of determining new pose parameters based on the above-mentioned distance error, referring to FIG. 4 , may include: using Lie algebraic space to characterize the above-mentioned initial pose parameters; The pose increment determines the new pose parameters.

In some embodiments, the step of determining the pose increment according to the distance error may include: using robust estimation to determine the pose increment according to the distance error. Determine the new pose parameters based on the above pose increments, including: determine the rotation increment and translation increment in Euclidean space through exponential mapping, and determine the new rotation matrix and new translation based on the above rotation increments and the above translation increments vector.

The method for establishing a reference feature library provided in this embodiment is used to produce the reference feature library of this embodiment, and the reference feature library can be applied to the method for estimating pose parameters of a three-dimensional object in this embodiment, especially in step S503. Referring to FIG. 6 , the method for establishing a reference signature database includes steps S601 to S604.

Step S601, acquire images of a three-dimensional object under multiple viewing angles, and obtain multiple frames of images of the three-dimensional object.

In this embodiment, the multiple frames of images may include images of at least part of the three-dimensional object or more viewing angles, and each viewing angle may include one or more frames of images. In some embodiments, images from 3 to 5 viewing angles of a three-dimensional object may be included.

Step S602, extracting image geometric features of the three-dimensional object in the above-mentioned multiple frames of images.

In some embodiments, the features are obtained by a feature detection and description algorithm, and the specific information includes their coordinates in the image coordinate system and feature vectors calculated from adjacent gray levels.

In some embodiments, the image geometric features are key corner features and/or straight line features of the three-dimensional object.

In some embodiments, extracting the image geometric features of the three-dimensional object in the above multiple frames of images may include: extracting key corner points and/or straight line segments of the three-dimensional object on the above multiple frame images; based on the extracted key corner points and And/or the local gray level of the straight line segment determines the key corner point and/or the feature vector of the straight line segment extracted above.

Step S603, according to the 3D model of the 3D object, use the back projection method to determine the 3D space coordinates corresponding to the geometric features of the image.

In this embodiment, one or more geometric features of each frame of image as a whole correspond to three-dimensional space coordinates.

Step S604, forming a reference feature library including the correspondence between the image geometric features of the three-dimensional object and the three-dimensional space coordinates.

In some implementations, as an example, a method for estimating pose parameters of a three-dimensional object includes: 1) offline stage: construct a reference feature library according to the method shown in Figure 6; 2) initialization stage: follow the method shown in Figure 5 The method determines the pose parameters corresponding to the initial frame image; 3) pose parameter update stage: determine the pose parameters corresponding to the image frames after the initial frame image according to the method shown in Figure 2 .

In some embodiments, it can be judged whether the image is the initial frame image, and when the image is the initial frame image, enter the initialization stage, and determine the pose parameters corresponding to the initial frame image according to the method as shown in Figure 5; when the image is not the initial frame When the image is generated, enter the pose parameter update stage, and determine the pose parameters corresponding to the image frame according to the method shown in Figure 2.

A visual device provided by the present application, the visual device includes: a memory, a processor, and a computer program stored on the above-mentioned memory and operable on the above-mentioned processor; when the above-mentioned computer program is executed by the above-mentioned processor, any of the above-mentioned implementations can be realized. The steps of the method for estimating a pose parameter of a three-dimensional object according to an example or an implementation manner.

A computer-readable storage medium provided by the present application. The above-mentioned computer-readable storage medium stores a three-dimensional object pose parameter estimation program. The steps of the attitude parameter estimation method.

Explanation of experiment and experiment result

In this embodiment, a continuously moving cube is used as the test object, and the image resolution is 640*480 pixels. The initial frame image is initialized to obtain the initial pose parameters, and the pose parameters of the next frame and above are used as the initial pose parameters of the current frame, and the two-dimensional and three-dimensional linear feature matching and pose parameter estimation are performed. All experiments were performed on a central processing unit (CPU) of Performed on a laptop with i5-5200Hq (2.2GHz) and 8GB of RAM.

Take the imaging images of the cube at three different viewing angles as reference images to build a reference feature library. The multi-frame reference image is shown in FIG. 10 . The matching results of key corner point features between the initial frame image and the reference feature library are shown in Figure 11. The image in the lower right corner is the initial frame image, and the corresponding point features are connected by straight lines. The cube 3D model is projected to the initial Frame the 2D image plane.

The linear feature matching and pose parameter estimation results of subsequent frames are shown in Figure 12. The straight line segment in the three-dimensional space of the cube is re-projected onto the image plane according to the estimated pose parameters, the projected straight line basically coincides with the image straight line, and the orientation of the rectangular coordinate system represents the target pose. The comparison curve of continuous pose parameter estimation results and marked reference values is shown in Figure 13, and the estimated curve is basically consistent with the reference curve. Calculate the root mean square error between the pose estimation value and the reference value, as shown in Table 1. In addition, the average processing time of a single frame is 43ms, which meets the real-time requirements.

Table 1 Cube pose parameter error

The above results show that the algorithm of the present invention can accurately complete two-dimensional and three-dimensional linear feature matching, and can accurately estimate the pose parameters of three-dimensional objects, and has low computational complexity.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element.

The serial numbers of the above embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in various embodiments of the present invention.

Embodiments of the present invention have been described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementations, and the above-mentioned specific implementations are only illustrative, rather than restrictive, and those of ordinary skill in the art will Under the enlightenment of the present invention, many forms can also be made without departing from the gist of the present invention and the protection scope of the claims, and these all belong to the protection of the present invention.

Claims (10)

1. A three-dimensional object pose parameter estimation method, is characterized in that, comprises: Acquiring the pose parameters of the three-dimensional object determined based on the previous frame image of the three-dimensional object as initial pose parameters for determining the corresponding pose parameters of the current frame image of the three-dimensional object; Projecting the 3D space straight line segment of the 3D object onto the 2D image plane of the current frame image based on the initial pose parameters to obtain the projection of the 3D space straight line segment on the 2D image plane straight line; determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and determining a distance error between the projected straight segment and the closest image straight segment; judging whether the distance error satisfies a preset condition; If the preset condition is not met, determine a new pose parameter based on the distance error, and use the new pose parameter as the initial pose parameter; If the preset condition is satisfied, the initial pose parameter is used as the pose parameter of the current frame image of the three-dimensional object. 2. The method for estimating pose parameters of a three-dimensional object according to claim 1, wherein, based on the initial pose parameters, the three-dimensional space straight line segment of the three-dimensional object is projected onto the two-dimensional image plane of the current frame image Above, before obtaining the projected straight line segment of the three-dimensional space straight line segment on the two-dimensional image plane, it also includes: Determine the angle between the line of sight of the current frame image and the plane where the three-dimensional space straight segment is located, judge the projection visibility of the three-dimensional space straight line segment based on the angle, and remove the blocked three-dimensional space straight line segment. 3. The method for estimating the pose parameters of a three-dimensional object according to claim 1, wherein the image straight line segment closest to the projected straight line segment on the two-dimensional image plane is determined, and the distance between the projected straight line segment and The distance error between the closest image straight line segments, including: Obtaining a plurality of control points by sampling on the projected straight line segment; determining the vertical direction of the projected straight line segment; In the plurality of control points, the preset search range at each control point is bidirectionally searched along the vertical direction, and the pixel point with the maximum likelihood ratio is used as a candidate corresponding point to obtain a plurality of candidate corresponding points; Determine distance errors between the plurality of candidate corresponding points and the projected straight line segment. 4. The three-dimensional object pose parameter estimation method according to claim 1, wherein determining new pose parameters based on the distance error comprises: Using a Lie algebraic space to characterize the initial pose parameters; determining a pose increment according to the distance error; New pose parameters are determined based on the pose increments. 5. the three-dimensional object pose parameter estimation method according to claim 4, is characterized in that, Determining the pose increment according to the distance error includes: determining the pose increment by using robust estimation according to the distance error; Determining a new pose parameter based on the pose increment includes: determining a rotation increment and a translation increment in Euclidean space through exponential mapping, and determining a new rotation matrix and a new rotation matrix based on the rotation increment and the translation increment The new translation vector. 6. A method for estimating pose parameters of a three-dimensional object, comprising: Acquiring an initial frame image of the three-dimensional object; extracting image geometric features of the three-dimensional object in the initial frame image; matching the extracted image geometric features with the reference feature database of the three-dimensional object to obtain the three-dimensional space coordinates corresponding to the extracted image geometric features, wherein the reference feature database contains The corresponding relationship between the image geometric features extracted from the image and the three-dimensional space coordinates; Based on the 3D model of the 3D object and the 2D image coordinates and 3D space coordinates corresponding to the extracted image geometric features, using an N-point perspective method to determine the initial pose parameters of the 3D object; Projecting the 3D straight line segment of the 3D object onto the 2D image plane of the initial frame image based on the initial pose parameters, to obtain the projection of the 3D straight line segment on the 2D image plane straight line; determining the closest image straight segment of the projected straight segment on the two-dimensional image plane, and determining a distance error between the projected straight segment and the closest image straight segment; judging whether the distance error satisfies a preset condition; If the preset condition is not met, determine a new pose parameter based on the distance error, and use the new pose parameter as the initial pose parameter; If the preset condition is met, the initial pose parameter is used as the pose parameter of the three-dimensional object. 7. The method for estimating pose parameters of a three-dimensional object according to claim 6, wherein the image geometric features are key corner features and/or straight line features of the three-dimensional object. 8. The three-dimensional object pose parameter estimation method according to claim 6, wherein extracting the image geometric features of the three-dimensional object in the initial frame image comprises: extracting key corner points and/or straight line segments of the three-dimensional object on the initial frame image; The feature vectors of the extracted key corner points and/or straight line segments are determined based on local gray levels of the extracted key corner points and/or straight line segments. 9. A visual device, characterized in that the visual device comprises: a memory, a processor and a computer program stored on said memory and executable on said processor; When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 8 are realized. 10. A computer-readable storage medium, characterized in that, a three-dimensional object pose parameter estimation program is stored on the computer-readable storage medium, and when the three-dimensional object pose parameter estimation program is executed by a processor, it is realized as claimed in claim 10. Steps of the three-dimensional object pose parameter estimation method described in any one of 1 to 8.