CN115049737A - Pose marking method, device and system and storage medium - Google Patents

Abstract

The embodiment of the invention discloses a pose marking method, a pose marking device, a pose marking system and a storage medium. The method comprises the steps that an obtained sequence frame image comprises a reference object and a target object, the sequence frame image comprises a reference frame image and a current frame image, and a first pose conversion relation is obtained based on feature points of the reference object, projection points in the reference frame image and projection points in the current frame image; and determining a second position and posture conversion relation based on the characteristic points of the target object, the projection points in the reference frame image and the first position and posture conversion relation. The characteristic points of the reference object and the characteristic points of the target object are stable and reliable, and a pose transformation matrix can be accurately calculated; based on the calculated second pose conversion relation and the projection point of the target object in any frame of image, automatically determining and marking the pose of the feature point of the target object without manual marking; when the target object is positioned and analyzed based on the pose of the target object, for example, the robot is used for grabbing the target object, so that grabbing efficiency and precision can be improved.

Description

A pose labeling method, device, system and storage medium

technical field

The embodiments of the present invention relate to a robot intelligent detection technology, and in particular, to a pose labeling method, device, system and storage medium.

Background technique

The pose of the target object is often used as an important technology in the field of robot intelligent detection. At present, manual labeling methods are often used to label the pose data of target objects. In the process of robot operation, a large amount of coordinate data of the target object needs to be collected. For example, a large amount of coordinate data of the target object is collected by using visual servoing business processes such as the robot to identify the target object and grasp the target object. Manually labeling the pose of the target object based on massive coordinate data consumes a lot of labor and time costs, and the labeling accuracy of the pose data is poor.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a pose labeling method, device, system and storage medium, so as to achieve the effect of improving the labeling efficiency and accuracy of poses.

In a first aspect, an embodiment of the present invention provides a pose labeling method, which includes:

Receive sequence frame images collected by a photographing device at different angles, and determine a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images includes a reference object and a target object;

Based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image, determine the difference between the current frame image coordinate system and the reference frame image coordinate system The first pose conversion relationship between;

Based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first attitude transformation relationship, determine the second coordinate system between the target object coordinate system where the target object is located and the current frame image coordinate system Pose transformation relationship;

Based on the second pose conversion relationship and the projection point of the target object in any frame image captured by the photographing device, the pose labeling is performed on the target object.

In a second aspect, an embodiment of the present invention further provides a pose labeling device, the device comprising:

An image determination module, configured to receive sequence frame images collected by a photographing device at different angles, and determine a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images includes a reference frame image objects and target objects;

The first attitude conversion relationship determination module is used to determine the current frame image based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image. The first pose conversion relationship between the frame image coordinate system and the reference frame image coordinate system;

The second pose transformation relationship determination module is used to determine the target object coordinate system where the target object is located based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship and the second pose transformation relationship between the current frame image coordinate system;

The pose labeling module is configured to label the target object on the basis of the second pose conversion relationship and the projection point of the target object in any frame of images captured by the photographing device.

In a third aspect, an embodiment of the present invention further provides a pose labeling system, including a robot, a target object, and a reference object, wherein the robot includes a robotic arm, a photographing device, a memory, and a processor; the robotic arm drives the photographing The device moves, so that the photographing device captures a sequence of frame images at multiple angles; wherein, each frame of the sequence of frame images includes a reference object and a target object, the target object includes a plurality of feature points, and the reference object including multiple feature points;

Wherein, a computer program exists in the memory and can be run on a processor, and the processor implements the pose labeling method according to any one of the first aspects when the processor executes the computer program.

In a fourth aspect, an embodiment of the present invention further provides a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, implement the pose according to any one of the first aspect labeling method.

The technical solution of the embodiment of the present invention is to receive the sequence frame images collected by the photographing device at different angles, each frame image of the sequence frame image includes the reference object and the target object, so that the reference object assists the photographing device in photographing; determine the sequence frame image Based on the reference frame image and the current frame image in the reference frame, the coordinates of the current frame image are determined based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image The first pose transformation relationship between the frame and the reference frame image coordinate system; based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship, determine the location of the target object. The second pose transformation relationship between the target object coordinate system and the current frame image coordinate system. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be calculated accurately; The second pose transformation relationship and the projection point of the target object in any frame of images automatically determine the pose of the feature points of the target object, and automatically mark the pose of the feature points of the target object, without manual annotation, reducing labor costs, and Improve the reliability of pose calculation; further, when positioning and analyzing the target object based on the pose of the target object, the positioning accuracy can be improved, especially in the field of robot grasping the target object, which can greatly improve the grasping efficiency of the target object and grasping accuracy.

Description of drawings

1 is a schematic flowchart of a pose labeling method according to Embodiment 1 of the present invention;

2 is a schematic flowchart of a pose labeling method according to Embodiment 2 of the present invention;

3 is a schematic diagram of a pose conversion relationship between coordinate systems according to Embodiment 2 of the present invention;

FIG. 4 is a schematic logical diagram of pose labeling according to Embodiment 2 of the present invention;

5 is a schematic structural diagram of a pose labeling device according to Embodiment 3 of the present invention;

FIG. 6 is a schematic structural diagram of a pose labeling system according to Embodiment 4 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Example 1

FIG. 1 is a schematic flowchart of a pose labeling method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of automatically labeling poses, and the method can be performed by a pose labeling device, wherein the system can be performed by software And/or hardware implementation, and is generally integrated in a robot or an electronic device with a pose labeling function, this embodiment is explained by taking a robot as an example. Specifically, as shown in FIG. 1, the method may include the following steps:

S110. Receive the sequence frame images collected by the photographing device at different angles, and determine the reference frame image and the current frame image in the sequence frame images.

Among them, the photographing device is installed on the robotic arm of the robot, and is used to collect the sequence frame images during the operation of the robot. The photographing device may be a camera, a camera or a laser, and may also be other devices with an image capturing function. It should be noted that when the camera is used to capture the sequence frame images, the robotic arm drives the camera to move to different shooting points according to a pre-planned path, so that the camera device captures the sequence frame images at different angles.

In a possible embodiment, the pre-planned path may be determined according to the working path of the robot, the length of the robotic arm, the installation position of the photographing device and/or the distance the robot moves between adjacent working points. In another possible embodiment, the path may be determined according to coordinate data of multiple feature points in the target object to be marked and/or coordinate data of multiple feature points in the reference object. It should be noted that the planning method of the path of the robotic arm is not limited to the above two methods, and can also be generated based on other methods.

Among them, the sequence frame image can be understood as a frame sequence composed of several images according to time. Each frame image of the sequence frame image includes a reference object and a target object. The target object can be understood as the object to be grasped or identified, which can be a regular object or an irregular object. For example, in the field of construction, when a robot is used to lay wall tiles or wallpaper, the target object can be the wall tile or wallpaper to be grabbed; in the field of logistics and transportation, robots are used to classify express delivery, and the target object can be the express delivery to be identified. The target object may include one or more stable feature points.

It should be noted that the reference object is generally located in the world coordinate system and is stable. The reference object includes one or more stable feature points, which are used to assist the shooting device to collect sequence frame images at different angles, so that each Both reference and target objects are included in the images. Optionally, the reference object is a two-dimensional code or a barcode.

It should be noted that, before collecting the sequence frame images, at least one feature point of the reference object and the target object is determined respectively, so that each frame image of the collected sequence frame images includes the projection point of the reference object and the projection point of the target object. In an optional embodiment, the feature points of the reference object may be determined according to a pre-planned path of the robotic arm. In another optional embodiment, the feature points of the reference object may be randomly distributed, and may also be determined according to other laws. The feature points of the target object can include the vertex of the target object, the center point of each edge, the center point of each face, and can also include other feature points, such as the third point of each edge, the non-face of each face Heart point and so on.

Wherein, the reference frame and the current frame may be two frames of images arbitrarily selected from the sequence of frame images. It can be known from the foregoing description that both the current frame image and the reference frame image include a reference object and a target object. The current frame image includes the projection point of the reference object and the projection point of the target object, and the reference frame image includes the projection point of the reference object and the projection point of the target object.

S120, determining the difference between the current frame image coordinate system and the reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image The first pose transformation relationship between them.

Before calculating the first pose transformation relationship, determine the coordinate system where the reference object is located, that is, the world coordinate system, define the coordinate system where the reference frame image is located as the reference frame image coordinate system, and define the coordinate system where the current frame image is located as the current frame image coordinate system. Frame image coordinate system.

Optionally, the method for determining the first pose conversion relationship includes: based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image, determining the world coordinate system where the reference object is located and the reference frame. The third pose transformation relationship between the image coordinate systems; based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image, determine the third position between the world coordinate system and the current frame image coordinate system. Four pose transformation relationships; according to the third pose transformation relationship and the fourth pose transformation relationship, determine the first pose transformation relationship.

It should be noted that the projection point of the reference object refers to the projection of the feature point of the reference object in the image collected by the photographing device, that is, the feature point of the reference object corresponds to the projection point of each frame image in the sequence frame images.

Specifically, the method for determining the third pose conversion relationship includes: acquiring the pose of the feature point of the reference object under the world coordinate system and the first coordinate data of the projection point of the reference object under the reference frame image coordinate system; The pose and first coordinate data of the feature points of the object in the world coordinate system determine the third pose transformation relationship between the world coordinate system and the reference frame image coordinate system.

Specifically, the fourth method for determining a pose conversion relationship includes: obtaining the pose of the feature point of the reference object in the world coordinate system and the second coordinate data of the projection point of the reference object in the current frame image coordinate system; according to the reference The pose and second coordinate data of the feature points of the object in the world coordinate system determine the fourth pose transformation relationship between the world coordinate system and the current frame image coordinate system.

It should be noted that the above pose refers to the six-degree-of-freedom pose of the feature point of the reference object in the world coordinate system, and the six-degree-of-freedom pose may include the x, y, z of the feature point of the reference object along the world coordinate system. The movement degrees of freedom for movement in the three coordinate axes and the rotational degrees of freedom for rotation around these three coordinate axes. The first coordinate data refers to the two-dimensional coordinates of the projection point of the reference object in the reference frame image coordinate system, and the second coordinate data refers to the two-dimensional coordinates of the projection point of the reference object in the current frame image coordinate system. The third pose transformation relationship can be understood as the transformation relationship between the world coordinate system and the reference frame image coordinate system, and the fourth pose transformation relationship can be understood as the transformation relationship between the world coordinate system and the current frame image coordinate system. Therefore, the world coordinate system is used as the transformation reference, and the first pose transformation relationship between the reference frame image coordinate system and the current frame image coordinate system is determined according to the third pose transformation relationship and the fourth pose transformation relationship.

S130. Determine the second coordinate system between the target object coordinate system where the target object is located and the current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first attitude transformation relationship Pose transformation relationship.

Before calculating the second pose transformation relationship, the coordinate system where the target object is located is defined as the target object coordinate system. Optionally, the method for determining the second pose conversion relationship includes: determining the distance between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image. The fifth pose transformation relationship is determined; based on the first pose transformation relationship and the fifth pose transformation relationship, the second pose transformation relationship is determined.

Specifically, the fifth method for determining a pose conversion relationship includes: acquiring the pose of the feature point of the target object in the target object coordinate system and the third coordinate data of the projection point of the target object in the reference frame coordinate system, based on the target object The pose and third coordinate data of the feature points of the object in the target object coordinate system are used to calculate the fifth pose transformation relationship between the target object coordinate system and the reference frame coordinate system.

It should be noted that the above pose refers to the six-degree-of-freedom pose of the feature point of the target object in the target object coordinate system, and the six-degree-of-freedom pose may include the x, y of the feature point of the target object along the target object coordinate system. , the movement degrees of freedom of movement in the three coordinate axes of z and the rotational degrees of freedom of rotation around these three coordinate axes. The third coordinate data refers to the two-dimensional coordinates of the projection point of the target object in the reference frame image coordinate system. The fifth pose transformation relationship can be understood as the transformation relationship between the reference frame coordinate system and the target object coordinate system, and the first pose transformation relationship can be understood as the transformation relationship between the reference frame image coordinate system and the current frame image coordinate system. Therefore, the reference frame image coordinate system is used as the transformation reference, and the second pose transformation relationship between the target object coordinate system and the current frame image coordinate system is determined according to the first pose transformation relationship and the fifth pose transformation relationship.

S140. Perform pose annotation on the target object based on the second pose conversion relationship and the projection point of the target object in any frame of images captured by the photographing device.

Wherein, any frame image captured by the photographing device is located in the coordinate system of the current frame image. In the current frame image coordinate system, determine the coordinate data of the projection point of the target object in the image captured by the shooting device, and determine the target according to the second pose transformation matrix and the coordinate data of the projection point of the target object in any frame image captured. The pose of the feature point of the target object corresponding to the projection point of the object in the target object coordinate system.

It should be noted that the coordinate data of the projection point of the target object in the captured image refers to the two-dimensional coordinates of the projection point of the target object in the current frame image coordinate system, and the pose refers to the feature point of the target object in The 6DOF pose in the target object coordinate system. Therefore, according to the coordinate data of the projection point of the target object in the captured image and the second pose transformation matrix, the six-degree-of-freedom pose of the feature point of the target object in the target object coordinate system can be determined, and the features of the target object can be automatically marked point pose.

In the technical solution provided by this embodiment, the sequence frame images collected by the photographing device from different angles are received, and each frame image of the sequence frame image includes the reference object and the target object, so that the reference object assists the photographing device in photographing; the sequence frame image is determined Based on the reference frame image and the current frame image in the reference frame, the coordinates of the current frame image are determined based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image The first pose transformation relationship between the frame and the reference frame image coordinate system; based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship, determine the location of the target object. The second pose transformation relationship between the target object coordinate system and the current frame image coordinate system. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be calculated accurately; The second pose transformation relationship and the projection point of the target object in any frame of images automatically determine the pose of the feature points of the target object, and automatically mark the pose of the feature points of the target object, without manual annotation, reducing labor costs, and Improve the reliability of pose calculation; further, when positioning and analyzing the target object based on the pose of the target object, the positioning accuracy can be improved, especially in the field of robot grasping the target object, which can greatly improve the grasping efficiency of the target object and grasping accuracy.

Embodiment 2

FIG. 2 is a schematic flowchart of a pose labeling method according to Embodiment 2 of the present invention. The technical solution of this embodiment is refined on the basis of the foregoing embodiment. The method of determining the transformation relationship of each pose and the method of labeling the pose are detailed. For the part not described in detail in the method embodiment, please refer to the above-mentioned embodiment. Referring specifically to Figure 2, the method may include the following steps:

S210: Receive the sequence frame images collected by the photographing device at different angles, and determine the reference frame image and the current frame image in the sequence frame images.

S220 , based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image, determine a third pose transformation relationship between the world coordinate system where the reference object is located and the reference frame image coordinate system.

Optionally, the third method for determining the pose conversion relationship includes: acquiring an internal parameter matrix obtained by pre-calibration of the photographing device; The projection point and the current pose transformation relationship between the world coordinate system and the reference frame image coordinate system, calculate the first relationship equation; iteratively solve the first relationship equation, if the current pose transformation relationship under the current iteration number converges, the The current pose transformation relationship under the current iteration number is used as the third pose transformation relationship.

第i个二维投影点对应的世界坐标系下的参考物体的三维特征点记为

世界坐标系和参考帧图像坐标系之间的当前位姿转换关系记为T_w2r，则第一关系方程的表达式为：It should be noted that the internal parameter matrix may be determined according to the parameters in the description of the photographing device, may also be determined according to the accuracy of the pose annotation, or may be determined in other ways. Specifically, the two-dimensional projection points of N reference objects are extracted on the reference frame image, and the i-th two-dimensional projection point is denoted as

The three-dimensional feature point of the reference object in the world coordinate system corresponding to the i-th two-dimensional projection point is recorded as

The current pose transformation relationship between the world coordinate system and the reference frame image coordinate system is recorded as T_w2r, then the expression of the first relationship equation is:

Further, the nonlinear optimization method is used to solve Equation 1, and the initial value of T_w2r is solved under the framework of RANSAC (RANdom Sample Consensus) T ₀ , and LM (Levenberg-Marquardt, based on Levenberg-Marquardt) is adopted. Special) algorithm iteratively solves the first relationship equation, that is, optimizes T_w2r, if the current pose transformation relationship under the current iteration number reaches a stable state, the current pose transformation relationship under the current iteration number is used as the third pose transformation relation.

Optionally, the calculation formula for optimizing T_w2r using the LM algorithm is:

即Δx_k是第k次和第k-1次代次数下的当前位姿转换关系T_w2r的变化量，或者说Δx_k是优化变量；m是信赖区间的半径。Among them, x _k is the current pose transformation relationship T_w2r under the kth iteration number, if k=0,

That is, Δx _k is the change amount of the current pose conversion relationship T_w2r under the kth and k-1th generation times, or Δx _k is the optimization variable; m is the radius of the confidence interval.

ρ是评价指标。若ρ＞0.75，m＝2m，若ρ＜0.25，m＝0.5m。并且，如果ρ大于第一阈值，认为可基于公式2求解Δx_k，令x_k＝x_k-Δx_k-1，根据公式2求解Δx_k，即根据公式2求解第k次和第k-1次迭代次数下的当前位姿转换关系T_w2r的变化量，如果变化量大于或等于第二阈值，确定当前迭代次数下的当前位姿转换关系未收敛，令x_k＝x_k+1-Δx_k，重新计算ρ，并基于公式2求解第k+1次和第k次代次数下的当前位姿转换关系T_w2r的变化量，基于新确定的变化量确定当前迭代次数(第k+1次)下的当前位姿转换关系是否收敛，如果未收敛，继续迭代计算，直至当前迭代次数下的当前位姿转换关系收敛，将当前迭代次数下的当前位姿转换关系。in,

ρ is the evaluation index. If ρ>0.75, m=2m, if ρ<0.25, m=0.5m. And, if ρ is greater than the first threshold, it is considered that Δx _k can be solved based on formula 2, let x _k =x _k -Δx _k-1 , and Δx _k is solved according to formula 2, that is, the kth and k-1th times are solved according to formula 2 The change amount of the current pose transformation relationship T_w2r under the number of iterations, if the change amount is greater than or equal to the second threshold, it is determined that the current pose transformation relationship under the current number of iterations has not converged, let x _k =x _k+1 -Δx _k , recalculate ρ, and solve the variation of the current pose conversion relationship T_w2r under the k+1th and kth generation times based on formula 2, and determine the current iteration number (k+1th) based on the newly determined variation. Whether the current pose transformation relationship of , is converged, if not, continue iterative calculation until the current pose transformation relationship under the current iteration number converges, and convert the current pose transformation relationship under the current iteration number.

S230 , determining a fourth pose transformation relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image.

Optionally, the fourth method for determining the pose conversion relationship includes: acquiring an internal parameter matrix pre-calibrated by a photographing device; The two-dimensional projection point in the frame image and the current pose transformation relationship between the world coordinate system and the current frame image coordinate system, calculate the second relationship equation; iteratively solve the second relationship equation, if the current number of iterations is The current pose transformation relationship converges, and the current pose transformation relationship under the current iteration number is used as the fourth pose transformation relationship.

第i个二维投影点对应的世界坐标系下的参考物体的三维特征点记为

世界坐标系和当前帧图像坐标系之间的当前位姿转换关系记为T_w2c，则第二关系方程的表达式为：Similar to the foregoing steps, the internal parameter matrix may be determined according to the parameters in the description of the photographing device, or may be determined according to the accuracy of the pose annotation, or may be determined in other ways. Specifically, the two-dimensional projection points of N reference objects are extracted on the current frame image, and the i-th two-dimensional projection point is denoted as

The three-dimensional feature point of the reference object in the world coordinate system corresponding to the i-th two-dimensional projection point is recorded as

The current pose transformation relationship between the world coordinate system and the current frame image coordinate system is recorded as T_w2c, then the expression of the second relationship equation is:

Similar to the previous steps, the nonlinear optimization method is used to solve Equation 3. The initial value of T_w2c is solved under the RANSAC (RANdomSampleConsensus, random sampling consistent) framework, and the initial value of T_w2c is T ₀ . Quarte) algorithm iteratively solves the second relationship equation, that is, optimizes T_w2c. If the current pose transformation relationship under the current iteration number reaches a stable state, the current pose transformation relationship under the current iteration number is regarded as the fourth position. Pose transformation relationship. It should be noted that the calculation formula of the LM algorithm optimization T_w2c is consistent with the formula 2. In this step, x _k in the LM algorithm is the current pose transformation relationship T_w2c under the kth iteration number, and the second relationship is calculated by using the formula 2. The equation is iteratively solved, and if the current pose transformation relationship under the current iteration number converges, the current pose transformation relationship under the current iteration number is used as the fourth pose transformation relationship.

S240. Determine the first pose transformation relationship according to the third pose transformation relationship and the fourth pose transformation relationship.

Optionally, the third pose transformation relationship and the fourth pose transformation relationship are vector-multiplied to obtain the first pose transformation relationship.

Figure 3 is a schematic diagram of the pose transformation relationship between the coordinate systems. The reference object in Figure 3 is a two-dimensional code, and the coordinate system where the two-dimensional code is located is the world coordinate system. The third pose transformation relationship between the world coordinate system and the reference frame image coordinate system is T_w2r, and the fourth pose transformation relationship between the world coordinate system and the current frame image coordinate system is T_w2c, the current frame image coordinate system and the reference frame. The first pose transformation relationship between the image coordinate systems T_c2r=T_w2r*T_w2c ^-1 .

S250. Determine a fifth pose transformation relationship between the coordinate system of the target object and the coordinate system of the reference frame image according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image.

Optionally, the fifth method for determining a pose conversion relationship includes: acquiring an internal parameter matrix pre-calibrated by a photographing device; The two-dimensional projection point in the frame image is used to calculate the third relationship equation; the third relationship equation is iteratively solved. If the current pose transformation relationship under the current number of iterations converges, the current pose transformation relationship under the current number of iterations is calculated. as the fifth pose conversion relationship.

第j个二维投影点对应的目标物体坐标系下的目标物体的三维特征点记为

世界坐标系和参考帧图像坐标系之间的当前位姿转换关系记为T_o2r，则第三关系方程的表达式为：Specifically, the two-dimensional projection points of M target objects are extracted on the reference frame image, and the j-th two-dimensional projection point is denoted as

The three-dimensional feature point of the target object in the target object coordinate system corresponding to the jth two-dimensional projection point is recorded as

The current pose transformation relationship between the world coordinate system and the reference frame image coordinate system is recorded as T_o2r, then the expression of the third relationship equation is:

Similar to the previous steps, the nonlinear optimization method is used to solve Equation 4, and the initial value of T_o2r is solved under the framework of RANSAC (RANdomSampleConsensus) T ₀ , and LM (Levenberg-Marquardt, based on Levenberg-Marquardt) is used. (Quart) algorithm iteratively solves the third relationship equation, that is, optimizes T_o2r. If the current pose transformation relationship under the current iteration number reaches a stable state, the current pose transformation relationship under the current iteration number is regarded as the fifth position. Pose transformation relationship. It should be noted that the calculation formula of T_o2r optimized by the LM algorithm is consistent with formula 2. In this step, x _k in the LM algorithm is the current pose transformation relationship T_o2r under the kth iteration number, and formula 2 is used for the third relationship. The equation is iteratively solved, and if the current pose transformation relationship under the current iteration number converges, the current pose transformation relationship under the current iteration number is used as the fifth pose transformation relationship.

S260. Determine the second pose transformation relationship based on the first pose transformation relationship and the fifth pose transformation relationship.

Optionally, the determining the second pose conversion relationship based on the first pose conversion relationship and the fifth pose conversion relationship includes: combining the first pose conversion relationship and the first pose conversion relationship. The five pose transformation relationships are multiplied by vectors to obtain the current pose transformation relationship between the target object coordinate system and the current frame image coordinate system; according to the internal parameter matrix pre-calibrated by the camera, the three-dimensional feature points of the target object, the target object The two-dimensional projection point of the feature point of the object in the current frame image, the current pose transformation relationship between the coordinate system of the target object and the coordinate system of the current frame image, calculate the fourth relation equation; Iteratively solves, if the current pose transformation relationship under the current iteration number converges, the current pose transformation relationship under the current iteration number is used as the second pose transformation relationship.

With reference to Figure 3, the fifth pose transformation relationship between the target object coordinate system and the reference frame image coordinate system is T_o2r, and the first pose transformation relationship between the reference frame image coordinate system and the current frame image coordinate system is T_c2r, then The second pose transformation relationship between the current frame image coordinate system and the target object coordinate system T_o2c=T_o2r*T_c2r ^-1 .

第j个二维投影点对应的目标物体坐标系下的目标物体的三维特征点记为

世界坐标系和当前帧图像坐标系之间的当前位姿转换关系记为T_o2c，则第四关系方程的表达式为：Specifically, the two-dimensional projection points of M target objects are extracted on the current frame image, and the j-th two-dimensional projection point is denoted as

The three-dimensional feature point of the target object in the target object coordinate system corresponding to the jth two-dimensional projection point is recorded as

The current pose conversion relationship between the world coordinate system and the current frame image coordinate system is recorded as T_o2c, then the expression of the fourth relationship equation is:

The expression of the fourth relational equation can also be in the following form:

Similar to the previous steps, the nonlinear optimization method is used to solve Equation 5 or Equation 6, and the initial value of T_o2c is T ₀ under the framework of RANSAC (RANdomSample Consensus), and LM (Levenberg-Marquardt, based on Levin Berg-Marquardt) algorithm iteratively solves the third relationship equation, that is, optimizes T_o2c. If the current pose transformation relationship under the current iteration number reaches a stable state, the current pose transformation relationship under the current iteration number as the fifth pose transformation relationship. It should be noted that the calculation formula for optimizing T_o2c of the LM algorithm is consistent with the formula 2. In this step, x _k in the LM algorithm is the current pose transformation relationship T_o2c under the kth iteration number, and formula 2 is used to determine the fourth relationship. The equation is iteratively solved, and if the current pose transformation relationship under the current iteration number is converged, the current pose transformation relationship under the current iteration number is used as the second pose transformation relationship.

S270. Perform pose annotation on the target object based on the second pose conversion relationship and the projection point of the target object in any frame of images captured by the photographing device. Optionally, based on the second pose conversion relationship and the projection point of the target object in any frame of images shot by the photographing device, performing pose labeling on the target object, including: according to any frame of images photographed by the photographing device. The projection point, the second pose conversion relationship, and the internal parameter matrix obtained by the pre-calibration of the photographing device, obtain the pose of the feature points of the target object in the image captured by the shooting device, and perform pose labeling on the feature points of the target object .

Specifically, any frame of image photographed by the photographing device is located in the coordinate system of the current frame image. In the current frame image coordinate system, the projection point of the target object in the captured image is multiplied according to the projection point of the target object in any frame image captured, the second pose conversion relationship and the internal parameter matrix pre-calibrated by the capturing device, The pose of the feature points of the target object in the image captured by the photographing device is obtained, that is, the six-degree-of-freedom pose of the feature points of the target object in the target object coordinate system is obtained, and the pose labeling of the target object is performed.

Figure 4 shows the logical schematic diagram of pose annotation. The above process is explained as a whole with reference to FIG. 4 . The path of the robotic arm is determined in advance and the internal parameter matrix of the shooting device is obtained. The robotic arm drives the shooting device to collect the sequence frame images according to the pre-planned path, and selects the reference frame image and the current frame from the collected sequence frame images; the current frame image and the sequence frame image The image includes the target object and the reference object. According to the projection point of the target object and the projection point of the reference object in the current frame image and the sequence frame image, as well as the feature point of the target object and the feature point of the reference object, the RANSAC framework and the LM algorithm are used. Calculate the pose transformation between the target object coordinate system and the current frame image coordinate system; when the camera captures a new image, determine the six Degree of freedom pose, until the camera has collected all the images and completed the pose annotation.

The technical solution provided in this embodiment accurately calculates the position between the coordinate system of the target object and the coordinate system of the current frame image according to the internal reference matrix, the feature points of the reference object, the feature points of the target object, and the projection points in the sequence frame image. Attitude conversion relationship; further based on the projection point of the target object in any frame image captured by the camera, according to the internal parameter matrix of the camera, and the pose transformation, accurately calculate and mark the six-degree-of-freedom pose of the feature points of the target object without manual labor. Labeling can reduce labor costs; further, when positioning and analyzing the target object based on the pose of the target object, the positioning accuracy can be improved, especially in the field of robot grasping the target object, which can greatly improve the grasping efficiency and grasping efficiency of the target object. Take precision.

Embodiment 3

FIG. 5 is a schematic structural diagram of a pose labeling device according to Embodiment 3 of the present invention. Referring to FIG. 5 , the system includes: an image determination module 310 , a first pose transformation relationship determination module 310 , a second pose transformation relationship determination module 330 , and a pose labeling module 340 .

The image determination module 310 is configured to receive the sequence frame images collected by the photographing device at different angles, and determine the reference frame image and the current frame image in the sequence frame images, wherein each frame image of the sequence frame images including the reference object and the target object;

The first attitude conversion relationship determination module 320 is used to determine the feature point of the reference object, the projection point of the feature point of the reference object in the reference frame image, and the projection point of the feature point of the reference object in the current frame image. The first pose conversion relationship between the current frame image coordinate system and the reference frame image coordinate system;

The second pose transformation relationship determining module 330 is configured to determine the coordinates of the target object where the target object is located based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship The second pose transformation relationship between the frame and the current frame image coordinate system;

The pose labeling module 340 is configured to label the target object on the basis of the second pose conversion relationship and the projection point of the target object in any frame of images captured by the photographing device.

In the technical solution provided by this embodiment, the sequence frame images collected by the photographing device from different angles are received, and each frame image of the sequence frame image includes the reference object and the target object, so that the reference object assists the photographing device in photographing; the sequence frame image is determined Based on the reference frame image and the current frame image in the reference frame, the coordinates of the current frame image are determined based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image The first pose transformation relationship between the frame and the reference frame image coordinate system; based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship, determine the location of the target object. The second pose transformation relationship between the target object coordinate system and the current frame image coordinate system. Since the feature points of the reference object and the feature points of the target object are stable and reliable, the first pose transformation matrix and the second pose transformation matrix can be calculated accurately; The second pose transformation relationship and the projection point of the target object in any frame of images automatically determine the pose of the feature points of the target object, and automatically mark the pose of the feature points of the target object, without manual annotation, reducing labor costs, and Improve the reliability of pose calculation; further, when positioning and analyzing the target object based on the pose of the target object, the positioning accuracy can be improved, especially in the field of robot grasping the target object, which can greatly improve the grasping efficiency of the target object and grasping accuracy.

Optionally, the first pose conversion relationship determination module 320 is further configured to, based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image, determine the difference between the world coordinate system where the reference object is located and the The third pose conversion relationship between the reference frame image coordinate systems;

Determine the fourth pose conversion relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image;

The first pose transformation relationship is determined according to the third pose transformation relationship and the fourth pose transformation relationship.

Optionally, the first pose conversion relationship determining module 320 is further configured to acquire an internal parameter matrix pre-calibrated by the photographing device;

According to the internal reference matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the reference frame image, and the current pose transformation relationship between the world coordinate system and the reference frame image coordinate system , calculate the first relation equation;

The first relationship equation is iteratively solved, and if the current pose conversion relationship under the current iteration number is converged, the current pose conversion relationship under the current iteration number is used as the third pose conversion relationship.

Optionally, the first pose conversion relationship determining module 320 is further configured to acquire an internal parameter matrix pre-calibrated by the photographing device;

According to the internal reference matrix, the three-dimensional feature points of the reference object, the two-dimensional projection points of the feature points of the reference object in the current frame image, and the current pose transformation relationship between the world coordinate system and the current frame image coordinate system , calculate the second relation equation;

The second relationship equation is iteratively solved, and if the current pose conversion relationship under the current iteration number is converged, the current pose conversion relationship under the current iteration number is used as the fourth pose conversion relationship.

Optionally, the second pose conversion relationship determination module 330 is further configured to determine the target object coordinate system and the the fifth pose transformation relationship between the reference frame image coordinate systems;

The second pose transformation relationship is determined based on the first pose transformation relationship and the fifth pose transformation relationship.

Optionally, the second pose conversion relationship determination module 330 is further configured to acquire an internal parameter matrix obtained by pre-calibration of the photographing device;

Calculate a third relational equation according to the internal reference matrix, the three-dimensional feature points of the target object, and the two-dimensional projection points of the feature points of the target object in the reference frame image;

The third relationship equation is iteratively solved, and if the current pose conversion relationship under the current iteration number is converged, the current pose conversion relationship under the current iteration number is used as the fifth pose conversion relationship.

Optionally, the second pose transformation relationship determining module 330 is further configured to perform vector multiplication of the first pose transformation relationship and the fifth pose transformation relationship to obtain the target object coordinate system and the current frame image. The current pose transformation relationship between coordinate systems;

The internal parameter matrix pre-calibrated according to the camera, the three-dimensional feature points of the target object, the two-dimensional projection points of the feature points of the target object in the current frame image, the current coordinate system between the target object coordinate system and the current frame image coordinate system Pose transformation relationship, calculate the fourth relationship equation;

The fourth relationship equation is iteratively solved, and if the current pose conversion relationship under the current iteration number is converged, the current pose conversion relationship under the current iteration number is used as the second pose conversion relationship.

Optionally, the first pose transformation relationship determining module 320 is further configured to perform vector multiplication of the third pose transformation relationship and the fourth pose transformation relationship to obtain the first pose transformation relationship.

Optionally, the pose labeling module 340 is further configured to, according to the projection point of the target object in any frame image shot by the shooting device, the second pose conversion relationship and the internal parameter matrix pre-calibrated by the shooting device, obtain the The pose of the feature points of the target object in the image captured by the photographing device is used to mark the pose of the feature points of the target object.

Optionally, the photographing device includes any one of the following: a camera, a camera, and a laser; the reference object includes any one of the following: a two-dimensional code and a barcode.

Embodiment 4

FIG. 6 is a schematic structural diagram of a pose labeling system according to Embodiment 4 of the present invention. Figure 6 shows a block diagram of an exemplary robot suitable for use in implementing embodiments of the present invention. The robot shown in FIG. 6 is only an example, and should not impose any limitations on the functions and scope of use of the embodiments of the present invention.

The pose labeling system shown in Figure 6 includes a robot, a target object and a reference object. The robot includes a robotic arm, a photographing device, a memory (not shown in the figure), and a processor (not shown in the figure). The robotic arm drives the photographing device to move, so that the photographing device captures a sequence of frame images from multiple angles; wherein, each frame of the sequence of frame images includes a reference object and a target object, and the target object includes multiple frames. feature points, and the reference object includes multiple feature points. As shown in FIG. 6 , the robot further includes: a base and an end tool; the robotic arm is mounted on the base, the end tool is mounted on the end of the robotic arm, and the end tool is mounted on one side of the end tool. filming device.

The processor executes various functional applications and data processing by running the program stored in the memory, for example, implementing a pose labeling method provided by the embodiment of the present invention, and the method includes:

Receive sequence frame images collected by a photographing device at different angles, and determine a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images includes a reference object and a target object;

Based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image, determine the difference between the current frame image coordinate system and the reference frame image coordinate system The first pose conversion relationship between;

Based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first attitude transformation relationship, determine the second coordinate system between the target object coordinate system where the target object is located and the current frame image coordinate system Pose transformation relationship;

Based on the second pose conversion relationship and the projection point of the target object in any frame image captured by the photographing device, the pose labeling is performed on the target object.

Of course, those skilled in the art can understand that the processor can also implement the technical solution of the pose labeling method provided by any embodiment of the present invention.

Embodiment 5

Embodiment 5 of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements a pose annotation provided by the embodiment of the present invention, and the method includes:

Receive sequence frame images collected by a photographing device at different angles, and determine a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images includes a reference object and a target object;

Based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image, determine the difference between the current frame image coordinate system and the reference frame image coordinate system The first pose conversion relationship between;

Based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first attitude transformation relationship, determine the second coordinate system between the target object coordinate system where the target object is located and the current frame image coordinate system Pose transformation relationship;

Based on the second pose conversion relationship and the projection point of the target object in any frame image captured by the photographing device, the pose labeling is performed on the target object.

Of course, in the computer-readable storage medium provided by the embodiment of the present invention, the computer program stored on the storage medium is not limited to the above method operations, and can also execute the related operations in the pose labeling method provided by any embodiment of the present invention. operate.

The computer storage medium in the embodiments of the present invention may adopt any combination of one or more computer-readable mediums. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or a combination of any of the above. More specific examples (a non-exhaustive list) of computer readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), Erasable Programmable Read Only Memory (EPROM or Flash), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device.

The computer-readable signal medium may include the first pose transformation relationship, the second pose transformation relationship, the projection point of the target object in the captured image, etc., and carry computer-readable program codes therein. The first pose transformation relationship, the second pose transformation relationship, the projection point of the target object in the captured image, etc. A computer-readable signal medium can also be any computer-readable medium, other than a computer-readable storage medium, that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, system, or device .

Program code embodied on a computer readable medium may be transmitted using any suitable medium, including - but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations of the present invention may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional procedural languages, or a combination thereof. Programming Language - such as "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

It is worth noting that, in the above embodiments of the pose labeling device, the modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, each function The specific names of the units are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present invention.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (14)

1. A pose labeling method is characterized by comprising the following steps:

receiving sequence frame images acquired by a shooting device at different angles, and determining a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images comprises a reference object and a target object;

determining a first pose conversion relation between a current frame image coordinate system and a reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image and the projection points of the feature points of the reference object in the current frame image;

determining a second pose conversion relation between a target object coordinate system where the target object is located and a current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image and the first pose conversion relation;

and marking the pose of the target object based on the second pose conversion relation and the projection point of the target object in any frame of image shot by the shooting device.

2. The method according to claim 1, wherein determining the first pose conversion relationship between the coordinate system of the current frame image and the coordinate system of the reference frame image based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image, and the projection points of the feature points of the reference object in the current frame image comprises:

determining a third pose conversion relation between a world coordinate system where the reference object is located and a reference frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image;

determining a fourth pose conversion relation between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image;

and determining the first attitude conversion relation according to the third attitude conversion relation and the fourth attitude conversion relation.

3. The method according to claim 2, wherein the determining a third pose transformation relationship between the world coordinate system of the reference object and the reference frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the reference frame image comprises:

obtaining an internal reference matrix obtained by pre-calibrating a shooting device;

calculating a first relation equation according to the internal reference matrix, the three-dimensional characteristic points of the reference object, the two-dimensional projection points of the characteristic points of the reference object in the reference frame image and the current pose conversion relation between the world coordinate system and the reference frame image coordinate system;

and carrying out iterative solution on the first relation equation, and if the current pose conversion relation under the current iteration times is converged, taking the current pose conversion relation under the current iteration times as the third pose conversion relation.

4. The method according to claim 2, wherein the determining a fourth pose conversion relationship between the world coordinate system and the current frame image coordinate system based on the feature points of the reference object and the projection points of the feature points of the reference object in the current frame image comprises:

acquiring an internal reference matrix obtained by pre-calibrating a shooting device;

calculating a second relation equation according to the internal reference matrix, the three-dimensional characteristic points of the reference object, the two-dimensional projection points of the characteristic points of the reference object in the current frame image and the current pose conversion relation between the world coordinate system and the current frame image coordinate system;

and carrying out iterative solution on the second relation equation, and if the current position and posture conversion relation under the current iteration times is converged, taking the current position and posture conversion relation under the current iteration times as the fourth position and posture conversion relation.

5. The method according to claim 1, wherein the determining a second pose transformation relationship between the target object coordinate system in which the target object is located and the current frame image coordinate system based on the feature points of the target object, the projection points of the feature points of the target object in the reference frame image, and the first pose transformation relationship comprises:

determining a fifth pose conversion relation between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image;

determining the second pose transformation relationship based on the first pose transformation relationship and the fifth pose transformation relationship.

6. The method according to claim 5, wherein the determining a fifth pose conversion relationship between the target object coordinate system and the reference frame image coordinate system according to the feature points of the target object and the projection points of the feature points of the target object in the reference frame image comprises:

acquiring an internal reference matrix obtained by pre-calibrating a shooting device;

calculating a third relation equation according to the internal reference matrix, the three-dimensional characteristic points of the target object and the two-dimensional projection points of the characteristic points of the target object in the reference frame image;

and carrying out iterative solution on the third relation equation, and if the current pose conversion relation under the current iteration times is converged, taking the current pose conversion relation under the current iteration times as the fifth pose conversion relation.

7. The method of claim 5, wherein the determining the second pose translation relationship based on the first pose translation relationship and the fifth pose translation relationship comprises:

performing vector multiplication on the first pose conversion relation and the fifth pose conversion relation to obtain a current pose conversion relation between a target object coordinate system and a current frame image coordinate system;

calculating a fourth relation equation according to a reference matrix obtained by calibrating in advance by a camera, the three-dimensional characteristic point of the target object, the two-dimensional projection point of the characteristic point of the target object in the current frame image, the target object coordinate system and the current pose conversion relation between the current frame image coordinate system and the target object coordinate system;

and carrying out iterative solution on the fourth relation equation, and if the current pose conversion relation under the current iteration times is converged, taking the current pose conversion relation under the current iteration times as the second pose conversion relation.

8. The method of claim 2, wherein the determining the first pose translation relationship based on the third pose translation relationship and the fourth pose translation relationship comprises:

and carrying out vector multiplication on the third attitude conversion relation and the fourth attitude conversion relation to obtain the first attitude conversion relation.

9. The method according to claim 1, wherein the pose labeling of the target object based on the second pose transformation relation and the projection point of the target object in any frame of image captured by the capturing device comprises:

and obtaining the pose of the characteristic point of the target object in the image shot by the shooting device according to the projection point of the target object in any frame of image shot by the shooting device, the second pose transformation relation and the internal reference matrix obtained by pre-calibration of the shooting device, and carrying out pose marking on the characteristic point of the target object.

10. The method of claim 1, wherein the camera comprises any one of: a camera, a camera and a laser; and/or, the reference object comprises any one of: two-dimensional codes and bar codes.

11. A pose labeling apparatus, comprising:

the image determining module is used for receiving sequence frame images acquired by a shooting device under different angles and determining a reference frame image and a current frame image in the sequence frame images, wherein each frame image of the sequence frame images comprises a reference object and a target object;

the first pose conversion relation determining module is used for determining a first pose conversion relation between a current frame image coordinate system and a reference frame image coordinate system based on the feature points of the reference object, the projection points of the feature points of the reference object in the reference frame image and the projection points of the feature points of the reference object in the current frame image;

the second pose conversion relation determining module is used for determining a second pose conversion relation between a target object coordinate system where the target object is located and a current frame image coordinate system based on the feature point of the target object, the projection point of the feature point of the target object in the reference frame image and the first pose conversion relation;

and the pose marking module is used for marking the pose of the target object based on the second pose conversion relation and the projection point of the target object in any frame of image shot by the shooting device.

12. A pose marking system comprises a robot, a target object and a reference object, wherein the robot comprises a mechanical arm, a shooting device, a memory and a processor; the method is characterized in that the mechanical arm drives the shooting device to move so that the shooting device can acquire sequence frame images at a plurality of angles; each frame of image of the sequence frame image comprises a reference object and a target object, the target object comprises a plurality of characteristic points, and the reference object comprises a plurality of characteristic points;

a computer program stored in a memory and executable on a processor, the processor implementing the pose labeling method according to any one of claims 1 to 10 when executing the computer program.

13. The system of claim 12, wherein the robot further comprises: a base and a tip tool;

the arm is installed on the base, end tool is installed at the end of the arm, and one side of the end tool is provided with the shooting device.

14. A storage medium containing computer-executable instructions, which when executed by a computer processor implement the pose annotation method according to any one of claims 1-10.

Images