CN118322213A - Vision-based kinematic calibration method for industrial robots in large workspaces - Google Patents

Abstract

The vision-based kinematic calibration method for industrial robots in large-scale workspaces solves the problem that the existing vision-based robot kinematic calibration method can only improve the absolute positioning accuracy of the robot in a limited space due to the field of view limitation, and belongs to the field of robot kinematic calibration technology. The present invention includes: according to the configuration of the industrial robot, ArUco markers are arranged in the workspace; according to the set shooting path, a monocular vision system is used to capture ArUco marker images to generate a fully covered ArUco map; the robot uses a monocular vision system to capture ArUco marker images in different postures, constructs an absolute position model of the robot end, and obtains the measured posture; according to the robot joint angle and the nominal value of the kinematic parameters, the error matrix of the nominal posture of the end and the measured posture is calculated; a posture error model is established, and the robot kinematic parameter error is obtained according to the error matrix.

Description

Vision-based kinematic calibration method for industrial robots in large workspaces

Technical Field

The invention relates to a vision-based kinematic calibration method for an industrial robot in a large-size workspace, and belongs to the technical field of robot kinematic calibration.

Background technique

With the demand for high-speed, high-precision, and large-load industrial robots in advanced manufacturing, the requirements for the absolute positioning accuracy of robots are becoming higher and higher. Research on improving the absolute positioning accuracy of robots through calibration technology has become a hot topic. Vision-based robot kinematic calibration technology is favored because of its low cost and simple operation. At present, robot kinematic calibration mainly relies on specific markers or feature points (such as chessboards and precision balls) as calibration references. In the paper "Kinematic identification of industrial robot using end-effector mounted monocular camera bypassing measurement of 3-d pose[J].IEEE/ASME Transactions on Mechatronics 27.1(2021):383-394.", a kinematic identification method for industrial robots is introduced. This method uses a monocular camera mounted on the end effector. It does not require three-dimensional pose measurement and directly uses the two-dimensional image of the chessboard calibration plate to simplify the process to a single-level estimation. However, due to the camera field of view, the robot calibration space is limited. In the paper "Anovel vision-based calibration framework for industrial robotic manipulators[J]. Robotics and Computer-Integrated Manufacturing 73(2022):102248.", a novel visual calibration framework is proposed, which uses a single camera fixed on the outside and ArUco markers at the end of the robot to calibrate the industrial robotic arm. However, this method cannot take into account the camera field of view and measurement distance, and its flexibility in application in large-scale workspaces is limited.

In summary, the current vision-based robot kinematic calibration methods still have a common problem: the camera field of view limits the robot's range of movement during the calibration process, causing the improvement of the robot's absolute positioning accuracy to be mainly concentrated in the restricted measurement area. Therefore, when the robot is used in a large workspace, its absolute positioning accuracy is difficult to meet the application requirements of high precision in the entire space.

Summary of the invention

In view of the problem that the existing vision-based robot kinematics calibration method can only improve the absolute positioning accuracy of the robot in a limited space due to the limitation of the field of view, the present invention provides a vision-based industrial robot kinematics calibration method in a large-size workspace.

The present invention provides a vision-based kinematic calibration method for an industrial robot in a large-size workspace, comprising:

ArUco markers are laid out in the workspace according to the industrial robot configuration;

According to the set shooting path, the monocular vision system is used to capture ArUco tagged images, and the image stitching technology is used to generate a fully covered ArUco map from the ArUco tagged images;

The robot uses a monocular vision system to capture ArUco marked images in different positions, identifies the end position of the robot according to the fully covered ArUco map, and constructs an absolute position model of the end position of the robot;

According to the robot joint angle and the nominal value of kinematic parameters, the terminal nominal posture is calculated, and the terminal measurement posture is obtained using the robot terminal absolute position model, and the error matrix between the terminal nominal posture and the measurement posture is obtained;

The posture error model is established, and the robot kinematic parameter error is obtained according to the error matrix.

Preferably, according to the industrial robot configuration, the method for laying out ArUco markers in the workspace comprises:

Establish the objective function F(O) of the layout plan:

F(O)＝α·V(T)+β·U(D)+γ·P(R)-δ·C(S)

Among them, O represents the layout of the mark,

V(T) represents the visibility parameter of ArUco marker, and α represents the weight of V(T);

U(D) represents the uniformity parameter of ArUco marker placement, and β represents the weight of U(D);

P(R) represents the average positioning accuracy of the robot at all positions within its motion range, and γ represents the weight of P(R);

C(S) represents the layout cost, and δ represents the weight of C(S);

A genetic algorithm is used to optimize the layout scheme O of ArUco markers to maximize the objective function F(O). During the optimization process, the position, number and direction of the markers are continuously updated iteratively until a layout scheme that maximizes F(O) is found.

As a preference,

Where D represents the set of label densities of each test point in the workspace, and σ ² (D) is the variance of D.

As a preference,

Where, _vi represents the visibility of the ith ArUco marker, which depends on the angle and distance between the ArUco marker and the camera, _Ai represents the visible area of the ith ArUco marker in the camera’s field of view, _Atotal represents the total area of the camera’s field of view, and n represents the total number of ArUco markers.

As a preference,

Where m is the total number of positioning operations, _dj is the positioning error in the jth operation, and _dmax is the maximum allowed error distance.

Preferably, the method of capturing ArUco tagged images using a monocular vision system according to a set shooting path, and generating a fully covered ArUco map from the ArUco tagged images using image stitching technology includes:

According to the set closed-loop shooting path, shoot ArUco mark images in sequence to ensure that the same ArUco mark exists in adjacent ArUco mark images;

Determine the position of each ArUco marker, and use the L-M algorithm to minimize the reprojection error of the target's ArUco marker corner points to obtain the initial ArUco map;

The objective function of the reprojection error is

and

Among them, Ψ(δ,γ ^t ,γ _i ·c _j ) represents the projection of the corner point from the three-dimensional space coordinates to the pixel coordinates, δ represents the camera intrinsic parameter matrix, γ ^t represents the camera extrinsic parameter, γ _i represents the transformation matrix from the marker coordinate system to the world coordinate system, and c _j represents the three-dimensional coordinates of the marker corner point. represents the pixel coordinates of the marked corner points, K is the total number of images in the closed-loop shooting path, _xk is the pose of the kth image, _zk is the pose transformation from the kth image to the k+1th image, is the operator between places.

As a preferred method, the method of constructing the absolute position model of the robot end is:

Based on the fully covered ArUco map and the ArUco marker images captured by the robot using the monocular vision system at different positions, the coordinate transformation relationship between the ArUco image coordinate system and the camera coordinate system is established;

The hand-eye calibration model is used to calculate the relative posture relationship between the robot end coordinate system and the camera coordinate system, thereby determining the absolute position model of the robot end.

Preferably, a posture error model is established, and the error matrix is processed by Python programming to obtain the robot kinematic parameter error.

The beneficial effects of the present invention are:

The present invention not only greatly reduces the application cost of high-precision calibration technology by adopting a monocular vision system and an ArUco marker map, but also simplifies the calibration process, making the system installation and operation simpler and faster. This feature makes the present invention particularly suitable for production environments that require frequent calibration or adjustment, saving users a lot of time and economic costs. The present invention effectively solves the problem that the traditional visual calibration method is limited by the camera field of view, and can achieve higher-precision robot positioning in a larger workspace. This improvement not only improves production efficiency, but also improves the control quality, and is particularly suitable for application scenarios with extremely high positioning accuracy requirements. The calibration method of the present invention is applicable to various industrial robot models and brands and has good versatility. The flexible design of the ArUco marker map allows the calibration system to be adjusted according to actual needs and has good scalability. At the same time, the enhanced information and redundancy provided by the method ensure that a high calibration accuracy can be maintained even when some markers are blocked or damaged, thereby enhancing the robustness of the system in complex environments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flowchart of a specific implementation of the present invention;

FIG2 is a schematic diagram of a kinematic model of a robot according to the present invention;

FIG3 is a schematic diagram of establishing an ArUco map according to the present invention;

FIG. 4 is a schematic diagram of a robot kinematics calibration platform according to the present invention.

FIG. 5 is a schematic diagram of regional verification of the robot positioning accuracy of the present invention.

FIG. 6 is a comparison diagram of the positioning accuracy of the robot before and after calibration in a specific example of the present invention.

Detailed ways

The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

A vision-based kinematic calibration method for an industrial robot in a large-size workspace in this embodiment includes the following steps:

Step 1: ArUco markers are laid out in the workspace according to the industrial robot configuration and the layout is optimized.

Step 2: Use a monocular vision system to capture ArUco tagged images according to the set shooting path, and use image stitching technology to generate a fully covered ArUco map from the ArUco tagged images;

Step 3: The robot uses a monocular vision system to capture ArUco marker images in different positions, identifies the end position of the robot according to the fully covered ArUco map, and constructs an absolute position model of the end position of the robot;

Step 4: Calculate the nominal pose of the end according to the robot joint angle and the nominal value of the kinematic parameters, and use the absolute position model of the robot end to obtain the end measurement pose, and obtain the error matrix between the nominal pose of the end and the measurement pose;

Step 5: Establish a posture error model and obtain the robot kinematic parameter error based on the error matrix.

This implementation not only greatly reduces the application cost of high-precision calibration technology by using a monocular vision system and ArUco marker maps, but also simplifies the calibration process, making system installation and operation simpler and faster. This feature makes the present invention particularly suitable for production environments that require frequent calibration or adjustment, saving users a lot of time and economic costs.

After this implementation, the layout of the on-site equipment (industrial robots, workbenches, processing equipment, etc.) is determined, and the appropriate industrial camera is selected according to the working distance and measurement accuracy. The installation position and angle of the camera and the end of the robot are carefully designed to optimize the quality of image capture;

Place ArUco markers with unique IDs at appropriate locations within the robot's workspace and use image stitching technology to generate an ArUco map (calibration benchmark) that fully covers the workspace. During the capture process, at least one complete ArUco marker is completely within the camera's field of view;

The robot posture is selected to cover the entire working range of the robot as much as possible, taking into account the robot's motion characteristics and the capture capability of the visual system. The robot performs motion at the predetermined posture points and collects the six joint angle data of the industrial robot in each posture; at the same time, the industrial camera installed at the end of the robot captures the image containing the ArUco marker. The image data of each posture point and the robot's joint angle data are recorded and stored for subsequent data analysis and processing;

The layout of the ArUco markers in step 1 is as follows:

First, in the workspace of the industrial robot, the layout of the ArUco markers is optimized according to the specific configuration and operation requirements of the robot. In this embodiment, the layout of the markers follows the following principles:

(1) Marker layout uniformity: By rationally planning the position and spacing of markers, we ensure that the markers cover the entire workspace and maintain similar marker density at different locations. This process is achieved by optimizing the objective function ArUco marker layout uniformity U(D), where

D represents the set of label densities of each inspection point in the workspace, σ ² (D) is the variance of D;

(2) ArUco marker visibility: When choosing the placement of the marker, consider the camera parameters and field of view to avoid placing the ArUco marker in the occluded area of the robot or other objects. The total visibility V(T) of the ArUco marker is calculated by the following formula to ensure that the camera can accurately monitor and locate each marker;

(3) Robot pose change: In order to increase the robot’s pose range, ArUco markers are placed in a variety of positions, including non-planar positions. This not only increases the pose range, but also increases the robustness and stability of the calibration, which is particularly suitable for complex workspaces and multi-pose operations.

Taking into account the visibility, uniformity and positioning accuracy of the markers, the optimization objective function is defined:

F(O)＝α·V(T)+β·U(D)+γ·P(R)-δ·C(S)

Among them, F(O) is the optimization objective function, O represents the layout of the mark. α represents the weight of V(T), β represents the weight of U(D), γ represents the weight of P(R), and P(R) represents the average positioning accuracy of all positions of the robot within its range of motion. Calculate, where m is the total number of positioning operations, d _j is the positioning error in the jth operation, and d _max is the maximum allowed error distance. C(S) represents the placement cost, which is calculated by the formula Calculate , where _ck is the placement cost of the kth token and δ represents the weight of C(S).

The genetic algorithm is used to optimize the layout scheme O of ArUco markers to maximize the objective function F(O). During the optimization process, the position, number and direction of the markers are continuously updated iteratively until a layout scheme that maximizes F(O) is found.

The establishment of the ArUco map in step 2 is as follows:

(1) The robot captures ArUco markers in different poses to ensure that the same marker exists in adjacent images. In robot pose 1, when the robot detects markers 1 and 3 in the image at the same time, the pose relationship between the camera and marker 1 can be established. And the posture relationship between markers 3 In this way, we can derive the pose transformation matrix between marker 1 and marker 3 in pose 1 In pose 2, when marker 3 and marker 4 are detected at the same time, the pose transformation matrix between them can be determined In addition, since there is a common marker 3, the pose transformation matrix between marker 4 and marker 1 can be calculated Similarly, by using the common markers in the images captured at each location, we can determine the relative position of any two ArUco markers in space. By specifying one of the ArUco coordinate systems as the reference world coordinate system, we can determine the relative positions of all ArUco markers to the reference coordinate system.

(2) Due to insufficient lighting, fast camera movement, low resolution, and poor focus, the pixel coordinates of the QR code corners become inaccurate, resulting in errors in the calculated position matrix. In short, observation noise causes simple projection relationships to be very inaccurate and the cumulative error is large. Therefore, it is necessary to determine the precise location of each ArUco when the camera observation data is inaccurate. This is an optimization problem to minimize the reprojection error:

Among them, Ψ(δ,γ ^t ,γ _i ·c _j ) represents the projection of the corner point from the three-dimensional space coordinates to the pixel coordinates, δ represents the camera intrinsic parameter matrix, γ ^t represents the camera extrinsic parameter, γ _i represents the transformation matrix from the marker coordinate system to the world coordinate system, and c _j represents the three-dimensional coordinates of the marker corner point. represents the pixel coordinates of the marked corner points, K is the total number of images in the closed-loop shooting path,

(3) At the same time, considering that the increase in the number of image stitching will lead to the accumulation of errors, in order to alleviate this problem, a closed-loop image acquisition route is set and the overall error is optimized, and the closed-loop constraint formula is introduced:

where K is the total number of images in the closed loop, _xk is the pose of the kth image, _zk is the pose transformation from the kth image to the k+1th image, is the operator between places.

In step 3, the robot uses a monocular vision system to capture ArUco marker images in different positions.

The specific construction of the robot end absolute position model in step 3 is:

(1) The measurement system consists of multiple ArUco markers in space, an industrial robot, and an industrial camera installed at the end of the industrial robot. The camera captures and estimates the posture of the ArUco marker through image recognition technology, and establishes the coordinate transformation relationship between the ArUco code image coordinate system and the camera coordinate system. Among them, the PnP algorithm is used to determine the 6-DOF posture of the camera relative to the calibration plate. In this algorithm, the 3D to 2D correspondence relationship of the four corner points indicated by the ArUco marker is described by the following formula:

(2) Using the hand-eye calibration model, the relative position relationship between the KUKA kr500 robot terminal coordinate system and the Daheng industrial camera coordinate system is calculated, thereby determining the absolute position model of the robot terminal. The hand-eye matrix obtained by calibration in this step of this embodiment is:

The error matrix of the measured pose is obtained in step 4, specifically:

(1) Establish a kinematic model based on the theoretical DH parameters of industrial robots and calculate the nominal position of the robot end in each posture;

(2) Using the absolute position model of the robot end to obtain the end measurement posture, and obtain the error matrix between the end nominal posture and the measured posture;

In step 5, a posture error model is established, and the error matrix is processed by Python programming to obtain the robot kinematic parameter error.

In order to verify the effectiveness and superiority of the present invention, the calibration method based on the ArUco map is compared with the traditional calibration method based on the checkerboard. The operation space of the industrial robot is further divided into 9 areas, and 10 verification points are randomly set in each area. By comparing the positioning accuracy of the two methods in different areas and analyzing the error distribution, the advantages of the present invention are further demonstrated.

The identified robot kinematic parameter errors are shown in the table:

Table 1 Kinematic parameter error results

The processed data are analyzed to evaluate the improvement effect of the method of the present invention on the absolute positioning accuracy of the robot. Special attention is paid to the error distribution in different areas and the comparison results with the traditional method, so as to fully verify the effectiveness and application value of the present invention.

In summary, the present invention successfully proposes and implements an efficient method for kinematic calibration of industrial robots in large-scale workspaces using ArUco maps and monocular vision systems. By innovatively combining the widespread deployment of ArUco markers, advanced image stitching technology, precise image processing and marker recognition, as well as advanced absolute position model construction and kinematic parameter calibration technology, the present invention significantly improves the positioning accuracy and operating efficiency of industrial robots in large-scale work areas. The system is not only easy to operate and cost-effective, but also has good adaptability and scalability, and can meet the needs of various high-precision manipulation tasks. Through the implementation of the present invention, it can provide strong technical support for various industrial applications, especially in the fields of aerospace, automobile manufacturing and precision machining, and has broad application prospects and significant economic and social value.

Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims (10)

1. A vision-based kinematic calibration method for an industrial robot in a large-size workspace, characterized in that the method comprises: ArUco markers are laid out in the workspace according to the industrial robot configuration; According to the set shooting path, the monocular vision system is used to capture ArUco tagged images, and the image stitching technology is used to generate a fully covered ArUco map from the ArUco tagged images; The robot uses a monocular vision system to capture ArUco marked images in different positions, identifies the end position of the robot according to the fully covered ArUco map, and constructs an absolute position model of the end position of the robot; According to the robot joint angle and the nominal value of kinematic parameters, the terminal nominal posture is calculated, and the terminal measurement posture is obtained using the robot terminal absolute position model, and the error matrix between the terminal nominal posture and the measurement posture is obtained; The posture error model is established, and the robot kinematic parameter error is obtained according to the error matrix. 2. The method for kinematic calibration of a large-size workspace based on vision according to claim 1 is characterized in that, according to the configuration of the industrial robot, the method for laying out ArUco markers in the workspace comprises: Establish the objective function F(O) of the layout plan: F(O)＝α·V(T)+β·U(D)+γ·P(R)-δ·C(S) Among them, O represents the layout of the mark, V(T) represents the visibility parameter of ArUco marker, and α represents the weight of V(T); U(D) represents the uniformity parameter of ArUco marker placement, and β represents the weight of U(D); P(R) represents the average positioning accuracy of the robot at all positions within its motion range, and γ represents the weight of P(R); C(S) represents the layout cost, and δ represents the weight of C(S); A genetic algorithm is used to optimize the layout scheme O of ArUco markers to maximize the objective function F(O). During the optimization process, the position, number and direction of the markers are continuously updated iteratively until a layout scheme that maximizes F(O) is found. 3. The vision-based kinematic calibration method for industrial robots in large-size workspaces according to claim 2 is characterized in that: Where D represents the set of label densities of each test point in the workspace, and σ ² (D) is the variance of D. 4. The vision-based kinematic calibration method for industrial robots in large-size workspaces according to claim 2 is characterized in that: Where, _vi represents the visibility of the ith ArUco marker, which depends on the angle and distance between the ArUco marker and the camera, _Ai represents the visible area of the ith ArUco marker in the camera’s field of view, _Atotal represents the total area of the camera’s field of view, and n represents the total number of ArUco markers. 5. The vision-based kinematic calibration method for industrial robots in large-size workspaces according to claim 2, characterized in that: Where m is the total number of positioning operations, _dj is the positioning error in the jth operation, and _dmax is the maximum allowed error distance. 6. The method for kinematic calibration of a large-size workspace based on vision according to claim 1 is characterized in that the method of capturing ArUco marker images using a monocular vision system according to a set shooting path and generating a fully covered ArUco map from the ArUco marker images using image stitching technology comprises: According to the set closed-loop shooting path, shoot ArUco mark images in sequence to ensure that the same ArUco mark exists in adjacent ArUco mark images; Determine the position of each ArUco marker, and use the L-M algorithm to minimize the reprojection error of the target's ArUco marker corner points to obtain the initial ArUco map; The objective function of the reprojection error is and Among them, Ψ(δ,γ ^t ,γ _i ·c _j ) represents the projection of the corner point from the three-dimensional space coordinates to the pixel coordinates, δ represents the camera intrinsic parameter matrix, γ ^t represents the camera extrinsic parameter, γ _i represents the transformation matrix from the marker coordinate system to the world coordinate system, and c _j represents the three-dimensional coordinates of the marker corner point. represents the pixel coordinates of the marked corner point, K is the total number of images in the closed-loop shooting path, _xk is the pose of the kth image, _zk is the pose transformation from the kth image to the k+1th image, and ⊕ is the operator between positions. 7. The vision-based kinematic calibration method for industrial robots in large-size workspaces according to claim 1 is characterized in that the method for constructing the absolute position model of the robot end is: Based on the fully covered ArUco map and the ArUco marker images captured by the robot using the monocular vision system at different positions, the coordinate transformation relationship between the ArUco image coordinate system and the camera coordinate system is established; The hand-eye calibration model is used to calculate the relative posture relationship between the robot end coordinate system and the camera coordinate system, thereby determining the absolute position model of the robot end. 8. The vision-based kinematic calibration method for an industrial robot in a large-size workspace according to claim 1 is characterized in that a posture error model is established, and the error matrix is processed by Python programming to obtain the robot kinematic parameter error. 9. A computer-readable storage device storing a computer program, wherein the computer program, when executed by a processor, implements a vision-based kinematic calibration method for an industrial robot in a large-size workspace as described in any one of claims 1 to 8. 10. A vision-based kinematic calibration device for an industrial robot in a large-size workspace, comprising a storage device, a processor, and a computer program stored in the storage device and executable on the processor, wherein the processor executes the computer program to implement the vision-based kinematic calibration method for an industrial robot in a large-size workspace as described in any one of claims 1 to 8.