CN115284292A - Mechanical arm hand-eye calibration method and device based on laser camera - Google Patents

Abstract

A method and device for hand-eye calibration of a robotic arm based on a laser camera, a calibration tool is connected to the end arm of the robotic arm, a laser camera is set on the base of the robotic arm, and the calibration tool is provided with a calibration plate that can change the posture, The laser camera is used to collect the intensity map and point cloud data of the calibration board, keep the robotic arm still, adjust the pose of the calibration board, and obtain the position information of the camera coordinate system point cloud and the point cloud position information of the world coordinate system under different poses of the calibration board. Simultaneously solve the pose transformation relationship between the laser camera and the end of the robotic arm, and complete the hand-eye calibration. The invention reduces multiple intermediate processes in the existing calibration process, does not require RGB images and the position and attitude information of the mechanical arm, reduces the introduction error of the mechanical arm and point cloud data, and improves the calibration accuracy, and is especially suitable for live working robots. The hand-eye calibration can be quickly completed, which is safe, fast and reliable.

Description

Method and device for hand-eye calibration of robotic arm based on laser camera

technical field

The invention belongs to the technical field of robots, relates to hand-eye calibration of robots, and relates to a method and device for hand-eye calibration of a mechanical arm based on a laser camera.

Background technique

The robot has a multi-joint multi-degree-of-freedom mechanical arm, and performs related operations by controlling the position of the end of the mechanical arm. In the prior art, the spatial position of the robot is generally judged by machine vision. In order to establish a relationship between the camera (ie, the eye of the robot) and the coordinate system at the end of the robot arm (ie, the hand of the robot), it is necessary to determine the position of the robot arm and the camera. The coordinate system is calibrated, and this calibration process is also called hand-eye calibration.

Usually, the hand-eye relationship of a robot is divided into two types: eye-in-hand and eye-to-hand, among which eye-in-hand means that the eye is on the hand, and the visual system of the robot moves with the end of the mechanical arm; and eye-to-hand The eye is next to the hand, the position of the robot's vision system and the robot base is relatively fixed, and it will not move in the world coordinate system. In the eye-in-hand relationship, the camera is set at the end of the robot arm, and the calibration plate is set at a fixed position next to the robot. The positional relationship between the robot base and the calibration plate remains unchanged, and the amount to be solved is the coordinate system of the camera and the end of the robot arm. pose relationship. Under the eye-to-hand relationship, the calibration board is set at the end of the robot arm, and the camera is set at a fixed position next to the robot. The pose relationship between the end of the robot arm and the calibration board is always the same, and the amount to be solved is the coordinate system of the camera and the robot base pose relationship between them. Regardless of the calibration method, when solving the pose relationship, it is necessary to let the robotic arm move, thereby changing the position of the camera or the calibration board, and then constructing the solution matrix of different coordinate system poses, which requires a larger venue. Facilitate the movement of the mechanical arm. At the same time, since the above two calibration methods rely on the movement of the manipulator to change the positional relationship between the camera and the calibration board to construct the pose solution matrix, during the movement of the manipulator, by obtaining the pose data of the end effector of the manipulator As the parameters of the pose solution matrix, the error of the robot arm will be introduced into the calibration solution due to factors such as the assembly error of the robot arm itself and the assembly error of the calibration board itself, which will affect the accuracy of the calibration.

The common hand-eye calibration of the robotic arm in the prior art is accomplished by detecting the checkerboard pattern with a camera, which requires the use of RGB images, which involves camera internal parameters and introduces camera internal reference errors. There are also plans to use point cloud cameras for hand-eye calibration. In the absence of RGB images, ICP algorithms are used to complete the hand-eye calibration of 3D point cloud cameras. However, the ICP method needs to use as many regular objects as possible for matching in a large space. At the same time, the mechanical arm also needs to move, and the error of the mechanical arm will also be introduced in the calibration.

The live working robot is a special robot for distribution network line operations. It mainly replaces manual high-voltage cable lapping and dismantling at high altitude to complete a series of high-risk operations. The working environment is outdoors and changes frequently. Most of the existing robotic arm hand-eye calibration methods The robot arm is required to complete the calibration by changing the pose in a relatively stable environment, and the live working robot needs to move frequently. On the one hand, it is easy to cause the movement error of the robot arm after a long time of use. On the other hand, different working environments may require updating the hand-eye calibration , the working environment of the live working robot is not suitable for on-site calibration by the existing calibration method. If it is calibrated on site, the surrounding cables may be affected by the pose change of the manipulator.

Contents of the invention

The problem to be solved by the present invention is: in the prior art, a large space is required to complete the calibration work when performing the hand-eye calibration of the mechanical arm. At the same time, due to the basic principle of the calibration method, the camera’s internal reference or the error of the mechanical arm itself will be introduced into the calibration solution, which will affect the calibration accuracy. . Especially for live working robots, the existing calibration methods require many conditions, which is not conducive to the rapid hand-eye calibration of live working robots.

The technical solution of the present invention is: a laser camera-based mechanical arm hand-eye calibration method, a calibration tool is connected to the end arm of the mechanical arm, a laser camera is set on the base of the mechanical arm, and a convertible position is provided on the calibration tool. The calibration plate of the posture, the laser camera is used to collect the intensity map and point cloud data of the calibration plate, including the following steps:

1) The point cloud data and intensity map are collected by the laser camera facing the calibration plate, and the point cloud position information P in the camera coordinate system and the real point cloud position information Q in the world coordinate system are obtained from the collected point cloud data;

2) For the obtained intensity map, extract the feature points of the calibration plate, select three points that are not collinear as the target point, and obtain the point cloud position information P of the corresponding target point;

3) Keep the mechanical arm still and adjust the pose of the calibration plate. The different poses include displacement or rotation in the horizontal, vertical, and depth directions. The adjustment includes at least one displacement in the depth direction, and repeat steps 1) and 2. ) to collect the intensity map and point cloud data of the calibration plate in different poses, and combine the obtained point cloud position information P of the target point with the real point cloud position information Q under the corresponding pose, and solve the problem to obtain the laser camera and the manipulator The pose conversion relationship at the end completes the hand-eye calibration.

Further, solve the pose transformation relationship between the laser camera and the end of the robotic arm as follows:

表示在位置m₁采集得到真实世界下的点云位置信息，

是真实世界下的点云深度数据，

表示在位置m₁的真实世界下的点云强度数据；Among them, the subscripts m ₁ and m ₂ represent different poses of the calibration board, and M represents the pose transformation relationship between the laser camera and the end of the robotic arm,

Indicates that the point cloud position information in the real world is collected at position m ₁ ,

is the point cloud depth data in the real world,

Represents the point cloud intensity data under the real world at position m ₁ ;

The pose conversion relationship M is obtained by simultaneously solving the point cloud data of the calibration board under two poses with different depths.

Further, when performing steps 1) and 2), collect the point cloud data of the calibration board at the same pose m for N times, and calculate the average point cloud information:

Among them, P _{m, n} represents the point cloud data collected at the nth time of pose m, P _m represents the average point cloud information at pose m, and the average point cloud information is used as the point cloud position of the camera coordinate system of the target point The information participates in the solution of the pose transformation relationship.

Further, the calibration work includes a bracket and a calibration plate, the bracket includes a connecting piece, an xyz three-way displacement platform, a rotating platform and a calibration plate clamping piece, the connecting piece is used to connect the calibration tooling to the mechanical arm, and the calibration plate clamps The parts are used to fix the calibration plate, and the xyz three-way displacement platform and the rotating platform are used to adjust the displacement or rotation of the calibration plate in the horizontal, vertical and depth directions.

Further, the mechanical arm is a mechanical arm of a live working robot.

The present invention also provides a hand-eye calibration device for a mechanical arm based on a laser camera, including a laser camera, a calibration tool and a calibration module. The calibration tool is arranged on the end arm of the robot, and the laser camera is arranged on the base of the robot. The calibration The tooling is provided with a calibration plate that can change the pose, and the calibration module is connected to the laser camera data. The calibration module includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, the above-mentioned In the calibration calculation step of the method, the calibration module receives the data collected by the laser camera, and outputs pose conversion relationship information for hand-eye calibration.

The method of the present invention does not require the movement of the mechanical arm in the whole process, and the calibration relationship can be solved by collecting a small amount of target position information of the calibration plate at different positions through the hand-eye calibration tooling that can change the pose and based on the laser point cloud data. The present invention directly utilizes point cloud data without using a color camera for calibration, which reduces the need for calibration environment, the style of the calibration board, the pose information of the manipulator, the number of fixed calibration positions in the real physical world, and the errors of the manipulator and laser camera. Information needs and reliance. The present invention places all required position and rotation information on the design of the calibration tool, thus avoiding the introduction of error information of the mechanical arm.

At the same time, the present invention also proposes a corresponding solution to the problem that point clouds at different positions in the same frame of data collected by the laser camera may have errors. Through the three-dimensional information of the same point in multiple frames, the average value is calculated in each dimension.

The present invention reduces multiple intermediate processes in the existing calibration process, does not require RGB images and mechanical arm pose information, reduces the introduction error of the mechanical arm and point cloud data, and improves the calibration accuracy. The invention is especially suitable for live working robots, without any action of the mechanical arm, and without considering the error of the mechanical arm itself, the hand-eye calibration can be quickly completed at the work site, which is safe, fast and reliable.

Description of drawings

Fig. 1 is a schematic flow chart of the method of the present invention.

Fig. 2 is a schematic diagram of the configuration of the laser camera and the calibration tool on the mechanical arm in the method of the present invention.

FIG. 3 is a side view of FIG. 2 .

Fig. 4 is a structural schematic diagram of the calibration tool in the method of the present invention.

Fig. 5 is a schematic diagram of two embodiments of the calibration plate in the method of the present invention.

Detailed ways

The present invention proposes a hand-eye calibration method based on a manipulator and a laser camera. The purpose is to avoid introducing errors of the manipulator as much as possible, and the calibration environment is not limited; at the same time, the accurate information of the real physical world can also be imported into the point cloud data. .

As shown in Figure 1 and Figures 2 and 3, the present invention connects a calibration tool to the end arm of the mechanical arm, a laser camera is arranged on the base of the mechanical arm, and the calibration tool is provided with a calibration plate that can change the pose, The laser camera is used to collect the intensity map and point cloud data of the calibration plate, including the following steps:

1) Collect point cloud data and intensity maps by facing the calibration plate with a laser camera. The laser sensor of the laser camera used in the present invention should be able to output a laser intensity map, and the laser intensity map can be converted into a simple grayscale image without the need for a color camera, which reduces the link that will introduce errors in the calibration process, that is, the impact of camera distortion on The impact of the calibration process. From the collected point cloud data, the point cloud position information P in the camera coordinate system and the real point cloud position information Q in the world coordinate system are obtained. The coordinate system of P and the real point cloud position information Q are different. The origin of the coordinate system of the point cloud position information P is at the optical center of the sensor; the coordinate point of the real point cloud position information Q is at the zero point of the tooling scale and the plane of the calibration plate the upper left corner of the . In addition, the accuracy of the point cloud position information P is different from that of the real point cloud position information Q. The point cloud data reflected by the laser on the plane is uneven; the point cloud data in the real world itself is a plane.

2) From the obtained intensity map, extract the feature points of the calibration plate, select three points that are not collinear as the target points, and obtain the point cloud position information P of the corresponding target points. The selection of the feature point is generally the center of the calibration plate, or the corner point. These points are easy to be the target points for subsequent processing due to the obvious color characteristics and size information.

3) Keep the mechanical arm still and adjust the pose of the calibration board. The different poses include displacement or rotation in the horizontal, vertical and depth directions. The horizontal, vertical and depth here refer to the plane facing the camera as Horizontal and vertical planes, and the plane perpendicular to this plane is the depth. The adjustment includes at least one displacement in the depth direction, repeat steps 1) and 2) to collect the intensity map and point cloud data of the calibration plate under different poses, and obtain the point cloud position information P of the target point and the corresponding pose. The real point cloud position information Q is combined, and the point cloud information collected in different poses is subtracted to solve the pose transformation relationship between the laser camera and the end of the robotic arm, and complete the hand-eye calibration. In the method of the present invention, the mechanical arm is only used as a connecting piece, and must not move during the calibration process, thereby avoiding the introduction of the mechanical arm tooling error and the mechanical arm control error in the calibration process.

In order to further reduce the influence of errors in the point cloud data, the present invention performs subtraction between two different frames, and at the same time uses the data on the depth and the data on the point cloud intensity map as accurate information, and finally obtains a method for solving the transformation matrix. Using the point cloud information to solve the pose transformation relationship between the laser camera and the end of the robotic arm is as follows:

表示在位置m₁采集得到真实世界下的点云位置信息，

是真实世界下的点云深度数据，

表示在位置m₁的真实世界下的点云强度数据；由具有不同深度的两个位姿下的标定板点云数据联立求解，得到位姿转换关系M。Among them, the subscripts m ₁ and m ₂ represent different poses of the calibration board, and M represents the pose transformation relationship between the laser camera and the end of the robotic arm,

Indicates that the point cloud position information in the real world is collected at position m ₁ ,

is the point cloud depth data in the real world,

Represents the point cloud intensity data in the real world at position m ₁ ; the pose transformation relationship M is obtained by simultaneously solving the point cloud data of the calibration board under two poses with different depths.

According to the above formula, M is obtained by solving the point cloud intensity map and point cloud depth data.

R refers to the rotation matrix, which is obtained by multiplying the rotation matrices of the three coordinate axes. t refers to the translation vector, which respectively refers to the translation distance in three directions.

Compared with the prior art, the present invention does not need to obtain the pose data of the end effector of the manipulator, and puts all the position and rotation information that needs to be used in the design of the calibration tool, so that the error information of the manipulator can not be introduced . The prior art needs to adjust the six-degree-of-freedom pose of the robotic arm, but the present invention only needs to translate at least once in the "depth". The existing eye-to-hand calibration needs to collect the data of the same point in different poses as the calibration data. The present invention only needs to select three points that are not collinear in each frame, that is, the target point in the calibration process.

Further, when performing steps 1) and 2), collect the point cloud data of the calibration board at the same position m for N times, and calculate the average point cloud information:

Among them, P _{m, n} represents the point cloud data obtained at the nth acquisition of pose m, P _m represents the average point cloud information at pose m, and the average point cloud information is used as the point cloud position information of the target point to participate in the position The solution of attitude transformation relationship. The present invention uses laser point cloud data to perform hand-eye calibration. Compared with the existing calibration method of collecting RGB images by means of a color camera, it does not need to acquire internal parameters of the camera, and simplifies the calibration process. But this may also introduce errors in the point cloud data, so the present invention uses the method of calculating the average value to reduce the error impact of the point cloud data. The information of each point in the point cloud is three-dimensional, so the multi-frames collected by the laser camera In the three-dimensional information corresponding to the same point, the average value is calculated on each dimension.

An embodiment as shown in Figure 4, the calibration work of the present invention comprises a support and a calibration plate, and the support includes a connector, an xyz three-way displacement platform, a rotating platform and a calibration plate clamp, and the connector is used to connect the calibration tooling to the On the mechanical arm, the calibration plate holder is used to fix the calibration plate, and the xyz three-way displacement platform and the rotating platform are used to adjust the displacement or rotation of the calibration plate in the horizontal, vertical and depth directions. Under this structure, the technical requirements of the calibration tooling are that the number of clamping parts of the calibration plate is as small as possible, the overall stability is high, and the error of displacement and rotation adjustment is small.

As shown in FIG. 5 , the present invention has no special requirements on the form of the calibration plate, and a commonly used checkerboard or round hole calibration plate can be used.

The present invention also provides a hand-eye calibration device for a mechanical arm based on a laser camera, including a laser camera, a calibration tool and a calibration module. The calibration tool is arranged on the end arm of the robot, and the laser camera is arranged on the base of the robot. The calibration The tooling is provided with a calibration plate that can change the pose, and the calibration module is connected to the laser camera data. The calibration module includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, the above-mentioned In the calibration calculation step of the method, the calibration module receives the data collected by the laser camera, and outputs pose conversion relationship information for hand-eye calibration.

The whole calibration process of the present invention does not require the error information of the manipulator, the error information of the sensor, the pose information of the manipulator or the internal reference of the camera. This is the key to the difference between the present invention and the prior art. The present invention controls the error in the designed calibration tooling , the calibration work can be completed only by using tooling with high precision requirements in the depth direction. There is no special requirement for the calibration environment and site. For the displacement or rotation accuracy in the depth direction, it depends on the accuracy of hand-eye calibration and the accuracy that the tooling can design. For example, in the current robot application, if the accuracy of the tooling accuracy in depth reaches 0.1mm, then the accuracy of our hand-eye calibration can reach the millimeter level. The present invention is biased towards the eye-to-hand calibration method, and the camera does not move, but the existing eye-to-hand calibration technology reads the pose information of the robotic arm itself to bring out the calibration board information, which will naturally Introduce arm error, which is mostly ignored in the prior art. The method of the present invention isolates the influence of the assembly error of the mechanical arm itself and the assembly error of the tooling itself. Errors caused by calibrated tooling or calibration plate, sensor distortion, etc. For example, the calibration plate has an accuracy of 0.1mm, and the final calibration result still produces an error of 3mm, which is caused by the accuracy of the point cloud data itself. At the same time, the present invention does not need to adjust the movement of the manipulator and change different poses, and avoids the influence of the control error of the manipulator on the hand-eye calibration. Especially for special robots such as live working robots, the calibration method that does not require the movement of the manipulator is especially suitable for it. Calibration is carried out at the outdoor high school operation site. At the same time, since the mechanical arm does not move, the present invention will introduce another problem, where does the accurate position information in the real world come from? For this, the present invention adopts the designed three-direction + rotation calibration tool and uses point cloud data to Two frames of point cloud data are subtracted to realize simultaneous point cloud data and real-world location information. At the same time, the tooling designed by the present invention cannot pay attention to all three dimensions, and the requirements for the calibration tooling are too high, and the industrial value will be lost. For this, the present invention chooses the depth dimension information of the point cloud and cooperates with the laser intensity map to calibrate, reducing additional Two-dimensional data import.

The following two tables have shown the calibration embodiment of the present invention, and the operation process is as follows:

1. Fix the calibration tooling at the end of the robot arm.

2. Set any position of the calibration board, and adjust the height so that the camera can capture the calibration board; select any three target points and record their corresponding point cloud position information.

3. Adjust the pose of the calibration board on the tooling, including at least the change in depth; select the three corresponding target points in step 2, and record the corresponding point cloud position information.

4. Calculate the transformation matrix according to the solution formula of the pose transformation relationship.

5. Remove the calibration tooling.

6. Install the execution end at the end of the robot arm, such as a needle-shaped tool, and input the shape parameters of the tool into the transformation matrix.

7. Use the camera to shoot any target, and manually select the location point.

8. Manipulate the robotic arm to touch the point selected in step 7 with the end of the needle tool.

9. Measure the distance between the touch point and the position point, and output the verification deviation.

Table 1 and Table 2 are the direction results output by the above embodiments. Table 1 shows the measurement results of the accuracy of the robot arm touching a fixed object under different pitch angles of the camera to the touch point. Touching the accuracy measurement results of a fixed object, it can be seen that the calibration method of the present invention can achieve good calibration accuracy when the mechanical arm does not move.

Table 1 (unit: mm)

Table 2 (unit: mm)

Target pitch angle (unit: degree) shooting distance X-direction deviation Y direction deviation Z direction deviation 0 700 <1 1.3 -0.3 0 700 1.3 1.4 -0.4 0 700 <1 1.8 0.2 0 700 1 1 0.2 0 800 4.2 1.2 -1.5 0 800 4.6 1.8 -3 0 800 4.2 1.3 -2.1 0 800 3.1 1.5 -3.2 0 900 2.2 -1.7 1 0 900 1.6 -3.1 -1.8 0 900 2.7 -3.3 -1.6 0 900 1.3 -1.8 -2.8 0 1000 7.1 -0.2 1.2 0 1000 9 -1.3 0.8 0 1000 5.3 -2.3 1.6 0 1000 8.2 -1.7 1.2 0 1100 4.3 -3.8 0 0 1100 6 -3.1 1 0 1100 5.6 -2.2 1.9 0 1100 5.9 -4.6 0.4 0 1200 7.5 -2.7 0 0 1200 5.3 -2.9 0.7 0 1200 6.2 -3.5 0.7 0 1200 5.8 -3.3 -0.3

Claims (6)

1. The hand-eye calibration method of the mechanical arm based on the laser camera is characterized in that a calibration tool is connected on the end arm of the mechanical arm, a laser camera is set on the base of the mechanical arm, and the calibration tool is provided with a variable pose Calibration plate, the laser camera is used to collect the intensity map and point cloud data of the calibration plate, including the following steps: 1) The point cloud data and intensity map are collected by the laser camera facing the calibration plate, and the point cloud position information P in the camera coordinate system and the real point cloud position information Q in the world coordinate system are obtained from the collected point cloud data; 2) For the obtained intensity map, extract the feature points of the calibration plate, select three points that are not collinear as the target point, and obtain the point cloud position information P of the corresponding target point; 3) Keep the mechanical arm still and adjust the pose of the calibration plate. The different poses include displacement or rotation in the horizontal, vertical, and depth directions. The adjustment includes at least one displacement in the depth direction, and repeat steps 1) and 2. ) to collect the intensity map and point cloud data of the calibration plate in different poses, and combine the obtained point cloud position information P of the target point with the real point cloud position information Q under the corresponding pose, and solve the problem to obtain the laser camera and the manipulator The pose conversion relationship at the end completes the hand-eye calibration. 2. The hand-eye calibration method of the mechanical arm based on the laser camera according to claim 1 is characterized in that solving the pose conversion relationship between the laser camera and the end of the mechanical arm is specifically:

Among them, the subscripts m ₁ and m ₂ represent different poses of the calibration board, and M represents the pose transformation relationship between the laser camera and the end of the robotic arm,

Indicates that the point cloud position information in the real world is collected at position m ₁ ,

is the point cloud depth data in the real world,

Represents the point cloud intensity data under the real world at position m ₁ ; The pose conversion relationship M is obtained by simultaneously solving the point cloud data of the calibration board under two poses with different depths. 3. The method for calibrating hands and eyes of a mechanical arm based on a laser camera according to claim 1, wherein when carrying out steps 1) and 2), the point cloud data of the same pose m of the calibration plate is collected for N times, and the average is calculated. Point cloud information:

Among them, P _{m, n} represents the point cloud data collected at the nth time of pose m, P _m represents the average point cloud information at pose m, and the average point cloud information is used as the point cloud position of the camera coordinate system of the target point The information participates in the solution of the pose transformation relationship. 4. The hand-eye calibration method of a robotic arm based on a laser camera according to claim 1, wherein the calibration work includes a bracket and a calibration plate, and the bracket includes a connector, an xyz three-way displacement platform, a rotating platform, and a calibration plate clamped The connecting piece is used to connect the calibration tool to the mechanical arm, the calibration plate clamp is used to fix the calibration plate, the xyz three-way displacement platform and the rotating platform are used to displace the calibration plate in the horizontal, vertical and depth directions Adjust or turn to adjust. 5. The laser camera-based hand-eye calibration method for a robotic arm according to any one of claims 1-4, wherein the robotic arm is a robotic arm of a live working robot. 6. The hand-eye calibration device of the mechanical arm based on the laser camera is characterized in that it comprises a laser camera, a calibration tool and a calibration module, the calibration tool is arranged on the end arm of the robot arm, the laser camera is arranged on the base of the robot arm, and the calibration tool There is a calibration board that can change the pose, and the calibration module is connected to the laser camera data. The calibration module includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, the claim is realized. In the calibration calculation step of the method described in any one of 1-5, the calibration module receives the data collected by the laser camera, and outputs pose conversion relationship information for hand-eye calibration.

Images