CN105157725B - A kind of hand and eye calibrating method of two-dimensional laser visual sensor and robot - Google Patents

Abstract

The invention discloses a hand-eye calibration method for a two-dimensional laser vision sensor and a robot, comprising the following steps: step A, establishing a mathematical model of the calibration algorithm; step B, formulating the implementation steps of hand-eye calibration; wherein establishing the mathematical model of the calibration algorithm The steps realize the derivation of the mathematical model of the hand-eye calibration algorithm; the steps of formulating the implementation operation of the hand-eye calibration solve the specific implementation operation process in the hand-eye calibration process of the two-dimensional laser sensor and the robot. The method has the advantages of simplicity, practicality, flexibility, good precision and the like.

Description

A hand-eye calibration method for two-dimensional laser vision sensor and robot

technical field

The present invention relates to a hand-eye calibration technology for laser sensors and robots, in particular to a two-dimensional laser vision sensor and a hand-eye calibration method for robots. And the two-dimensional laser sensor is the measuring device. The attitude change of the robot drives the laser sensor to measure several fixed points in space, and records the attitude of the robot and the coordinate values of the space point in the robot base coordinate system and the laser sensor measurement coordinate system respectively. The algorithm solves the transformation relationship between the sensor measurement coordinate system and the robot end flange coordinate system, that is, the hand-eye calibration of the two-dimensional laser sensor.

Background technique

Because the vision system has good detection performance and positioning performance, the development of robot vision system has become a hot spot and focus in the field of robot research. Due to the abundant information obtained, high sensitivity, high precision, and no contact with the workpiece, the visual sensing method has attracted more and more attention.

At present, the images collected by visual sensors include images based on natural light, artificial ordinary light, and structured light images with lasers as active light sources. In some special industrial environments, such as strong arc light, dust, smoke and other adverse interference factors in the welding site, the performance of traditional CCD cameras has been seriously disturbed, and traditional CCD cameras cannot perform well in this environment. To complete the task, practicality is poor. In contrast, the two-dimensional laser sensor is based on the principle of triangulation, and uses a linear laser beam to measure the cross-sectional profile of an object. An optical filter with the same wavelength as the laser is used to filter out all stray light including arc light. The sensor The internal integrated optical receiving device and CMOS planar detector only receive and form the image of the laser stripes. The advantage of this two-dimensional laser sensor is that it does not use any moving parts, is durable, and is not disturbed by arc light, smoke, splashes, etc.

As an active light source, laser has the advantages of high energy, high brightness, good monochromaticity, and strong anti-interference ability. Therefore, two-dimensional laser vision sensors have great development prospects. The CCD camera is an area array vision sensor, and the two-dimensional laser sensor is a line array vision sensor. As one of the core technologies in the field of inspection and artificial intelligence, machine vision will undoubtedly greatly improve the flexibility and efficiency of robot operations. Among them, the mapping relationship between the visual coordinate system and the robot end joint coordinate system must be known through hand-eye calibration. The calibration accuracy largely determines the robot’s operating accuracy. Therefore, the vision system is calibrated, and the calibrated vision It becomes a technical problem to be solved that the system can obtain higher precision.

Contents of the invention

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and provide a two-dimensional laser vision sensor and a hand-eye calibration method for a robot. The hand-eye calibration method includes establishing a mathematical model of a calibration algorithm, formulating the implementation steps of hand-eye calibration, and having Simple, practical, flexible, good precision and so on.

The object of the present invention is achieved through the following technical solutions: a two-dimensional laser vision sensor and a hand-eye calibration method for a robot, the hand-eye calibration method adopts a two-dimensional laser sensor and a robot that are integrated with an optical receiving device and a CMOS plane detector (including the robot controller and the teaching box), in which the two-dimensional laser sensor is fixedly installed on the end flange of the robot through a bracket to form an Eye-in-hand hand-eye system; the base coordinate system of the robot is {Base}, the end of the sixth joint of the robot The flange coordinate system is {End}, the two-dimensional laser sensor measurement coordinate system is {M}, the purpose of hand-eye calibration is to solve the transformation matrix of the coordinate system {M} relative to the coordinate system {End}

When the linear laser beam emitted by the two-dimensional laser sensor used in this method is projected onto the surface of the measured object, the laser beam will form an image consistent with the surface contour of the measured object. P laser sampling points, and then the sensor returns the Z _M axis and X _M axis coordinate values of these P sampling points relative to the sensor measurement coordinate system {M};

The method also uses a computer to obtain the measurement data of the two-dimensional laser sensor, complete the entry of the calibration data, and execute the calibration algorithm operation;

This method also requires some other components, such as calibration plates.

The hand-eye calibration method of described two-dimensional laser vision sensor and robot comprises the following steps:

Step A, establishing a mathematical model of the calibration algorithm;

Step B, formulate the implementation steps of hand-eye calibration;

Described step A comprises the following steps:

A1) Obtain the coordinates of a certain point P ₁ in the robot base coordinate system {Base} as ^B P ₁ , ^B P ₁ = [ ¹ x ₁ , ¹ y ₁ , ¹ z ₁ ,1] ^T , the acquired point P ₁ is at The coordinates in the measurement coordinate system {M} of the two-dimensional laser sensor are ^M P ₁ , ^M P ₁ =[ ^M x ₁ , ^M y ₁ , ^M z ₁ ,1] ^T . Note the hand-eye matrix, that is, the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End} is Note that the transformation relationship of the coordinate system {End} relative to the coordinate system {Base} is

A2) According to the transformation relationship between the coordinate systems, it can be known that Left-multiply both sides of the equation by the matrix Inverse of , we get:

In the formula, is the inverse of the transformation matrix of the coordinate system {End} relative to the coordinate system {Base}, ^B P ₁ is the coordinate of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

remember get:

In the formula, T is a column matrix, ^B P ₁ is the coordinates of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

Expand formula (2) to get:

In the formula, is a column matrix, is the specific form of the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, is a column matrix, representing the coordinates of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

Further expand the formula (3) to get:

According to the principle and characteristics of the two-dimensional laser sensor, it can be known that the y-axis coordinate value ^M y ₁ of the spatial point P ₁ in the measurement coordinate system {M} of the two-dimensional laser sensor is always 0, so the formula (4) is obtained.

See formula (3) for the meaning of the variables in the formula;

A3) Similar to a certain point P ₁ in space, similarly, for certain points P ₂ and P ₃ in space, there are:

See formula (4) for meanings of symbols in formula (5) and formula (6);

From formula (4), formula (5), formula (6) can get:

Refer to formula (3) - formula (6) for the meaning of the symbols in the formula;

Formula (7) can be written in the following form:

Further transformed into:

Refer to formula (7) for meanings of symbols in formula (8) and formula (9);

So far, r ₁₁ , r ₁₃ , Δx have been solved;

Similarly, from formula (4), formula (5), formula (6) can get:

In the formula, for inverse of.

So far, r ₁₁ , r ₁₃ , Δx and r ₂₁ , r ₂₃ , Δy and r ₃₁ , r ₃₃ , Δz have been solved;

A4) To completely determine the hand-eye relationship matrix Also need to solve r ₁₂ , r ₂₂ , r ₃₂ , according to the nature of the attitude matrix:

and the vector are unit vectors and are orthogonal to each other.

In the formula, the symbols are the transformation matrix representations of the coordinate system {M} relative to the robot end flange coordinate system {End}.

F:

r ₁₂ , r ₂₂ , r ₃₂ can be solved from equation (13), and the obtained vector Perform unitization processing to obtain the normalized Vector, so far solve the hand-eye relationship matrix

A5) From the coordinate pose transformation relation:

In the formula, ¹ P' is based on the calibration result The deduced theoretical coordinate value of a point P in the coordinate system {Base}, ^MP is the coordinate of the point in the coordinate system {M}, comparing ^BP ' with ^BP can test the hand-eye calibration Accuracy, the results of hand-eye calibration using only spatial points P ₁ , P ₂ , and P ₃ may have large errors. In order to improve the accuracy and error tolerance of the calibration algorithm, N (N>3) spatial points are selected. Choose 3 points from it, a total of The combination with the smallest error is selected as the calibration result.

In the step A2), according to the transformation relationship between the coordinate systems, it is obtained Left-multiply both sides of the equation by the matrix Inverse of , we get:

remember get:

Expand formula (2) to get:

Further expand the formula (3) to get:

According to the principle and characteristics of the two-dimensional laser sensor, it can be known that the y-axis coordinate value ^M y ₁ of the spatial point P ₁ in the measurement coordinate system {M} of the two-dimensional laser sensor is always 0, so the formula (4) is obtained.

In the formula, is a column matrix, is the specific form of the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, is a column matrix, representing the coordinates of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

In the step A3), similar to a certain point P ₁ in space, other certain points P ₂ and P ₃ in space are taken, similarly:

From formula (4), formula (5) and formula (6) can get:

Write formula (7) in the following form:

Further transformed into:

Refer to formula (7) for meanings of symbols in formula (8) and formula (9);

So far, r ₁₁ , r ₁₃ , Δx have been solved;

Similarly, from formula (4), formula (5), formula (6) can get:

In the formula, for inverse of

So far, r ₁₁ , r ₁₃ , Δx, r ₂₁ , r ₂₃ , Δy, r ₃₁ , r ₃₃ and Δz have been solved.

Described step B comprises the following steps:

B1) Place the calibration board at a suitable position in the space, operate the robot so that the laser line emitted by the two-dimensional laser sensor installed on its end flange is projected on a certain point on the calibration board, which is marked as point P, according to the calibration The geometric relationship between the plate and the laser light sensor reads the coordinate value ^M P of point P in the laser sensor measurement coordinate system {M} at this time; keep the robot still and read and record the tool coordinate system at the end of the robot at this time The pose of {End} relative to the robot base coordinate system {Base}

B2) Operate the robot so that the TCP of another established tool coordinate system reaches the point P, and read the coordinate ^B P of point P in the robot base coordinate system {Base}. Will get ^MP , ^B P as a set of calibration data;

B3) Steps B1), B2) are repeated to obtain N groups of different calibration data;

B4) Input the N sets of calibration data obtained in step B3) into the calibration program, and obtain the hand-eye calibration result through computer calculation

Compared with the prior art, the present invention has the following advantages and effects:

1. The present invention has strong practicability, flexible and convenient use, and high accuracy of hand-eye calibration.

2. The hand-eye system of the present invention, which is suitable for the two-dimensional laser vision sensor and the robot, makes good use of the laser vision sensor to replace the traditional CCD vision, and meets the application requirements of the robot in special occasions.

Description of drawings

Fig. 1a is a schematic diagram of obtaining the coordinates of the spatial calibration point in the sensor measurement coordinate system and the current posture of the robot in the present invention.

Figure 1b is a partial enlarged view of the laser sensor.

Fig. 2 is a schematic diagram of coordinates of acquired spatial calibration points in the robot base coordinate system in the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

Example

A two-dimensional laser sensor is installed on the end flange of the robot (including robot controller and teaching box) through the mounting bracket. The sensor communicates with the computer, and the computer receives the measurement data returned by the sensor.

When the laser beam emitted by the two-dimensional laser sensor is projected onto the surface of the measured object, the laser beam will form an image consistent with the surface contour of the measured object. There are a series of continuous and uniformly distributed P laser sampling points on the laser beam. Then the sensor returns the Z-axis and X-axis coordinate values of the P sampling points in the laser beam relative to the sensor measurement coordinate system.

Combined with the calibration board, laser sensors and robots are used to obtain the data required for hand-eye calibration. The method also uses a computer to obtain the measurement data of the two-dimensional laser sensor, completes the entry of the calibration data, executes the calibration algorithm, and uses these data to calculate and solve the hand-eye relationship.

In this embodiment, the attachment of a calibration plate is also used.

As shown in Fig. 1a and Fig. 1b, described step A (establishing the mathematical model of calibration algorithm) comprises the following steps:

A1) Obtain the coordinates of a point P ₁ in the robot base coordinate system {Base}1 as ^B P ₁ , ^B P ₁ = [ ¹ x ₁ , ¹ y ₁ , ¹ z ₁ ,1] ^T , and obtain point P ₁ The coordinates in the measurement coordinate system {M}2 of the two-dimensional laser sensor are ^M P ₁ , ^M P ₁ =[ ^M x ₁ , ^M y ₁ , ^M z ₁ ,1] ^T . Note the hand-eye matrix, that is, the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}3 is Note that the transformation relationship of the coordinate system {End} relative to the coordinate system {Base} is

In the formula, is the inverse of the transformation matrix of the coordinate system {End} relative to the coordinate system {Base}, ^B P ₁ is the coordinate of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

in:

All twelve variables in are unknown; Also a 4x4 matrix.

Its value is known and can be read directly from the robot teaching box 4 .

A2) According to the transformation relationship between the coordinate systems, it can be known that Left-multiply both sides of the equation by the matrix Inverse of , we get:

In the formula, is the inverse of the transformation matrix of the coordinate system {End} relative to the coordinate system {Base}, ^B P ₁ is the coordinate of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

remember get:

In the formula, T is a column matrix, ^B P ₁ is the coordinates of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

Expand formula (2) to get:

In the formula, is a column matrix, is the specific form of the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, is a column matrix, representing the coordinates of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}.

Further expand the formula (3) to get:

According to the principle and characteristics of the two-dimensional laser sensor, it can be known that the y-axis coordinate value ^M y ₁ of the spatial point P ₁ in the measurement coordinate system {M} of the two-dimensional laser sensor is always 0, so the formula (4) is obtained.

See formula (3) for the meaning of the variables in the formula;

A3) Similar to a certain point P ₁ in space, similarly, for certain points P ₂ and P ₃ in space, there are:

See formula (4) for meanings of symbols in formulas (5) and (6);

From formula (4), formula (5), formula (6) can get:

Refer to formula (3) - formula (6) for the meaning of the symbols in the formula;

Formula (7) can be written in the following form:

Further multiply both sides of the equation by the inverse of the matrix to get:

Refer to formula (7) for meanings of symbols in formula (8) and formula (9);

So far, r ₁₁ , r ₁₃ , Δx have been solved;

Similarly, from formula (4), formula (5), formula (6) can get:

In the formula, for inverse of.

So far, r ₁₁ , r ₁₃ , Δx and r ₂₁ , r ₂₃ , Δy and r ₃₁ , r ₃₃ , Δz have been solved;

A4) To completely determine the hand-eye relationship matrix It is also necessary to solve r ₁₂ , r ₂₂ , and r ₃₂ , according to the properties of the attitude matrix:

vector are unit vectors and are orthogonal to each other.

In the formula, the symbols are the transformation matrix representations of the coordinate system {M} relative to the robot end flange coordinate system {End}.

F:

r ₁₂ , r ₂₂ , r ₃₂ can be solved from equation (13), and the obtained vector according to and Perform unitization processing to obtain unitized Vector, so far solve the hand-eye relationship matrix

A5) From the coordinate pose transformation relation:

In the formula, ^BP ' is the theoretical coordinate value of a certain point P in the coordinate system {Base} derived from the calibration results, ^MP is the coordinate of the point in the coordinate system {M}, and ^BP ' and The accuracy of hand-eye calibration can be tested by comparing with ^B P . The result of hand-eye calibration using only spatial points P ₁ , P ₂ , and P ₃ may have large errors. In order to improve the accuracy and error tolerance of the calibration algorithm, select 5 spatial points, select 3 points from them, a total of The combination with the smallest error is selected as the calibration result.

Described step B (making the implementation operation step of hand-eye calibration) comprises the following steps:

B1) As shown in Figure 1, the calibration plate 5 is placed at a suitable position in the space, and the robot is operated through the robot teaching box 4 so that the laser line emitted by the two-dimensional laser sensor 2 installed on its end flange 3 is projected At a point P on the calibration plate 5, the calibration plate 5 is within the range of the sensor. As can be seen from the geometric relationship in Figure 1, find out the minimum value Z of the Z _M axis in the current sampling data returned by the sensor 2. _min is the Z-axis coordinate of point P in the sensor measurement coordinate system {M}2, and the X _-M axis coordinate value corresponding to Z _min is the X _-M axis coordinate of P point in the coordinate system {M}, so that Read the coordinate value ^M P of point P in the laser sensor measurement coordinate system {M}2; keep the robot still, read it from the robot teaching box 4 and write it down, indicating that at this time the robot end flange tool coordinate system {End }3 relative to the robot base coordinate system {Base}1 pose Euler angles α, β, γ and offsets Δf _x , Δf _y , Δf _z . Can be obtained by the following formula:

In the formula, is the pose matrix of the tool coordinate system {End} relative to the robot base coordinate system {Base};

Among them: sα and cα represent sinα and cosα respectively, and the rest of the symbols can be deduced by analogy;

B2) As shown in Figure 2, operate the robot to make another established TCP of the tool coordinate system {Tool}6 reach the point P, and read the coordinate ^B of point P in the robot base coordinate system {Base}1 p. Will get ^MP , ^B P as a set of calibration data;

B3) Steps B1), B2) are repeated to obtain N groups of different calibration data;

B4) Input the N sets of calibration data obtained in step B3) into the calibration program, and obtain the hand-eye calibration result through computer 7 according to the mathematical model of the calibration algorithm

The above-mentioned embodiment is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited by the above-mentioned embodiment, and any other changes, modifications, substitutions, combinations, Simplifications should be equivalent replacement methods, and all are included in the protection scope of the present invention.

Claims (1)

1. A hand-eye calibration method of a two-dimensional laser vision sensor and robot, characterized in that, comprising the following steps: Step A, establishing a mathematical model of the calibration algorithm; Step B, formulate the implementation steps of hand-eye calibration; Described step B comprises the following steps: B1) Place the calibration board at a suitable position in space, operate the robot so that the laser line emitted by the two-dimensional laser sensor installed on its end flange is projected on the calibration board at point P, according to the distance between the calibration board and the laser light sensor Read the coordinate value ^M P of point P in the laser sensor measurement coordinate system {M} at this time; keep the robot still to read and write down the relative distance between the robot end flange tool coordinate system {End} and the robot The pose of the base coordinate system {Base} B2) Operate the robot to make another established TCP of the tool coordinate system arrive at the P point, read the coordinate ^B P of the P point in the robot base coordinate system {Base}, and obtain ^{the MP} , ^B P as a set of calibration data; B3) Steps B1), B2) are repeated until different calibration data of N groups of settings are obtained; B4) Input the N sets of calibration data obtained in step B3) into the calibration program, and obtain the hand-eye calibration result through computer calculation Described step A comprises the following steps: A1) Obtain the coordinate ^B P ₁ of a certain point P ₁ in the robot base coordinate system {Base}, ^B P ₁ = [ ¹ x ₁ , ¹ y ₁ , ¹ z ₁ ,1] ^T , and obtain the point P ₁ in two The coordinates in the three-dimensional laser sensor measurement coordinate system {M} are ^M P ₁ , ^M P ₁ = [ ^M x ₁ , ^M y ₁ , ^M z ₁ ,1] ^T , record the hand-eye matrix, that is, the coordinate system {M} is relative to The transformation matrix of the robot end flange coordinate system {End} is Note that the transformation relationship of the coordinate system {End} relative to the coordinate system {Base} is A2) According to the transformation relationship between the coordinate systems, it is obtained: Left-multiply both sides of the equation by the matrix Inverse of , we get: In the formula, is the inverse of the transformation matrix of the coordinate system {End} relative to the coordinate system {Base}, ^B P ₁ is the coordinate of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}; remember get: In the formula, T is the column matrix, ^B P ₁ is the coordinate of a certain point P ₁ in the robot base coordinate system {Base}, is the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, ^M P ₁ is the coordinate of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}; Expand formula (2) to get: In the formula, is a column matrix, is the specific form of the transformation matrix of the coordinate system {M} relative to the robot end flange coordinate system {End}, is a column matrix, representing the coordinates of point P ₁ in the two-dimensional laser sensor measurement coordinate system {M}; Further expand the formula (3) to get: According to the principle and characteristics of the two-dimensional laser sensor, it can be known that the y-axis coordinate value ^M y ₁ of the spatial point P ₁ in the two-dimensional laser sensor measurement coordinate system {M} is always 0, so the formula (4) is obtained; A3) For some points P ₂ and P ₃ in space, there are: From formula (4), formula (5) and formula (6): Write formula (7) in the following form: The formula (8) is further transformed into: So far, r ₁₁ , r ₁₃ and Δx have been solved; From formula (4), formula (5) and formula (6): So far, r ₁₁ , r ₁₃ , Δx, r ₂₁ , r ₂₃ , Δy, r ₃₁ , r ₃₃ and Δz have been solved; In the formula, for inverse of A4) In order to determine the hand-eye relationship matrix Solve for r ₁₂ , r ₂₂ and r ₃₂ according to the properties of the attitude matrix: vector and are unit vectors and are orthogonal to each other; In the formula, and Both are the transformation matrix representations of the coordinate system {M} relative to the robot end flange coordinate system {End}, so we get: r ₁₂ , r ₂₂ , and r ₃₂ are obtained by formula (13), and the obtained vector and Perform unitization processing to obtain the normalized and Vector, so far, solve the hand-eye relationship matrix A5) Obtained from the coordinate pose transformation relation: Among them, ^BP ' is the theoretical coordinate value of a point P in the coordinate system {Base} derived from the calibration results, ^MP is the coordinate of the point in the coordinate system {M}, and ^BP ' and The accuracy of hand-eye calibration can be tested by comparing with ^B P . The result of hand-eye calibration using only spatial points P ₁ , P ₂ , and P ₃ may have large errors. In order to improve the accuracy and error tolerance of the calibration algorithm, select N space points, N>3, select 3 points from them, a total of combination, select The combination with the smallest error among the combinations is used as the calibration result.