JPH01193902A - Coordinate system calibration method for robot with hand vision - Google Patents

Abstract

PURPOSE:To precisely and simply attain the calibration of a coordinate system by using a coordinate system calibration board where plural recognition patterns which can be identified are drawn on an outer periphery. CONSTITUTION:The coordinate system calibration board 4 where plural recognition patterns which can be identified on the outer periphery is placed on a prescribed work area where calibration is to be executed, and they are image- picked up by an image-pickup device 3. An image processor 10 calculates the positions of the recognition patterns in a visual coordinate system 12. Whenever a robot controller 9 gives a prescribed move quantity and moves the terminal part of the robot 1, the recognition pattern is image-picked up, and the same recognition patterns are visually recognized, whereby the positions are calculated. Thus, both coordinate systems can be made to correspond from the move quantity in the robot coordinate system 11 given to the terminal part of the robot 1 and the relative move quantity of the recognition pattern in the visual coordinate system 12, and a calibration work can simple and precisely be executed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a method of calibrating a visual coordinate system and a robot coordinate system in a robot with hand vision, which is equipped with an imaging device for vision at the hand portion of the robot.

Here, the visual coordinate system refers to a coordinate system that expresses the position of an object within the field of view of an imaging device, and the robot coordinate system refers to a coordinate system used to control the operation of a robot.

[Conventional technology]

In robots with hand vision, the scale and direction of the visual coordinate system and the robot coordinate system are generally different. Therefore, the coordinates of the visual coordinate system obtained by the imaging device cannot be directly used as the coordinates of the robot coordinate system, and the visual coordinate system A transformation from the system to the robot coordinate system is required, so
It is necessary to calibrate the coordinate system that associates both zataneito.

Conventionally, when calibrating the visual coordinate system and the robot coordinate system, as described in JP-A-61-131887, a recognition target for calibration is placed at an arbitrary position within the work area, and the robot is Operate the reference point of the robot's hand to match the position of the recognition target, memorize the position of the hand at this time in the robot coordinate system, and then take an image of the same recognition target with the imaging device to find the position in the visual coordinate system. By calculating this method or by executing the robot's program operation, a recognition target for calibration similar to the above is automatically placed at a position where the position of the reference point of the robot's hand is known in the robot coordinate system, and this recognition target is The visual coordinate system and the robot coordinate system were associated from the results obtained by one of the methods of capturing an image with an imaging device and calculating the position in the visual coordinate system.

[Problem to be solved by the invention]

The above conventional technology had the following problems.

That is, in a method in which a robot is operated to match the reference point of the robot's hand to the position of an arbitrarily placed recognition target for calibration, accurate positioning is difficult and errors occur.

Furthermore, since the recognition target for calibration consists of small blocks,
If the work surface is uneven, the recognition target for calibration cannot be placed stably, resulting in tilting. For these reasons, the accuracy of the conversion coefficients between coordinate systems varies depending on the calibration process.3.

There was a drawback.

In addition, the method of automatically arranging the recognition target for calibration by executing the robot's program operation is such that when the direction of the imaging device or the size of the field of view relative to the reference point of the robot's hand changes, the robot for calibration Because the motion program must be changed, it takes time to create the program, and it is not suitable for robots with hand vision that require calibration of the coordinate system every time the field of view changes.

The purpose of the present invention is to solve the problems of these conventional techniques,
An object of the present invention is to provide a highly accurate and simple method for calibrating the coordinate system of a robot with hand vision.

[Means to solve the problem]

In order to achieve the above object, the present invention includes a tool for acting on a target object and an image capturing device for capturing an image of the target object in the hand portion of the robot, and recognizes the target object from the image data of the image capturing device. an image processing device that calculates the position of the object;
4. A robot controller with hand vision that controls the robot's movements.

In a robot, a coordinate system calibration plate on which a plurality of discernible recognition patterns are drawn on a circumference of a specific radius is subjected to a predetermined operation such that at least one recognition pattern falls within the field of view of the imaging device. the robot control device applies a predetermined amount of movement to the hand portion of the robot;
Every time the hand part of the robot moves, the image processing device calculates the position of the same recognition pattern captured within the field of view of the imaging device, and calculates the position of the same recognition pattern captured in the field of view of the imaging device, and calculates the position of the same recognition pattern captured in the field of view of the imaging device, and calculates the position of the same recognition pattern captured in the field of view of the imaging device. Correspondence between the visual coordinate system that expresses the position of the object in the image processing device and the robot coordinate system used to control the movement of the robot, based on the relative movement amount of the recognition pattern calculated by visual recognition by the image processing device. It is characterized by attaching.

[Effect]

In the present invention, a coordinate system calibration plate (hereinafter simply referred to as a calibration plate) on which a plurality of discernible recognition patterns are drawn on a circumference of a specific radius is placed in a predetermined work area where calibration is to be performed. The image is captured by an imaging device attached to the hand of the robot, and the position of the recognition pattern captured within the field of view is calculated in the visual coordinate system by an image processing device. Then, each time the robot control device moves the robot's hand by applying a predetermined amount of movement to the robot's hand, a recognition pattern is captured, and the same recognition pattern is visually recognized to calculate its position. In this way, it becomes possible to associate both coordinate systems based on the amount of movement in the robot coordinate system given to the robot's hand and the relative movement amount in the visual coordinate system of the recognition pattern, and the reference for the robot's hand becomes possible. There is no need to match the point with the position of the recognition target for calibration and calculate the position of the recognition target in the robot coordinate system.

In addition, since the calibration plate has a plurality of distinguishable recognition patterns drawn on the circumference, if the calibration plate is placed vertically below the robot's hand, the imaging device will be able to locate the reference point of the robot's hand. Regardless of the direction or size of the field of view, one or more recognition patterns can always be captured within the field of view.
Furthermore, even if the field of view moves due to the movement of the robot, it is possible to visually recognize the same recognition pattern. This makes it easy to create a robot motion program for calibration.

In addition, since the calibration plate with a large area is placed on the work surface, it will not tilt due to small irregularities on the work surface.
Visual coordinate values can be calculated accurately, and calibration work can be performed easily and accurately.

〔Example〕

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a sketch of an embodiment of the present invention, in which 1 is a robot, 2 is an example of a tool provided at the hand portion of the robot 1, and 3 is a visual representation of the hand portion of the robot 1. A virtual image device (hereinafter referred to as a camera) provided as
shows a calibration plate placed on the work surface 5. 9 is a robot control device that controls the operation of the robot 1; 10 is a camera 3;
Although the details are not shown, the robot control device 9 includes an operation panel for inputting and selecting programs such as playback operations and coordinate system calibration operations by the operator; During the playback operation, the robot 1's movement commands and the hand 2 are issued based on the teaching data read from the memory and the visual coordinate data from the image processing device 10.
During the calibration operation, the robot 1 sends an operation command based on the calibration data read from the memory, and calculates the distance between the two coordinate systems based on the amount of movement in the robot coordinate system and the amount of movement in the visual coordinate system. The controller is comprised of a processor that calculates conversion coefficients, and a control section that controls each axis of the robot 1 and the servo drive system of the hand 2 according to instructions from the processor. The image processing device 10 also includes an image memory that stores image data, an image processor that visually recognizes an object within the field of view of the camera 3 from the image data, and calculates the position of the object. , all of which have well-known hardware configurations. This figure shows an example in which the robot 1 is a horizontal articulated robot.
The invention is applicable to any other type of robot.

11 is the robot coordinate system (
OR-Xλ, YA% ZR), 12 indicates the visual coordinate system (Or

Figure 2 is a plan view showing the relationship between the calibration plate 4 and the field of view 13 of the camera 3, Figure 2 (B) is a side view of the hand portion of the robot 1 and the calibration plate 4, and 21 is a side view of the robot 1. 31 is the optical axis of the optical system of the camera 3, and 8 is a protrusion provided at the center point C of the calibration plate 4 for gripping the calibration plate 4 with the hand 2. Figure 2 (C) shows the calibration plate 4.
FIG.

As shown in FIG. 2 (4) to (C), the calibration plate 4 includes
n recognition patterns 41, 42, . . . , 4n are calibrated at appropriate intervals on a circumference with a radius approximately equal to the distance d between the rotation center axis 21 of the wrist portion of the robot 1 and the optical axis 31 of the camera 3. It is drawn in a color that contrasts with the background 40 of the work board 4. More specifically, each of the n recognition patterns is, for example, 41a, 41 in FIG. 2(C).
As shown in b, 41c, the center of gravity of the pattern G,,G,,
..., are drawn in concentric circles with contrasting colors alternately with Gn as the center, and inside the innermost circle, for example, 41c, each mark is drawn in order to distinguish it from each adjacent recognition pattern 42.4n. Small circles 4 with different numbers for each pattern
01a is drawn in a contrasting color. Small circles 401a and 402a for identifying these patterns
, 402b, -, 40na, 40nb.

40nc, 40nd, the center of gravity position is the center of gravity Gl r Gt + of each recognition pattern 41, 42°..., 4n.
. . , Gn, it is also possible to detect the direction of each pattern 41 , 42 , . . . , 4n. In this example, each pattern has 41
a * 41 b and 41 c were used as concentric circles, but we also made sure that the centers of gravity of multiple similar figures other than circles coincided with each other, such as 71 a , 71 b , and 71 c in Figure 2). You may. In addition, in this embodiment, the small circles 401a, 402a,..., 4 inside the innermost circles 41c, 42c,..., 4nc of each pattern
It was identified as an adjacent pattern by the number of 0nd, but the second
Figures or codes with different shapes and areas, such as 701, 702, and 703 in the figure) may be written for identification.

The calibration plate 4 on which such a recognition pattern is drawn is held by the protrusion 8 by the hand 2 of the robot 1, and is placed on the work surface 5 on which the coordinate system is to be calibrated by a programmed operation of the robot 1. After placing the calibration plate 4 on the work surface 5, the camera 3 is placed in the calibration position, that is, in a state where the area to be calibrated is within the field of view 13, and the camera 3 is set at the height at which the calibration is to be performed as shown in FIG. 2(B). Move the hand part of the robot 1 to the position shown below. At this time, in camera 3, as shown in Fig. 2 (5)
and imaged as in the field of view 13 in (C). In this visual field 13, two small circles 402a for pattern identification,
The center of gravity of the concentric recognition pattern 42 with 402b inside is determined by the image processing device 10 as 42a and 42b.
, calculate the center of gravity position for each day of 42c, and select the circle 42 with the largest area among those that appear as complete circles.
It is obtained by setting the calculated value of the center of gravity position of b as the center of gravity position G of the recognition pattern. The center of gravity of these patterns can be determined by dividing adjacent patterns drawn with contrast and concentric circles within the same pattern into brightness and darkness using a certain brightness from the captured image to obtain a binary image. The contour is extracted and calculated using a known centroid calculation algorithm.

In addition, when multiple recognition patterns are recognized within the visual field depending on the size and position of the visual field, for example, as in the visual field 16 of the prisoner in Figure 2, there are recognition patterns 41 and 42 whose center of gravity position can be calculated within the visual field. There are cases where two items are recognized. In the procedure for associating coordinate systems described below, it is necessary to associate the pattern recognized within the field of view when the field of view moves with the pattern recognized before the field of view moves. small circles 401a, 402
By using a, -, 40nd, it is possible to associate the same pattern even within different fields of view.

A particular pattern 42 recognized in this manner is focused on and used in a simplified manner as a recognition pattern 6 from FIG. 3 onwards.

Hereinafter, the procedure for associating the visual coordinate system with the robot coordinate system will be explained using FIGS. 3 to 6. 3 and 4, in the position of the robot 1 vertically above the calibration plate 4 as shown in FIG. 2(B), the camera 3
The center of gravity of the recognition pattern 6 in the visual field 13 of OV-
G is calculated in the visual coordinate system 12 of XV and YV.

Next, while keeping the direction and height of camera 3 the same,
If the hand part of the robot 1 is moved by ΔXλ along the Y-axis of the robot coordinate system 11 within the range where the recognition pattern 6 can be seen within the field of view of the camera 3, the center of gravity of the recognition pattern 6 will be moved within the field of view 14 of the camera 3. The position is OFI −Xrr +
It is calculated as G, 1 in the visual coordinate system of YFl. Furthermore, while keeping the direction and height of the camera 3 the same, the hand portion of the robot 1 is moved along the Y axis of the robot coordinate system 11.
Move ΔXJI, return to the initial position, and then move camera 3
The hand part of the robot 1 is moved along the Y axis of the robot coordinate system 11 within the range where the recognition pattern 6 is visible within the field of view.
When the recognition pattern 6 is moved by YR, the center of gravity of the recognition pattern 6 in the field of view 15 of the camera 3 is calculated as Gtt in the visual coordinate system of O, -X, t.YVt. In these, the recognition pattern 6 is fixed to the robot coordinate system 11, the camera 3 is moved by ΔXR2ΔYR within the robot coordinate system 11, and the respective center of gravity positions G, , Gt, are determined. The amount of movement is -ΔXR1 when the camera 3 is fixed in the field of view 13 and the recognition pattern 6 is expressed in the robot coordinate system 11.
This is equivalent to moving by -ΔYR and calculating the respective center of gravity positions as Gll + on in the visual field 13 as shown in FIG. The center of gravity position G of the recognition pattern shown in Fig. 5!
The relationship between set+ I on and the robot coordinate system 11 and visual coordinate system 12 is shown in FIG.

In the figure, the distance and direction between the centroid positions of the recognition patterns in the visual coordinate system 12 correspond to the amount of movement of the robot. That is, in FIG. 6, the arrow -ΔXR-ΔYR corresponds to the movement amount of the robot's hand in the robot coordinate system 11, but in the visual coordinate system 12, it corresponds to the center of gravity position G.
, and G□ is a line segment that visually recognizes -ΔxR, and the straight lines p and F that connect the center of gravity position G and Gtt are line segments that visually recognize -ΔYR. Actually, in the visual coordinate system 12, these line segments PI' + P2
' are the X-axis components P, VxP, and VxP of the visual coordinate system 12, respectively.
It is calculated as only VxY axis components P and 'yP, and from the relationship between these and -ΔXR2-ΔYR, the correspondence between the visual coordinate system 12 and the robot coordinate system 11 can be determined. In FIG. 6, regarding the line segment PIV, the mapping of PIX to the X-axis and Y-axis of the robot coordinate system 11 @P1'X-asα
・AXX * PIVc' gklα・AXY and mapping P of P1'1 to the X-axis and Y-axis of the robot coordinate system 11
1'!・Blood α・AYX, p, j,・μsα・AYY
Since the amount of movement of the robot from G to G2 in the robot coordinate system 11 is -ΔxR in the X-axis direction and 0 in the Y-axis direction, the following equation holds true.

-ΔX7B-PIVx-casα・Axx+P1VY-
ghlα・AYx...(1)0-P1'z-si
nαaAzy-PW-asα・AYY ・-
+21 Here, α is the angle that the visual coordinate system 12 makes with respect to the robot coordinate system 11, Axx, AxY, AYx, AY
Y is the respective X of the visual coordinate system 12 and the robot coordinate system 11.

This is a coefficient indicating the scale ratio of the Y axis, where the first letter of the subscript indicates the axis of the visual coordinate system 12 and the second letter indicates the axis of the robot coordinate system 11.

Similarly, regarding line segment P, p, l P:'Y
The mapping of to the X-axis and Y-axis of the robot coordinate system 11 is determined, and the amount of movement of the robot from G to ON in the robot coordinate system 11 is 0 in the X-axis direction and -ΔYR in the Y-axis direction. Therefore, the following formula holds true.

op,, cmα・Axx+Pty --α・AY
x...-(3+-ΔYR=P2S bit α"X
Y P:'y'cmα'Ary −= (41
From formula fil~(4), the coefficients representing the following scale ratios are found.

Furthermore, in the visual coordinate system 12, since the ratio between the X-axis component and the Y-axis component is constant, the following equation holds true.

Axx/Ayx = Axy/Ayy
=-・i9) Substitute equations (5) to (8) into equation (9) to obtain the following equation.

The angle α formed between the coordinate axes of the robot coordinate system 11 and the visual coordinate system 12 is determined from the equation αQ.

The conversion coefficient Axx between the coordinates obtained in this way,
By storing the values of AYx, Axy, Ayy, and α in the memory in the control device 9, the control device 9 uses this stored data to calculate the object imaged by the camera 3 in the same work area. It becomes possible to convert the positional deviation in the visual coordinate system 12 into a deviation amount in the robot coordinate system 11.

In the above embodiment, the recognition pattern drawn on the calibration plate is
Each figure consists of a plurality of similar figures arranged so that their center of gravity coincides, and among the figures whose entire shape can be completely seen within the field of view of the camera 3, the position of the center of gravity of the figure with the largest area is determined. Since the calculated value is used as the position of the center of gravity of the recognition pattern, even if the size of the field of view changes, it is possible to always select a figure that matches the size of the field of view and calculate the position of the center of gravity of the recognition pattern. Although this further improves the calculation accuracy of visual coordinate values, it is not an essential requirement of the present invention.

Furthermore, according to the above embodiment, all the calibration work, including the placement of the calibration plate in the work area, can be performed automatically by executing the robot program. The calibration plate may be placed directly under the hand portion by hand, and strict alignment between the reference point of the robot's hand and the calibration plate is not required.

〔Effect of the invention〕

As described above, according to the present invention, a calibration plate on which a plurality of distinguishable recognition patterns are drawn on the circumference is placed in the work area to be calibrated, and a predetermined movement amount is applied to the hand portion of the robot.
Each time the robot's hand moves, the position of the same recognition pattern is visually recognized, and the amount of movement in the robot coordinate system and the amount of relative movement in the visual coordinate system are correlated to perform calibration between the two-seat species. Therefore, there is no need to match the reference point of the robot's hand to the position of the recognition target for calibration, as in the past.
Therefore, errors in alignment do not occur, and since the calibration plate with a large area is placed on the work surface, the posture of the calibration plate is not affected by small irregularities on the work surface, and the conversion coefficients between coordinates can be accurately adjusted. It can be found well and stably.

Furthermore, since multiple distinguishable recognition patterns are drawn on the calibration plate on the circumference, the recognition pattern can be drawn within the field of view regardless of the direction of the camera relative to the reference point of the robot's hand and the size of the field of view. Therefore, when creating a program for the robot's calibration operation, there is no need to consider changes in the direction of the camera or the size of the field of view, making the program easy to create. For these reasons, it is possible to provide a simple and highly accurate coordinate system calibration means for robots with hand vision whose field of view changes depending on the work area and requires frequent calibration.

[Brief explanation of the drawing]

Figure 1 is a sketch showing an embodiment of the present invention, Figure 2 (5)
, (B), and (C) are a plan view, a side view, and a partially enlarged view of the calibration plate showing the relationship between the calibration plate and the field of view of the camera in this example, and FIG. 2(D) is a partial enlarged view of the calibration plate. A partially enlarged view showing another example, FIG. 3 is a sketch showing the calibration operation of the robot in this example, FIG. 4 is an explanatory diagram of the relationship between the coordinates in FIG. 3, and FIG. 5 is the coordinates in FIG. 4. FIG. 6 is an explanatory diagram of the process of determining the relationship between coordinates and the conversion coefficient between coordinates in this embodiment. Trobot 2 Hand (tool) 3 Camera (imaging device) 4.7 Calibration plate 5 Work surface 6 Recognition pattern 8 Grasping protrusion 9
Robot control device 10・Image processing device 11...Robot coordinate system 12...Visual coordinate system 13-16...
Vision Agent Patent Attorney Katsuo Ogawa

Claims (1)

[Claims] 1. The hand portion of the robot is equipped with a tool that acts on an object and an imaging device that takes an image of the object, and the robot recognizes the object from the imaging data of the imaging device and captures the object. A robot with hand vision that has an image processing device that calculates the position of an object and a robot control device that controls the motion of the robot,
A coordinate system calibration board on which a plurality of discernible recognition patterns are drawn on a circumference of a specific radius is arranged in a predetermined work area so that at least one recognition pattern is within the field of view of the imaging device. The robot control device applies a predetermined movement amount to the robot's hand, and each time the robot's hand moves, the image processing device detects the position of the same recognition pattern captured within the field of view of the imaging device. The position of the object is expressed in the image processing device from a predetermined movement amount calculated by the robot control device and a relative movement amount of the recognition pattern calculated by visual recognition by the image processing device. A method for calibrating a coordinate system of a robot with hand vision, the method comprising associating a visual coordinate system with a robot coordinate system used to control the motion of the robot. 2. The recognition pattern drawn on the coordinate system calibration board is composed of a plurality of similar figures arranged so that their center of gravity coincides with each other. Coordinate system calibration method for robots with hand vision. 3. Grip the coordinate system calibration plate with the tool provided at the hand of the robot, apply a predetermined movement amount to the hand of the robot by the robot control device, and place the calibration plate in a predetermined work area. A method for calibrating a coordinate system of a robot with hand vision according to claim 1 or 2, characterized in that: