WO2024257136A1 - Robot device and method for controlling same - Google Patents

Abstract

Provided is a robot device comprising: a robot hand to which a plurality of finger parts are provided; an arm part that drives the robot hand; a plurality of gripping parts that follow the plurality of finger parts and can grip an object; at least one identification part on at least one of the plurality of gripping parts; and a control unit that uses position information regarding the identification part and drives the robot hand via the arm part to position the plurality of gripping parts with respect to the object. Thus, positional alignment of the object and the robot hand can be efficiently performed.

Description

Robot device and control method thereof

The present invention relates to a robot device and a method for controlling a robot device.

A robot device is used that has a robot hand (end effector) for grasping an object and an arm that drives the robot hand, and that can detect the amount of positional deviation of the robot hand relative to the arm from the image capture results of the robot hand (see, for example, Patent Document 1). In this technology, it is desirable to efficiently align the object and the robot hand.

JP 2017-124468 A

According to a first aspect of the present invention, there is provided a robot device comprising: a robot hand provided with a plurality of finger portions; an arm portion for driving the robot hand; a plurality of gripping portions capable of gripping an object in response to the plurality of finger portions; at least one identification portion possessed by at least one of the plurality of gripping portions; and a control portion for driving the robot hand via the arm portion using position information of the identification portion to position the plurality of gripping portions relative to the object.

According to a second aspect, there is provided a method for controlling a robot device including a robot hand provided with a plurality of finger portions and an arm portion for driving the robot hand, the method including: causing a plurality of gripping portions capable of gripping an object to follow the plurality of finger portions; storing position information of at least one identification portion possessed by at least one of the plurality of gripping portions; and using the position information of the identification portion to drive the robot hand via the arm portion and position the plurality of gripping portions relative to the object.

1A is a perspective view showing the robot device of the first embodiment, FIG. 1B is a perspective view showing a modified example of a finger, and FIG. 1C is a diagram showing a main part of a modified example of a robot hand. 1A is a side view showing the robot hand of FIG. 1A, FIG. 1B is a front view showing the fingers of the robot hand in the -Y direction, and FIG. 1C is a diagram showing an example of an image within the field of view of an imaging device. FIG. 2 is a diagram illustrating an example of a robot device and a control device for a robot hand. (A) is a plan view showing the tool fixed to the fingers, (B) is an image taken of the state shown in Figure 4(A), (C) is an image taken of the state where the gripping center of the tool is aligned with the center of the workpiece, and (D) is an image taken of the state where the workpiece is gripped by the tool. (A) is an image taken when the angle between the center line of the workpiece and the center line of the tool is large, (B) is an image taken after the robot hand has been rotated relative to the workpiece, (C) is an image taken when the gripping center of the tool is aligned with the center of the workpiece, and (D) is a plan view showing another example of a marking member. 1A is a flowchart showing an example of a method for detecting a mark member, FIG. 1B is a flowchart showing an example of a method for gripping a workpiece with a robot hand, and FIG. 1C is a flowchart showing a modified example of the gripping method. 7A and 7B are side views showing the robot device, and 7C and 7D are images captured in the cases of FIGS. 7A and 7B, respectively. 8A is a side view showing the main parts of the robot hand when gripping another workpiece, (B) is a diagram showing an image obtained by an imaging device in the state shown in FIG. 8A, and (C) is a cross-sectional view showing the case where a workpiece is gripped at two different heights. 9(A) is a plan view showing the state in which the tool is holding multiple workpieces, (B) is a side view showing the tool in the case of FIG. 9(A), (C) is a cross-sectional view showing an example of the tip of a tool that is not capable of holding multiple workpieces, (D) is a side view showing the tip of the tool in FIG. 9(C), and (E) is an enlarged view showing another example of the tip of the tool. FIG. 11 is a perspective view showing a robot device according to a second embodiment. 11 is a diagram showing an example of a screen captured by the imaging device of FIG. 10 . 12A is a side view showing a modified robot hand, FIG. 12B is a front view showing the finger in FIG. 12A in the -Y direction, and FIG. 12C is a diagram showing an example of an image within the field of view of the imaging device in FIG. 12A.

[First embodiment]
The first embodiment will be described below with reference to FIGS. 1(A) to 6(C).
Fig. 1(A) shows an articulated robot device 2 according to this embodiment. Note that the robot device 2 may be of any configuration, such as a horizontal articulated robot device (SCARA type). In Fig. 1(A), as an example, an X-axis and a Y-axis are taken as being parallel to and perpendicular to the installation surface of the robot device 2, and a Z-axis is taken as being perpendicular to the installation surface. The installation surface is an approximately horizontal plane, and the upward direction of the installation surface (the direction opposite to the vertical direction) is the + direction of the Z-axis. A coordinate system consisting of positions on the X-axis, Y-axis, and Z-axis is called the robot coordinate system (X, Y, Z).

In FIG. 1(A), the robot device 2 comprises a robot main body 4, a robot hand 6 joined to the robot main body 4, a control device 10 that controls the operation of the robot device 2, and a control device 12 that controls the operation of the robot hand 6. The robot main body 4 comprises a base 14, a rotating part 16 connected to the base 14, and a first robot arm 18A, a second robot arm 18B, and a third robot arm 18C that are connected to the rotating part 16 in a displaceable manner in sequence. The robot hand 6 is joined to the tip 20A of the third robot arm 18C. The robot hand 6 can also be called an end effector.

As an example, the rotating unit 16 can control the rotation angle θ1 around an axis parallel to the Z axis, the first robot arm 18A can control the swivel angle φ1, the second robot arm 18B can control the swivel angle φ2, and the third robot arm 18C can control the swivel angle φ3 and the rotation angle θ2 of the tip end 20A (robot hand 6) around an axis parallel to the Z axis, for example. The third robot arm 18C has a rotating unit 20C that changes the rotation angle θ2 of the tip end 20A and a tilt drive unit 20B that changes the swivel angle φ3. The rotating unit 20C and the tilt drive unit 20B each have a drive motor and an encoder that detects the rotation angle. The rotating unit 16 and the other robot arms 18A and 18B similarly have drive motors and encoders. Therefore, the robot hand 6 attached to the third robot arm 18C can be controlled in four degrees of freedom (including angles), including at least the position in the X direction (direction along the X axis; same below), Y direction, and Z direction, and the rotation angle around the Z axis.

The origin 58A of the robot coordinate system (X, Y, Z) is, for example, the center of the base 14, but the origin 58A can be set or changed to any position. As an example, the robot hand 6 is driven by the rotating unit 16 and the robot arms 18A-18C so as to move parallel to the Z axis. Since the robot hand 6 has at least five encoders, it is also possible to control the position of the robot hand 6 with three degrees of freedom and the angle with two degrees of freedom. Also, instead of the robot device 2, the robot hand 6 may be joined to a robot device (not shown) that is capable of controlling the position of the robot hand 6 with three degrees of freedom and the angle with three degrees of freedom.

The robot device 2 also includes an imaging device 22 and a lighting device (not shown). As an example, the imaging device 22 is attached to the robot hand 6, but the imaging device 22 may be provided at a position other than the robot hand 6, such as the side of the robot arm 18C, a measurement position on the installation surface of the robot device 2, or a wall or ceiling of the room in which the robot device 2 is installed. As an example, the imaging device 22 is a camera that captures normal two-dimensional images, and its focal position is set near an area including the tips of the tools 8b and 8c (described later) fixed in the robot hand 6, and its focal depth is set relatively deep. The imaging device 22 may include an autofocus lens. When the focal position of the imaging device 22 is near an area including the tips of the tools 8b and 8c, when the tips of the tools 8b and 8c are in contact with or close to the placement surface FL of the workpiece 48, the workpiece 48 is also at approximately the focal position of the imaging device 22. However, a three-dimensional camera (stereo camera) or the like may be used as the imaging device 22.

In addition, the robot hand 6 of this embodiment has two fingers 38A, 38B that are driven to open and close, and members (hereinafter referred to as tools) 8b, 8c for grasping an object are fixed to the inner surfaces of the fingers 38A, 38B by bolts B1 or the like (see FIG. 2(A)). As an example, the tools 8b, 8c (grasping parts) are supported so as to protrude forward of the robot hand 6 and have tips that protrude downward, and an object can be grasped between the tips. The shape of the tips of the tools 8b, 8c may be any shape depending on the object. The tools 8b, 8c are made of metal, but some or all of the tools 8b, 8c may be made of synthetic resin or the like. By closing the fingers 38A, 38B, the tools 8b, 8c close in response to the movement of the fingers 38A, 38B, and an object can be grasped or supported between the tips of the movable parts 8b, 8c. In addition, by opening the fingers 38A and 38B, the tools 8b and 8c open in response to this movement, allowing the tools 8b and 8c to be removed from the object.

It is also possible that one tool 8b is fixed and only the other tool 8c is movable. In other words, it is sufficient that the tools 8b and 8c are capable of opening and closing relatively to each other. In this case, the finger 38A to which the fixed tool 8b is fixed may also be fixed. Also, three or more tools can be used instead of the tools 8b and 8c. In this case, only at least one of the three or more tools may be movable and the other tools may be fixed. When three or more tools are held by the robot hand 6, the number of fingers 38A, etc. may be the same as the number of the tools. Furthermore, when some of the three or more tools are fixed, the fingers 38A, etc. supporting the fixed tools may be fixed.

In addition, instead of fixing the tools 8b and 8c to the fingers 38A and 38B, fingers 38C and 38D whose tips 38Cb and 38Db are processed to have a tool-like shape may be used as shown in FIG. 1B. In the following, the objects to be processed (objects to be gripped) gripped by the tools 8b and 8c are multiple workpieces 48 placed on a mounting surface FL that is substantially parallel to the horizontal plane. As an example, the workpieces 48 are bolts having a shaft 48a and a head 48b. The shape of the objects to be processed is arbitrary, and the size of the objects to be processed is arbitrary as long as it has a portion that can be gripped by the tools 8b and 8c. Furthermore, even if the workpiece is, for example, a long string-like or cable-like object, it can be gripped by the tools 8b and 8c as long as it is within the range that can be gripped by the diameter of the cross-sectional shape.

Fig. 2(A) shows the robot hand 6 in Fig. 1(A), and Fig. 2(A) shows a state in which the robot hand 6 is joined to the lower end of the tip 20A of the robot arm 18C so that the opening and closing direction of the tools 8b, 8c is parallel to the X-axis (X-direction). In Fig. 2(A), the robot hand 6 comprises a hand main body 28 joined to the lower end of the tip 20A, a connecting part 36 connected to an operating part 32 of the hand main body 28, a pair of fingers 38A, 38B that are opened and closed in the X-direction by the operating part 32, and tools 8b, 8c fixed to the fingers 38A, 38B. The tools 8b, 8c are opened and closed in the X-direction by the fingers 38A, 38B. The fingers 38A, 38B are made of metal, for example.

The hand body 28 also has a frame 30 having a flat joint (joint) joined to the tip 20A, and an operating unit 32 connected to the frame 30 via a linear slider 34 so as to be movable relative to the tip 20A in the Z direction. The imaging device 22 is supported on the +Y side of the frame 30 via a support member 50. The joint of the frame 30 is mechanically (fixed by a screw, fixed by a movable claw mechanism, fixed by a permanent magnet, etc.) or electrically (fixed by an electromagnet, etc.) joined to the tip 20A. Furthermore, the joint is provided with connections such as electrical wiring, compressed air piping, and/or vacuum suction piping as necessary. The joint of the frame 30 is provided with positioning parts (not shown) such as multiple protrusions or recesses so that the position and rotation angle relative to the tip 20A are predetermined values.

The imaging device 22 is also placed slightly diagonally upward (in the +Y and +Z directions) with respect to the tools 8b and 8c. As shown in FIG. 2C, the center of the image 22G captured within the field of view of the imaging device 22 is taken as the origin 58B, the x-axis is taken parallel to the longitudinal direction of the image 22G passing through the origin 58B, and the y-axis is taken parallel to the lateral direction of the image 22G. The coordinate system (x, y) is called the coordinate system of the imaging device 22. The position of the origin 58B is arbitrary. In the state of FIG. 2C, the x-axis and y-axis are respectively parallel to the X-axis and Y-axis of the robot coordinate system (X, Y, Z), but the position and angle of the coordinate system (x, y) of the imaging device 22 with respect to the robot coordinate system (X, Y, Z) changes when the robot hand 6 is driven by the robot arms 18A to 18C.

Furthermore, the position of the imaging device 22 is set so that the tips of the images 8bI, 8cI of the tools 8b, 8c are located on the -y side of the origin 58B in the image 22G. Also, since the magnification of the imaging device 22 is known, when the Z positions of the tools 8b, 8c in the robot coordinate system (X, Y, Z) are set to a certain position, the control device 10 can divide the interval (distance) in the x and y directions on the image 22G of the imaging device 22 by that magnification to obtain the interval (approximation) in the x and y directions on the robot coordinate system (X, Y, Z) that corresponds to that interval.

In addition, two expandable coil springs (hereinafter referred to as elastic members) 44A, 44B are connected in a direction parallel to the Z axis between the joint (upper plate portion) of the frame 30 and both side surfaces in the X direction of the connecting portion 36. Hereinafter, the elastic members 44A, 44B are collectively referred to as the elastic member 44. A suspension mechanism 46 (buffer mechanism) is configured that includes the linear slider 34 and the elastic member 44 and supports the movable portion (the portion including the operating portion 32, the fingers 38A, 38B, and the tools 8b, 8c) in a manner that allows elastic displacement in the Z direction relative to the frame 30. In addition, a stopper 52 for limiting the displacement in the Z direction of the operating portion 32 (movable portion) relative to the frame 30 within a predetermined range is configured by a U-shaped limiting member 52a provided on the frame 30 and a pin 52b provided on the operating portion 32 that is displaceable in the Z direction between the limiting member 52a. When the tools 8b and 8c are not gripping the workpiece, the stopper 52 is set so that the pin 52b is positioned in the middle of the limiting member 52a.

The suspension mechanism 46 may have any configuration, such as a mechanism that uses a leaf spring instead of a tension coil spring as the elastic member 44. In this embodiment, the above-mentioned movable part is supported by the suspension mechanism 46. As another configuration, for example, a suspension mechanism may be used in which the operating part 32 is fixed to the frame 30 and the tools 8b and 8c are supported so as to be elastically displaceable in the Z direction relative to the operating part 32.

Furthermore, a proximity sensor 56 is provided above the operating part 32 inside the frame 30 to detect the distance from the operating part 32 without contact. From the detection result of the proximity sensor 56, the relative displacement of the movable part (operating part 32 and tools 8b, 8c, etc.) with respect to the frame 30 can be detected, and from this detection result, it can be detected whether the tips of the tools 8b, 8c have come into contact with the mounting surface FL of the workpiece 48. If the mounting surface FL has a certain degree of conductivity, a contact sensor that detects the conduction between the tools 8b, 8c and the mounting surface FL may be used instead of the proximity sensor 56. Also, for example, a proximity sensor that detects whether the tips of the tools 8b, 8c have come close to the mounting surface FL may be provided on the side of the fingers 38A, 38B, and the proximity sensor may be used to detect that the tips of the tools 8b, 8c have come close to the mounting surface FL.

Also, as an example, when the robot device 2 starts operating, for example, when the proximity sensor 56 detects that the tips of the tools 8b, 8c have come into contact with the mounting surface FL of the workpiece 48, the value of the position on the Z axis of the robot coordinate system (X, Y, Z) (hereinafter also referred to as the Z position) may be preset to 0. This preset information may be supplied, for example, to a coordinate calculation unit 11C (see FIG. 3) described below. Thereafter, the control devices 10 and 12 may recognize the distance in the Z direction between the tips of the tools 8b, 8c and the mounting surface FL based on the value of the Z position of the robot coordinate system (X, Y, Z).

Furthermore, by detaching the frame 30 from the tip 20A and joining the frame 30 of a robot hand equipped with tools 8b, 8c of a different shape or fingers 38A, 38B of a different shape to the tip 20A, it is possible to replace the robot hand 6 joined to the tip 20A of the robot arm 18C. Also, in the robot hand 6 joined to the tip 20A, the tools 8b, 8c fixed to the fingers 38A, 38B can be replaced with tools of a different shape depending on the shape of the object to be processed, etc.

Fig. 2(B) is a front view of Fig. 2(A) as viewed in the -Y direction, and in Fig. 2(B), the operating unit 32 has a finger drive unit 13C (see Fig. 3) including a rotary motor, a conversion unit (not shown) that decelerates the amount of rotation of the finger drive unit 13C and converts it into linear motion to open and close the fingers 38A, 38B, and a gap detection unit 13B (see Fig. 3) that detects the position and gap between the fingers 38A, 38B from the rotation angle of the finger drive unit 13C. The pair of fingers 38A, 38B opens and closes the tools 8b, 8c in the opening and closing direction (here, the X direction) by the operating unit 32.

In this embodiment, the fingers 38A, 38B are driven by the actuator 32 so that the distance between the tools 8b, 8c in the opening and closing direction changes. The actuator 32 may operate the fingers 38A, 38B so that the tools 8b, 8c move in the X direction independently of each other.
Furthermore, when an object is not being gripped or supported, as an example, the -X direction finger 38A (tool 8b) waits at the outermost position (-X direction) (hereinafter also referred to as the standby position) within the movable range in the X direction, and the +X direction finger 38B (tool 8c) waits at the outermost position (+X direction) (hereinafter also referred to as the standby position) within the movable range in the X direction. When gripping an object with the tools 8b and 8c, the fingers 38A and 38B move from the standby position to close the tools 8b and 8c, and when releasing the tools 8b and 8c from the object, the fingers 38A and 38B move toward the standby position.

Next, Figure 3 shows the control device 10 of the robot device 2 and the control device 12 of the robot hand 6. In Figure 3, the control device 10 has a control information input/output unit 11A that inputs and outputs control information (such as the type of tool to be used and the position of the object to be processed) with the operator, a main control unit 11B that transmits and receives control information with the control device 12 of the robot hand 6 and controls the operation of the entire device, and a coordinate calculation unit 11C that processes the detection results of at least five-axis encoders that detect the movements of the robot arms 18A-18C, etc., and calculates the coordinates and rotation angles of the tips of the tools 8b and 8c held by the robot hand 6 at the tip of the third robot arm 18C.

The control device 10 further includes an image processing unit 11D that processes the imaging signal of the imaging device 22 attached to the robot hand 6 to determine, for example, the positional relationship between the tip of the tool 8b, 8c held by the robot hand 6 and an object, an arm control unit 11E that controls the movement of the rotation unit 16 and the robot arms 18A to 18C based on the processing results of the coordinate calculation unit 11C and the image processing unit 11D, and a memory unit 11F that stores various data. The image processing unit 11D may be provided in the control device 12 on the robot hand 6 side. The memory unit 11F may include, for example, any one of a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and a non-volatile memory such as a USB (Universal Serial Bus) memory. The memory unit 11F stores the programs and various setting values of the control device 10.

The control device 12 also has a hand control unit 13A that transmits and receives control information between the main control unit 11B and the arm control unit 11E of the control device 10 and controls the operation of the entire robot hand 6, a finger drive unit 13C that opens and closes the fingers 38A and 38B, and a spacing detection unit 13B that detects the position and spacing of the fingers 38A and 38B. The control device 12 also has a finger gripping force detection unit 13D that detects the reaction force (force by the fingers 38A and 38B) from the tools 8b and 8c against the fingers 38A and 38B as a finger gripping force FF, a contact detection unit 13E that processes the detection signal of the proximity sensor 56 in FIG. 2 and detects whether the tools 8b and 8c held by the robot hand 6 have contacted the placement surface FL, and a memory unit 13F. The configuration of the memory unit 13F is the same as that of the memory unit 11F, and the memory unit 13F stores the programs and various setting values of the control device 12.

As an example, the finger grip force detection unit 13D detects the finger grip force FF from the current flowing through the rotation motor unit of the finger drive unit 13C, and the hand control unit 13A controls the operation of the fingers 38A, 38B (finger drive unit 13C) so that the detected finger grip force FF is below a preset level (standard value). This makes it possible to prevent damage to the tools 8b, 8c, the fingers 38A, 38B, and/or the object. Note that instead of the finger grip force detection unit 13D, a sensor capable of detecting the finger grip force FF, such as a strain gauge or a pressure sensor provided on the fingers 38A, 38B, can be used. The control information input to the hand control unit 13A includes, for example, a command to start gripping an object with the tools 8b and 8c and a command to release the grip, and the control information output from the hand control unit 13A to the main control unit 11B includes, for example, information indicating completion of gripping of the object by the tools 8b and 8c, and information indicating completion of releasing grip of the object by the tools 8b and 8c (completion of removal of the tools from the object).

Next, an example of the basic operation of the robot device 2 of this embodiment will be described. First, FIG. 4(A) shows the fingers 38A, 38B, tools 8b, 8c, and the workpiece 48 to be grasped, as viewed from the operating unit 32 side of the robot hand 6 in FIG. 2(A). The workpiece 48 has a small diameter shaft 48a and a large diameter head 48b. In the robot device 2, as an example, the tips of the tools 8b, 8c are in contact with the mounting surface FL of the workpiece 48 (or are set at a position slightly higher than the mounting surface FL) and the robot hand 6 (tools 8b, 8c) is moved in the X and Y directions to place the shaft 48a of the workpiece 48 between the tips of the tools 8b, 8c, and then the fingers 38A, 38B are closed to grasp the shaft 48a of the workpiece 48 between the tools 8b, 8c.

In FIG. 4A, small flat marking members 62A and 62B made of metal or the like, on which marks MA and MB are formed, are fixed to the tips of tools 8b and 8c (portions that can be imaged in image 22G of imaging device 22). As an example, the midpoint of the line segment connecting the centers of marks MA and MB is regarded as the gripping center 58C of tools 8b and 8c. The gripping center 58C is set slightly inward from the center (midpoint) of the tips of tools 8b and 8c. Marking members 62A and 62B are fixed to the upper surfaces of tools 8b and 8c, for example, by adhesion or glue, in a state that does not interfere with the opening and closing of tools 8b and 8c. In this embodiment, marks MA and MB have the same circular shape, but marks MA and MB may have different shapes. Furthermore, for example, if the shape of a part of tools 8b and 8c is distinctive, that part of the shape can be used instead of marks MA and MB or marking members 62A and 62B.

The marks MA and MB can be any mark or marker, such as a symbol other than a circle, such as a cross or polygon, an alphabet or number, or a symbol containing predetermined information, such as a barcode. The marks MA and MB can also be an identification code, such as an April Tag or a QR Code (registered trademark) (two-dimensional barcode). Such an identification code or recognition code can be used to recognize and manage whether the tools 8b and 8c being used for the object to be grasped (workpiece) are of the correct shape. The April Tag is a visual mark (marker) for applications that detect the position and orientation of an object in three dimensions. The April Tag is also an example of an AR (Augmented Reality) marker.

In this embodiment, the tools 8b, 8c are approximately line-symmetrical with respect to a center line 8cl in a plane approximately parallel to the plane including the Y-axis and Z-axis, and the marks MA, MB are fixed to the tools 8b, 8c at positions approximately line-symmetrical with respect to the center line 8cl. In addition, since the tools 8b, 8c are inclined with respect to a plane perpendicular to the optical axis of the imaging device 22 (see FIG. 2(A)), the marks MA, MB of the mark members 62A, 62B fixed to the tools 8b, 8c are formed on the mark members 62A, 62B in a shape that can be easily identified from the image of the imaging device 22 when the image is captured by the imaging device 22.

Figure 4(B) shows image 22G obtained by imaging tools 8b, 8c and workpiece 48 in the state of Figure 4(A) with imaging device 22 of Figure 2(A). In Figure 4(B), images 8bI, 8cI of tools 8b, 8c, images 62AI, 62BI of mark members 62A, 62B, images MAI, MBI of marks MA, MB, and image 48I of workpiece 48 are displayed in image 22G. Information of image 22G is supplied to image processing unit 11D of Figure 3, and image processing unit 11D regards the center of images MAI, MBI of the two marks as image 58CI of grip center 58C of tool 8. Also, image processing unit 11D regards the center of the image of shaft portion 48a in image 48I of workpiece 48 as image 48cI of the center of workpiece 48, as an example. Furthermore, the image processing unit 11D calculates the distance δx in the x direction and the distance δy in the y direction on the image 22G between the image 48cI of the center of the workpiece 48 and the image 58CI of the gripping center of the tools 8b and 8c, and further divides these distances (distances) by the magnification of the imaging device 22 to calculate the distances converted into the distances in the X direction and the Y direction on the robot coordinate system (X, Y, Z), and supplies the calculated distances to the arm control unit 11E via the main control unit 11B.

In response to this, the arm control unit 11E drives the rotating unit 16 and the robot arms 18A to 18C so as to offset the interval, so that the image of the center of the workpiece 48 almost coincides with the image 58CI of the gripping center of the tools 8b and 8c, as shown in the image 22G of the imaging device 22 in FIG. 4C. If the image of the center of the workpiece 48 and the image 58CI of the gripping center of the tools 8b and 8c are significantly misaligned in this state, the robot arms 18A to 18C, etc. may be driven again so that the image of the center of the workpiece 48 and the image 58CI of the gripping center coincide with each other. Thereafter, by closing the tools 8b and 8c, the shaft portion of the workpiece 48 is gripped between the tools 8b and 8c so that the image 48I of the workpiece is sandwiched between the images 8bI and 8cI of the tools in the image 22G in FIG. 4D. 4C, even if the center line of the workpiece 48 (a straight line passing through the center of the shaft portion 48a) and the center lines of the tools 8b and 8c are inclined to some extent, by closing the tools 8b and 8c, the center line of the workpiece 48 and the center line of the tools 8b and 8c become substantially parallel, and the workpiece 48 can be held between the tools 8b and 8c. After that, the robot arms 18A to 18C, etc. are driven to transport the workpiece 48 held by the tools 8b and 8c of the robot hand 6 to a workpiece storage box (not shown) or the like, and the fingers 38A and 38B (tools 8b and 8c) are opened to separate the workpiece 48 from the workpiece 48, thereby completing the movement of the workpiece 48.
The position where the workpiece 48 is gripped does not necessarily have to be the center 48c of the workpiece. The tools 8b and 8c may be positioned so that an appropriate gripping position for each workpiece coincides with the gripping center of the tools 8b and 8c. For example, the center of gravity of the workpiece may coincide with the gripping center of the tools 8b and 8c.

Next, an example of a control method for the robot device 2 of this embodiment will be described with reference to the flowcharts of Figures 6(A) to (C). First, as a preparation step, in step 102 of Figure 6(A), the tools 8b and 8c are fixed (attached) to the fingers 38A and 38B of the robot hand 6 as shown in Figure 4(A). Furthermore, in step 104, the mark members 62A and 62B on which the marks MA and MB are formed are attached to the tips of the tools 8b and 8c. Note that step 104 may be performed first, and then step 102. Then, in step 106, as shown in Figure 4(A) (the Z positions of the tips of the tools 8b and 8c may be arbitrary), the image capturing device 22 captures images of the marks MA and MB of the mark members 62A and 62B at the tips of the tools 8b and 8c.

Furthermore, in step 108, the image processing unit 11D calculates the position (xm, ym) of the center of the image of the marks MA, MB (image of the gripping center 58C or gripping position) on the coordinate system (x, y) of the imaging device 22 from the obtained image 22G, and stores the obtained position (xm, ym) in the memory unit 11F. Note that if the relative positions of the tools 8b, 8c with respect to the imaging device 22 hardly change, the imaging of the marks MA, MB by the imaging device 22 can be omitted thereafter, and the stored position (xm, ym) can be used as the gripping position. However, if there is a possibility that the relative positions of the tools 8b, 8c with respect to the imaging device 22 may change due to vibration or short-term changes over time, steps 106 and 108 may be executed repeatedly.

After that, when the robot device 2 transports the multiple workpieces 48 in FIG. 1(A) to a workpiece storage box (not shown) or the like, in step 112 in FIG. 6(B), the tips of the tools 8b and 8c held by the robot hand 6 are brought into contact with the placement surface FL of the workpiece 48, as shown in FIG. 2(A). At this time, it can be detected from the detection signal of the proximity sensor 56 that the tips of the tools 8b and 8c have come into contact with the placement surface FL. If the Z position is preset to 0 when the tips of the tools 8b and 8c come into contact with the placement surface FL in advance, the robot hand 6 may be driven via the robot arms 18A to 18C, etc., simply so that the Z position becomes 0 on the robot coordinate system (X, Y, Z). The fact that the tips of the tools 8b and 8c have come into contact with the placement surface FL may be determined by image processing, for example, from an image captured by an imaging device (not shown) capturing an image of the vicinity of the placement surface FL from the side direction of the robot hand 6. Furthermore, the operator may visually determine that the tips of the tools 8b and 8c have come into contact with the mounting surface FL, and set the Z position at that time to 0. Also, in step 112, the height of the tips of the tools 8b and 8c may be set to a position that is higher than the mounting surface FL by a predetermined value δZ (e.g., a height at which they do not come into contact with the workpiece 48).

Then, in step 114, with the tips of the tools 8 b, 8 c raised by a predetermined value δZ above the mounting surface FL via the robot arms 18A to 18C etc. (if the heights of the tools 8 b, 8 c were set to that predetermined value in step 112, then the heights are kept the same), the robot hand 6 is driven in the X and Y directions to a position where the workpiece 48 falls within the image 22G of the imaging device 22, and the workpiece 48 is imaged by the imaging device 22.
Then, in step 116, the image processing unit 11D calculates the distances δx, δy in the x and y directions on the image 22G from the center of the workpiece image 48I to the gripping center image 58CI (gripping center), as shown in FIG. 4(B), and calculates the distance (amount of positional deviation) on the robot coordinate system (X, Y, Z) from this distance.

Furthermore, in step 118, the robot hand 6 is moved by the robot arms 18A to 18C, etc. so as to offset the positional deviation, and then the tips of the tools 8b and 8c are brought into contact with the placement surface FL or brought close to the placement surface FL. Then, in step 120, the workpiece 48 is gripped by the tools 8b and 8c of the robot hand 6. Then, in step 122, the robot arms 18A to 18C, etc. are driven to move the robot hand 6, and the workpiece 48 gripped by the tools 8b and 8c of the robot hand 6 is transported to a storage box or the like, and the tools 8b and 8c are detached from the workpiece 48. After that, in step 124, if there are remaining workpieces 48, the process proceeds to step 112, and the next workpiece 48 is transported from the placement surface FL to a storage box or the like. In step 124, when there are no remaining workpieces 48, the transport process ends.

If the image capturing result in step 114 indicates that the angle between the workpiece 48 and the tools 8b, 8c is large, the rotation angle correction process in FIG. 6(C) may be executed. In step 128 in FIG. 6(C), the image processing unit 11D calculates the angle Δθ between the image 48clI of the center line of the workpiece and the image 8clI of the center line of the tool on the image 22G of the imaging device 22, as shown in FIG. 5(A). The angle θ is an angle around an axis that is approximately parallel to the Z axis. Then, in step 130, if the angle Δθ is smaller than a predetermined angle (e.g., 30 degrees), the workpiece 48 can be grasped by the tools 8b, 8c, so the process proceeds to step 116 without correcting the rotation angle, and the workpiece 48 is grasped by the tools 8b, 8c.

On the other hand, in step 130, if the angle θ is equal to or greater than the predetermined angle, the process proceeds to step 132, where the robot arms 18A-18C, etc. are driven to offset the angle θ, and the robot hand 6 (tools 8b, 8c) is rotated around an axis parallel to the Z axis. As a result, as shown in FIG. 5B, the image 48cI of the center line of the workpiece 48 becomes parallel to the images of the center lines of the tools 8b, 8c. After this, the process proceeds to step 116. Note that if the shape of the workpiece is substantially circular and can be grasped without angle correction, there is no need for rotation angle correction, and the rotation angle correction process in FIG. 6C can be omitted. Then, in FIG. 5B, the image processing unit 11D calculates the distances δx, δy in the x and y directions from the center 48cI of the workpiece image 48I to the image 58CI of the gripping center, and calculates the distance (positional deviation) on the robot coordinate system (X, Y, Z) from this distance. Furthermore, in step 118, the robot hand 6 is moved so as to offset the amount of positional deviation, and in step 120, the tools 8b and 8c are closed, as shown by the tool images 8bI and 8cI in FIG. 5(C), so that the workpiece 48 can be grasped by the tools 8b and 8c.

In this manner, according to this embodiment, the tools 8b and 8c are fixed so that they can be opened and closed by the fingers 38A and 38B of the robot hand 6. In addition, the images of the marks MA and MB of the mark members 62A and 62B attached to the tools 8b and 8c are detected, and from this detection result, for example, an image 58CI of the tool's gripping center 58C is obtained, and the robot hand 6 is driven so as to align this image 58CI with the center of the image 48I of the workpiece 48, thereby efficiently and accurately aligning the robot hand 6 (tool) with respect to the workpiece 48. Note that the tool's gripping center 58C may be aligned with any position suitable for gripping the workpiece 48. As a result, the tools 8b and 8c of the robot hand 6 can efficiently and accurately grip the workpiece 48.

In addition, when the angle between the workpiece 48 and the tools 8b, 8c is equal to or greater than a predetermined angle, the robot hand 6 (tool) can be rotated to offset that angle, thereby efficiently aligning the workpiece 48 and the tools 8b, 8c.
In addition, since the tips of the tools 8b and 8c are brought into contact with the mounting surface FL of the workpiece 48 to align the tools 8b and 8c with respect to the workpiece 48, alignment in the Z direction can be performed automatically. Therefore, the tools 8b and 8c can be efficiently aligned with respect to the workpiece 48 by using a two-dimensional image captured by a normal imaging device 22.

As described above, the robot device 2 of this embodiment includes a robot hand 6 provided with multiple fingers 38A, 38B, a rotating unit 16 and robot arms 18A-18C (arm units) that drive the robot hand 6, multiple tools 8b, 8c (gripping units) that can grasp a workpiece 48 in response to the multiple fingers 38A, 38B, mark members 62A, 62B (identification units) possessed by the multiple tools 8b, 8c, and a main control unit 11B (control unit) that uses position information of the mark members 62A, 62B to drive the robot hand 6 via the arm unit and position the multiple tools 8b, 8c relative to the workpiece 48.

The control method for the robot device 2 includes a robot hand 6 provided with multiple fingers 38A, 38B, a rotating unit 16 that drives the robot hand 6, and robot arms 18A-18C (arm units), and includes steps 104 and 112 of causing multiple tools 8b, 8c (gripping units) capable of gripping a workpiece 48 to follow the multiple fingers 38A, 38B, step 108 of storing position information of mark members 62A, 62B possessed by the multiple tools 8b, 8c, and steps 114, 116, and 118 of driving the robot hand 6 via the arm unit using the position information of the mark members 62A, 62B to position the multiple tools 8b, 8c relative to the workpiece 48.

According to this embodiment, the positions (grip centers) of the tools 8b, 8c are determined from the position information of the mark members 62A, 62B possessed by the tools 8b, 8c. Therefore, even if the shapes of the tools 8b, 8c or the fingers 38A, 38B are difficult to distinguish, or even if it is difficult to distinguish the shape of the tools 8b, 8c or the fingers 38A, 38B from the shape of the workpiece 48, the alignment of the workpiece 48 with the robot hand 6 can be performed efficiently and with high precision.
In addition, tools 8b, 8c shaped according to the object to be processed can be fixed to fingers 38A, 38B, and tools 8b, 8c can be opened and closed using fingers 38A, 38B, so that the object to be processed can be grasped and moved efficiently or without damaging the object to be processed using tools shaped according to the object to be processed.

In the above embodiment, the workpiece 48 held by the tools 8b and 8c is moved. However, it is also possible to perform various operations such as pressing down, twisting, pulling out, peeling off, or attaching the processing object held or supported by the tools 8b and 8c held by the robot hand 6 in the above embodiment.
In the above embodiment, the two fingers 38A and 38B are movable, but for example, only one tool 8b may be movable by one finger 38A, and the other tool 8c may be fixed. In this case, the other finger 38B may be fixed. Then, in the robot hand 6, only one finger 38A is operated to open and close the tools 8b and 8c. In this case, the mark member 62A is attached only to the tool 8b, and the mark member 62B may not be provided to the fixed tool 8c. In this case, the position of the grip center of the tools 8b and 8c may be obtained from the detection result of the mark MA of one mark member 62A.

In addition, in the above-described embodiment, the mark members 62A, 62B (marks MA, MB) are imaged and detected by the imaging device 22, but the positions of the mark members 62A, 62B may be detected by any other detection method (for example, a method using a gap sensor, etc.).
In the above embodiment, the workpiece is gripped by opening and closing the tools 8b, 8c with the fingers 38A, 38B of the robot hand 6, but the workpiece may be gripped directly with the fingers 38A, 38B of the robot hand 6. In this case, mark members 62A, 62B (marks MA, MB) may be attached to the fingers 38A, 38B, the positions of the gripping centers of the fingers 38A, 38B may be obtained from detection information of the mark members 62A, 62B, and the fingers 38A, 38B may be positioned with respect to the workpiece using this position, and the workpiece may be gripped by the fingers 38A, 38B.

Next, the above-described embodiment can be modified as follows. First, in the above-described embodiment, mark members 62A, 62B are attached to the top (ends) of tools 8b, 8c. Alternatively, as shown in FIG. 5(D), mark members 62C, 62D with an L-shaped cross section may be fixed to the outer side of tools 8b, 8c, and marks MA, MB may be formed on mark members 62C, 62D. In this configuration, even with mark members 62C, 62D, the workpiece can be more easily gripped by tools 8b, 8c.

Next, in the above-described embodiment, a proximity sensor 56 detects that the tips of the tools 8b, 8c held by the robot hand 6 have come into contact with the mounting surface FL of the workpiece 48, and in this state, the workpiece 48 and the tips of the tools are aligned in the Z direction.
As another method, as shown in FIG. 7A, the robot arms 18A-18C of the robot device 2 are driven to bring the tips of the tools 8b and 8c fixed to the fingers 38A and 38B into contact with the placement surface FL of the workpiece 48, and the tools 8b and 8c are moved to a position where they sandwich the shaft of the workpiece 48. In this state, the tools 8b and 8c are imaged by the imaging device 22 provided on the robot hand 6, and an image 22G shown in FIG. 7C is obtained. The image processing unit 11D processes the image 22G in FIG. 7C to obtain the x- and y-direction positions of the image 48I of the workpiece passing through the image 58CI of the gripping center in the images 8bI and 8cI of the tools 8b and 8c. In this state, the Z position of the robot coordinate system (X, Y, Z) is set to 0.

In this case, as shown in Figures 7(A) and (B), when the height of the imaging device 22 changes, the position of the image 48I of the workpiece 48 on the image 22G moves in the y direction due to the parallax of the imaging device 22. Therefore, as shown in Figure 7(B) from the state of Figure 7(A), the robot arms 18A-18C, etc. are driven to raise the position of the robot hand 6 (imaging device 22 and tools 8b, 8c) by a predetermined distance Za in the +Z direction. In this state, the imaging device 22 captures images of the workpiece 48 and tools 8b, 8c, and the image 22G shown in Figure 7(D) is obtained. The image processing unit 11D processes the image 22G in Figure 7(D) to determine the amount of movement Δy in the y direction of the image 48I of the workpiece relative to the state of Figure 7(C). The Z position Za and amount of movement Δy at this time are stored.

Then, conversely, with the Z position of the robot hand 6 (tools 8b, 8c) set to Za, the robot hand 6 is positioned so that the center of the workpiece image 48I is shifted by Δy in the y direction from the center of the tool images 8bI, 8cI on the image 22G. Then, by lowering the Z position of the robot hand 6 to 0, the shaft of the workpiece 48 can be positioned between the tools 8b, 8c as shown in FIG. 7(A), and the workpiece 48 can be grasped by the tools 8b, 8c.

Next, as shown in FIG. 8(A), assume that a workpiece 48A having a truncated cone-shaped shaft 48Aa and a disk-shaped head 48Ab is gripped by tools 8Bb, 8Bc with flat tips, with the shaft 48Aa standing on the mounting surface FL. At this time, the tools 8Bb, 8Bc to which the mark members 62A, 62B are attached are moved to both sides of the shaft 48Aa of the workpiece 48A. In this case, as shown in FIG. 8(B), in the image 22G captured by the imaging device 22, the image 48AbI of the head part of the workpiece image 48AI covers part of the images MAI, MBI of the marks in the mark members 62A, 62B. This may make it difficult to accurately calculate the gripping center between the tools 8Bb, 8Bc from the mark images MAI, MBI. Therefore, as an example, the background color of the mark members 62A, 62B can be set to a color different from the mounting surface FL of the workpiece 48A, and the images 62AaI, 62BaI of the outer edge portions of the mark members 62A, 62B can be used as positional references, and the positions of the gripping centers of the tools 8Bb, 8Bc can be found from the positions of the images 62AaI, 62BaI of the edge portions.

Furthermore, the robot device 2 of the above embodiment is equipped with a finger gripping force detection unit 13D and a gap detection unit 13B that detects the gap between the fingers 38A, 38B. For this reason, for example, in FIG. 4(A), when there is nothing between the tools 8b, 8c and the fingers 38A, 38B are closing the tools 8b, 8c, the finger gripping force increases when the tools 8b, 8c come into contact, and the hand control unit 13A can recognize that the tools 8b, 8c have come into contact. In this state, the gap between the fingers 38A, 38B is stored as an initial value, and when the fingers 38A, 38B are subsequently opened to open the tools 8b, 8c, the gap between the tools 8b, 8c can be found by subtracting the initial value from the gap between the fingers 38A, 38B.

As a result, as shown in FIG. 8(C), it is possible to detect the distance d between the tools 8Bb, 8Bc when the shaft portion 48Aa of the workpiece 48A is gripped by the tools 8Bb, 8Bc at a position of height Z1 from the mounting surface FL, and the distance d' between the tools 8Bb, 8Bc when the shaft portion 48Aa is gripped by the tools 8Bb, 8Bc at a position of height Z2 from the mounting surface FL. From the distances d, d', the hand control unit 13A or main control unit 11B can recognize the diameter (size) and/or its distribution (shape), etc. of the shaft portion 48Aa of the workpiece gripped by the tools 8Bb, 8Bc.

Furthermore, by using the detection value of the interval detection unit 13B, it is also possible to measure the length of the workpiece 48 or object gripped by the tools 8Bb, 8Bc, etc. At this time, in order to measure the length more accurately, the detection value of the interval detection unit 13B may be calibrated by gripping a workpiece of known length in advance, such as a gauge block, with the tools 8Bb, 8Bc, etc. In this way, by using the method of gripping the workpiece 48 with the tools 8Bb, 8Bc, etc. of this embodiment, it is possible to have a function for measuring the length of an object such as the workpiece 48. Furthermore, by using the detection value of the interval detection unit 13B, it is also possible to determine whether the workpiece is gripped by the tools 8Bb, 8Bc, etc., or whether the workpiece is not gripped by the tools 8Bb, 8Bc, etc.

Furthermore, the interval between the mark images MAI, MBI of the mark members 62A, 62B captured by the imaging device 22 when the workpiece is gripped by the tip of the tool or when the tip of the tool is closed may be used. In this case, as an example, the intermediate value between the interval between the mark images when one workpiece is gripped by the tool and the interval between the mark images when the tip of the tool is closed is stored in advance as a reference value. Then, when the interval between the mark images is narrower than the reference value, the main control unit 11B can determine that the tool is in a non-gripping state where the workpiece is not being gripped by the tool, and when the interval between the mark images is wider than the reference value, it can determine that the tool is in a gripping state where the workpiece is being gripped by the tool.

Next, as shown in FIG. 9(A), assume that there are multiple workpieces 48 between tools 8Cb, 8Cc held by the robot hand 6 in FIG. 1(A). In this case, as shown by the arrows, when tools 8Cb, 8Cc are closed, there is a risk that two workpieces 48, for example at positions P1 and P2, may be gripped between tools 8Cb, 8Cc. FIG. 9(B) is a side view of the two cases in FIG. 9(A). It can be seen from FIG. 9(B) that the two workpieces 48 face in opposite directions and are overlapped in the height direction relative to the placement surface between tools 8Cb, 8Cc. In this case, even if tool 8C is moved by the robot hand 6, it may be difficult to accurately place the two workpieces 48 in a storage box (not shown) or the like.

In order to prevent multiple workpieces 48 from being mistakenly gripped by a tool in this way, as an example, as shown in FIG. 9(C), a pin B2 may be provided at one of the tips of tools 8Cb, 8Cc, and a guide through hole 8Cf for passing pin B2 may be provided at the other. FIG. 9(D) is a side view of FIG. 9(C). In this case, when tools 8Cb, 8Cc are closed, pin B2 fits into through hole 8Cf, and another workpiece 48 is prevented from being pinched between tools 8Cb, 8Cc by pin B2. In other words, because the shape of the tips of tools 8Cb, 8Cc is such that they cannot grip multiple workpieces 48, only one workpiece 48 can be gripped between tools 8Cb, 8Cc.

Also, as shown by arrow E1 in FIG. 9(E), the tips of tools 8Db, 8Dc may be formed into a cone shape or a triangular plate shape with a gradually narrowing width. In this case, the distance between tools 8Db, 8Dc when one workpiece 48 is held by tools 8Db, 8Dc is de1. In contrast, as shown by arrow E2 in FIG. 9(E), when two workpieces 48 are held between tools 8Db, 8Dc, the distance de2 between tools 8Db, 8Dc is greater than distance de1, so that the hand control unit 13F, etc., can recognize that multiple workpieces 48 are held between tools 8Db, 8Dc.

9E, the tools 8Eb and 8Ec may be provided at their tips with notches 8Ed and 8Ee into which one workpiece 48 can be accommodated, so that the tool 8E can be prevented from gripping a plurality of workpieces 48.
1C, the operating unit 32 may be composed of a frame 32b and an operating body 32a that opens and closes the fingers 38A and 38B within the frame 32b. A load sensor 32d (e.g., a load cell, a pressure sensor, or a weight meter such as a strain gauge) may be provided between the frame 32b and the bottom surface of the operating body 32a, and a preload may be applied to the load sensor 32d side from the top of the frame 32b by an elastic member 32c such as a compression coil spring. In this case, the load detected by the load sensor 32d changes depending on the weight of the workpiece held by the tools 8b and 8c, so the weight of the workpiece held by the tools 8b and 8c may be obtained from the detection value of the load sensor 32d. The load sensor 32d may be provided, for example, at the tip of the tools 8b and 8c.

Second Embodiment
The second embodiment will be described with reference to Figures 10 and 11. In Figures 10 and 11, parts corresponding to those in Figures 1(A), 4(A) and 4(B) are given the same reference numerals and detailed description thereof will be omitted.
Fig. 10 shows a robot device 2A of this embodiment. In Fig. 10, the robot device 2A includes a robot main body 4, a robot hand 6 joined to the robot main body 4, a control device 10 that controls the operation of the robot device 2A, and a control device 12 that controls the operation of the robot hand 6.

The robot hand 6 has the same configuration as the robot hand 6 in FIG. 1(A), but is different in that the imaging device 22 is not provided on the side of the robot hand 6 in FIG. 10. That is, the tools 8b and 8c are fixed to the fingers 38A and 38B of the robot hand 6, and the tools 8b and 8c are opened and closed by the fingers 38A and 38B, so that the tools 8b and 8c grip the workpiece 48 (here, a bolt). In addition, imaging devices 22A and 22B for imaging an area including the tip of the tool 8 held by the robot hand 6 are installed on the ceiling or side wall of the room in which the robot device 2A is installed. In FIG. 10, two imaging devices 22A and 22B are installed, but three or more imaging devices may be installed. Furthermore, for example, if there is a position where the entire range in which the tips of the tools 8b and 8c of the robot hand 6 move can be viewed, only one imaging device may be installed at that position.

As an example, the imaging devices 22A and 22B are normal cameras, and the focal positions of the imaging devices 22A and 22B are set, for example, near the placement surface FL of the workpiece 48, and the focal depth is set relatively wide. Note that a three-dimensional camera (stereo camera) may be used as the imaging devices 22A and 22B. In the image processing unit 11D (see FIG. 3) of the control device 10 of this embodiment, the imaging signals of the imaging devices 22A and 22B are processed to calculate the positional deviation amount between the tools 8b and 8c and the workpiece 48. The rest of the configuration is the same as in the first embodiment, and the mark members 62A and 62B are fixed to the upper surfaces of the tips of the tools 8b and 8c held by the robot hand 6, and the marks MA and MB are formed on the mark members 62A and 62B.

In the robot device 2A of this embodiment, when the tools 8b and 8c held by the robot hand 6 grip the workpiece 48 on the placement surface FL, an image from an imaging device (here, imaging device 22B) capturing images of the workpiece 48 to be gripped and the tips of the tools 8b and 8c is used. FIG. 11 shows an example of an image 22BG captured by imaging device 22B. In FIG. 11, an image 6I of the robot hand 6, images 8bI and 8cI of the tools, images 62AI and 62BI of the mark members 62A and 62B at the tips of the images 8bI and 8cI, an image 48I of the workpiece at the position P1 to be gripped, and an image 48I of another workpiece are formed in the image 22BG. At this time, as an example, the tips of the tools 8b and 8c are in contact with the placement surface FL of the workpiece 48.

In this embodiment, the image processing unit 11D also determines the positions of the mark images MAI, MBI in the mark member images 62AI, 62BI on the coordinate system (x, y) of the imaging device 22B, and determines the position of the grip center image 58CI of the tools 8b, 8c from these positions. Furthermore, the image processing unit 11D determines the position of the center 48cI of the image 48I of the workpiece to be gripped, and determines the intervals δx, δy in the x and y directions between this center 48cI and the grip center image 58CI. Furthermore, the intervals δx, δy are converted to intervals on the robot coordinate system (X, Y, Z) by dividing the intervals δx, δy by the known magnification of the imaging device 22B. Thereafter, the robot arms 18A-18C, etc. are used to drive the robot hand 6 so as to offset the interval, thereby moving the tools 8b, 8c so as to sandwich the workpiece 48, and in this state the workpiece 48 can be gripped by the tools 8b, 8c.

In this embodiment as well, the marks MA, MB of the mark members 62A, 62B provided on the tools 8b, 8c are detected, and the gripping centers of the tools 8b, 8c are obtained from the detection results, thereby enabling efficient and highly accurate alignment of the workpiece 48 and the robot hand 6A (tools 8b, 8c). In addition, when the rotation angle between the workpiece image 48I and the tool 8 image is large, the robot hand 6A may be rotated to correct the rotation angle.
In this embodiment, calibration may be performed in advance between the coordinate system (x, y) of the imaging devices 22A and 22B and the robot coordinate system (X, Y, Z). In this case, calibration may be performed so that the x-axis and y-axis of the coordinate system (x, y) are parallel to the X-axis and Y-axis of the robot coordinate system, respectively.

Next, in the above-described embodiment, the tools 8b and 8c are fixed to the fingers 38A and 38B, but as shown in a modified robot hand 6A in Fig. 12(A), the fingers 38E and 38F may open and close the movable parts 9b and 9c of the tweezers 9 (tweezers-type tool) to grip a workpiece 48 with the tweezers 9. Note that in Figs. 12(A) to (C), parts corresponding to Figs. 1 and 2(A) to (C) are given the same or similar reference numerals, and detailed description thereof will be omitted.
The tweezers 9 may be any type, such as commercially available, custom-made, self-made, or a modified version of a commercially available product. The tweezers 9 have a pair of flexible, elongated, flat movable parts (also called gripping parts, contact parts, or opening/closing parts) 9b and 9c that are manufactured to be openable and closable about a fulcrum 9a. By pressing a force point approximately midway between the fulcrum 9a of the tweezers 9 and the tips (points of action) of the movable parts 9b and 9c inward with the fingers 38E and 38F, the movable parts 9b and 9c close to grip or support a workpiece 48 between the tips. As an example, the fulcrum 9a of the tweezers 9 is held by a fixed part 42 of the robot hand 6, which will be described later.

Figure 12(A) shows a robot hand 6A of this modified example, in which the robot hand 6A is joined to the lower end of the tip 20A of the robot arm 18C so that the opening and closing direction of the movable parts 9b, 9c of the tweezers 9 it holds is the X direction. In Figure 12(A), the robot hand 6 comprises a hand main body 28 joined to the lower end of the tip 20A, a tweezers holding part 26 connected to an operating part 32 of the hand main body 28 and holding the tweezers 9, and a pair of fingers 38E, 38F opened and closed in the X direction by the operating part 32. The movable parts 9b, 9c of the tweezers 9 held by the tweezers holding part 26 are opened and closed in the X direction by the fingers 38E, 38F. In other words, the movable parts 9b, 9c of the tweezers 9 are opened and closed following the movement of the fingers 38E, 38F.

The configuration of the hand body 28 is the same as that of the first embodiment, and the imaging device 22 is fixed to the frame 30 of the hand body 28 via a support member 50. The tweezers holding part 26 has a connecting part 38 connected to the -Y direction side of the operating part 32 via a connecting part 36, a rotating part 40 fixed to the connecting part 38 so as to be rotatable within a predetermined range (for example, several degrees) around an axis parallel to the Z axis, and a fixed part 42 fixed to the bottom surface of the rotating part 40. The rotating part 40 is provided with a stopper 54 for determining the rotation angle. The fixed part 42 has a fixed plate fixed to the bottom surface of the rotating part 40 and a pressing plate that clamps the fulcrum 9a of the tweezers 9 against the fixed plate. The pressing plate is fixed to the fixed plate with a plurality of bolts B1, so that the fulcrum 9a of the tweezers 9 can be stably held by the fixed part 42. At this time, the tips of the movable parts 9b and 9c of the tweezers 9 protrude in the +Y direction of the fingers 38E and 38F through the gap between the fingers 38E and 38F. The method of holding the tweezers 9 is arbitrary.

The imaging device 22 is also arranged slightly diagonally above (in the +Y and +Z directions) the movable parts 9b and 9c of the tweezers 9. As shown in FIG. 12C, the images 9bI and 9cI of the movable parts 9b and 9c of the tweezers 9 are set to be located on the -y direction side of the origin 58B in the image 22G of the imaging device 22. Even in this modified example, a suspension mechanism 46 (shock absorber mechanism) is configured that includes the linear slider 34 and the elastic member 44 and supports the movable part (the part including the operating part 32, the fingers 38E and 38F, the tweezers holding part 26, and the tweezers 9) in the Z direction so that it can be elastically displaced relative to the frame 30. The detection result of the proximity sensor 56 can detect the relative displacement of the movable part (the operating part 32, the tweezers 9, etc.) relative to the frame 30, and this detection result can detect whether the tip of the tweezers 9 has come into contact with the placement surface FL of the workpiece 48.

The connecting portion 36 and the connecting portion 38 are connected so as to be detachable by, for example, a permanent magnet provided inside. The connecting portion 38 of the tweezers holding portion 26 is provided with a groove 38Aa. The tweezers holding portion 26 can be installed (stored) on a tool stand (not shown) by aligning the groove 38Aa with the base of the tool stand and installing the second connecting portion 38 on the base. At this time, multiple tweezers holding portions 26 with tweezers of different shapes fixed to the fixing portion 42 may be prepared in advance and installed on the tool stand. Then, by simply moving the robot hand 6 to detach and connect the connecting portion 36 and the connecting portion 40, the tweezers held by the robot hand 6 (the tweezers holding portion 26) can be easily replaced with tweezers appropriate for the object.

12(B) is a front view of Fig. 12(A) as viewed in the -Y direction, and in Fig. 12(B), a pair of fingers 38E, 38F open and close movable parts 9b, 9c of tweezers 9 in the opening and closing direction (here, X direction) by operating part 32. Also, hemispherical protrusions 38Ea, 38Fa, for example, are provided on the inside of fingers 38E, 38F, and protrusions 38Ea, 38F come into contact with the force points (or points between fulcrum 9a and action points 9d, 9e) of movable parts 9b, 9c of tweezers 9 to push movable parts 9b, 9c in the +X and -X directions, thereby closing movable parts 9b, 9c and allowing the tips of movable parts 9b, 9c to grasp, pinch, or support an object. Thereafter, by moving the protrusions 38Ea, 38Fa in the -X and +X directions, the movable parts 9b, 9c open, and the tweezers 9 can be removed from the object. Note that the operating part 32 may operate the fingers 38E, 38F so that the fingers 38E, 38F move in the X direction independently of each other.

Furthermore, the protrusions 38Ea, 38Fa of the fingers 38E, 38F are formed, for example, from a material with low frictional resistance (such as Teflon (registered trademark)). Therefore, the tweezers 9 held by the tweezers holding part 26 can move (slide) slightly in the Z direction and Y direction relative to the fingers 38E, 38F. Note that in this embodiment, the suspension mechanism 46 displaces the tool including the operating part 32 and the tweezers holding part 26 as a unit, so the protrusions 38Aa, 38Ba may be made of a material with high frictional resistance. Note that it is not necessary to provide the protrusions 38Ea, 38Fa on the inside of the fingers 38E, 38F.

In this modified example, the mark members 62A and 62B on which the marks MA and MB similar to those in the example of FIG. 4A are formed are attached to the tips of the movable parts 9b and 9c of the tweezers 9, and the images of the mark members 62A and 62B are captured by the imaging device 22, and the gripping positions of the centers of the movable parts 9b and 9c can be obtained using the images. In the image 22G of FIG. 12C, the images 62AI and 62BI of the mark members and the images MAI and MBI of the marks are formed, and for example, the center positions (or positions shifted by a predetermined amount) of the images MAI and MBI can be regarded as images of the gripping positions. Then, the robot hand 6A is positioned so that the gripping position is aligned with the center of the workpiece 48, and the fingers 38E and 38F are closed, so that the movable parts 9b and 9c of the tweezers 9 can grip the workpiece 48 efficiently and accurately. In this modified example, if the shape of the movable parts 9b and 9c of the tweezers 9 is partly distinguishable, it is possible to use the distinguishable part instead of the marks MA and MB or the mark members 62A and 62B.

Next, the robot hand 6, 6A in the above-described embodiment can be joined to the robot arm of a robot device of any configuration, such as a vertical multi-joint type robot device 2, a horizontal multi-joint type (SCARA type), a parallel link type, or an orthogonal type. Furthermore, the robot hand 6, 6A is not limited to the above-described robot device, but can also be applied to various robot devices (e.g., assembly robots, human-collaborative robots, etc.) that have a joint structure similar to that of other robot arms.

The present specification also describes the following aspects of the invention.
1) A robot device comprising: a robot hand provided with a plurality of finger portions, an arm portion that drives the robot hand, a plurality of gripping portions capable of gripping an object in response to the plurality of finger portions, at least one identification portion provided in at least one of the plurality of gripping portions, and a control portion that drives the robot hand via the arm portion using position information of the identification portion to position the plurality of gripping portions with respect to the object. 2) The robot device described in 1, wherein the plurality of gripping portions are connected to the plurality of finger portions so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close. 3) The robot device described in 1, wherein the plurality of gripping portions are a plurality of movable portions that are supported so as to protrude in a direction intersecting a direction in which the plurality of finger portions open and close, and are opened and closed by the opening and closing motion of the plurality of finger portions. 4) The robot device according to any one of 1 to 3, further comprising an imaging device that images the identifier, and the control unit determines position information of the identifier from an imaging result of the imaging device and determines position information related to gripping positions of the multiple grippers from the position information. 5) The robot device according to 4, further comprising: an imaging device that images the identifier, and an image of at least a portion of the object, within a same field of view of the imaging device, and the control unit determines difference information between gripping positions of the multiple grippers and a gripped position of the object using position information of at least a portion of the image of the identifier and at least a portion of the image of the object. 6) The robot device according to 4 or 5, further comprising: a plurality of the grippers, each of which has two identifiers, and is capable of capturing images of the two identifiers within a same field of view of the imaging device, and the control unit determines distance information between the two identifiers from position information of the images of the two identifiers, and uses the distance information to distinguish between a gripping state in which the object is gripped between the multiple grippers and a non-gripped state in which the object is not gripped between the multiple grippers.

7) The robot device according to any one of 4 to 6, wherein the plurality of gripping sections have two of the identification sections and are capable of capturing images of the two identification sections within the same field of view of the imaging device, and the control section determines distance information of the two identification sections from position information of the images of the two identification sections and determines the size of the gripped position of the object gripped between the plurality of gripping sections using the distance information. 8) The robot device according to any one of 1 to 7, wherein the plurality of gripping sections are shaped to be able to grip only one of the objects. 9) The robot device according to any one of 1 to 8, wherein the plurality of finger sections and the plurality of gripping sections each have two. 10) The robot device according to any one of 1 to 9, further comprising a weighing scale that determines weight information of the object gripped by the plurality of gripping sections. 11) The robot device according to any one of 1 to 10, wherein the identification section is a member on which an identification mark is formed. 12) The robot device according to 11, wherein the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed. 13) The robot device according to any one of 1 to 10, in which the identification unit is a part of the gripping unit. 14) A method for controlling a robot device including a robot hand provided with multiple finger units and an arm unit for driving the robot hand, the method including: moving multiple gripping units capable of gripping an object relative to the multiple finger units; storing position information of at least one identification unit of at least one of the multiple gripping units; and using the position information of the identification unit to drive the robot hand via the arm unit to position the multiple gripping units relative to the object. 15) The method for controlling a robot device according to 14, in which the multiple gripping units are connected to the multiple finger units so as to protrude in a direction intersecting a direction in which the multiple finger units open and close. 16) The method for controlling a robot device according to 14, in which the multiple gripping units are supported so as to protrude in a direction intersecting a direction in which the multiple finger units open and close, and are multiple movable units that are opened and closed by the opening and closing operation of the multiple finger units.

17) A method for controlling a robot device described in any one of claims 14 to 16, comprising capturing an image of the identification part with an imaging device, determining position information of the identification part from the imaging results of the imaging device, and determining position information regarding the gripping positions of the multiple gripping parts from the position information. 18) A method for controlling a robot device described in 17, wherein capturing an image of the identification part with the imaging device includes capturing an image of at least a portion of the identification part and an image of at least a portion of the object within the same field of view of the imaging device, and comprising determining difference information between the gripping positions of the multiple gripping parts and the gripped position of the object using position information of the image of at least a portion of the identification part and the image of at least a portion of the object. 19) The method for controlling a robot device according to any one of 17 to 18, wherein the plurality of gripping units have two of the identifiers, and capturing an image of the identifiers with the imaging device includes capturing images of the two identifiers within a same field of view of the imaging device, determining distance information of the two identifiers from position information of the images of the two identifiers, and using the distance information to distinguish between a gripping state in which the object is gripped between the plurality of gripping units and a non-gripped state in which the object is not gripped between the plurality of gripping units. 20) The method for controlling a robot device according to any one of 17 to 19, wherein the plurality of gripping units have two identifiers, and capturing an image of the identifiers with the imaging device includes capturing images of the two identifiers within a same field of view of the imaging device, determining distance information of the two identifiers from position information of the images of the two identifiers, and using the distance information to determine a size of a gripped position of the object gripped between the plurality of gripping units. 21) The method for controlling a robot device according to any one of claims 14 to 20, in which the plurality of gripping parts are shaped to grip only one of the objects. 22) The method for controlling a robot device according to any one of claims 14 to 21, in which the plurality of finger parts and the plurality of gripping parts are each two in number. 23) The method for controlling a robot device according to any one of claims 14 to 22, including determining weight information of the object gripped by the plurality of gripping parts. 24) The method for controlling a robot device according to any one of claims 14 to 23, in which the identification part is a member on which an identification mark is formed. 25) The method for controlling a robot device according to claim 24, in which the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed. 26) The method for controlling a robot device according to any one of claims 14 to 23, in which the identification part is shaped as a part of the gripping part.

2, 2A...Robot device, 6, 6A...Robot hand, 8b, 8c...Tool, 9...Tweezers, 10...Control device for robot device, 12...Control device for robot hand, 18A-18C...Robot arm, 16...Rotary part, 22, 22A, 22B...Imaging device, 28...Hand main body, 32...Operating part, 38A, 38B...Fingers, 46...Suspension mechanism, 48...Workpiece

Claims (26)

a robot hand having a plurality of finger portions;
An arm unit that drives the robot hand;
A plurality of gripping portions capable of gripping an object in response to the plurality of finger portions;
At least one identification portion included in at least one of the plurality of grip portions;
a control unit that drives the robot hand via the arm unit using position information of the identification unit to position the plurality of gripping units with respect to the object;
A robot device comprising: The robot device according to claim 1, wherein the multiple gripping parts are connected to the multiple finger parts so as to protrude in a direction intersecting the direction in which the multiple finger parts open and close. The robot device according to claim 1, wherein the multiple gripping parts are supported so as to protrude in a direction intersecting the direction in which the multiple finger parts open and close, and are multiple movable parts that are opened and closed by the opening and closing movements of the multiple finger parts. an imaging device for imaging the identification part,
The robot device according to claim 1 , wherein the control unit obtains position information of the recognition unit from an imaging result of the imaging device, and obtains position information relating to gripping positions of the plurality of gripping units from the position information. An image of at least a part of the identification part and an image of at least a part of the object can be captured within the same field of view of the imaging device,
5. The robot device according to claim 4, wherein the control unit calculates difference information between gripping positions of the plurality of grippers and a gripped position of the object, using position information of at least a portion of an image of the identification unit and at least a portion of an image of the object. The plurality of gripping portions each have two of the identification portions,
The imaging device is capable of capturing images of the two identification parts within the same field of view,
5. The robot device according to claim 4, wherein the control unit determines distance information between the two identification parts from position information of the images of the two identification parts, and uses the distance information to distinguish between a gripping state in which the object is gripped between the plurality of gripping parts and a non-gripping state in which the object is not gripped between the plurality of gripping parts. The plurality of gripping portions each have two of the identification portions,
The imaging device is capable of capturing images of the two identification parts within the same field of view,
5. The robot device according to claim 4, wherein the control unit determines distance information between the two identification parts from position information of the images of the two identification parts, and determines a size of a grasped position of the object grasped between the plurality of gripping parts by using the distance information. The robot device according to claim 1, wherein the multiple gripping parts are shaped to be able to grip only one of the objects. The robot device according to claim 1, wherein the number of the finger portions and the number of the gripping portions are each two. The robot device according to claim 1, further comprising a weight scale that determines weight information of the object being held by the multiple grippers. The robot device according to any one of claims 1 to 10, wherein the identification part is a member on which an identification mark is formed. The robot device according to claim 11, wherein the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed. The robot device according to any one of claims 1 to 10, wherein the identification part is shaped as a part of the grip part. A method for controlling a robot device including a robot hand provided with a plurality of finger units and an arm unit that drives the robot hand, comprising:
A plurality of gripping portions capable of gripping an object are caused to move relative to the plurality of finger portions;
storing position information of at least one identifier included in at least one of the plurality of gripping portions;
Using the position information of the identification unit, drive the robot hand via the arm unit to position the plurality of gripping units with respect to the object;
A method for controlling a robot device comprising: The method for controlling a robot device according to claim 14, wherein the plurality of gripping parts are connected to the plurality of finger parts so as to protrude in a direction intersecting the direction in which the plurality of finger parts open and close. The method for controlling a robot device according to claim 14, wherein the plurality of gripping parts are supported so as to protrude in a direction intersecting the direction in which the plurality of finger parts open and close, and are a plurality of movable parts that are opened and closed by the opening and closing movements of the plurality of finger parts. capturing an image of the identification part with an imaging device;
determining position information of the identification part from an imaging result of the imaging device;
The method for controlling a robot apparatus according to claim 14, further comprising: determining position information relating to gripping positions of a plurality of said gripping parts from said position information. The imaging of the identification part by the imaging device includes:
capturing an image of at least a part of the identification part and an image of at least a part of the object within the same field of view of the imaging device;
18. The method for controlling a robot device according to claim 17, further comprising: calculating difference information between gripping positions of the plurality of gripping units and a gripped position of the object, using position information of at least a portion of an image of the identification unit and at least a portion of an image of the object. The plurality of gripping portions each have two of the identification portions,
capturing an image of the identification part with the imaging device includes capturing images of two of the identification parts within the same field of view of the imaging device;
18. The control method for a robot device according to claim 17, comprising: determining distance information between the two identification parts from position information of images of the two identification parts; and using the distance information to distinguish between a gripping state in which the object is gripped between a plurality of the gripping parts and a non-gripping state in which the object is not gripped between the plurality of the gripping parts. The plurality of gripping portions each have two of the identification portions,
capturing an image of the identification part with the imaging device includes capturing images of two of the identification parts within the same field of view of the imaging device;
18. The method for controlling a robot device according to claim 17, further comprising: determining distance information between the two identification parts from position information of the images of the two identification parts; and determining a size of a grasped position of the object grasped between the plurality of gripping parts using the distance information. The method for controlling a robot device according to claim 14, wherein the multiple gripping parts are shaped to be able to grip only one of the objects. The method for controlling a robot device according to claim 14, wherein the number of finger portions and the number of gripping portions are each two. The method for controlling a robot device according to claim 14, further comprising determining weight information of the object being gripped by the multiple grippers. The method for controlling a robot device according to any one of claims 14 to 23, wherein the identification unit is a member on which an identification mark is formed. The method for controlling a robot device according to claim 24, wherein the color of the member on which the identification mark is formed is different from the color of the placement surface on which the object is placed. The method for controlling a robot device according to any one of claims 14 to 23, wherein the identification part is shaped as a part of the grip part.

Images