JPH06203166A - Multi-dimensional position measurement, control device, and learning method - Google Patents

Abstract

(57) [Summary] [Object] The present invention relates to a device for measuring and controlling the position of a specific object, for example, placed in a three-dimensional space, and an object from an image processing device or a one-dimensional distance measuring device. The position is measured, or an object placed in the three-dimensional space is grasped. [Structure] Set the position of the measurement target grasped by the robot etc. to 3
The image information (X, Y, α, S) obtained by measuring with an image processing device such as a TV camera or more, and inputting the acquired image information (X, Y, α, S) into a learning device such as a neural network, For example, 6-dimensional position information (X, Y, Z, of the measurement object that can be calculated by forward kinematics from the arm length information)
Learn as α, β, γ) as teacher data. In addition, a plurality of markers are attached to the object to be grasped, and feature quantities such as the position of the center of gravity (X, Y) and area (S) of the markers on the screen are input to the neural network, and the corresponding robot is supported. Joint angle, hand effector position, posture angle (X, Y, Z, α,
β, γ) is learned as teacher data, and an object placed at an arbitrary position is grasped by the learning result.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional position measurement and control device for a position of a specific object placed in a three-dimensional space, or a learning method for the three-dimensional measurement and control device.

In recent years, with the rapid development of computers and sensor technology, various sensors, for example, vision sensors of image processing devices such as video cameras are used to recognize the external environment, and based on the recognized information, Expectations are rising for "intelligent robots" that can judge and act on their own.

In particular, similar to human vision, a visual control technique for recognizing the surrounding environment by using an optical sensor such as the above vision sensor to grasp and assemble an object is
One of the most important technologies for improving robot intelligence
Is one.

[0004]

2. Description of the Related Art FIG. 7 is a diagram for explaining a conventional method for measuring a three-dimensional position. When measuring the three-dimensional position (x, y, z) of an object placed in the three-dimensional space, in the conventional method, as shown in the figure, measurement is performed from multiple observation points (O _a , 0 _b ). It is common to measure the linear distance (x, y, z) to the object (P) and then the principle of triangulation.

As shown, the observation lens node
And a pair of observation optical systems whose distance from the observation surface is f,
The optical axis should be vertical and parallel to the line connecting the nodes.
, An image of each of the reference points P on the respective observation planes, arranged at intervals of L
But each (x_a, y_a), (x_b, y _b)
Then, the coordinate value (x, y, z) of the control point P is x = x_aL / x_a-x_b y = fL / x_a-x_b z = y_aL / x_a-x_b, y_bL / x_a-x_b Given in.

[0006]

However, in the above-mentioned conventional method, since the distance is measured by using the reflection of light or the like, it is difficult to determine where the object to be measured is, so-called, The method of determining corresponding points has become a major issue.

Therefore, another method is used to find out where the target object is, for example, the feature points of the object are autocorrelated with neighboring image information, and the object having a large autocorrelation value is targeted. It was necessary to know in advance, such as

Further, in the case of a device utilizing reflection or the like, when the posture of the object changes, reflected light cannot be obtained, and even the true posture of the target object cannot be detected or the distance to the measurement target is erroneously measured. There were problems such as things.

In view of the above-mentioned conventional drawbacks, the present invention provides a robot or the like with an object to be measured, and determines the three-dimensional coordinate value (or in each dimension) of the object gripped by the robot from the joint angle and arm length of the robot. The 6-dimensional coordinate values including the posture angles α, β, γ of) are obtained, and the object or a plurality of markers attached to the object is captured using a vision sensor such as a television camera, Image information obtained from the vision sensor is measured, and the measurement information {for example, 2
The dimensional coordinates (X, Y) and the angle α of the robot's inertial axis,
The learning device such as a neural network is made to learn the relationship between the object area S} and the three-dimensional coordinate value or the six-dimensional coordinate value of the object held by the robot, and the television camera or the like. A device that measures the orientation of the target object from the image information obtained from the vision sensor, or
An object of the present invention is to provide a control device that causes a robot to grip a target object placed at an arbitrary position, or a learning method thereof.

[0010]

FIG. 1 is a diagram showing an embodiment of the present invention, FIG. 2 is a diagram for explaining a learning method by a neural network of the present invention, and FIG. 3 is a diagram of the present invention. FIG. 5 is a diagram showing an example of another learning device, FIG. 4 is a diagram showing another embodiment of the present invention, and FIG. 5 is a diagram illustrating marker feature amounts on a vision sensor observation surface according to the present invention. FIG. 6 is a diagram showing a method for approaching a grasped object according to the present invention. The above problems can be solved by the three-dimensional position measurement _and control device and the learning method thereof configured as follows.

(1) Object 1 measured by the robot 1 or the like
Image processing device such as three or more TV cameras at position 0
2, or image information acquired by measuring with a one-dimensional distance measuring device 2a using a laser or the like, or information such as an object position obtained from the image information, or distance measurement information is learned by a neural network or the like. Type in device 3,
The multi-dimensional position information (X, Y, Z, α, of the measurement object that can be calculated from the angle of each joint of the robot 2 and the arm length information)
β, γ) is learned as teacher data, and the learned device 3 is used to input image information of the measurement object 10 or distance measurement information, or
Configured to measure multidimensional position of 0.

(2) From the image information from the image processing device 2 such as a television camera, using the above multidimensional position measuring device,
The X coordinate, the Y coordinate, the inclination angle (α), the area (S), etc. of the center of gravity of a certain measurement object 10 on each image are input to the learning device 3, and the input measurement object is input. object
The joint angles of the robot 1 etc. corresponding to the 10 positions (θ
1, θ2, 〜) and multidimensional position information (X, Y, Z, α, β, γ) that can be calculated from the arm length are used as teacher data.

(3) Using the multidimensional position measuring device, the hand position of the robot 1 is randomly set within the visual field of the image processing device 2 group such as the TV camera, and the hand position is set to the set hand position. The corresponding joint angle (θ
1, θ2, ~) and multidimensional position information (X, Y, Z, α, β, γ) that can be calculated from the arm length are used as teacher data.

(4) When the joint angles of the robot 1 are set by a predetermined number of divisions using the multidimensional position measuring device, the hand position obtained by the set joint angle is the television camera. Each joint angle is divided and set so as to be within the visual field of the image processing device 2 group such as, and the joint angle (θ1, θ2,
~) And multidimensional position information (X,
Y, Z, α, β, γ) are used as teacher data.

(5) At least one of the three or more image processing devices 2 such as television cameras is installed in a direction to maintain a predetermined angle with the image processing device 2 such as another television camera. To do.

(6) Object to be gripped by the robot 1 or the like 1
A plurality of markers are attached to 1, and the positions of the attached markers are adjusted by three or more image processing devices such as TV cameras 2, or a one-dimensional distance measuring device 2a using a laser or the like
And the like, and the acquired image information, or information such as an object position obtained from the image information, or distance measurement information is input to a learning device 3 such as a neural network,
The three-dimensional position information (X, Y, Z, α, of the measurement object that can be calculated from the angle of each joint of the robot 2 and the arm length information)
β, γ) is learned as teacher data, and the learning object 3 learned is used to grasp the grasped object placed at an arbitrary position and posture.
Image information of 11 or distance measurement information or is input, and the object to be grasped 11 is grasped.

(7) When gripping an object placed in any of the above positions and postures, the learning device 3 causes the learning device 3 to set a target point at a position separated by a predetermined distance d from the target position of the gripped object 11. Then, the hand effector 12 of the robot 1 is controlled so as to approach the grasped object 11 after reaching the target point.

[0018]

In other words, in the present invention, the object to be measured is held by the robot or the like, and the three-dimensional coordinate value (or each value) of the object grasped by the robot is determined from the joint angles θ1, θ2, ... Of the robot and the arm length. 6-dimensional coordinate values including posture angles α, β, γ in three dimensions) are obtained from forward kinematics calculation (forward kinematics), and the object is captured using a vision sensor such as a television camera, The image information obtained from the vision sensor is measured, and the measurement information, for example, the two-dimensional coordinates (X, Y) of the center of gravity of the object, the hand direction, for example, the direction α of the principal axis of inertia, and the area S of the object are measured. And a learning device such as a neural network to learn the relationship between the three-dimensional coordinate value of the object held by the robot or the six-dimensional coordinate,
From the image information two-dimensional coordinates (X, Y) obtained from the vision sensor of the television camera, the hand direction α, the object area S, etc., the posture of the target object, that is, the three-dimensional direction of the object The coordinate values or the six-dimensional coordinate values including the posture angles α, β, γ in each dimension are measured.

In particular, for example, two or more TV cameras are installed in front of the object, and one or more TV cameras are installed in a direction perpendicular to them, for example, in the depth direction (X-axis direction). By increasing the image information, it is possible to improve the measurement accuracy of, for example, the three-dimensional position of the measurement target object.

Therefore, according to the present invention, it is possible to accurately measure the position of the object to be measured within the visual field of the television camera, and the posture change, which is difficult with the conventional method, can be obtained in the direction of the main axis of the image. (The above α _A ~) and the area (the above S
_A ~) can be used for detection.

Further, since the learning device, for example, the neural network has a function of interpolating the points other than the points that were not taught, it is possible to obtain an appropriate representative point without creating teacher data for the entire space. The position of the target object can be measured only by teaching.

In the method according to the present invention, the position is measured by learning the correspondence between the three-dimensional position of the object to be measured and the image information.
The learning device is not limited to the neural network, and other learning device that can learn the three-dimensional position may be used at the first learning.

Further, when the space to be measured becomes wider, the size of the neural network becomes too large for learning with one neural network. Therefore, by dividing the coordinate space of the hand into a plurality of pieces of a predetermined size and learning them in different neural networks, the neural network can be downsized, and the learning time can be shortened. By doing so, the range covered by one neural network is narrowed, so that the interpolation accuracy is improved even if the neural network is small.

Further, one neural network includes
Teaching the whole space sparsely and teaching other parts of the neural network only a certain part improves the measurement accuracy in a specific place, and by using multiple neural networks, It also becomes possible to improve.

When the environment around the robot (manipulator) and the work object are known to some extent, such as assembly work, a marker for identifying each of them is attached to the surface of the work object based on its shape, color and the like.

Next, the hand effector of the robot is fixed to the work space while holding the work object, and the joint angle of the robot operating as a manipulator, or the joint angle or arm From the length etc., the position of the hand effector (X, Y,
Z) and attitude angle (α, β, γ) are collected as teacher data, and at the same time, the position of the center of gravity of the marker on the vision sensor observation surface, the area, the main axis direction, etc. are observed by the vision sensor.

The above work is performed by changing the posture of the hand effector at some points in the space, and the feature amount of the obtained marker, the joint angle of the robot, or the position and posture of the hand effector ( The relationship of (X, Y, Z, α, β, γ) is learned by a learning device such as the hierarchical neural network shown in FIG.

Then, the barycentric position and area of the marker on the observation surface of the vision sensor, the main axis direction, etc. are measured with respect to the work object having an arbitrary position and posture angle in the space,
By inputting these feature quantities into a learning device such as the above-mentioned hierarchical neural network, the joint angle for gripping the work target, or the position and posture angle of the work target can be obtained as output. As a result, the work target can be gripped.

At this time, if the joint angle and the position and posture angle of the work object obtained from the learning device are instructed to the robot, the movement and direction of each arm of the robot cannot be specified. There is a risk of touching the work object or grasping the work object from an unexpected direction.

Therefore, in the present invention, the learning device is used to set a position separated by an arbitrary position d from the front position of the object obtained from the target position and posture angle of the hand effector,
As the first target value, after the hand effector of the robot is positioned at the target point, the object is constantly gripped from the desired direction by moving in parallel to the front direction of the gripped object. become.

[0031]

Embodiments of the present invention will be described in detail below with reference to the drawings. FIG. 1 described above is a diagram showing an embodiment of the present invention.
FIG. 2 is a diagram for explaining a learning method by the neural network of the present invention, FIG. 3 is a diagram showing an example of another learning device of the present invention, and FIG. 4 is another embodiment of the present invention. FIG. 5 is a diagram for explaining the marker feature amount on the vision sensor observation surface according to the present invention, and FIG. 6 is a diagram showing a method for approaching a gripped object according to the present invention.

In the present invention, in a measuring device for measuring the position of a specific object, for example, in a three-dimensional space,
The position of the measurement object 10 held by the robot 1 etc. is measured by the image processing device 2 etc. such as three or more TV cameras, and the acquired image information (X, Y, α, S) and distance measurement information are For example, the 6-dimensional position information (X, Y, Z, α, of the object to be measured, which can be input to the learning device 3 such as a neural network and calculated from the angle of each joint of the robot and the arm length information.
Learn β, γ) as teacher data. In addition, one or more of the image processing devices 2 such as the above television cameras are provided with a means to be installed in a direction substantially perpendicular to the other television cameras 2 or a plurality of markers are attached to the gripping object 11.
The position information of the marker is measured by the image processing device 2 such as a television camera, and the acquired image information and the grasped object 11
Based on the result of learning the information such as the joint angle of the robot 1 grasped by the teacher data with the teacher data, a means for grasping the object 11 placed in a predetermined space is necessary for implementing the present invention. It is a means. The same reference numerals indicate the same objects throughout the drawings.

Hereinafter, referring to FIGS. 1 to 4 and FIGS. 5 and 6, the three-dimensional position measurement of the present invention, the configuration of the control device, the learning operation, the object position measuring method, the object gripping method, etc. will be described in detail. To explain.

First, in FIG. 1, a robot 1 having an arm having three joints and a plurality of television cameras (A, B, ...) 2 are installed outside the robot 1. Each of the television cameras A, B, and 2 has an object 10 held by the robot 1.
, And two-dimensional position coordinates (X _A , Y _A , X _B , Y _B , ...) Of the hand are obtained by image processing.

The two-dimensional position coordinates (X _A , Y _A , X _B , Y _B, ...)
For example, the barycentric position of the obtained image may be used. Furthermore, the direction of the hand, for example, the direction of the principal axis of inertia α _A, α
_B, ~ can also be obtained. Further, as shown in FIG. 1, an object 10 such as a particular rod is held, and the position of its center of gravity is
You may measure the direction of a stick.

At this time, the angle of each joint of the robot 1 can be measured by using an angle encoder or the like (not shown). Also, the arm length of the robot 1 is known when the robot 1 is designed.
From (θ _1, θ _2, θ _3, ~) and the arm length, enter the three-dimensional coordinates (X, Y, Z) of the tip of the robot 1 or the attitude angles α, β, γ on each coordinate axis. And the 6-dimensional coordinates can be obtained by a simple operation (the above-mentioned forward kinematics).

At this time, the object in the depth direction (X-axis direction)
Even if the position of 10 changes, two TV cameras A and B 2
Compared to the case where the position of the object 10 changes in the YZ direction, the image obtained from does not show a large change. Therefore, in order to accurately measure the three-dimensional position by the above method, It is necessary to make image processing highly accurate.

In order to perform image processing with high accuracy, an expensive image processing device and homogenization of illumination (usually, since illumination is accompanied by inhomogeneity, for example, if a shadow of a threshold value or more is generated,
The object 10 becomes smaller and may be erroneously recognized as being present at a distance), which leads to an increase in the size of the device and is not realistic.

Therefore, in the present invention, as shown in FIG. 3, another television camera C 2 is placed at a position forming an angle of approximately 90 degrees with the line connecting the television cameras A and B 2, and this television camera C 2 is used. , The change in the depth direction (X-axis direction) is captured as an image.

As described above, the television cameras A, B,
The above image information captured by C 2 (X _A , Y _A α _A, S _A, X _B , Y _B α
_B, SB _, ...) are input to the neural network 3 as shown in FIG. 2, and the joint angles (θ _1, θ) of the robot 1 are input.
_2, θ _3, ~) and the arm length
Dimensional coordinates (X, Y, Z) or attitude angles α, β, γ on each coordinate axis
Then, using the 6-dimensional coordinates as teacher data, the mutual positional relationship is learned by, for example, the neural network 3.

Various learning devices 3 are conceivable, but in the embodiment shown in FIG. 2, a three-layer hierarchical neural network is taken as an example. As the above-mentioned teacher data, the three-dimensional information (or six-dimensional information) of the object obtained from the forward kinematics of the robot 1 or the joint angle (θ _1, θ _2, θ _3, ...) Is used.

At this time, if the joint coordinate values of the robot 1 are randomly changed and the robot 1 is pointed at, the hand of the robot 1 moves to a random position.
, A plurality of three-dimensional coordinate values are randomly obtained.

In this way, a large number of learning data can be randomly acquired and learning can be performed. After learning, when the gripping target object 10 is presented in the range of the taught space, the neural network 3 outputs the three-dimensional position of the gripping target object 10 or the joint angle. It becomes possible to measure and move the position and posture angle of the hand effector 12 of the robot 1 to a position where an object is gripped.

When the robot 1 grips an object,
A specific example will be described with reference to FIGS. FIGS. 5 and 6 show a hand effector by an articulated robot (manipulator: hereinafter referred to as a robot) 1 of the present invention.
FIG. 12 is a diagram for explaining the operation in the case of performing an object gripping work with 12.

First, in the learning process, as shown in FIG. 1, a plurality of markers having different colors, for example, three markers (circles) in the present embodiment are provided on the surface of the object 11 to be grasped. With the attached and articulated robot 1 gripping the gripped object 11, the hand effector 12 is moved to any position (X, Y,
Z), the attitude angle (α, β, γ), and a plurality of vision sensors, for example, the TV camera 2, measures the barycentric position on the TV camera observation surface of each circle as image information, and at the same time, The joint angles of the articulated robot 1 are also collected at several points by changing the position and posture angle of the hand effector 12, and the barycentric position of the circle (marker) is used as input data, The position (X, Y, Z) and posture angle of the object 10 that can be calculated by forward kinematics from each joint angle of the above-mentioned articulated robot 1, or the joint angle and arm length.
The hierarchical neural network 3 is trained using (α, β, γ) as teacher data.

Next, in the execution process, with respect to the gripping object 11 placed at an arbitrary position, the marker attached to the gripping object 11 and the position of the center of gravity on the observation surface of the TV camera 2 indicated by the circle are described. By measuring and inputting them into the learned neural network 3, the output will be the joint angle for gripping the gripping object 11, or the position and posture angle of the hand effector 12. .

However, this joint angle, the position of the hand effector,
Even if the posture angle is instructed to the articulated robot 1 as it is, the direction of movement of the articulated robot 1 and the like cannot be specified. Therefore, as shown in FIG. There is a risk that the hand effector 12 of the articulated robot 1 approaches from the unexpected direction and collides with the object 10 depending on the approaching direction to 11 and the like.

Therefore, in the present invention, as shown in FIG. 6B, first, the target position of the hand effector 12 is calculated using the joint angles and the like obtained from the neural network 3.

This target position (approach point) is a hand effector when the position information of the hand effector 12 obtained from the neural network 3 is (X, Y, Z, α, β, γ) ^T. It is assumed that 12 points are separated by a fixed distance d in the front axis direction.

This target position (approach point) is set to (X ", Y",
If Z ", α, β, γ) ^T , then (X", Y ", Z", α, β, γ) ^T ＝ (Xd cosα sin β, Yd sin
α sin β, Zd cos β, α, β, γ) ^T.

Regarding the method of deriving the above equation, "Robot Engineering" published by Sokabo, Shigeo Hirose, 1987, 12, 15, 1st edition, Chapter 8 "Preparation for Manipulator Analysis", 8 -6, "Indication of direction", 8-6-2, Euler angles, P142-145.

The control method of the articulated robot 1 according to the present invention uses the position information (X, Y, Z, α, β, γ) ^{T of} the hand effector 12 obtained from the neural network to calculate the target position. (Approach point) (X ", Y", Z ", α, β, γ) ^T
Calculate this calculated target position (approach point)
To the above articulated robot 1, and after the hand effector 12 of the robot 1 reaches the target position (approach point), the hand effector 1
By moving the distance d in the axial direction of 2 by the above distance d, the gripping operation of the object 11 to be gripped can always be performed from a fixed direction, that is, the front direction of the hand effector 12 of the robot 1.

In the above measuring and grasping work of the object, the installation positions of the television cameras A and B 2 do not have to be exactly facing the object, and the position of the third television camera C 2 is also It does not have to be at a right angle to the TV cameras A and B 2. Since it is sufficient to install the TV cameras A, B, C, ~ 2 in the direction in which the posture and position changes of the object can be captured more, the installation positions of the TV cameras A, B, C, ~ 2 need not be accurate. .

In the above embodiment, a television camera (vision sensor) 2 is used to measure the position of the object 10 held by the robot 1 or the like and obtain information to be input to the neural network 3. However, the present invention is not limited to this, but the distance measurement information (X, X, measured by the one-dimensional distance measuring device based on the conventional triangulation principle described in FIG. 7 is used.
Needless to say, it may be Y, Z).

Further, only the three-dimensional position of the object 10 is measured,
If the posture of the object 10 is not measured, it is only necessary to install the two TV cameras 2 in the front direction of the object 10 and the direction perpendicular to the front direction.

Furthermore, when the posture of the object 10 changes
Also, the main axis direction α of the image_A~, Area S _AUse information from
By doing so, the posture change of the object can be detected.
Not only position but also 3D posture can be measured
It

FIG. 4 shows a plurality of neural networks.
3 shows the case where the position of an object is measured.
In general, as the space to be measured becomes wider, the size of the neural network 3 becomes too large for one neural network 3 to learn.

Therefore, by dividing the coordinate space of the hand into a plurality of areas of a predetermined size and allowing the neural networks 3 to learn by different ones, the neural network 3 can be downsized and the learning time can be shortened. Can be achieved.

By doing so, the range covered by one neural network 3 is narrowed, so that the interpolation accuracy is improved even if the neural network 3 is small.

In addition, one neural network 3 is sparsely taught about the entire space, and another neural network is
In section 3, it is possible to improve the measurement accuracy at a specific location by teaching only certain parts to the nectar, and to improve the overall accuracy by using multiple neural networks 3.

As described above, the three-dimensional position measuring device, the three-dimensional position control device, and the learning method according to the present invention include a measuring object grasped by a robot and the like, three or more television cameras for grasping the grasped object. The image information (X, Y, α, S) and distance measurement information obtained by measuring with the image processing device of the above are input to a learning device such as a neural network, and the angle of each joint of the robot and the arm length are input. From this information, for example, 6-dimensional position information (X, Y, Z, α, β, γ) of a measurement object that can be calculated by forward kinematics is learned as teacher data. In addition, one or more of the image processing devices such as the above television cameras are installed in a direction substantially perpendicular to the other television cameras. Further, when grasping an object, a plurality of markers are attached to the object, and the position of the center of gravity of each marker on the television camera observation plane is input as a feature amount to a neural network for learning, and the learned neural When grasping an object using a network,
From the position and posture angle of the hand effector obtained from the neural network, the robot is instructed to a point that is a certain distance d in the approach direction as a target point, and the hand effector of the robot sets the target point to the instructed point. After reaching, distance d
It is characterized by moving only to approach an object.

[0062]

As described above in detail, according to the present invention, it is possible to accurately measure the position of the object to be measured within the visual field of the television camera, which is difficult with the conventional method. The change is also information in the direction of the main axis of the image (α _A ~ above ₎
Alternatively, it can be detected by using the area (S _A ~ described above).

Further, since the learning device, for example, the neural network has a function of interpolating the points other than the points that were not taught, even if the teacher data is not created for the entire space, appropriate representative points can be obtained. The position of the target object can be measured only by teaching.

In the method according to the present invention, the position measurement is performed by learning the correspondence between the three-dimensional position of the object to be measured and the image information.
The learning device is not limited to the neural network, and other learning device that can learn the three-dimensional position may be used at the first learning.

Further, when the space to be measured becomes wider, the size of the neural network becomes too large for learning with one neural network. Therefore, by dividing the coordinate space of the hand into a plurality of pieces of a predetermined size and learning them in different neural networks, the neural network can be downsized, and the learning time can be shortened. By doing so, the range covered by one neural network is narrowed, so that the interpolation accuracy is improved even if the neural network is small.

Further, in one neural network,
Teaching the whole space sparsely and teaching other parts of the neural network only a certain part improves the measurement accuracy in a specific place, and by using multiple neural networks, It also becomes possible to improve.

When an object is grasped, a marker is attached to the object and the position of the center of gravity thereof is used as the feature amount, so that the image of the television camera is not easily affected by illumination or other objects, and the object itself Compared to when measuring
Accurate measurement can be performed.

Further, when gripping an object, the robot is set with a point distant by a certain distance d from the position of the hand effector obtained from the neural network in the approach direction of the hand effector as the approach target point. And the hand effector controls to actually approach the object after reaching the approach target point, it is possible to always perform the gripping operation of the object from a fixed direction.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of the present invention.

FIG. 2 is a diagram illustrating a learning method using a neural network according to the present invention.

FIG. 3 is a diagram showing an example of another learning device of the present invention.

FIG. 4 is a diagram showing another embodiment of the present invention.

FIG. 5 is a diagram for explaining marker feature amounts on a Byron sensor observation surface according to the present invention.

FIG. 6 is a diagram showing a method for approaching a grasped object according to the present invention.

FIG. 7 is a diagram illustrating a conventional method for measuring a three-dimensional position.

[Explanation of symbols]

1 Robot 10 Object to be measured 11 Object to be grasped, Object to be grasped 12 Hand effector 2 TV cameras A, B, C, ~ 3 Neural network (Networ)
k 1, 2, ~n) image information _{(X A, Y A, α} A, S A, ~) distance measurement information teacher data (X, Y, Z, α , β, γ) of the markers theta ₁ ~ robot joint Horn

Claims (7)

[Claims] 1. A measuring object (10) held by a robot (1) or the like.
Image processing device (2) such as three or more TV cameras
Or image information () obtained by measuring with a one-dimensional distance measuring device using a laser or the like, or information such as an object position obtained from the image information, or distance measurement information ()
Is input to a learning device (3) such as a neural network, and multidimensional position information (X, Y, Z, of the measurement target that can be calculated from the angle of each joint of the robot (2) and arm length information).
α, β, γ) is learned as teacher data (), and the learned learning device (3) is used to input image information of the measurement object (10) or distance measurement information (, or). hand,
A multidimensional position measuring device characterized by measuring a multidimensional position of the measurement object (10). 2. Using the above multi-dimensional position measuring device, from the image information () from the image processing device (2) such as a television camera, the X at the center of gravity or the like of a certain measuring object (10) on each image.
Coordinate, Y coordinate, inclination angle (α), area (S) of the object
Etc. are input to the learning device (3), and the joint angles (θ1, θ2, ~) of the robot (1) etc. corresponding to the input position of the measurement object (10), and the arm length A learning method characterized by using multidimensional position information (X, Y, Z, α, β, γ) that can be calculated as teacher data (). 3. The hand position of the robot (1) is randomly set in the visual field of the image processing device (2) group such as the television camera by using the multidimensional position measuring device, and the hand position of the robot (1) is set. Joint angle of the robot (1) etc. corresponding to the hand position
A learning method characterized in that (θ1, θ2, ~) and multidimensional position information (X, Y, Z, α, β, γ) that can be calculated from the arm length are used as teacher data (). 4. When setting each joint angle of the robot (1) by a predetermined number of divisions using the multidimensional position measuring device, the hand position obtained by the set joint angle is the television set. Each joint angle is divided and set so that it is within the visual field of the image processing device (2) group such as a camera, and the joint angle (θ1, θ
2, ~), and multidimensional position information that can be calculated from arm length
A learning method characterized in that (X, Y, Z, α, β, γ) is the teacher data (). 5. At least one of the three or more image processing devices (2) such as a television camera is installed in a direction to maintain a predetermined angle with another image processing device such as another television camera. The multidimensional position measuring device and the learning method according to any one of claims 1 to 4. 6. A gripping target (11) gripped by a robot (1) or the like.
A plurality of markers () are attached to the camera, and the positions of the attached markers () are set to three or more image processing devices such as TV cameras (2), or a one-dimensional distance measuring device (2a) using a laser or the like. Input the image information () acquired by the above, or the information such as the object position obtained from the image information or the distance measurement information () to the learning device (3) such as a neural network, and the robot ( The angle of each joint in 2),
Three-dimensional position information (X, Y, Z, α, β, γ) of the measuring object that can be calculated from the arm length information is learned as teacher data (), and using the learned device (3), The image information of the gripping target object (11) placed at an arbitrary position or posture, or the distance measurement information (, or) is input, and the gripping target object (1
A multi-dimensional position control device characterized by holding 0). 7. When performing a gripping operation for an object placed in the arbitrary position and orientation, the learning device (3) separates a predetermined distance (d) from the target position of the gripping object (11). The target position is the position where the hand effector (12) of the robot (1)
A multi-dimensional position control device characterized by controlling so as to approach the gripped object (10) after reaching the target point.