JP7718949B2 - Robotic systems using complementary measurement and positioning systems. - Google Patents

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a continuation-in-part of U.S. patent application Ser. No. 16/146,640, entitled "Supplementary Metrology Position Coordinates Determination System for Use with a Robot," filed on September 28, 2018, which is a continuation-in-part of U.S. patent application Ser. No. 16/104,033, entitled "Robot System with Supplementary Metrology Position Coordinates Determination System," filed on August 16, 2018, both of which are incorporated herein by reference in their entireties.

This disclosure relates to robotic systems, and more particularly to systems for determining the coordinates of a robot's end tool position.

Robotic systems are increasingly being used for manufacturing and other processes. Various types of robots available include articulated robots, selective compliance articulated robot arm (SCARA) robots, Cartesian robots, cylindrical robots, and spherical robots. As an example of components that may be included in a robot, a SCARA robot system (e.g., which may be a type of articulated robot system) typically has a base to which a first arm can be rotatably coupled, and a second arm can be rotatably coupled to the end of the first arm. In various configurations, an end tool can be coupled to the end of the second arm (e.g., for performing a specific task and/or inspection operation). Such systems may include position sensors (e.g., rotary encoders) that are used to determine and control the position of the arm and, accordingly, the position of the end tool. In various embodiments, such systems can have a placement accuracy of approximately 100 microns, although this is limited by several factors (e.g., the performance of the rotary encoder combined with the mechanical stability of the robotic system).

U.S. Pat. No. 4,725,965, the entirety of which is incorporated herein by reference, discloses several calibration techniques for improving the accuracy of SCARA systems. As described in the '965 patent, a technique is provided for calibrating a SCARA-type robot having a first rotatable arm and a second rotatable arm carrying an end tool. This calibration technique relies on the fact that a kinematic model can be used to control a SCARA robot. If this kinematic model is accurate, the arm can be positioned in a first and second angular configuration, such that the end tool carried by the second arm maintains the same position in both the first and second angular configurations. To calibrate the kinematic model, the arm is positioned in a first configuration that positions the end tool above a fixed reference point. The arm is then positioned in a second angular configuration that nominally realigns the end tool with the reference point. The kinematic model error is calculated from the shift in the position of the end tool relative to the reference point when the arm is switched from the first angular configuration to the second angular configuration. The kinematic model is then compensated according to the calculated error. These steps are repeated until the error reaches zero, at which point the kinematic model of the SCARA robot is considered calibrated.

As further described in the '965 patent, calibration techniques can include the use of a specific camera. For example, in one embodiment, the reference point can be the center of the viewing area of a stationary television camera (i.e., positioned on the ground below the end tool), and the camera's output signals can be processed to determine the shift in position of the end tool from the center of the camera's viewing area when the link is switched from a first configuration to a second configuration. In another embodiment, a second arm can carry the camera, and the technique can involve first positioning the arm in a first angular configuration in which a second predetermined interior angle is measured between the arm, thereby centering the camera carried by the second arm directly above the fixed reference point. The arm can then be positioned in a second angular configuration in which an interior angle equal to the second predetermined interior angle is measured between the arm, thereby nominally centering the camera again above the reference point. The camera's output signals can then be processed to determine the shift in position of the reference point observed by the camera when the arm is switched from the first angular configuration to the second angular configuration. The error in the camera's known position is then determined according to the shift in the position of the reference point observed by the camera. These steps are repeated as part of the calibration process until the error approaches zero.

While techniques such as those described in the '965 patent can be used to calibrate robotic systems, in some applications, the use of such techniques may be undesirable (e.g., they may require significant time and/or may not provide the desired level of accuracy in all possible orientations of the robot during a particular operation). Robotic systems that can achieve improvements in these areas are desirable (e.g., to increase the reliability, repeatability, speed, etc. of position determination during workpiece measurement and other processes).

This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

A supplemental metrology positioning system is provided for use with a robot as part of a robotic system. The robot (e.g., an articulated robot, a SCARA robot, a Cartesian coordinate robot, a cylindrical coordinate robot, a spherical coordinate robot, etc.) includes a movable arm configuration and a motion control system. The movable arm configuration includes a first arm section, a second arm section, and an end tool mounting configuration for mounting an end tool. The first arm section is mounted to a first rotary joint at a proximal end of the first arm section. The first rotary joint has a first axis of rotation. The first arm section has a second rotary joint positioned at a distal end of the first arm section. The second rotary joint has a second axis of rotation. The second arm section is mounted to a second rotary joint at a proximal end of the second arm section and is configured to rotate about the second rotary joint. The end tool mounting configuration is positioned near the distal end of the movable arm configuration. The motion control system is configured to control the end tool position of the end tool to a level of accuracy defined as robot accuracy based, at least in part, on sensing and controlling the angular positions of the first and second arm sections about the first and second rotary joints using rotation sensors included in the robot. The supplemental metrology position determination system includes first and second two-dimensional (2D) scales, a first camera, a second camera, and a metrology processing unit. The first and second two-dimensional (2D) scales are coupled to the movable arm structure at first and second 2D scale coupling locations, respectively. Each 2D scale includes a nominally planar substrate and a plurality of imageable elements distributed on the planar substrate. The first camera is for acquiring a first 2D scale image at an image acquisition time. The first camera defines a first reference position and is coupled to the movable arm structure at a first camera coupling location. The second camera is for acquiring a second 2D scale image at an image acquisition time. The second camera defines a second reference position and is coupled to the movable arm arrangement at a second camera coupling position. The metrology processor is configured to determine a first relative position of the first 2D scale based at least in part on a first image of the first 2D scale acquired by the first camera at the time of first image acquisition, and to determine a first relative position of the second 2D scale based at least in part on a first image of the second 2D scale acquired by the second camera at the time of first image acquisition.

The metrology processor may further be configured to determine coordinates of the end tool position at the time of first image acquisition based at least in part on the determined first relative positions of the first and second 2D scales. The first 2D scale coupling position may be on the first rotary joint, and a change in the relative position of the first 2D scale may occur due to movement of the first rotary joint in a direction perpendicular to the first axis of rotation during rotation. The metrology processor may further be configured to determine an angular orientation of the first 2D scale based at least in part on a first image of the first 2D scale acquired by the first camera at the time of first image acquisition. The first 2D scale coupling position may be on the first arm, and at least one of bending or twisting of the first arm may cause a change in the relative position of the first 2D scale (e.g., with respect to the first camera and/or a corresponding first reference position). The metrology processor may be further configured to determine a second relative position of the first 2D scale based at least in part on a second image of the first 2D scale acquired by the first camera at the time of second image acquisition, and to determine a second relative position of the second 2D scale based at least in part on a second image of the second 2D scale acquired by the second camera at the time of second image acquisition. The metrology processor may be further configured to determine measurement position coordinates of the first end tool position at the time of first image acquisition based at least in part on the determined first relative positions of the first and second 2D scales, and to determine measurement position coordinates of the second end tool position at the time of second image acquisition based at least in part on the determined second relative positions of the first and second 2D scales. The metrology processor may be further configured to determine a dimension related to the distance between the first and second end tool positions using the determined measurement position coordinates of the first and second end tool positions. This dimension is the distance between a first and second surface location on the workpiece, and a contact point of the end tool may contact the first surface location on the workpiece at the time of the first image acquisition and the second surface location on the workpiece at the time of the second image acquisition. The end tool may be at least one of a touch probe or a scanning probe used to measure the workpiece.

Also disclosed is a method for operating a supplemental metrology position determination system for use with a robot, the method comprising: operating a first camera to acquire a first image at a first 2D scale at a first image acquisition time, the first camera being coupled to a movable arm configuration of the robot at a first camera coupling position and defining a first reference position, the first 2D scale being coupled to the movable arm configuration at a first 2D scale coupling position; and operating a second camera to acquire a first image at a second 2D scale at the first image acquisition time, the second camera being coupled to the movable arm configuration of the robot at a second camera coupling position. The method can be summarized as including: operating a second camera coupled to the movable arm arrangement at a second 2D scale coupling position, the second camera being coupled to the movable arm arrangement at a second 2D scale coupling position and defining a second reference position; determining a first relative position of the first 2D scale based, at least in part, on a first image of the first 2D scale acquired by the first camera at the time of first image acquisition; and determining a first relative position of the second 2D scale based, at least in part, on a first image of the second 2D scale acquired by the second camera at the time of first image acquisition.

The method may further include determining a measurement position coordinate of the first end tool position at the time of the first image acquisition based at least in part on the determined first relative positions of the first and second 2D scales. The method may further include operating the first camera to acquire a second image at the first 2D scale at the time of the second image acquisition, operating the second camera to acquire a second image at the second 2D scale at the time of the second image acquisition, determining a second relative position of the first 2D scale based at least in part on the second image at the first 2D scale acquired by the first camera at the time of the second image acquisition, and determining a second relative position of the second 2D scale based at least in part on the second image at the second 2D scale acquired by the second camera at the time of the second image acquisition. The method may further include determining a measurement position coordinate of a first end tool position at the time of the first image acquisition based at least in part on the determined first relative position of the first and second 2D scales, and determining a measurement position coordinate of a second end tool position at the time of the second image acquisition based at least in part on the determined second relative position of the first and second 2D scales. The method may further include using the determined measurement position coordinates of the first and second end tool positions to determine a dimension related to the distance between the first and second end tool positions. The dimension may be a distance between a first and second surface position on the workpiece. A contact point of the end tool may contact a first surface position on the workpiece at the time of the first image acquisition and a second surface position on the workpiece at the time of the second image acquisition. The method may further include coupling first and second 2D scales to the movable arm configuration at first and second 2D scale coupling positions, respectively; coupling a first camera to the movable arm configuration at a first camera coupling position; and coupling a second camera to the movable arm configuration at a second camera coupling position.

Also disclosed is a supplemental metrology positioning system for use with a robot including a movable arm configuration with an end tool mounting configuration for mounting an end tool, and a motion control system configured to control the end tool position of the end tool. The supplemental metrology positioning system can be summarized as including first and second two-dimensional (2D) scales, a first camera, a second camera, and a metrology processing unit. The first and second two-dimensional (2D) scales are configured to be coupled to the movable arm configuration of the robot at first and second 2D scale coupling positions, respectively. Each 2D scale includes a nominally planar substrate and a plurality of imageable elements distributed on the planar substrate. The first camera is configured to acquire a first 2D scale image at the image acquisition time. The first camera defines a first reference position and is configured to be coupled to the movable arm configuration at the first camera coupling position. The second camera is configured to acquire a second 2D scale image at the image acquisition time. The second camera defines a second reference position and is configured to be coupled to the movable arm arrangement at a second camera coupling position. The measurement processor is configured to determine a first relative position of the first 2D scale based at least in part on a first image of the first 2D scale acquired by the first camera at the time of first image acquisition, and to determine a first relative position of the second 2D scale based at least in part on a first image of the second 2D scale acquired by the second camera at the time of first image acquisition. The measurement processor determines measurement position coordinates of the first end tool position at the time of first image acquisition based at least in part on the determined first relative positions of the first and second 2D scales.

The first 2D scale binding location may be on a first rotary joint of the movable arm arrangement, the first rotary joint having a first axis of rotation, and movement perpendicular to the first axis of rotation during rotation of the first rotary joint may cause a change in the relative position of the first 2D scale. The metrology processor may be further configured to determine the angular orientation of the first 2D scale based, at least in part, on a first image of the first 2D scale acquired by the first camera at the first image acquisition time. The first 2D scale binding location may be on a first arm portion of the movable arm arrangement, and at least one of bending or twisting of the first arm portion may cause a change in the relative position of the first 2D scale.

The supplemental metrology position determination system described herein can be added to an existing robot that already includes a measurement system (e.g., see block 140 in FIG. 1 ) with rotary encoders included at each robot rotary joint. This can measure/determine where the end tool position is at the end of the robot arm, referred to herein as "robot accuracy," and the accuracy of the included encoders may be limited/lower than desired. The present invention aims to provide a supplemental metrology system (e.g., that can be attached to an existing robot by attaching additional cameras and scales to the robot arm) that can achieve improved accuracy in determining the end tool position at the end of the robot (i.e., the end of the robot arm). More specifically, the existing robot's encoders only measure the rotation of the rotary joints, and the robot system/model may assume that all joints rotate perfectly while the arm remains perfectly straight. For various reasons, this may not be the case (e.g., the arm may be heavy, causing the arm/joint to bend/twist, the end tool located at the end of the arm may be heavy, the joint may not rotate perfectly, etc.), which may result in a "wobble" or "slop" in the joint movement (or other movement perpendicular to the expected joint/axis of rotation), or some amount of bending/twisting of the arm, etc. The present invention adds a camera and 2D scale attached to the joint and/or robot arm, and monitors/images the 2D scale with the camera to detect normal rotational movement and undesired movement (e.g., wobble, slop, bending, twisting, etc.), and adds the determination of the undesired movement to the calculations/models for determining the end tool position of the end tool at the end of the robot, providing greater accuracy than would be possible using only the robot's rotational encoder. In some embodiments, such techniques can achieve accuracy in the range of 10 microns or less (e.g., versus the 100 micron accuracy of some conventional robotic systems). Such increased accuracy may be particularly desirable for certain applications (e.g., measuring workpieces, precision drilling holes in workpieces, precise manipulation and placement of extremely small workpieces or other elements, etc.).

FIG. 1 is a block diagram of a first exemplary embodiment of a robotic system including an articulated robot and a supplemental metrology positioning system. FIG. 2 is an isometric view of a second exemplary embodiment of a robotic system similar to the robotic system of FIG. 1 . FIG. 1 is a top view of a portion of a robot system. FIG. 1 is a side view of a portion of a robotic system. FIG. 1 is an isometric view of a first example embodiment of an incremental 2D scale. FIG. 10 is an isometric view of a second example embodiment of an incremental 2D scale. FIG. 1 is an isometric view of an example embodiment of an absolute 2D scale. FIG. 10 is a block diagram of a third exemplary embodiment of a robotic system including an articulated robot and a supplemental metrology positioning system. FIG. 10 is an isometric view of a fourth exemplary embodiment of a robotic system similar to the robotic system of FIG. 8 including an articulated robot. FIG. 1 is a side view of a portion of a robotic system. FIG. 1 is a side view of a portion of a robotic system. FIG. 1 is a flow diagram illustrating an exemplary embodiment of a routine for operating a robotic system including an articulated robot and a supplemental metrology positioning system. FIG. 1 is a flow diagram illustrating an exemplary embodiment of a routine for operating a robotic system including an articulated robot and a supplemental metrology positioning system. FIG. 10 is a flow diagram illustrating an example embodiment of a routine for determining end tool position that can utilize a robot position sensor during a first portion of the movement timing and a supplemental metrology position determination system during a second portion of the movement timing. FIG. 10 is a flow diagram illustrating an exemplary embodiment of a routine for operating a supplemental metrology position determination system utilized with a robot.

1 is a block diagram of a first exemplary embodiment of a robot system 100 including an articulated robot 110 and a supplemental metrology positioning system 150. The articulated robot 110 includes a movable arm configuration MAC and a robot movement control and processing system 140. In the example of FIG. 1, the movable arm configuration MAC includes first and second arm sections 121 and 122, first and second rotary joints 131 and 132 (e.g., included as part of the first and second movement mechanisms), position sensors SEN1 and SEN2, and an end tool configuration ETCN. The first arm section 121 is mounted to the first rotary joint 131 at a proximal end PE1 of the first arm section 121. A first rotary joint 131 (e.g., positioned at an upper end of the support base portion BSE) has an axis of rotation RA1 aligned along the z-axis direction, such that the first arm portion 121 moves nominally about the first rotary joint 131 in an x-y plane perpendicular to the z-axis. A second rotary joint 132 is positioned at a distal end DE1 of the first arm portion 121. The second rotary joint 132 has an axis of rotation RA2 aligned nominally along the z-axis direction. The second arm portion 122 is mounted to the second rotary joint 132 at a proximal end PE2 of the second arm portion 122, such that the second arm portion 122 moves nominally about the second rotary joint 132 in an x-y plane nominally perpendicular to the z-axis. In various embodiments, position sensors SEN1 and SEN2 (e.g., rotary encoders) can be utilized to determine the angular position (i.e., in the x-y plane) of the first and second arm portions 121 and 122 about the first and second rotary joints 131 and 132, respectively.

In various embodiments, the end tool configuration ETCN can include a Z movement mechanism 133 (e.g., included as part of the third movement mechanism), a Z arm portion ZARM (e.g., shown as the third arm portion), a position sensor SEN3, and an end tool coupling portion ETCP (e.g., included as part of the end tool mounting configuration ETMC) coupled to the end tool ETL. In various embodiments, the end tool ETL can include an end tool sensing portion ETSN and an end tool stylus ETST having a contact point CP (e.g., for contacting the surface of the workpiece WP). The Z movement mechanism 133 is positioned near the distal end DE2 of the second arm portion 122. The Z movement mechanism 133 (e.g., a linear actuator) is configured to move the Z arm portion ZARM up and down in the z-axis direction. In some embodiments, the Z arm portion ZARM can also be configured to rotate about an axis parallel to the z-axis direction. In either case, the end tool ETL is coupled to the end tool interface ETCP and has a corresponding end tool position ETP with corresponding coordinates (e.g., x, y, and z coordinates). In various embodiments, the end tool position ETP can correspond to or be near the distal end DE3 of the Z-arm portion ZARM (e.g., at or near the end tool interface ETCP).

The robot's motion control system 140 is configured to control the end tool position ETP of the end tool ETL with a level of precision defined as the robot precision. More specifically, the motion control system 140 is generally configured to control the coordinates of the end tool position ETP with the robot precision based, at least in part, on sensing and controlling the angular positions (i.e., in the x-y plane) of the first and second arm portions 121 and 122 about the first and second rotary joints 131 and 132, respectively, using position sensors SEN1 and SEN2. In various embodiments, the motion control and processing system 140 can include first and second rotary joint control and sensing units 141 and 142, which can receive signals from position sensors SEN1 and SEN2, respectively, to sense the angular position of the first and second arm units 121 and 122, and/or provide control signals (e.g., to motors, etc.) at the first and second rotary joints 131 and 132 to rotate the first and second arm units 121 and 122.

Generally, robot precision is related to certain assumptions about the robot's movements (e.g., it may be related to models, such as kinematic models and/or geometric models, and/or the corresponding calculations used to determine the end tool position). For example, the determination of the end tool position according to robot precision may be based on the known lengths of the first and second arm portions 121 and 122, which are generally assumed to be horizontal and straight and without bending or twisting, and the rotations about the first and second rotary joints 131 and 132, which are assumed to be precise, about their respective axes of rotation. However, in some cases, some arm portions may bend or twist, and/or the movement of the rotary joints may be perpendicular to their respective axes of rotation. For example, the distal ends DE1 and DE2 of the first and second arm sections 121 and 122 may experience vertical displacement or deflection (e.g., due to weight and/or different orientations of the arm sections and/or end-tool configurations, etc.) and/or may experience undesired movement during rotation about the first and/or second rotary joints 131 and 132 (e.g., movement perpendicular to the respective axes of rotation). As described in more detail below, improved accuracy in determining end-tool position or other robot movements/configurations can be achieved through the use of a supplemental metrology and positioning system capable of determining, measuring, and/or otherwise accounting for such undesired movement (e.g., bending or twisting of the arm sections, movement of the rotary joints perpendicular to the axes of rotation, etc.) in accordance with the principles disclosed herein. It will be appreciated that even small improvements in accuracy may be highly desirable for certain applications (e.g., for robotic measurement and control operations, such as measuring workpieces or precision drilling of workpieces).

Additionally, the motion control system 140 is generally configured to control the z-coordinate of the end tool position ETP with robotic precision based, at least in part, on sensing and controlling the linear position (i.e., along the z-axis) of the Z-arm portion ZARM using the Z-movement mechanism 133 and position sensor SEN3. In various embodiments, the motion control and processing system 140 can include a Z-arm movement mechanism control and sensing unit 143, which can receive signals from position sensor SEN3 to sense the linear position of the Z-arm portion ZARM and/or provide control signals to the Z-movement mechanism 133 (e.g., a linear actuator) to control the z-position of the Z-arm portion ZARM. As described in more detail below, in some embodiments, the supplemental metrology positioning system 150 can include a corresponding position sensor 163, which can provide similar information and operate similarly to position sensor SEN3 (or, for example, operate differently and/or be a sensor with higher precision than position sensor SEN3). In some embodiments, the supplemental metrology positioning system 150 may not include a corresponding position sensor 163, and instead may utilize sensed data from position sensor SEN3 transmitted to the supplemental metrology positioning system 150. In some embodiments, the supplemental metrology positioning system 150 may provide other sensed position information (e.g., regarding the relative positions of arm sections 121 and 122 and/or rotary joints 131 and 132) to the motion control and processing system 140 to more accurately determine the end tool position.

Additionally, the motion control and processing system 140 and/or the supplemental metrology positioning system 150 can receive signals from the end tool sensing unit ETSN. In various embodiments, the end tool sensing unit ETSN can include circuitry and/or configurations related to the operation of the end tool ETL to sense the workpiece WP. As described in more detail below, in various embodiments, the end tool ETL (e.g., a touch probe, a scanning probe, a camera, etc.) can be utilized to contact or otherwise sense surface locations/positions/points on the workpiece WP, and various corresponding signals can be received, determined, and/or processed by the end tool sensing unit ETSN, which can provide corresponding signals to the motion control and processing system 140 and/or the supplemental metrology positioning system 150. In various embodiments, the motion control and processing system 140 and/or the supplemental metrology position determination system 150 can include an end tool control and sensing unit 144, which can provide control signals to and/or receive sensing signals from the end tool sensing unit ETSN. In various embodiments, the end tool control and sensing unit 144 and the end tool sensing unit ETSN can be merged and/or indistinguishable. In various embodiments, the first and second rotary joint control and sensing units 141 and 142, the Z movement mechanism control and sensing unit 143, and the end tool control and sensing unit 144 can all provide outputs to and/or receive control signals from a robot position processor 145. As part of the robot motion control and processing system 140, the robot position processor 145 can control and/or determine the overall position of the articulated robot 110 and a corresponding end tool position ETP. In various embodiments, the articulated robot 110 may have a designated operable work volume OPV, which may additionally or alternatively be referred to as the end tool work volume ETWV, within which at least a portion of the end tool (e.g., contact point CP) may be moved (e.g., for measuring/inspecting a workpiece, etc.).

In the configuration of FIG. 1, the robot 110 is configured to move the movable arm configuration MAC to move at least a portion of the end tool ETL mounted in the end tool mounting configuration ETMC in at least two dimensions within the end tool working volume ETWV. The movement control system 140 is configured to control the end tool position ETP with a level of precision defined as the robot precision based, at least in part, on sensing and controlling the position of the movable arm configuration MAC (e.g., using one or more position sensors included in the robot 110).

In various embodiments, the supplemental metrology positioning system 150 may be included with or otherwise added to the articulated robot 110 (e.g., as part of a retrofit configuration for addition to an existing articulated robot 110, etc.). Generally, the supplemental metrology positioning system 150 may be utilized to improve the level of accuracy in determining the end tool position ETP. More specifically, as described in more detail below, the supplemental metrology positioning system 150 may be used to determine a relative position indicative of a measurement position coordinate of the end tool position ETP at an accuracy level better than the robot accuracy, which may be used to determine the measurement position coordinate.

As shown in FIG. 1, the supplemental measurement position determination system 150 can include cameras 161A, 161B, 162A, 162B, a sensor 163, scales 171A, 171B, 172A, 172B, and a measurement position coordinate processing unit 190. As shown in FIGS. 1 and 2 (i.e., the configuration of FIG. 2 is described in detail below), the cameras/scales are configured as four camera/scale sets including cameras 161A, 161B, 162A, 162B pointed at corresponding scales 171A, 171B, 172A, 172B, respectively. Scales 171A and 172A are located on the rotary joint of the robot 110, and scales 171B and 172B are located on the arm of the robot 110. The cameras 161A, 161B, 162A, and 162B and the scales 171A, 171B, 172A, and 172B are coupled to the robot 110 at respective coupling positions CL1 to CL8. More specifically, the camera 161A is coupled to a mounting bracket BK1A coupled to the support base unit BSE at a camera coupling position CL1. The scale 171A is coupled to the first rotary joint 131 at a 2D scale coupling position CL2 corresponding to the rotation axis RA1 of the first rotary joint 131. The camera 161B is coupled to the first arm unit 121 at a camera coupling position CL3. The scale 171B is coupled to the first arm unit 121 at a 2D scale coupling position CL4 adjacent to the second rotary joint 132. The camera 162A is coupled to a mounting bracket BK2A coupled to the first arm unit 121 at a camera coupling position CL5. Scale 172A is coupled to second rotary joint 132 at 2D scale coupling location CL6, which corresponds to rotation axis RA2 of second rotary joint 132. Camera 162B is coupled to second arm portion 122 at camera coupling location CL7. Scale 172B is coupled to second arm portion 122 at 2D scale coupling location CL8, which is proximate Z-arm portion ZARM. In various embodiments, coupling of each of the various components can be achieved using one or more coupling components, elements, mechanisms, and/or techniques (e.g., fasteners, bolts, clamps, adhesives, etc.).

1 and 2 (i.e., the configuration of FIG. 2 is described in more detail below), in various embodiments, camera 161A defines a reference position REF1, the optical axis OA1A of camera 161A is aligned with the rotation axis RA1 of the first rotary joint 131, and elements of scale 171A can be imaged by camera 161A. Camera 161B defines a reference position REF2, the optical axis OA1B of camera 161B is aligned with a position (e.g., a central position) of scale 171B, and elements of scale 171B can be imaged by camera 161B. Camera 162A defines a reference position REF3, the optical axis OA2A of camera 162A is aligned with the rotation axis RA2 of the second rotary joint 132, and elements of scale 172A can be imaged by camera 162A. Camera 162B defines a reference position REF4, the optical axis OA2B of camera 162B is aligned with the position (e.g., the central position) of scale 172B, and elements of scale 172B can be imaged by camera 162B.

Each of the four cameras 161A, 161B, 162A, and 162B is controlled by a respective imaging configuration control and processor (ICCCP) 181A, 181B, 182A, and 182B and provides an image signal to the processor. In some examples, a trigger unit 187 can coordinate the triggering of all of the cameras 161A, 161B, 162A, and 162B to simultaneously acquire images (e.g., corresponding to the current position of the robot 110 at a particular time to determine the position of an end tool at the end of the robot 110 at that time). In embodiments where a position sensor 163 is included (e.g., to sense the position of the Z-arm unit ZARM), it can be controlled by the sensing configuration control and processor (SCCPP) 183 and provide a position signal to the processor. In some embodiments, the collection and/or recording of position data is triggered by a signal from the trigger unit 187.

In various embodiments, each of the 2D scales 171A, 171B, 172A, and 172B includes a nominally planar substrate SUB and a plurality of imageable elements distributed on the substrate SUB. Each imageable element is located at known local x and y scale coordinates on the respective 2D scale. In various embodiments, each 2D scale can be an incremental scale or an absolute scale, as described in more detail below with reference to Figures 5-7.

In various embodiments, the trigger unit 187 and/or the measurement position coordinate processing unit 190 can be included as part of the external control system ECS (e.g., as part of an external computer, etc.). The trigger unit 187 can be included as part of the imaging and detection configuration control and processing unit 180. In various embodiments, the trigger unit 187 is configured to receive at least one input signal related to the end tool position ETP, determine the timing of a first trigger signal based on the at least one input signal, and output the first trigger signal to the cameras 161A, 161B, 162A, and 162B and the position sensor 163. In various embodiments, in response to receiving the first trigger signal, each of the cameras 161A, 161B, 162A, and 162B is configured to acquire a digital image of the corresponding 2D scale 171A, 171B, 172A, and 172B at the time of image acquisition. In various embodiments, the measurement position coordinate processing unit 190 is configured to input the acquired images and identify at least one imagable element included in each acquired image at 2D scale and each associated known 2D scale coordinate location. In various embodiments, the external control system ECS can also include a standard robot position coordinate mode unit 147 and a supplemental measurement position coordinate mode unit 192 to implement corresponding modes, which are described in more detail below.

In various embodiments, each imaging configuration control and processing unit 181A, 181B, 182A, and 182B can include a component (e.g., a subcircuit, routine, etc.) that periodically (e.g., at set timing intervals) activates image integration of the corresponding camera 161A, 161B, 162A, and 162B, and the first trigger signal can activate strobe light timing (e.g., each camera 161A, 161B, 162A, and 162B can include a strobe light) or other mechanism that effectively stops motion and correspondingly determines exposure within the integration period. In such embodiments, if the first trigger signal is not received during an integration period, the resulting image can be discarded; if the first trigger signal is received during an integration period, the resulting image can be saved and/or otherwise processed/analyzed to determine relative position, as described in more detail below.

In various embodiments, different types of end tool ETLs provide different types of outputs, which can be used in conjunction with the trigger unit 187. For example, in an embodiment where the end tool ETL is a touch probe used to measure a workpiece and outputs a touch signal when it touches the workpiece, the trigger unit 187 can be configured to receive the touch signal or a signal derived therefrom as at least one input signal and determine the timing of the first imaging trigger signal based on the touch signal. As another example, in an embodiment where the end tool ETL is a scanning probe used to measure a workpiece and provides workpiece measurement sample data corresponding to each sample timing signal, the trigger unit 187 can be configured to receive the sample timing signal or a signal derived therefrom as at least one input signal. As another example, in an embodiment where the end tool ETL is a camera used to provide workpiece measurement images corresponding to each workpiece image acquisition signal, the trigger unit 187 can be configured to receive the workpiece image acquisition signal or a signal derived therefrom as at least one input signal.

1, the supplemental metrology position determination system 150 is configured to operate such that the measurement position coordinate processor 190 determines (e.g., based on determining the image position of at least one imagable element identified in each acquired image) the relative position (e.g., including local x and y scale coordinates, which may indicate the scale's orientation, position, etc.) of each 2D scale 171A, 171B, 172A, and 172B to the corresponding reference position REF1, REF2, REF3, and REF4 of the corresponding camera 161A, 161B, 162A, and 162B. The determined relative positions can be used to determine the measurement position coordinates of the end tool position ETP at the time of image acquisition with a level of accuracy better than robot accuracy. In various embodiments, the supplemental metrology position determination system 150 can be configured to determine the measurement position coordinates of the end tool position ETP at the time of image acquisition based at least in part on the determined relative positions.

As discussed above, robotic accuracy may be related to the model (e.g., kinematic model, geometric model, etc.) and/or corresponding calculations or other processes used to determine the end tool position. In accordance with such robotic processes, the determination of the end tool position may be based on the known lengths of the first and second arm sections 121 and 122, which are assumed to be generally horizontal and straight, with no bending or twisting, and the rotations of the first and second rotary joints 131 and 132, which are assumed to be precise, about their respective axes of rotation. The presence of undesired motion (e.g., bending or twisting of the arm sections, movement of the rotary joints perpendicular to their respective axes of rotation, etc.) can lead to inaccurate robotic determination of the end tool position. In accordance with the principles disclosed herein, improved accuracy in the determination of the end tool position and/or other robotic movements/configurations can be achieved by using a supplemental metrology positioning system 150 that can determine, measure, and/or otherwise account for such undesired motion (e.g., bending or twisting of the arm sections, movement of the rotary joints perpendicular to their axes of rotation, etc.). For example, with respect to an exemplary kinematic and/or geometric model assumed by the robotic system (e.g., using a straight robot arm of a specified length and perfect rotation), additional measurements can be determined/added to such a model to determine more accurate position information. For example, rather than assuming that each of first and second arm portions 121 and 122 is straight, camera/scale combination 161B/171B (for first arm portion 121) and combination 162B/172B (for second arm portion 122) can provide position information/measurements representative of bending, twisting, etc. of arm portions 121 and 122. Similarly, rather than assuming perfect rotational movement about each of rotary joints 131 and 132, camera/scale combination 161A/171A (for first rotary joint 131) and combination 162A/172A (for second rotary joint 132) can provide position information/measurements representative of rotary joint movement perpendicular to each axis of rotation (as well as position information/measurements representative of more conventional angular orientations of each 2D scale, with a high level of accuracy). By including such information (e.g., as part of kinematic and/or geometric models, calculations, etc.) for determining the robot arm position and/or end tool position (e.g., at the distal end of the movable arm configuration MAC), etc., increased levels of accuracy can be achieved.

In certain embodiments, the supplemental metrology positioning system 150 can operate relatively independently (e.g., from the robot position processor 145) to improve the accuracy of its determinations (e.g., of end tool position, etc.). In other embodiments, the supplemental metrology positioning system 150 can operate in cooperation (e.g., with the robot position processor 145, and/or the control and sensing unit or other portions of the robot, and/or other systems) to improve the accuracy of its determinations. For example, the supplemental metrology positioning system 150 can receive specific information from the robot system (e.g., from the robot position processor, or the control and sensing unit, or other portions) to combine with, supplement, and/or add to determined position information (e.g., to determine end tool position, etc.). As another example, the supplemental metrology positioning system 150 can provide specific information to the robot system, or to other systems that can combine specific position information from the robot and supplemental systems, to combine with, supplement, and/or add to determined position information (e.g., to determine end tool position, etc.).

It will be appreciated that such a system may have several advantages over various alternative systems. For example, in various embodiments, a system as disclosed herein may be smaller and/or less expensive, and in some embodiments, more accurate, than alternative systems that utilize technologies such as laser trackers or photogrammetry to track robot movement/position. Also, while the disclosed system does not occupy or obscure any portion of the operable work volume OPV, alternative systems may include scales or fiducials on the ground, stage, or elsewhere within the area (e.g., the operable work volume) where operations and/or inspections, etc., may be performed on the workpiece. Furthermore, in various embodiments, by coupling all of the cameras and scales to the robot (e.g., including being coupled to moving parts of a movable arm configuration such as an arm section and rotary joint), no external structure or external couplings in the robot environment are required for the cameras or scales.

Figure 2 is an isometric view of a second illustrative embodiment of a robotic system 200 substantially similar to the robotic system 100 of Figure 1. It will be appreciated that some numbered components (e.g., 1XX or 2XX) in Figure 2 correspond to and/or have similar operation as, can be understood to be similar or identical to, or otherwise understood by analogy and as described below, identically or similarly numbered corresponding components (e.g., 1XX) in Figure 1. This numbering scheme for indicating elements having similar and/or identical design and/or function also applies to the other figures described below.

2 (i.e., similar to the configuration of FIG. 1), the supplemental metrology positioning system 150 includes cameras 161A, 161B, 162A, and 162B, each of which is aimed at a corresponding scale 171A, 171B, 172A, and 172B and is attached to each arm section 121 and 122, rotary joint 131 and 132, etc. In various embodiments, different reference axes and lines may be designated to refer to specific movements, coordinates, and angles of components of the articulated robot. For example, the first and second arm sections 121 and 122 may be designated with nominally horizontal centerlines CT1 and CT2, respectively, passing through the center of each arm section.

In various embodiments, the end tool configuration ETCN may be coupled to the second arm portion 122 near the distal end DE2 of the second arm portion 122 and may be specified to have an end tool axis EA of the end tool ETL that nominally intersects the centerline CT2 of the second arm portion 122. The end tool position ETP may be specified to have coordinates X2, Y2, and Z2. In various embodiments, the end tool ETL may have a contact point CP (e.g., at the end of the end tool stylus ETST for contacting the workpiece) that may be specified to have coordinates X3, Y3, and Z3. In embodiments in which the contact point CP of the end tool ETL does not move in the x or y directions relative to the rest of the end tool, the X3 and Y3 coordinates may in some cases be nominally equal to the X2 and Y2 coordinates, respectively. It will be appreciated that in instances where bending or twisting of arms 121 and 122 can be determined (i.e., in accordance with the principles disclosed herein), the resulting model (e.g., a kinematic model, a geometric model, etc.) may indicate that the X3 and Y3 coordinates are different from the X2 and Y2 coordinates. For example, a bending or twisting arm may correspondingly cause tilting of the end tool ETL, etc. The measurement/determination of the amount of bending/twisting can be included in the model to determine a more accurate indication of the coordinates X3 and Y3 relative to the coordinates X2 and Y2, etc.

In one specific example, each image captured by each camera 161A, 161B, 162A, and 162B can be analyzed by the measurement position coordinate processor 190 to determine the relative position (e.g., corresponding to the position, orientation, etc. of each scale 171A, 171B, 172A, and 172B). Such determination can be performed according to standard camera/scale image processing techniques (e.g., for determining the position, orientation, etc. of the camera relative to the scale). Various examples of such techniques are described in U.S. Patent Nos. 6,781,694; 6,937,349; 5,798,947; 6,222,940; and 6,640,008, each of which is incorporated herein by reference in its entirety. Various techniques can be used to determine the position of the field of view (e.g., corresponding to the camera's position) within the scale (e.g., within each 2D scale 171A, 171B, 172A, and 172B). This is described in more detail below with reference to Figures 5-7. In various embodiments, such determination may include identifying at least one imageable element included in the acquired image of each 2D scale and its associated known 2D scale coordinate location. Such determination may correspond to determining the relative position of each 2D scale 171A, 171B, 172A, and 172B with a corresponding reference position REF1, REF2, REF3, and REF4 (e.g., corresponding to and/or indicative of the position of the corresponding camera 161A, 161B, 162A, and 162B).

FIG. 3 is a top view of a portion of a robot system similar to that shown in FIGS. 1 and 2. More specifically, FIG. 3 shows a top view of first and second arm sections 121 and 122, first and second rotary joints 131 and 132, and scales 171A and 172A. Scale 171A is coupled at a 2D scale coupling location on first rotary joint 131, and scale 172A is coupled at a 2D scale coupling location on second rotary joint 132. Movement of rotary joint 131 perpendicular to rotation axis RA1 during rotation changes the relative position of scale 171A (e.g., along the x- and/or y-axes of the local camera coordinate system), and movement of rotary joint 132 perpendicular to rotation axis RA2 during rotation changes the relative position of scale 172A (e.g., along the x- and/or y-axes of the local camera coordinate system). As described above, cameras 161A and 162A are positioned to capture images of scales 171A and 172A, respectively. Based on these images, measurement position coordinate processor 190 determines the angular orientations of scales 171A and 172A, as well as the relative positions of scales 171A and 172A (e.g., determined in x- and y-axis coordinates in a local camera coordinate system, which may be referenced to a reference position REF that may be defined by the cameras as described above). Measurement position coordinate processor 190 uses this to detect any undesirable movement perpendicular to rotation axes RA1 and RA2. In various embodiments, measurement position coordinate processor 190 can use the determined relative positions of scales 171A and 172A to detect so-called "wobble," "rocking," or other movement of first and second rotary joints 131 and 132. Such movement would not typically be detected or revealed by standard robot encoders, thereby introducing measurement errors into determining the end tool position at the end of the robot arm.

In various embodiments, the supplemental metrology positioning system 150 is "self-contained" because it does not obtain rotational information of the first and second rotary joints 131 and 132 from robot encoders. Instead, the supplemental metrology positioning system 150 obtains images of the scales 171A and 172A and determines rotational information (e.g., angular orientation) of the first and second rotary joints 131 and 132. The supplemental metrology positioning system 150 also determines the relative position/displacement of the first and second rotary joints 131 and 132. In various embodiments, the rotational information (e.g., angular orientation) of the first and second rotary joints 131 and 132 determined by the supplemental metrology positioning system 150 may typically be more accurate than the angular orientations of the first and second rotary joints 131 and 132 determined by the robot encoders included in those rotary joints.

FIG. 4 is a side view of a portion of a robotic system similar to that shown in FIGS. 1 and 2. More specifically, FIG. 4 shows a side view of the first arm portion 121, the camera 161B, and the scale 171B. Although not shown in FIG. 4, the scale 171B is positioned near the second rotary joint 132 coupled to the second arm portion 122 (see, e.g., FIG. 2). The first arm portion 121, the second arm portion 122, and/or an end tool and/or other element coupled to the second arm portion 122 may be heavy enough to cause bending or twisting of the first arm portion 121, as shown by the dashed lines in FIG. 4. Such bending or twisting may change the relative position of the scale 171B, causing the lower portion of the scale 171B to move from an expected position P1 to a bent position P2. As the scale 171B moves relative to the camera 161B, a corresponding amount of movement/position change D12 can be detected/measured. More specifically, when arm 121 is bent as shown, the field of view of camera 161B shows a different portion of scale 171B (e.g., a portion closer to the top of scale 171B in the orientation shown), and by analyzing the images, the amount of bending of first arm portion 121 corresponding to the position of scale 171B can be determined (e.g., based at least in part on determining different positions of one or more imagable elements of the scale in the different images).

FIG. 5 is a diagram of an exemplary embodiment of incremental 2D scale 171/172, FIG. 6 is an isometric view of an exemplary embodiment of incremental 2D scale 171'/172', and FIG. 7 is an exemplary embodiment of absolute 2D scale 171"/172". In various embodiments, any of 2D scales 171/172, 171'/172', or 171"/172" may be used for or represent any of 2D scales 171A, 171B, 172A, and 172B in FIGS. 1 and 2, and/or any of 2D scales 871A, 871B, 872A, 872B, 873A, 873B, and 874A in FIGS. 8 and 9, as described in more detail below.

As shown in FIG. 5, the incremental 2D scale 171/172 includes an array of evenly spaced incremental imageable elements IIF distributed on a planar substrate SUB. In various embodiments, the incremental 2D scale 171/172 may have a specified periodicity (e.g., less than 100 microns, such that the periodic spacing between the incremental imageable elements IIF along each of the x-axis and y-axis may be less than 100 microns, as described in more detail below with reference to FIG. 6). In one specific embodiment, the 2D scale 171/172 may be specified to have a reference position (e.g., an origin position) at scale coordinates X0, Y0, and Z0, as described in more detail below with reference to the example of FIG. 6.

FIG. 6 is an isometric view of an exemplary embodiment of the incremental 2D scale 171'/172'. As shown in FIG. 6, the incremental 2D scale 171'/172' includes an array of evenly spaced incremental imageable elements IIF distributed on a planar substrate SUB. In various embodiments, the incremental 2D scale 171'/172' may have a periodicity of less than 100 microns (e.g., the periodic spacings XSP1 and YSP1 between the incremental imageable elements IIF along the x-axis and y-axis, respectively, may each be less than 100 microns). In various embodiments, position information determined using the incremental 2D scale 171'/172' may have an accuracy of at least 10 microns. Compared to the robot accuracy, which may be approximately 100 microns in certain embodiments, the accuracy determined using such a 2D scale may be at least 10 times the robot accuracy. In one specific embodiment, the incremental 2D scales 171'/172' can have an even higher periodicity of about 10 microns, achieving an accuracy of about 1 micron when the magnification of each camera is about 1 and interpolation is performed at 10x.

In various embodiments, the position of the field of view FOV of each camera (e.g., cameras 161A, 161B, etc.) within the incremental 2D scale 171'/172' can provide an indication of the relative position of the 2D scale 171'/172' with a corresponding reference position (e.g., reference position REF1, REF2, etc.). In various embodiments, each camera (e.g., cameras 161A, 161B, etc.) can be combined with the incremental 2D scale 171'/172' and used as part of a camera/scale image processing configuration. For example, the measurement position coordinate processing unit 190 can further determine the relative incremental position of the 2D scale 171'/172' and the corresponding reference position (e.g., reference position REF1 corresponding to and/or indicating the position of the corresponding camera 161A) based on the position of the field of view FOV within the incremental 2D scale 171'/172' (e.g., according to one or more positions and orientations of the incremental imageable elements IIF) indicated by the portion and orientation of the 2D scale 171'/172' in the acquired image (which may indicate, for example, the position and orientation of the 2D scale 171'/172' relative to each camera and reference position), as is known in the art of camera/scale image processing techniques (e.g., as described in the references incorporated herein as mentioned above). In various embodiments, the incremental 2D scales 171'/172' can be various sizes relative to the field of view FOV (e.g., the incremental 2D scales 171'/172' can be larger than the FOV so that as the 2D scales move relative to each camera, the captured image is included as part of the 2D scale; the 2D scales can be at least 2x, 4x, etc., the field of view FOV).

In various embodiments, the incremental positions indicated by 2D scales 171'/172' can be combined with other 2D scales, other sensors, and/or position information from articulated robot 110 to determine a relatively precise and/or absolute position (e.g., of the end tool). For example, sensors SEN1 and SEN2 (e.g., rotary encoders) on articulated robot 110 can indicate end tool position ETP with robot precision, and the incremental positions indicated by 2D scales 171'/172' (e.g., 2D scales 171A, 171B, 172A, and 172B) can be used to determine end tool position ETP with precision greater than the robot precision and/or to further refine the determined end tool position ETP. In one such embodiment, the measurement position coordinate processing unit 190 can be configured to identify one or more incrementally imageable elements IIF included in the acquired images of each 2D scale 171'/172' and determine the image positions of the one or more incrementally imageable elements IIF within the acquired images.

As described above with reference to FIG. 2, in one specific embodiment, according to a local scale coordinate system (e.g., which may be related to a corresponding local camera coordinate system and is different from the robot coordinate system, although conversions between various coordinate systems are possible), the 2D scale 171'/172' can be specified to have a reference position (e.g., an origin position) at X0, Y0, Z0 (e.g., the origin position can have values of 0, 0, 0). In such a configuration, the reference position REF1 (e.g., reference position REF1) can exist at relative coordinates X1, Y1, Z1, and the corresponding center of the field of view FOV (e.g., captured in the acquired image) can exist at relative coordinates X1, Y1, Z0. In various embodiments, in the scale coordinate system, all coordinates on the 2D scale have a Z position of Z0, and the corresponding reference position (e.g., reference position REF1, which can correspond to and/or indicate the position of camera 161A) has a different relative Z position with respect to the 2D scale, the corresponding Z position being Z1. In various embodiments, the center of the field of view FOV, at coordinates X1, Y1, is along the optical axis (e.g., optical axis OA1) of each camera (e.g., camera 161A), which in some configurations can be assumed to be nominally orthogonal to the 2D scale. The reference position REF1 is also along the optical axis and may have the same XY coordinates X1, Y1 as the center of the field of view FOV.

During operation, the acquired images can be analyzed by the measurement position coordinate processor 190 to determine the X1, Y1 coordinates corresponding to the center of each camera's field of view (FOV). In various embodiments, such determination can be performed according to standard camera/scale image processing techniques for determining the position of the field of view (e.g., corresponding to the camera position) within the scale (e.g., within the 2D scale 171'/172'). It will be appreciated that, in accordance with standard camera/scale image processing techniques, the reference/origin positions X0, Y0, Z0 need not be within the field of view (FOV) to make such a determination (i.e., the relative position can be determined from scale information for any position along the 2D scale 171'/172', provided in part by scale elements including evenly spaced incremental imageable elements IIF). In various embodiments, such determination can include identifying at least one imageable element included within the acquired images of the 2D scale and its associated known 2D scale coordinate location. Such determination may correspond to determining the relative position of the 2D scale 171'/172' with respect to a corresponding reference position (e.g., reference position REF1, REF2, etc.).

As described above, once the relative position of each of the 2D scales is determined, such information can be used for other position determination and/or control processes (e.g., for determining and/or controlling the end tool position ETP, etc.). As described above, in some embodiments, the relative position of each of the 2D scales can be initially expressed/determined in a local coordinate system (e.g., scale and/or camera coordinate system, etc.) and then transformed or otherwise processed with respect to the robot coordinate system. The end tool position ETP can be determined and/or controlled according to the robot coordinate system and/or other coordinate systems.

Figure 7 is an isometric view of an exemplary embodiment of absolute 2D scale 171"/172". In the example of Figure 7, like incremental 2D scale 171"/172", absolute 2D scale 171"/172" includes an array of evenly spaced incremental imageable elements IIF and also includes a set of absolute imageable elements AIF having a unique identifiable pattern (e.g., a 16-bit pattern). During operation, the position of the field of view FOV (i.e., contained in a captured image) within absolute 2D scale 171"/172" provides an indication of the absolute position between 2D scale 171"/172" and a corresponding reference position (e.g., reference position REF1 corresponding to and/or indicating the position of corresponding camera 161A). In the embodiment of FIG. 7 , the set of absolute imagable elements AIF are distributed on the substrate SUB at distances (e.g., intervals XSP2 and YSP2) that are less than the distance corresponding to the diameter of the field of view FOV of each camera (i.e., such that at least one absolute imagable element AIF is always included within the field of view). In operation, the measurement position coordinate processing unit 190 is configured to identify at least one absolute imagable element AIF included in an acquired image of the 2D scale 171"/172" based on the unique identifiable pattern of each absolute imagable element AIF, as part of a process for determining the absolute relative position of the 2D scale 171"/172" (e.g., corresponding to or otherwise indicating the relative position or orientation of the 2D scale 171"/172" with respect to each camera, a reference position, etc.).

A specific example of using absolute imagable elements AIF to determine relatively precise absolute positions is as follows. As shown in FIG. 7, an acquired image may show that the center of the field of view FOV is at the center of multiple incremental imagable elements IIF. Position information from the two included absolute imagable elements AIF indicates which section of the 2D scale 171"/172" the image includes, and therefore the included incremental imagable elements IIF of the 2D scale can also be identified. Therefore, by analyzing the acquired image using the measurement position coordinate processing unit 190, it is possible to precisely determine where the center of the field of view (i.e., coordinates X1, Y1, Z0) is within that section of the 2D scale (i.e., including the two absolute imagable elements and multiple incremental imagable elements IIF).

Figure 8 is a block diagram of a third exemplary embodiment of a robot system 800 including a robot 810 and a supplemental measurement position determination system 850. The robot 810 (e.g., an articulated robot) includes a movable arm configuration MAC' and a robot movement control and processing system 840. The supplemental measurement position determination system 850 may include at least cameras 862A, 862B, 863A, 863B, and 864A, scales 872A, 872B, 873A, 873B, and 874A, and a measurement position coordinate processing unit 890.

8, the movable arm configuration MAC' includes a lower support base portion BSE', arm portions 821-825, movement mechanisms 831-835, position sensors SEN1'-SEN5', and an end tool mounting configuration ETMC'. As described in more detail below and also shown in FIG. 9, each of the arm portions 821-825 can have a proximal end PE1-PE5 and a distal end DE1-DE5, respectively. In various embodiments, some or all of the arm portions 821-825 can be mounted to a respective movement mechanism 831-835 at a respective proximal end PE1-PE5 of each arm portion 821-825. In the example of FIG. 8, some or all of the movement mechanisms 831-835 (e.g., rotary joints and/or linear actuators with corresponding motors, etc.) enable movement (e.g., rotation, linear movement, etc.) of each arm portion 821-825 (e.g., about or along each rotation axis RA1'-RA5'). In various embodiments, position sensors SEN1'-SEN5' (e.g., rotary encoders, linear encoders, etc.) can be used to determine the position (e.g., angular orientation, linear position, etc.) of each arm portion 821-825.

In various embodiments, the movable arm configuration MAC' can have a portion designated as a terminal end (e.g., fifth arm portion 825). In the exemplary configuration of FIG. 8 , the end tool mounting configuration ETMC' is positioned near (e.g., at) the distal end DE5 of the fifth arm portion 825, which corresponds to the distal end of the movable arm configuration MAC' (e.g., designated as a terminal end). In various alternative embodiments, the terminal end of the movable arm configuration can be an element that is not an arm portion (e.g., a rotatable element, etc.), but at least a portion of the terminal end corresponds to the distal end of the movable arm configuration where the end tool mounting configuration ETMC' is located.

In various embodiments, the end tool mounting configuration ETMC' may include various elements for coupling and maintaining the end tool ETL near the distal end of the movable arm configuration MAC'. For example, in various embodiments, the end tool mounting configuration ETMC' may include an autojoint connection, a magnetic coupling, and/or other coupling elements known in the art for mounting the end tool ETL to a corresponding element. The end tool mounting configuration ETMC' may also include electrical connections (e.g., a power connection, one or more signal lines, etc.) for providing and/or transmitting power and/or signals to and from at least a portion of the end tool ETL (e.g., to and from the end tool sensing unit ETSN).

In various embodiments, the end tool ETL may include an end tool sensing portion ETSN and an end tool stylus ETST having a contact point CP (e.g., for contacting the surface of the workpiece WP). The fifth movement mechanism 835 is positioned near the distal end DE4 of the fourth arm portion 824. In various embodiments, the fifth movement mechanism 835 (e.g., a rotary joint with a corresponding motor) may be configured to rotate the fifth arm portion 825 about a rotation axis RA5'. In some embodiments, the fifth movement mechanism 835 may additionally or alternatively include a different type of movement mechanism (e.g., a linear actuator) configured to move the fifth arm portion 825 linearly (e.g., up and down). In either case, the end tool ETL is mounted (e.g., coupled) to the end tool mounting configuration ETMC' and has a corresponding end tool position ETP' with corresponding measurement position coordinates (e.g., x, y, and z coordinates in the robot coordinate system). In various embodiments, the end tool position ETP' may correspond to or be near the position of the end tool mounting configuration ETMC' (e.g., at or near the distal end DE5 of the fifth arm portion 825, which corresponds to the distal end of the movable arm configuration MAC').

The motion control system 840 is configured to control the end tool position ETP' of the end tool ETL with a level of precision defined as robot precision. More specifically, the motion control system 840 is generally configured to control the measured position coordinates (e.g., x, y, and z coordinates) of the end tool position ETP' with robot precision based, at least in part, on the use of motion mechanisms 831-835 and position sensors SEN1'-SEN5' to sense and control the positions of the arm sections 821-825. In various embodiments, the motion control and processing system 840 can include motion mechanism control and sensing units 841-845, which can receive signals from each position sensor SEN1'-SEN5', respectively, to sense the position (e.g., angular position, linear position, etc.) of each arm section 821-825 and/or provide control signals to each motion mechanism 831-835 (including, for example, rotary joints, linear actuators, motors, etc.) to move each arm section 821-825.

Additionally, the motion control and processing system 840 and/or the supplemental metrology position determination system 850 may receive signals from the end tool sensing unit ETSN. In various embodiments, the end tool sensing unit ETSN may include circuitry and/or configurations related to the operation of the end tool ETL to sense the workpiece WP. As described in more detail below, in various embodiments, the end tool ETL (e.g., a touch probe, a scanning probe, a camera, etc.) is utilized to contact or otherwise sense surface locations/positions/points on the workpiece WP, and various corresponding signals are received, determined, and/or processed by the end tool sensing unit ETSN, which may provide corresponding signals to the motion control and processing system 840 and/or the supplemental metrology position determination system 850. In various embodiments, the motion control and processing system 840 and/or the supplemental metrology position determination system 850 can include an end tool control and sensing unit 846, which can provide control signals to and/or receive sensing signals from the end tool sensing unit ETSN. In various embodiments, the end tool control and sensing unit 846 and the end tool sensing unit ETSN can be merged and/or indistinguishable. In various embodiments, the motion mechanism control and sensing units 841-845 and the end tool control and sensing unit 846 can all provide outputs to and/or receive control signals from a robot position processor 847. The robot position processor 847, as part of the robot motion control and processing system 840, can control and/or determine the overall position of the movable arm configuration MAC' and corresponding end tool position ETP' of the robot 810.

In various embodiments, the supplemental metrology positioning system 850 may be included with the robot 810 or otherwise added to the robot 810 (e.g., as part of a retrofit configuration for addition to an existing robot 810, etc. In various embodiments, the robot may be an articulated robot, a SCARA robot, a Cartesian coordinate robot, a cylindrical coordinate robot, a spherical coordinate robot, etc.). Generally, the supplemental metrology positioning system 850 may be used to improve the level of accuracy in determining the end tool position ETP'. More specifically, as described in more detail below, the supplemental metrology positioning system 850 may be used to determine measurement position coordinates indicative of the end tool position ETP' with a level of accuracy better than the robot accuracy.

As shown in FIG. 8, the supplemental metrology positioning system 850 includes five camera/scale sets including cameras 862A, 862B, 863A, 863B, and 864A, each pointed at a corresponding scale 872A, 872B, 873A, 873B, and 874A. Scales 872A, 873A, and 874A are located on the robot's rotary joints (e.g., at coupling locations corresponding to the axes of rotation of each rotary joint), and scales 872B and 873B are located on the robot's arm. More specifically, scale 872A is coupled to the rotary joint of the second movement mechanism 832 at a first 2D scale coupling location, scale 872B is coupled to the second arm portion 822 at a second 2D scale coupling location, scale 873A is coupled to the rotary joint of the third movement mechanism 833 at a third 2D scale coupling location, scale 873B is coupled to the third arm portion 823 at a fourth 2D scale coupling location, and scale 874A is coupled to the rotary joint of the fourth movement mechanism 834 at a fifth 2D scale coupling location. Each of the five cameras 862A, 862B, 863A, 863B, and 864A is controlled by and provides image signals to imaging configuration control and processing portions (ICCCPs) 882A, 882B, 883A, 883B, and 884A, respectively. The trigger unit 887, in some examples, can coordinate the triggering of all of the cameras to simultaneously acquire images (e.g., corresponding to the position of the robot at a particular time to determine the position of the end tool at the end of the robot at that time).

In various embodiments, the end tool working volume ETWV' comprises a range within which at least a portion of the end tool ETL can be moved. In the example of FIG. 8, the end tool working volume ETWV' is illustrated as including a range within which the contact point CP of the end tool ETL can be moved when inspecting a workpiece. In various embodiments, the robot 810 is configured to move the movable arm configuration MAC' to move at least a portion (e.g., contact point CP) of the end tool ETL mounted in the end tool mounting configuration ETMC' in at least two dimensions (e.g., x and y dimensions) within the end tool working volume ETWV'. In the example of FIG. 8, a portion (e.g., contact point CP) of the end tool ETL can be moved in three dimensions (e.g., x, y, and z dimensions) by the robot 810.

In various embodiments, as described above with reference to Figures 5-7, each of 2D scales 872A, 872B, 873A, 873B, and 874A includes a nominally planar substrate SUB and a plurality of imagable elements distributed on the substrate SUB. As used herein, the term "nominal" encompasses variations in one or more parameters that fall within acceptable tolerances. Each of the imagable elements is positioned at known local x and y scale coordinates on each 2D scale. In various embodiments, each 2D scale can be an incremental scale or an absolute scale.

In various embodiments, the imaging and detection configuration control and processing unit 880, the trigger unit 887, and/or the measurement position coordinate processing unit 890 can be included as part of an external control system ECS' (e.g., as part of an external computer, etc.). The trigger unit 887 can be included as part of the imaging and detection configuration control and processing unit 880. In various embodiments, the trigger unit 887 is configured to receive at least one input signal related to the end tool position ETP', determine the timing of a first trigger signal based on the at least one input signal, and output the first trigger signal to the cameras 862A, 862B, 863A, 863B, and 864A. In various embodiments, the cameras 862A, 862B, 863A, 863B, and 864A are each configured to acquire a digital image of the corresponding 2D scale 872A, 872B, 873A, 873B, and 874A at the time of image acquisition in response to receiving the first trigger signal. In various embodiments, the measurement position coordinate processor 890 is configured to input the acquired image and identify at least one imagable element included in the acquired image at 2D scale and an associated known 2D scale coordinate location. In various embodiments, the external control system ECS' can also include a standard robot position coordinate mode component 849 and a supplemental measurement position coordinate mode component 892 to implement corresponding modes, which are described in more detail below.

In various embodiments, each imaging configuration control and processing unit 882A, 882B, 883A, 883B, and 884A can include a component (e.g., a subcircuit, routine, etc.) that periodically (e.g., at set timing intervals) activates image integration of the corresponding camera 862A, 862B, 863A, 863B, and 864A, and a first trigger signal from trigger unit 887 can activate strobe light timing (e.g., each camera 862A, 862B, 863A, 863B, and 864A can include a strobe light) or other mechanism that effectively stops motion and correspondingly determines exposure during the integration period. In such embodiments, if the first trigger signal is not received during an integration period, the resulting image can be discarded; if the first trigger signal is received during an integration period, the resulting image can be saved and/or otherwise processed/analyzed to determine measurement position coordinates, as described in more detail below.

In various embodiments, different types of end tool ETLs may provide different types of outputs, which may be used in conjunction with the trigger unit 887. For example, in an embodiment where the end tool ETL is a touch probe used to measure a workpiece and outputs a touch signal when the workpiece is touched (e.g., when a contact point CP touches the workpiece), the trigger unit 887 may be configured to receive the touch signal or a signal derived therefrom as at least one input signal and determine the timing of the first trigger signal based thereon. As another example, in an embodiment where the end tool ETL is a scanning probe used to measure a workpiece and provides workpiece measurement sample data corresponding to each sample timing signal, the trigger unit 887 may be configured to receive the sample timing signal or a signal derived therefrom as at least one input signal. As another example, in an embodiment where the end tool ETL is a camera used to provide workpiece measurement images corresponding to each workpiece image acquisition signal, the trigger unit 887 may be configured to receive the workpiece image acquisition signal or a signal derived therefrom as at least one input signal.

In the exemplary embodiment of FIG. 8 , the supplemental metrology position determination system 850 is configured such that the measurement position coordinate processor 890 is operable to determine the relative position of each 2D scale 872A, 872B, 873A, 873B, and 874A and a corresponding reference position REF1', REF2', REF3', REF4', REF5' (e.g., corresponding to and/or indicative of the position of each camera 862A, 862B, 863A, 863B, and 864A) based, for example, on determining the image position of at least one imagable element identified in each acquired image. The determined relative positions can be used (e.g., by the supplemental metrology position determination system 850) to determine the measurement position coordinate of the end tool position ETP' at the time of image acquisition with a level of accuracy better than robot accuracy.

It will be appreciated that robotic systems such as those shown in FIGS. 1 and 8 may have several advantages over various alternative systems. For example, in various embodiments, systems such as those disclosed herein may be smaller and/or less expensive, and in some embodiments, more accurate, than alternative systems that utilize technologies such as laser trackers or photogrammetry to track robot movement/position. Additionally, while the disclosed systems do not occupy or obscure any portion of the end tool working volume ETWV or ETWV', alternative systems may include scales or fiducials on the ground, stage, or elsewhere within the area where operations and/or inspections, etc., may be performed on a workpiece (e.g., the end tool working volume ETWV or ETWV'). Furthermore, in various embodiments, by coupling all of the cameras and scales to the robot (including, for example, being coupled to the moving parts of a movable arm configuration such as an arm section and rotary joint), no external structure or external couplings in the robot environment are required for the cameras or scales.

FIG. 9 is an isometric view of a portion of a fourth illustrative embodiment of a robotic system 900 substantially similar to the robotic system 800 of FIG. 8. Similar to the numbering scheme described above, it will be appreciated that some named or numbered components (e.g., 8XX, 8XX', or 9XX) in FIG. 9 correspond to and/or have similar operation as, can be understood to be similar or identical to, or otherwise understood by analogy and as described below, identically or similarly named or numbered corresponding components (e.g., 8XX) in FIG. 8 or other figures. As noted above, this naming and numbering scheme for indicating elements having similar and/or identical design and/or function generally applies to the various figures of the present application (e.g., FIGS. 1 through 11).

As shown in FIG. 9 , the first arm portion 821 (e.g., the upper base portion) is mounted to a first movement mechanism 831 (e.g., including a rotary joint) at the proximal end PE1 of the first arm portion 821. The first movement mechanism 831 is positioned at the upper end of the lower support base portion BSE' and has a rotation axis RA1' such that the first arm portion 821 rotates in a nominally horizontal plane. In various embodiments, a position sensor SEN1' (e.g., a rotary encoder) can be used to determine the angular position (e.g., angular orientation) of the first arm portion 821.

A second movement mechanism 832 (e.g., including a rotary joint) is positioned near the distal end DE1 of the first arm portion 821. The second movement mechanism 832 has a rotation axis RA2'. The second arm portion 822 is mounted to the second movement mechanism 832 at the proximal end PE2 of the second arm portion 822 so as to move about the second movement mechanism 832. In various embodiments, a position sensor SEN2' (e.g., a rotary encoder) can be used to determine the angular position of the second arm portion 822.

A third movement mechanism 833 (e.g., including a rotary joint) is positioned at the distal end DE2 of the second arm portion 822. The third movement mechanism 833 has a rotation axis RA3'. The third arm portion 823 is mounted to the third movement mechanism 833 at the proximal end PE3 of the third arm portion 823 for movement about the third movement mechanism 833. In various embodiments, a position sensor SEN3' (e.g., a rotary encoder) can be used to determine the angular position of the third arm portion 823.

A fourth movement mechanism 834 (e.g., including a rotary joint) is positioned at the distal end DE3 of the third arm portion 823. The fourth movement mechanism 834 has a rotation axis RA4'. The fourth arm portion 824 is rotationally mounted to the fourth movement mechanism 834 at the proximal end PE4 of the fourth arm portion 824. In various embodiments, a position sensor SEN4' (e.g., a rotary encoder) can be used to determine the angular position of the fourth arm portion 824.

The fifth movement mechanism 835 can be located at the distal end DE4 of the fourth arm portion 824. As described above, in some embodiments, the fifth movement mechanism 835 (e.g., including a rotary joint) can be configured to rotate the fifth arm portion 825 about a rotation axis RA5'. In such a configuration, the fifth arm portion 825 can be mounted to the fifth movement mechanism 835 at the proximal end PE5 of the fifth arm portion 825. In some embodiments, the fifth movement mechanism 835 can additionally or alternatively include a different type of movement mechanism (e.g., a linear actuator) configured to move the fifth arm portion 825 linearly (e.g., up and down). In various embodiments, the fifth arm portion 825 can be designated as the terminal end of a movable arm configuration MAC', the distal end of which corresponds to the distal end DE5 of the fifth arm portion 825 at which the end tool mounting configuration ETMC' can be positioned. In embodiments in which the fifth movement mechanism 835 includes a rotary joint, the end tool ETL can be rotated accordingly in several configurations (e.g., in the xy plane orthogonal to the z-axis in some examples).

In various embodiments, different reference axes and lines may be designated to refer to specific movements, coordinates, and angles of components of the movable arm configuration MAC'. As some specific examples, as shown in FIG. 9, second and third arm sections 822 and 823 may be designated with centerlines CT2' and CT3', respectively, passing through the center of each arm section. It will be appreciated that the other arm sections 821, 824, and 825 may similarly have corresponding reference lines and/or axes, etc., to refer to specific movements, coordinates, and angles of components of the movable arm configuration MAC'.

In various embodiments, the end tool ETL can be mounted (e.g., coupled) to the end tool mounting configuration ETMC' near the distal end DE5 of the fifth arm portion 825. The end tool ETL can be specified to have an end tool axis EA (e.g., passing through the central axis and/or center axis of the stylus ETST) that can coincide with the fifth rotational axis RA5' of the fifth movement mechanism 835 and intersect with an extension of the fourth rotational axis RA4' of the fourth movement mechanism 834. In various embodiments, the end tool axis EA passes through an end tool position ETP'. The end tool position ETP' can be specified to have coordinates X2, Y2, Z2 (e.g., in the robot coordinate system). In various embodiments, the end tool ETL can have a contact point CP (e.g., the end of the end tool stylus ETST for contacting the workpiece) that can be specified to have coordinates X3, Y3, Z3.

In one specific example, each image captured by each camera 862A, 862B, 863A, 863B, and 864A can be analyzed by a measurement position coordinate processor 890 to determine the relative position (e.g., corresponding to the position, orientation, etc. of each scale 872A, 872B, 873A, 873B, and 874A). Such determination can be performed according to standard camera/scale image processing techniques (e.g., to determine the position, orientation, etc. of the camera relative to the scale). Examples of such techniques are described in the aforementioned incorporated U.S. Patent Nos. 6,781,694, 6,937,349, 5,798,947, 6,222,940, and 6,640,008. In various embodiments, such techniques may be used to determine the position of a field of view (e.g., corresponding to a camera's position, orientation, etc.) within a scale range (e.g., within each of 2D scales 872A, 872B, 873A, 873B, and 874A), as described above with reference to FIGS. 5-7 . In various embodiments, such determination may include identifying at least one imagable element included in a captured image of each of the 2D scales and its associated known 2D scale coordinate location. Such determination may correspond to determining the relative position of each of 2D scales 872A, 872B, 873A, 873B, and 874A to a corresponding reference position REF1′, REF2′, REF3′, REF4′, and REF5′ (e.g., corresponding to and/or indicating the position of each of cameras 862A, 862B, 863A, 863B, and 864A).

FIG. 10 is a side view of a portion of a robotic system similar to that shown in FIGS. 8 and 9. More specifically, FIG. 10 shows a side view of second, third, and fifth arm sections 822, 823, and 825, second, third, and fourth movement mechanisms 832, 833, and 834, and scales 872A, 873A, and 874. Scale 872A is coupled to the rotary joint of the second movement mechanism 832 at a first 2D scale coupling location, scale 873A is coupled to the rotary joint of the third movement mechanism 833 at a second 2D scale coupling location, and scale 874A is coupled to the rotary joint of the fourth movement mechanism 834 at a third 2D scale coupling location. Movement of the rotary joint of the second movement mechanism 832 perpendicular to the rotation axis RA2' during rotation changes the relative position of scale 872A (e.g., along the x- and/or y-axes of the camera coordinate system). Movement of the third movement mechanism 833 perpendicular to the axis of rotation RA3′ during rotation of the rotary joint changes the relative position of the scale 873A (e.g., along the x- and/or y-axes of the camera coordinate system). Movement of the fourth movement mechanism 834 perpendicular to the axis of rotation RA4′ during rotation of the rotary joint changes the relative position of the scale 874A (e.g., along the x- and/or y-axes of the local camera coordinate system). Cameras 862A, 863A, and 864A are positioned to capture images of the elements included in scales 872A, 873A, and 874A, respectively. Based on these images, the measurement position coordinate processing unit 890 determines the angular orientations of the scales 872A, 873A, and 874A and also determines the relative positions of the scales 872A, 873A, and 874A (e.g., in x- and y-axis coordinates of the local camera coordinate system). This is used by the measurement position coordinate processor 890 to detect any unwanted movement perpendicular to each of the rotation axes RA2', RA3', and RA4'. The determined relative positions of scales 872A, 873A, and 874A can be used by the measurement position coordinate processor 890 to detect so-called "wobble," "rocking," or other movement of the rotary joints of the second, third, and fourth movement mechanisms 832, 833, and 834. Such movement would not normally be detected or revealed by standard robot encoders, and would therefore introduce measurement errors into determining the position of the end tool at the end of the robot arm.

FIG. 11 is a side view of a portion of a robotic system similar to that shown in FIGS. 8 and 9. More specifically, FIG. 11 shows a side view of the second arm portion 822, the camera 862B, and the scale 872B. Although not shown in FIG. 11, the scale 872B is positioned near a rotary joint of the third movement mechanism 833, which is coupled to the third arm portion 823 (see, for example, FIG. 9). In various embodiments, the second arm portion 822, the third arm portion 823, and/or other components coupled to the third arm portion 823 may be heavy enough to cause bending or twisting of the second arm portion 822, as shown by the dashed lines in FIG. 11. Such bending or twisting may change the relative position of the scale 872B, causing the scale 872B to move from an expected position P1' to a bent position P2'. As the scale 872B moves relative to the camera 862B, a corresponding change in movement/position D12' can be detected/measured. More specifically, when second arm portion 822 is bent as shown, the field of view of camera 862B shows a different portion of scale 872B (e.g., a portion closer to the top of scale 872B in the orientation shown), and by analyzing that image, the amount of bending of second arm portion 822 corresponding to the position of scale 872B can be determined (e.g., based at least in part on determining different positions of one or more imagable elements of the scale in different images, etc.).

12A and 12B are flow diagrams illustrating exemplary embodiments of routines 1200A and 1200B for operating a robotic system including an articulated robot and a supplemental metrology position determination system. As shown in FIG. 12A, decision block 1210 determines whether the robotic system is to operate in supplemental metrology position coordinate mode. In various embodiments, selection and/or activation of supplemental metrology position coordinate mode or standard robot position coordinate mode can be performed by a user and/or automatically by the system in response to specific operations and/or commands. For example, in one embodiment, the supplemental metrology position coordinate mode can be initiated (e.g., automatically or pursuant to user selection) when the articulated robot is moved to a specific location (e.g., when the end tool is moved from a general area where assembly or other operations are performed to a specific area where workpiece inspection operations are typically performed and supplemental metrology position coordinate mode is utilized). In various embodiments, such a mode can be implemented by an external control system ECS (e.g., the external control system ECS of FIG. 1 using standard robot position coordinate mode portion 147 and supplemental metrology position coordinate mode portion 192). In various embodiments, the hybrid mode may be implemented to operate independently or as part of the supplemental measurement position coordinate mode and/or as a switch between modes, as described in more detail below with reference to FIG. 13.

If, at decision block 1210, it is determined not to operate the robot system in the supplemental metrology position coordinate mode, the routine proceeds to block 1220 and operates the robot system in the standard robot position coordinate mode. As part of the standard robot position coordinate mode, the articulated robot's position sensors (e.g., rotary encoders) are used to control and determine articulated robot movements and corresponding end tool positions with robot accuracy (e.g., based at least in part on the accuracy of the articulated robot's position sensors). In general, the robot position coordinate mode may correspond to an independent and/or standard operating mode of the articulated robot (e.g., a mode in which the articulated robot operates independently when the supplemental metrology position coordinate determination system is inactive or otherwise not provided).

If the robot system is to operate in the supplemental measurement position coordinate mode, the routine proceeds to block 1230 and receives at least one input signal (i.e., at a trigger unit) related to the end tool position of the articulated robot. Based on the at least one input signal, the routine determines the timing of a first trigger signal and outputs the first trigger signal to cameras of the supplemental measurement position determination system. In response to receiving the first trigger signal, each camera acquires a corresponding 2D scale digital image at the image acquisition time. In block 1240, the acquired images are received (e.g., at a measurement position coordinate processor) and, for each image, at least one imagable element included in the acquired 2D scale image and each associated known 2D scale coordinate position are identified.

At block 1250, the relative position of each 2D scale is determined based on determining the image position of at least one imagable element identified in each acquired image. At block 1260, the determined position information (e.g., including the determined relative positions and/or other related determined position information) is used for a specified function (e.g., for determining measurement position coordinates for an end tool position, workpiece measurement, position control of an articulated robot, etc.). As part of such an operation or others, the routine then proceeds to point A, where in various embodiments the routine may end, or may continue as described in more detail below with reference to FIG. 12B.

As shown in FIG. 12B, routine 1200B may continue from point A to block 1270. As described in more detail below, as part of routine 1200B, the determined position information (e.g., from block 1260) may correspond to or be used to determine a first surface position on the workpiece. A second surface position on the workpiece may then be determined (e.g., as part of workpiece measurement). In block 1270, at least one second input signal related to the end tool position is received (e.g., at a trigger) and the timing of a second trigger signal is determined based on the at least one second input signal. The second trigger signal is output to cameras of the supplemental metrology position determination system. Each camera, in response to receiving the second trigger signal, acquires a second digital image at a corresponding 2D scale at the second image acquisition time point.

In block 1280, the acquired images are received (e.g., by a measurement position coordinate processor) and, in each image, at least one second imagable element included in the second acquired image of the 2D scale and an associated second known 2D scale coordinate location are identified. In block 1290, a second relative position of each 2D scale is determined based on determining the second image location of the at least one second imagable element identified in each second acquired image.

In block 1295, the determined relative positions and/or associated position information are used to determine a dimension of the workpiece corresponding to the distance between the first and second surface locations on the workpiece corresponding to the respective end tool positions (e.g., indicating contact points, etc.) at the time of the first and second image capture. It will be appreciated that rather than using a position sensor (e.g., a rotary encoder) on an articulated robot to determine the first and second surface locations on the workpiece with robotic precision, more accurate position information can be determined using techniques such as those described above.

FIG. 13 is a flow diagram illustrating one exemplary embodiment of a routine 1300 for determining end tool position, which can use different techniques during different portions of the move timing. Generally, during the move timing, one or more arms of the robot are moved from a first position to a second position (e.g., this may include rotating one or more arms from a first rotational orientation to a second rotational orientation about a movement mechanism, or moving the arms in other ways). As shown in FIG. 13, decision block 1310 determines whether to utilize hybrid mode for determining end tool position during the move timing. In various embodiments, hybrid mode may also represent a process that includes switching between a supplemental measurement position coordinate mode and a standard robot position coordinate mode, as described above with reference to FIG. 12A. If hybrid mode is not utilized, the routine proceeds to block 1320 and solely utilizes the robot's position sensors (e.g., rotary encoders, linear encoders, etc.) (e.g., of a movable arm configuration such as MAC or MAC′) to determine end tool position during the move timing.

If hybrid mode is used, the routine proceeds to block 1330 and uses a position sensor included in the robot (e.g., included in the robot's movable arm configuration MAC or MAC') to determine the end tool position during a first portion of the movement timing. During such operation, the supplemental metrology positioning system may not be used to determine the end tool position. In block 1340, the supplemental metrology positioning system is used to determine the end tool position during a second portion of the movement timing, which occurs after the first portion of the movement timing. It will be appreciated that such operation allows the system to perform an initial/fast/coarse movement of the end tool position during the first portion of the movement timing, and a highly accurate final/slow/fine movement of the end tool position during the second portion of the movement timing.

FIG. 14 is a flow diagram illustrating an exemplary implementation of a routine 1400 for operating a supplemental metrology position determination system for use with a robot. As shown in FIG. 14, in block 1410, a first camera is operated to capture a first image of a first 2D scale at a first image acquisition time point. The first camera is coupled to the movable arm arrangement of the robot at a first camera coupling position and defines a first reference position. The first 2D scale is coupled to the movable arm arrangement at a first 2D scale coupling position. For example, trigger unit 187 sends a control signal to camera 161A, causing camera 161A to capture a first image of scale 171A at the first image acquisition time point.

In block 1420, the second camera is operated to capture a first image of a second 2D scale at the first image acquisition time point. The second camera is coupled to the movable arm arrangement of the robot at a second camera coupling position and defines a second reference position. The second 2D scale is coupled to the movable arm arrangement at a second 2D scale coupling position. For example, the trigger unit 187 sends a control signal to camera 161B, causing camera 161B to capture a first image of scale 171B at the first image acquisition time point.

In block 1430, a first relative position of the first 2D scale is determined based, at least in part, on a first image of the first 2D scale acquired by the first camera at the time of first image acquisition. For example, the measurement position coordinate processing unit 190 determines the relative position of scale 171A based on a first image of scale 171A acquired by camera 161A in block 1410.

In block 1440, a first relative position of the second 2D scale is determined based, at least in part, on a first image of the second 2D scale acquired by the second camera at the time of first image acquisition. For example, the measurement position coordinate processing unit 190 determines the relative position of scale 171B based on a first image of scale 171B acquired by camera 161B in block 1420.

In some embodiments, after block 1440, measurement position coordinates of the first end tool position at the time of first image acquisition are determined based at least in part on the determined first relative positions of the first and second 2D scales. For example, the measurement position coordinate processor 190 determines measurement position coordinates (X2, Y2, Z2) of the end tool position ETP (e.g., see FIG. 2) based at least in part on the determined first relative positions of the scales 171A and 171B.

In some embodiments, after block 1440, the method further includes operating the first camera to acquire a second image at the first 2D scale at the second image acquisition time point; operating the second camera to acquire a second image at the second 2D scale at the second image acquisition time point; determining a second relative position of the first 2D scale based, at least in part, on the second image at the first 2D scale acquired by the first camera at the second image acquisition time point; and determining a second relative position of the second 2D scale based, at least in part, on the second image at the second 2D scale acquired by the second camera at the second image acquisition time point. For example, the acts described in association with blocks 1410, 1420, 1430, and 1440 may be repeated at a different time point (i.e., at the second image acquisition time point).

In some embodiments, the method further includes determining measurement position coordinates of a first end tool position at the time of first image acquisition based at least in part on the determined first relative positions of the first and second 2D scales, and determining measurement position coordinates of a second end tool position at the time of second image acquisition based at least in part on the determined second relative positions of the first and second 2D scales. For example, the measurement position coordinate processing unit 190 determines measurement position coordinates (X2a, Y2a, Z2a) of the end tool position ETP based at least in part on the determined first relative positions of scales 171A and 171B, and determines measurement position coordinates (X2b, Y2b, Z2b) of the end tool position ETP based at least in part on the determined second relative positions of scales 171A and 171B. In some embodiments, the method further includes determining a dimension related to the distance between the first and second end tool positions using the determined measurement position coordinates of the first and second end tool positions. For example, the measurement position coordinate processing unit 190 calculates the distance between the first and second end tool positions using the above-described measurement position coordinates (X2a, Y2a, Z2a) and measurement position coordinates (X2b, Y2b, Z2b) and determines a dimension related to the distance between the first and second end tool positions. This dimension may be or correspond to the distance between the first and second surface positions on the workpiece. In this case, the contact point of the end tool contacts the first surface position on the workpiece at the time of acquiring the first image, and contacts the second surface position on the workpiece at the time of acquiring the second image.

In some embodiments, prior to block 1410, the method further includes coupling first and second 2D scales to the movable arm configuration at first and second 2D scale coupling positions, respectively; coupling a first camera to the movable arm configuration at a first camera coupling position; and coupling a second camera to the movable arm configuration at a second camera coupling position. For example, the method includes coupling scale 171A to the first arm portion 121 at scale coupling position CL2, coupling scale 171B to the first arm portion 121 at scale coupling position CL4, coupling camera 161A to the support base portion BSE at camera coupling position CL1, and coupling camera 161B to the first arm portion 121 at camera coupling position CL3.

In other examples, it will be appreciated that the first and second cameras may be any of cameras 161A, 161B, 162A, 162B, 862A, 862B, 863A, 863B, 864A, etc., and the first and second 2D scales may be any of scales 171A, 171B, 172A, 172B, 872A, 872B, 873A, 873B, 874A, etc., and correspondingly any of the configurations described herein.

While the element name "2D scale" has been used in this disclosure to refer to relative scale elements and the like, it will be understood that this element name is merely exemplary and not limiting. This "2D scale" is described as being associated with x and y scale coordinates as part of a Cartesian coordinate system and including a nominally planar substrate. More generally, however, the element name 2D scale should be understood to refer to any reference scale including a plurality of elements or marks corresponding to known two-dimensional coordinates (e.g., precise and/or precisely calibrated positions in two dimensions) on the scale, provided that the scale is operable as disclosed herein. For example, such scale elements may be expressed and/or indexed in Cartesian, polar, or any other convenient coordinate system on the reference scale. Furthermore, such elements may include elements uniformly or non-uniformly distributed throughout the operating scale area and may include graduated or ungraduated scale marks, provided that the elements correspond to known two-dimensional coordinates on the scale and are operable as disclosed herein.

While the robotic systems and corresponding movable arm configurations disclosed and described herein are generally illustrated and described with reference to a particular number of arms (e.g., three arms, five arms, etc.), it will be understood that such systems are not so limited. In various embodiments, while including arms such as those described herein, robotic systems can also include fewer or more arms, if desired.

As described herein, it will be understood that the 2D scale and the camera used to image the scale may rotate relative to one another (e.g., the scale is mounted on a rotary joint, etc.) depending on the movement and/or position of the robotic system. It will be appreciated that methods known in the art (e.g., as disclosed in the references incorporated herein) can be used to accurately determine such relative rotation and/or perform the necessary coordinate transformations and/or analyze the relative position of the camera and scale in accordance with the principles disclosed herein with respect to such relative rotation. It will be understood that in various embodiments, the measured position coordinates referred to herein take such relative rotation into account. It will further be understood that in some embodiments, the measured position coordinates referred to herein include coordinate sets that include precise determinations and/or indications of such relative rotation.

While preferred embodiments of the present disclosure have been shown and described, numerous variations in the arrangement of elements and sequence of operation shown and described will be apparent to those skilled in the art based on this disclosure. Various alternative configurations can be used to implement the principles disclosed herein. Furthermore, various embodiments described above can be combined to provide additional embodiments. All U.S. patents and U.S. patent applications referenced herein are incorporated by reference in their entirety. Aspects of the above-described embodiments can be modified as necessary to provide still further embodiments using concepts from these various patents and applications.

These and other changes can be made to the embodiments in light of the foregoing description. In general, in the following claims, the terms used should not be construed as limiting the scope of the claims to the specific embodiments disclosed in the specification and claims, but should be construed as embracing all possible embodiments along with the full range of equivalents to which such claims are entitled.

Claims (21)

A robot,
A movable arm arrangement,
a first arm portion mounted to a first rotary joint having a first axis of rotation at a proximal end of the first arm portion and having a second rotary joint having a second axis of rotation located at a distal end of the first arm portion;
a second arm mounted to the second rotary joint at a proximal end thereof for rotation about the second rotary joint;
an end tool mounting structure for mounting an end tool, the end tool mounting structure being provided near a distal end of the movable arm structure;
a movable arm arrangement comprising:
a motion control system configured to control an end tool position, the position of the end tool, to a level of precision defined as robot precision based at least in part on sensing and controlling the angular positions of the first arm section and the second arm section about the first rotary joint and the second rotary joint, respectively, using rotation sensors included in the robot; and
a robot comprising:
1. A supplemental measurement position determination system, comprising:
a first two-dimensional (2D) scale and a second 2D scale coupled to the movable arm arrangement at a first 2D scale coupling location and a second 2D scale coupling location, respectively, each 2D scale including a nominally planar substrate and a plurality of imagable elements distributed on the planar substrate;
a first camera for acquiring an image at the first 2D scale at an image acquisition time, the first camera defining a first reference position and coupled to the movable arm arrangement at a first camera coupling position;
a second camera for capturing an image at the second 2D scale at the image capture time, the second camera defining a second reference position and coupled to the movable arm arrangement at a second camera coupling position;
A measurement processing unit,
determining a first relative position of the first 2D scale based at least in part on a first image of the first 2D scale acquired by the first camera at a first image acquisition time;
determining a first relative position of the second 2D scale based, at least in part, on a first image of the second 2D scale acquired by the second camera at the time of the first image acquisition;
a measurement processing unit configured as above;
a supplemental measurement position determination system comprising:
A robot system comprising: The robot system of claim 1, wherein the measurement processing unit is configured to determine measurement position coordinates of the end tool position at the time of acquiring the first image based, at least in part, on the determined first relative position of the first 2D scale and the determined first relative position of the second 2D scale. the first 2D scale coupling location is on the first rotary joint;
2. The robotic system of claim 1, wherein movement of the first rotary joint perpendicular to the first axis of rotation during rotation causes a change in the first relative position of the first 2D scale. The robot system of claim 3, wherein the measurement processing unit is configured to determine the angular orientation of the first 2D scale based, at least in part, on the first image of the first 2D scale acquired by the first camera at the time of the first image acquisition. the first 2D scale attachment location is on the first arm portion;
The robotic system of claim 1 , wherein at least one of bending or twisting of the first arm portion causes a change in the first relative position of the first 2D scale. The measurement processing unit
determining a second relative position of the first 2D scale based at least in part on a second image of the first 2D scale acquired by the first camera at a second image acquisition time;
determining a second relative position of the second 2D scale based, at least in part, on a second image of the second 2D scale acquired by the second camera at the second image acquisition time;
The robot system according to claim 1 , wherein the robot system is configured as follows: The measurement processing unit
determining a measurement position coordinate of a first end tool position at the time of the first image acquisition based at least in part on the determined first relative position of the first and second 2D scales;
determining a measurement position coordinate of a second end tool position at the time of the second image acquisition based at least in part on the determined second relative position of the first and second 2D scales;
The robot system according to claim 6 , wherein the robot system is configured as follows: The robot system of claim 7, wherein the measurement processing unit is configured to determine a dimension related to the distance between the first end tool position and the second end tool position using the determined measurement position coordinates of the first end tool position and the second end tool position. The robot system of claim 8, wherein the dimension is a distance between a first surface location and a second surface location on a workpiece, and the contact point of the end tool contacts the first surface location on the workpiece at the time the first image is acquired and contacts the second surface location on the workpiece at the time the second image is acquired. The robot system of claim 9, wherein the end tool is at least one of a touch probe or a scanning probe used to measure the workpiece. 1. A method for operating a supplemental metrology position determination system for use with a robot, comprising:
The robot
A movable arm arrangement,
a first arm portion mounted to a first rotary joint having a first axis of rotation at a proximal end of the first arm portion and having a second rotary joint having a second axis of rotation located at a distal end of the first arm portion;
a second arm mounted to the second rotary joint at a proximal end thereof for rotation about the second rotary joint;
an end tool mounting structure for mounting an end tool, the end tool mounting structure being provided near a distal end of the movable arm structure;
a movable arm arrangement comprising:
a motion control system configured to control an end tool position, the position of the end tool, to a level of precision defined as robot precision based at least in part on sensing and controlling the angular positions of the first arm section and the second arm section about the first rotary joint and the second rotary joint, respectively, using rotation sensors included in the robot;
the supplemental measurement position determining system;
a first two-dimensional (2D) scale and a second two-dimensional scale, each of the 2D scales including a nominally planar substrate and a plurality of imageable elements distributed on the planar substrate;
a first camera and a second camera,
The method comprises:
operating the first camera to acquire a first image at the first 2D scale at a first image acquisition time;
the first camera is coupled to the movable arm arrangement of the robot at a first camera coupling position to define a first reference position;
the first 2D scale is coupled to the movable arm arrangement at a first 2D scale coupling location;
operating the first camera;
operating the second camera to acquire a first image at the second 2D scale at the first image acquisition time;
the second camera is coupled to the movable arm arrangement of the robot at a second camera coupling location to define a second reference location;
the second 2D scale is coupled to the movable arm arrangement at a second 2D scale coupling location;
operating the second camera;
determining a first relative position of the first 2D scale based, at least in part, on the first image of the first 2D scale acquired by the first camera at the first image acquisition time;
determining a first relative position of the second 2D scale based, at least in part, on the first image of the second 2D scale acquired by the second camera at the time of the first image acquisition;
A method comprising: The method of claim 11, further comprising determining measurement position coordinates of a first end tool position at the time of the first image acquisition based, at least in part, on the determined first relative position of the first 2D scale and the determined first relative position of the second 2D scale. operating the first camera to acquire a second image at the first 2D scale at a second image acquisition time point;
operating the second camera to capture a second image at the second 2D scale at the second image capture time;
determining a second relative position of the first 2D scale based, at least in part, on the second image of the first 2D scale acquired by the first camera at the second image acquisition time;
determining a second relative position of the second 2D scale based, at least in part, on a second image of the second 2D scale acquired by the second camera at the second image acquisition time;
The method of claim 11 further comprising: determining a measurement position coordinate of a first end tool position at the time of the first image acquisition based at least in part on the determined first relative position of the first 2D scale and the determined first relative position of the second 2D scale;
determining a measurement position coordinate of a second end tool position at the time of the second image acquisition based at least in part on the determined second relative position of the first 2D scale and the determined second relative position of the second 2D scale;
The method of claim 13 further comprising: 15. The method of claim 14, further comprising determining a dimension related to the distance between the first end tool position and the second end tool position using the determined measurement position coordinates of the first end tool position and the second end tool position. the dimension is a distance between a first surface location and a second surface location on the workpiece;
16. The method of claim 15, wherein a contact point of the end tool contacts the first surface location on the workpiece at the time of the first image acquisition and contacts the second surface location on the workpiece at the time of the second image acquisition. coupling the first 2D scale and the second 2D scale to the movable arm arrangement at the first 2D scale coupling location and the second 2D scale coupling location, respectively;
coupling the first camera to the movable arm arrangement at the first camera coupling location;
coupling the second camera to the movable arm arrangement at the second camera coupling location;
The method of claim 11 further comprising: 1. A supplemental metrology positioning system for use with a robot comprising: a movable arm arrangement having an end tool mounting arrangement for mounting an end tool; and a motion control system configured to control an end tool position of the end tool,
a first two-dimensional (2D) scale and a second 2D scale configured to be coupled to the movable arm arrangement of the robot at a first 2D scale coupling location and a second 2D scale coupling location, respectively, each 2D scale including a nominally planar substrate and a plurality of imagable elements distributed on the planar substrate;
a first camera for acquiring an image at the first 2D scale at an image acquisition time, the first camera defining a first reference position and configured to be coupled to the movable arm arrangement at a first camera coupling position;
a second camera for acquiring an image at the second 2D scale at the image acquisition time, the second camera defining a second reference position and configured to be coupled to the movable arm arrangement at a second camera coupling position;
A measurement processing unit,
determining a first relative position of the first 2D scale based at least in part on a first image of the first 2D scale acquired by the first camera at a first image acquisition time;
determining a first relative position of the second 2D scale based, at least in part, on a first image of the second 2D scale acquired by the second camera at the time of the first image acquisition;
a measurement processing unit configured as above;
Equipped with
a supplemental measurement position determination system that determines measurement position coordinates of a first end tool position at the time of the first image acquisition based at least in part on the determined first relative position of the first 2D scale and the second 2D scale. the first 2D scale coupling location is on a first rotary joint of the movable arm arrangement;
20. The supplemental measurement position determination system of claim 18, wherein the first rotary joint has a first axis of rotation, and wherein movement of the first rotary joint perpendicular to the first axis of rotation during rotation causes a change in the first relative position of the first 2D scale. The supplemental measurement position determination system of claim 19, wherein the measurement processing unit is configured to determine the angular orientation of the first 2D scale based, at least in part, on the first image of the first 2D scale acquired by the first camera at the time of the first image acquisition. the first 2D scale binding location is on a first arm portion of the movable arm arrangement;
20. The supplemental measurement position determination system of claim 18, wherein at least one of bending or twisting of the first arm portion causes a change in the first relative position of the first 2D scale.