JPH06229732A - Spot light beam scanning three-dimensional visual sensor - Google Patents

Abstract

PURPOSE:To perform the three-dimensional measurement of an object having different shapes and existing over a wide range using a single three-dimensional visual sensor which avoids the problems of insufficient illuminance of stripe light due to beam expansion or curved central points of projecting stripe light. CONSTITUTION:The spot light beam scanning three-dimensional visual sensor comprises projectors 1-3 each having means for deflecting the scanning spot light beam being projected to an object W, a video camera unit 40, an image processor 10 connected with the video camera unit, and means 30 for causing the projectors to perform spot light beam deflection scanning at a period sufficiently shorter than the scanning period of the video camera and forming a stripe projecting region including the object at least partially. Fringe part of the object W is scanned concentrically to form a short and high illuminance light band having no bent in the line of central points. The light band is observed by means of the video camera thus realizing highly accurate three- dimensional measurement.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a visual sensor used in a manufacturing line for performing automated work using a robot or other various FA equipment, such as a work for assembling mechanical parts by a robot. Speaking of
The light of a predetermined shape is projected onto the object to be measured, which is observed by a video camera, and the three-dimensional position and orientation of the object to be measured,
The present invention relates to a three-dimensional visual sensor that measures a shape or the like (hereinafter, simply referred to as “three-dimensional measurement”).

[0002]

2. Description of the Related Art In the case of constructing a system for automating various works by using an industrial robot or various FA equipments, or in the case of making an intelligent robot intelligent,
The visual sensor plays a large role. In particular, in automation of work by a system including a robot, visual sensors are often used to measure the position, posture, shape, etc. of a work.

The visual sensor having such a purpose can be classified into a two-dimensional measuring one and a three-dimensional measuring one according to its function. The latter three-dimensional visual sensor is used when it is necessary to measure the shape and the like. A three-dimensional visual sensor generally comprises means for determining the three-dimensional position of a point on a two-dimensional image captured by a video camera. Various types have been proposed as the means, but most of them are based on the principle of triangulation. Above all, slit-shaped light is projected onto a target object from a predetermined direction to A typical example is one in which a light band having a higher brightness than that of the surrounding area is formed, and this is observed by a video camera to perform three-dimensional measurement of the object.

The three-dimensional visual sensor of such a system does not have the problem of corresponding point detection, which is unavoidable when the stereoscopic system is adopted, and it suffices to observe a light band of relatively high brightness. It has the basic advantage that it is not easily affected by the brightness of the surroundings and the color of the object.

However, one slit light projector installed at a fixed position projects slit light onto an object to be measured,
If you adopt the arrangement that observes this with video camera means,
When the position of the object to be measured changes and is out of the slit light projection range, there is a problem that position measurement cannot be performed. Therefore, when there are variations in the existing position of the object to be measured, a scanning type three-dimensional visual sensor in which the measurement range is expanded by scanning slit light is also used.

However, even with this scanning type three-dimensional visual sensor, it was impossible to measure the edge portion extending in the direction parallel to the optical band formed by the set slit light. . As a means to eliminate this drawback,
A method has been proposed in which two light projectors having different slit directions are used. Although this method provides a means for solving the problem of slit beam parallel part measurement in principle,
Increasing the size of the entire device by using two projectors is inevitable, and the light source of two projectors (laser oscillator)
It has a disadvantage that a mechanism for switching control is required.

Although a method of combining two laser beams using a half prism has been proposed, the use of a half prism causes a problem that a stronger light source is required since the light intensity is attenuated.

Further, in the prior art, it is usual to use a cylindrical lens (cylindrical lens) as a means for generating a slit light beam, and it is suitable for the distance between the projector and the object to be measured and the size of the object to be measured. In order to change the size of the slit light flux, it is necessary to replace the cylindrical lens, which is extremely inconvenient. When the slit light beam is formed by greatly expanding the light from the laser light source in order to project the slit light on the object to be measured that is relatively distant and has a relatively large size, the illuminance of the projected light band Is inevitable.

The fact that the projected light beam shape is a fixed slit shape that is oriented in a certain direction also causes a reason that sufficient light band illuminance cannot be obtained. That is, there are various sizes, shapes, postures, perspectives, etc. of the measured object, but the shape and the extending direction of the projection slit light flux are fixed, so that the object important for three-dimensional measurement. The object part (for example, a specific edge part) is not sufficiently irradiated with the slit luminous flux extending in the direction suitable for the three-dimensional measurement, and most of the luminous flux is not important for measurement of the object to be measured or the background (wall, floor). , The light energy is wasted by being projected onto non-measurement target articles, etc., and a slit light projection area extending in a direction suitable for three-dimensional measurement having sufficient illuminance necessary for measurement is not secured. Things are easy to happen.

Then, a large amount of light is projected onto a background or a portion where information useful for three-dimensional measurement of the object to be measured is not obtained, which itself easily obstructs observation by the video camera means and subsequent image processing. However, this is not preferable for efficiently performing accurate three-dimensional position measurement.

Further, as will be described later, since it is difficult to freely change the extending direction of the slit light, the center point array of the light band is curved at the end of the light band projected on the object to be measured. The problem was unavoidable (see description of action).

[0012]

DISCLOSURE OF THE INVENTION The present invention is intended to solve the above-mentioned problems in the conventional three-dimensional visual sensor, and particularly, to perform three-dimensional measurement of an object to be measured of various shapes. It should be possible to use a single visual sensor with a wide measurement range, and avoid the problem of insufficient illuminance of the optical band due to the expansion of the luminous flux due to slit luminous flux formation and the problem of center point curve of the projected optical band at that time. It is intended to provide a spot light beam scanning type three-dimensional visual sensor capable of performing the above.

[0013]

SUMMARY OF THE INVENTION The present invention is directed to a video camera device, an image processing device connected to the video camera device, and a strip scanning range at a period sufficiently shorter than the imaging scanning period of the video camera device. Various problems in the above prior art are solved by a spot light beam scanning type three-dimensional visual sensor provided with a light projecting device having a means for deflecting and scanning the inside with a spot light beam.

[0014]

The three-dimensional visual sensor of the present invention is different from the conventional slit light projection type three-dimensional visual sensor in that it has a flexible pattern shape, size and extension according to the distance, size, shape, etc. of the object to be measured. A major feature is that a band-shaped light projection region having a direction is formed by high-speed deflection scanning of a spot light beam. This will be described with reference to FIGS.

FIG. 1 schematically shows the basic structure of a light projecting device which can be used in a three-dimensional visual sensor according to the present invention. The spot light beam L emitted from a laser light source 1 is scanned by an X-scanner. 1 illustrates a state in which a projection light pattern is formed by performing high-speed deflection scanning by (X-axis direction deflector) 2 and Y-scanner (Y-axis direction deflector) 3. For convenience of explanation, it is assumed that the spot light beam draws a light band pattern on the image plane (a surface in which the pixel surface is virtually represented outside the camera lens) 4 of the camera used. 2M and 3M represent the deflection mirrors of the X-scanner 2 and the Y-scanner 3, respectively. If the deflection angles of the mirrors 2M and 3M of the X and Y scanners are controlled over time while controlling the turning on / off of the laser light source 1 to repeatedly perform deflection scanning, projection light of an arbitrary shape with an arbitrary projection range can be obtained. A pattern, that is, a light band H is formed. The scanner device itself capable of deflecting and scanning such a spot light beam is well known.

For example, by controlling both scanners 2 and 3, the spot light beam L is O → P1 → P2 on the image plane.
→ Q1 → Q2 → P3 → P4 → ・ ・ ・ ・ ・ ・ → Qh-1 → P
By repeatedly performing high-speed scanning along the path of h → O, it is possible to form a light projecting area substantially equivalent to the case where the slit light corresponding to the strip-shaped area OP1 Ph Qh-1 is projected.

That is, the spot light loci O of each adjacent spot
If the intervals of P1 and P2 Q1, P2 Q1 and Q2 P3, etc. are set to be sufficiently narrow, and the deflection scanning speed is set sufficiently higher than the imaging scanning speed on the pixel surface of the video camera, The camera has one light band H over the entire swath.
Will be observed as. The spot light beam scanning path for forming the light band is not limited to the one shown in the drawing. In short, as long as the specified scanning area is observed as a light band by the video camera used, the scanning path in the light band formation area is arbitrary, and the light path formation by the scanning path suitable for the control scanning method of the scanner system is performed. You can do it.

When the beam emitted from the laser light source 1 is extremely thin, the resolution of the video camera (scan line density)
It is also possible to expand the beam diameter (spot diameter) to some extent by using an appropriate lens system in consideration of the accuracy required for three-dimensional measurement. If the diameter of the projected spot light beam is set to an appropriate thickness within the range that does not impair the measurement accuracy, even if the scanning density for forming the projected light area is relatively low, the entire projected area will have uneven illuminance. It becomes easy for the video camera to recognize it as a non-light band.

FIG. 2 is a view for explaining that the deflection scanning range of the spot light beam (the swing angle of the scanner) is changed depending on the distance of the object to be measured. (A) and (b) show that the measured object W having the same size is relatively distant from the scanner, and is present in the vicinity (d).
a> db). Set the edge E of the measured object to 3
In order to set the scanning range that is necessary and sufficient for the dimension measurement light beam to traverse and does not perform unnecessary irradiation of the background portion, the deflection angle θa of the deflection mirror M in the case of (a) is set to
It can be seen that the deflection angle θb of the deflecting mirror M in the case of (b) should be made larger than that shown in the figure.

In the conventional method of forming a slit light beam with a cylindrical lens, various types of cylindrical lenses are prepared according to the assumed distance of the object to be measured, or the slit light beam size is changed. Although a special optical system is required, the fact that the object to be measured can be dealt with far and near simply by deflecting the deflection angle range of the deflection mirror is a characteristic feature of the three-dimensional visual sensor of the present invention. Is one of.

FIG. 3 is a diagram for explaining the function of limiting the projection range of the three-dimensional visual sensor of the present invention.

(A) schematically shows how the measurement light projection range is limited when there are a plurality of (two) objects to be measured. In such a case, in the three-dimensional visual sensor of the present invention, first, the optimum projection range F1 F2 for the measured object W1 is set and the measurement is performed, and then the optimum projection range F3 for the measured object W2 is set. F4
It is possible to easily perform the measurement by setting (displayed by a broken line) only by changing the scanning range of the scanner. In the conventional method of forming a slit light flux with a cylindrical lens, a large slit light flux corresponding to the range surrounded by F1 F4 in the figure is formed by allowing a decrease in illuminance, and information regarding each of the measured objects W1 and W2 is individually obtained. An optical system that extracts or shifts the slit light flux formation range over a wide range that corresponds to the size of the measurement target object and covers the entire assumed range of the measurement target object It is necessary to install it in the device. Even if the latter method is adopted, if there are large differences in the sizes of the multiple measured objects, the oversized slit light will be projected onto the smaller measured object, and It will not be used efficiently.

Further, for example, as shown in FIG. 3B, the measurement light is projected only on a specific portion (in this example, an annular portion of the end face) important for performing the three-dimensional measurement to perform the three-dimensional measurement. Can be easily realized by the three-dimensional visual sensor of the present invention. In the conventional method using the cylindrical lens, it is extremely difficult to freely form the light projection area as shown in FIG. 2B on the object to be measured having various shapes and sizes.

Furthermore, the measuring light projection method adopted by the three-dimensional visual sensor of the present invention has the feature that the extending direction of the light band can be freely selected. When used, it is possible to set a light projection area that is extremely advantageous in actual three-dimensional measurement. This will be described with reference to FIG. 4, taking as an example the case where a light projection region is set on the end surface of the cylindrical object W to be measured.

4 (a) and 4 (c) are enlarged schematic views showing a state near the light band Ga and the end point P of the light band by the conventional slit light beam projection method. (B),
(D) is an enlarged schematic view of the light band Ga formed by the projection area forming method adopted by the three-dimensional visual sensor of the present invention and the vicinity of the end point P of the light band. is there. In the conventional method, as shown in (a),
The optical band must be formed across the central portion of the end surface of the measured object, which is usually unnecessary for three-dimensional measurement, whereas the spot light scanning method adopted by the three-dimensional visual sensor of the present invention is used. According to this, as shown in (b), it is possible to easily form a projection light band by concentrating light on an edge portion that is important for three-dimensional measurement, so that the light source light quantity can be effectively used. Become. That is, it becomes possible to make the video camera read a bright optical band image useful for three-dimensional measurement by repeatedly performing high-speed scanning only on a portion essential for three-dimensional measurement.

Since the extending direction of the light band is fixed in the conventional method, the light band is generally formed as shown in (a) and is drawn in (b). As described above, it is extremely difficult to freely select the optical band extending direction orthogonal to the contour line of the measured object.

Therefore, the light band is narrowed to one side near the end point P applied to the edge of the object W to be measured as shown in (c), and as a result, the center point sequence Ha of the light band is curved. become.

In the three-dimensional measurement using the visual sensor, it is usual to use the position of the center point sequence of the light band as the basic data of the measurement. Therefore, such curvature of the center point sequence causes an error in the result of the three-dimensional measurement. Will cause. Performing error-corrected three-dimensional measurement in consideration of the curvature of the center point sequence Ha may not be constant in the size and direction of the curvature.
It's never easy.

On the other hand, if the light projection area is formed as a short light band orthogonal to the contour line of the object to be measured as shown in (b) by the control of the X and Y scanners, the portion (d) is obtained. As shown in the enlarged view, since it is possible to obtain a bright light band without thinning, the center line Hb is not curved near the end point Q of the light band. Measurement errors are avoided in principle.

As described above, the three-dimensional visual sensor of the present invention employs a method of forming the light projection area corresponding to the slit light projection by the light projecting device having the means for deflecting and scanning the spot light beam. As a result, the light band area can be formed flexibly in response to various changes in distance, size, presence direction, shape, etc. of the measured object without wasting the amount of light emitted from the light source and at the center of the light band. The accuracy of three-dimensional measurement is ensured by avoiding the problem of point sequence curvature.

[0031]

FIG. 5 is a diagram schematically showing an example of the system configuration of the three-dimensional visual sensor of the present invention. the system,
A laser light source 1 that emits light projected onto the object to be measured W, a spot light beam emitted from the laser light source is deflected and scanned at a high speed, and light bands H1, H2 ...
XY scanners 2 and 3 that sequentially form Hn, a controller 30 that controls these scanners, a CCD camera 40 that captures the light bands H1, H2, ... Hn that are projected and formed on the measured object W, the camera 40 and the image processing control device 10 connected to the controller 30. Reference numeral 4 is C
The image plane of the CD camera 40 is shown.

The image processing controller 10 has a central processing unit (CPU) 11, and the CPU 11 has a communication interface 12, an image processor 13, a console interface 14, a camera interface 15, and a TV monitor interface 16. A frame memory 17, a control software memory 18 formed of a ROM, a program memory 19 formed of a non-volatile RAM, and a data memory 20 are connected via a bus 21.

A CCD camera 40 is connected to the camera interface 15, and an image captured by the CCD camera 40 is converted into a grayscale grayscale image and stored in the frame memory 17. The image processor 13 processes the image stored in the frame memory 17 according to the control signal from the CPU 11 according to the program content.

A console 23 having a liquid crystal display section and a keyboard including various command keys and ten keys is connected to the console interface 14, and the console 23 is used to input / set various data and store application programs. Operations such as input, editing, registration, and execution can be performed. Further, the liquid crystal display unit can display a menu for inputting / setting various data, a program list, and the like.

The TV monitor interface 16 includes a TV
A monitor 24 is connected, and an image being taken by the CCD camera 40, an image stored in the frame memory 17, or the like can be switched and displayed on the screen of the monitor 24.

The control software memory 18 stores various programs for controlling the visual sensor system via the CPU 11, and the program memory 19 stores the various programs.
A program created by the user is stored. The data memory 20 is a memory used for temporary storage of calculation data and the like necessary for three-dimensional measurement and the like.

Then, the communication interface 12 is connected to an external device (not shown) such as a robot for performing work using the measurement result by the light projecting device or the visual sensor,
The CPU 1 receives signals from these external devices and
1 and also has a function of transmitting a signal for controlling each external device.

Although a considerable part of the system configuration of the present embodiment described above is basically the same as the conventional slit light beam projection type three-dimensional visual sensor, in correspondence with the features of the present invention,
It has the following differences.

(1) A light projecting device for forming an optical band for three-dimensional measurement is not a conventional one that converts a spot light beam emitted from a laser light source into a slit light beam by a cylindrical lens and projects it. The spot light beam is subjected to high-speed deflection scanning with the X and Y scanners 2 and 3 whose deflection is controlled via the controller 30 to form an optical band as a spot light scanning projection area on the object to be measured. That (The scanning speed of the spot light beam is CCD
The speed is set high enough to be recognized as a light band when the projected light pattern is observed by the camera. (2) The sensor coordinate system that connects the controller 30 of the scanners 2 and 3 and the camera coordinate system of the CCD camera 40 is set by calibration, and the projection direction of the spot light beam is data on the sensor coordinate system. Be able to describe.

(3) The shape type, position, etc. of the measured object are judged based on the two-dimensional normal image data of the measured object, and based on the judgment result and known data (size data of the measured object, etc.). A series of programs for determining the contents of the spot light scanning pattern and related setting data are stored in the control software memory 18 and the data memory 20.

A specific example of the procedure for specifying the contents of the spot light beam scanning pattern will be described in the description of the flowchart of FIG.

(4) A program for calculating the direction (direction of the central longitudinal section) of each light band formed by spot light beam scanning and related setting data are stored in the control software memory 18 and the data memory 20. thing.

(5) Conversion for obtaining the three-dimensional position of the end point or bending point of the light band formed on the object to be measured based on the light band direction data and the pixel value data of the end point or bending point. Data (distance conversion matrix data) is acquired by calibration, and data memory 2
Stored in 0. Since the optical band formed by the spot light beam scanning is equivalent to the conventional slit light flux, the calibration method is basically the same as that of the conventional visual sensor of the type that projects the slit light flux. is there.

(6) When the necessity of remeasurement is judged at the end of the three-dimensional measurement of the object to be measured, and when it is judged that the remeasurement is necessary, the projection is performed according to the purpose of the remeasurement. A program for selecting the light pattern, determining the scanning reference position, determining the constants of proportionality k and m, and performing remeasurement is stored in the memory 18 for control software (see the flowchart and the description of step ST18). ).

CPU for three-dimensional measurement using the three-dimensional visual sensor system having the above configuration and function
An outline of the processing of 11 will be described with reference to FIGS.

The object to be measured W illustrated in FIG. 5 is a work having an overall shape having a concentric projection S on one end face side of a cylindrical shape, and a cutout is formed in a part of the concentric projection S. It is assumed that U is formed. An enlarged view of the cutout portion U is shown in FIG. The work W shown in FIGS. 5 and 11 is referred to as a type [01], and three types of works (types [02] and [03] having other shapes (for example, prismatic shape, elliptic cylindrical shape, etc.) not shown are provided. [04]) out of order 3
It shall be the target of dimensional measurement. Here, the measurement process of the illustrated type [01] workpiece W will be described as an example.

As a concrete content of the three-dimensional measurement of the work W, it is assumed that the center P0 of the concentric projection S and the three-dimensional position of the notch U are measured.

First, when the three-dimensional visual sensor receives a signal from an external device (not shown) (for example, a production line) via the communication interface 12, the CPU 11 starts a series of processing for three-dimensional measurement work ( Start), an index α representing the number of light bands and an index β representing the number of measurements are α = 1,
Reset to β = 1 (step ST1). Then C
The object to be measured W is activated by activating the photographing operation of the CD camera 40.
Image is captured (step ST2). This image is designated as A. The image A is a normal image obtained when the measured object W is not projected by the spot light beam from the light projecting device.

The image A is analyzed by using the image processor 13, and data (two-dimensional data of a representative point representing the contour) for determining the type of work is acquired (step ST3). The data acquired from the image A is collated and compared with the data that characterizes the types [01] to [04] of the work W registered in advance to determine the type of the work W (step ST4). As described above, since the work W is of type [01], it is determined to be type [01].

When the type of the work W is determined, the process proceeds to the process of specifically selecting and determining the spot light scanning pattern.
First, a light band distribution shape that matches the work type [01] is selected from standard shapes registered in advance (step S5). The standard form of the light band distribution type is prepared according to the assumed shape of the work, the specific content of the three-dimensional measurement, and the like. Figure 7
One example is shown below. (I), (ii), (ii)
i), (iv), and (v) are standard shapes that are mainly selected when projecting and forming a light band in a circular, fan-shaped, square-shaped, rectangular, or elliptical area. In the case of the present embodiment, the first measurement (β =
In 1), it is assumed that a program for selecting the standard form (i) is set up in response to projecting and forming a radial optical band on the entire double circular end surface of the work W.

Next, with respect to the selected standard form, the spread coefficient k, the light band length coefficient 1, the light band number coefficient m, and the scanning reference position γ
By setting 0, the specific content of the spot light scanning pattern is determined (step S6). FIG. 8 to FIG. 10 are schematic diagrams illustrating the states of the spot light scanning pattern (light band distribution) when the values of k, l, and m are changed with respect to the selected standard form (i). .

First, the spread coefficient k is a coefficient that determines the spread of the light band distribution. As shown in FIG. 8, with k = 1 as a reference, if k <1, the light band distribution range is narrowed and k> 1
If so, it means expanding. For example, the size of the diameter data on the screen of the image A may be divided into three levels, and the value of k may be selected from 1, 0.8, and 1.2 accordingly. When setting the selection conditions concretely, it is desirable to give priority to the fact that the light band does not deviate from the object W to be measured, and to allow for selection with margin.

Similarly, the optical band length coefficient l designates the length of each optical band (the deflection angle width of the spot beam in the optical band length direction) as shown in FIG. m is FIG.
As shown in, it is a coefficient that specifies the number of light bands forming the light band distribution. In each figure, k = 1 and l =
1, 0.5, 1.5; each case of m = 1, 2 is shown.
Depending on the analysis result of the image A and the content of the three-dimensional measurement, k =
0.8 or k = 1.2, l = 0.5, m =
It goes without saying that it is rational to make it possible to specify a combination of 2 and the like in a program. For example, in the three-dimensional measurement work in which a small-sized object to be measured is observed in detail from a distance, the arrival of the scene is indicated by the image A.
, K = 0.8, l = 0.5, m =
It is conceivable to configure the program so that the combination of 2 is selected.

Along with the selection / determination of such coefficients k, l, and m, the scanning reference position γ0 serving as the representative point of the scanning pattern is obtained.
If the (deflection angle reference position of each scanner) is determined, the specific contents of the spot light beam scanning can be specified. For the first measurement (β = 1), it is practical to determine γ0 by appropriately combining the standard position data assumed in advance for the work W and the image A data. Here, the known reference data about the center P0 of the concentric protrusion S of the workpiece W (three-dimensional position data of the point P0 when the workpiece is present at the predicted standard position in the standard posture) is appropriately used as CCD camera pixel value data. The reference position γ0 (center of the optical band group) is determined by correction. Γ0 at the first measurement
In principle, it is difficult to exactly match the point P with the specific point (P0) on the work W, and it is not necessary. That is,
Since the light band has a length, three-dimensional measurement can be performed even if there is a slight deviation. When performing remeasurement, there is room to use the first three-dimensional measurement data,
There is a high possibility that a more accurate determination of γ 0 can be made (see step ST18 described later).

When step ST6 is executed according to the contents of the program described above, the spot light beam scanning for forming the first (α = 1) optical band is started, and the first spot on the work W in FIG. A light band H1 is formed (step S
T7). The CCD camera 40 captures an image while the light band H1 is being formed to capture an image in which the light band H1 is superimposed on a normal image. This is image B (step ST8).

After the acquisition of the image B is completed, the data of the difference image C (C = BA) is acquired in order to extract the image of the light band H1 projected on the work W (step ST9). Then, an appropriate number of center point sequences are extracted from the difference image C according to a predetermined rule (step ST10), and a polygonal line (including a case of one straight line) matching these center point sequences is obtained by approximation calculation (step S11). ), The end points and bending points included in the polygonal line are extracted (step ST12), and the three-dimensional positions of these end points / bending points are calculated. In the case of the work W shown in FIG. 5, for the first optical band H1, P11 corresponding to the outermost peripheral edge of the work W, P21 corresponding to the base peripheral edge of the concentric projection S, corresponding to the top peripheral edge of the concentric projection S. P31
Is extracted, and the three-dimensional position of each extraction point is calculated (step S13). The calculation of the three-dimensional position of the extraction point is performed in the same manner as in the case of the conventional slit light beam projection type three-dimensional visual sensor.
This is performed based on the data specifying the central longitudinal section of the light band H1 and the pixel value data of the extraction point. The data for specifying the central longitudinal section of the light band H1 is obtained in steps ST5 and ST6.
It is calculated based on the parameter determination content of the spot light beam scanning content determination parameter and the index α.

When the processing for the first light band H1 is completed, the prescribed number n (n = m × n0; where m is the coefficient of the number of light bands described with reference to FIG. 10 and n0 is shown in FIG. 8). It is determined whether or not the formation and processing of the light bands of the light band distribution standard form are completed (step ST1).
4). As long as there is an unirradiated / unprocessed light band, a negative determination is made, so 1 is added to the light band number index α (step S15), and the process returns to step ST7.

The processing for the second and subsequent light bands H2, H3, ... Hn (steps ST7 to ST14) is the same as the processing for H1 described above. However, as shown in FIG. 9, since the r-th optical band is formed across the notch U, the five end points / bend points P1r, P2r, U1, U2,
U3 is extracted and the three-dimensional position is calculated.

When steps ST7 to ST13 are completed for the n = n × m0th optical band, step ST
If the answer in step 14 is YES, the process proceeds to step ST16, and after the end points / bend points are sorted and classified, the target three-dimensional measurement data is calculated (step S).
T17).

In the example shown in FIGS. 5 and 11, the end points / bending points are P11, P12, ... Corresponding to the outermost peripheral edge of the work W.
P1r ... P1n and P2 corresponding to the peripheral edge of the concentric protrusion S
1, P22 ... P2r ... P2n and P31, P33 ... P3r-1, P3r + 1 ... P3n corresponding to the peripheral edge of the concentric projection S
And U1r, U2r, U3r corresponding to the notches U (see FIG. 9,
The case where the number of light bands applied to the notch is one is illustrated. )are categorized.

Then, the three-dimensional positions of the center P0 of the concentric projection S and the notch U are calculated based on the three-dimensional position data of the end points / bent points of each group (step ST17). P
To calculate 0, P31, P33 ... P3r-1, P3r + 1 ... P3n
Alternatively, the data of P11, P12 ... P1r ... P1n can be used. For example, when one group has a data loss (when the light band position is not proper and does not cross the edge portion, etc.), the other group may be used. Further, since the light band spacing is rough for the cutout portion U, at this stage, one or a combination of the data of U1, U2, and U3 is used as the position data of U.

When the first measurement (β = 1) is completed,
The necessity of remeasurement is judged (step ST18). The specific content of the program for this judgment can be extremely diverse depending on the content, accuracy, etc. of the data to be acquired, but as a typical case where re-measurement is required, (a) It is assumed that measurement failure due to defective formation of the optical band, and (b) acquisition of more accurate information or more detailed information. As an example of (a), a program is conceivable in which when the number of end points / bending points to be extracted is less than or equal to a predetermined number, it is determined that the optical zone is poorly formed and the measurement is performed again. Further, there is a case where more accurate three-dimensional information regarding the cutout portion U of the work W shown in FIGS. 5 and 11 is desired, which corresponds to the example of (B).

Here, it is assumed that there is no measurement failure and remeasurement is performed for the latter purpose. First, 1 is added to the measurement number index β (step ST19), and it is confirmed that the limit value β 0 of the measurement number set for coping with the abnormality of the system is not exceeded (step 20), The number of light bands index α is reset (α = 1; step 2
1) Return to step 5 and select the light band distribution standard form again. In selecting the standard type, the work W is of the type [01], the second measurement (β = 2), and the first measurement was not defective. The standard form of the type of (ii) of FIG. 7 shall be selected. That is, it is assumed that a program is formed so that fan-shaped light band distribution patterns are concentratedly formed in the cutouts U and detailed information of the cutouts U is acquired.

Using the data of P0 and U1, U2, and U3 obtained by the first measurement, the spread coefficient k, the light band length 1, the light band number coefficient m, and the reference position γ0 for the standard form [ii] are used. Is selected and determined (step ST6). The spread coefficient k in this case is a coefficient (θ = kθ) that represents how many times the fan-shaped opening angle θ is to be increased by the reference opening angle θ0.
0) is also considered.

When the spot light beam scanning content in the second measurement (β = 2) is determined, as shown in FIG. 11, optical bands Hu1, Hu2 ... Hun (indicated by broken lines in FIG. 11).
The image B is captured every time each of the images is formed and projected, and the difference image C = B-A corresponding to the light band image is acquired. Next, the center point sequence is detected, the end points and the inflection points are extracted by polygonal line approximation, and the three-dimensional position of each extraction point is calculated (step ST7 to step ST15 are repeated).

The end points / bending points extracted in the example shown in FIG. 11 are P'11, P'21, P'31, U'11, U'21, U'31.
, U'21, U'22, U'32 ...

The three-dimensional position data of these extraction points are sorted out.
It is classified (step ST16), and finally necessary three-dimensional information is calculated (step S17). In the second measurement, a large number of light bands are intensively formed in the fan-shaped notch U to obtain sufficient data on details, so compared to the first measurement (one or two light bands). Therefore, accurate and detailed three-dimensional information can be obtained by data calculation. The information obtained by calculation includes the three-dimensional position of the notch representative point,
The size (fan-shaped spread angle), depth (radial direction and cylindrical axis direction), etc. can be considered.

If the detailed information about the notch is obtained by the second measurement and the intended purpose of the three-dimensional measurement is achieved, it is determined in step ST18 that re-measurement is unnecessary, and the three-dimensional measurement 1 The work cycle ends (end). If the second measurement becomes defective due to lack of extraction points or the like, it is determined again in step ST18 that measurement is necessary, and the process from step ST19 is repeated. When remeasurement is performed due to measurement failure, the program is configured so that selection is performed in step ST6 so as to increase the possibility of eliminating the cause of measurement failure. For example, the values of k, l, and m may be maximized to reduce the possibility of insufficient extraction points.

If the third measurement is executed and the desired measurement purpose is achieved, one cycle of measurement work is completed, but if there is still a measurement failure, the measurement count index β becomes the limit value β 0. The measurement is repeated until it is reached. If the intended measurement purpose is not achieved even after performing the β0th measurement, a yes determination is made in step ST20, and necessary measures such as issuing an alarm are taken in consideration of the possibility of a system abnormality. After that (step ST22), the work cycle is ended (end). The specific value of β0 is determined in consideration of the probability of occurrence of measurement failure, the reliability of the program, the required measurement speed and the amount of increase in measurement time due to β0 increase, but it is usually set to about several times. It is considered to be something.

In the example described above, the spot light beam scanning content is determined by selecting the standard type and various parameters (k, l, m, γ 0), but sufficient information is available to specify the details of the spot light beam scanning content. Is known before the start of measurement, some or all of such a process is unnecessary. For example, the shape, size, and representative point P0 of the work W
Under the condition that the three-dimensional position, posture, etc. of the workpiece W are limited to a small variation range, and the rough data of each is known before the observation by the three-dimensional visual sensor, the accurate point 3 of the representative point P0 of the workpiece W is determined. If only the dimensional position is intended for measurement, it is also possible to specify the spot light beam scanning content in advance.

Generally speaking, when a three-dimensional visual sensor is used to support the work of an automatic machine such as a robot at a production site of a factory, etc., the shape, size, and range of existence of an object to be measured. Since it is rare that the posture and the like are completely unknown, it is considered that the program for determining the spot light beam scanning content and the program for determining the necessity of re-measurement are often sufficiently simple content.

[0072]

According to the three-dimensional visual sensor of the present invention,
Since the light projection required for three-dimensional measurement is performed by the light projecting device having the means for scanning the spot light beam at high speed, the distance, size, existence direction, shape, and finally required three-dimensional information of the object to be measured. It is possible to form an optical band that is freely adapted to the content, accuracy, etc.

Since the amount of light emitted from the light source can be concentrated on a portion useful for three-dimensional measurement to form a bright light band and observation can be performed by the video camera, work efficiency and measurement accuracy are excellent. Three-dimensional measurement is realized.

Further, since the extending direction of the light band can be freely selected according to the shape of the object to be measured, it is possible to avoid the phenomenon of bending of the center point sequence of the light band at the edge of the object to be measured. Therefore, the three-dimensional visual sensor of the present invention can contribute to the improvement of the accuracy of the three-dimensional measurement even from this aspect.

[Brief description of drawings]

FIG. 1 is a schematic diagram illustrating a basic configuration of a light projecting device that can be used for a three-dimensional visual sensor according to the present invention and spot light beam scanning for forming a light band.

FIG. 2 is a diagram illustrating that the deflection scanning range (scanner swing angle) of the spot light beam is changed according to the distance between the object to be measured.

FIG. 3 is a diagram illustrating a function of limiting a projection range in the three-dimensional visual sensor of the present invention.

FIG. 4 is a diagram schematically showing an example of a light band distribution and a state in the vicinity of an end point of the light band separately for a conventional technique and a three-dimensional visual sensor of the present invention.

FIG. 5 is a system configuration 1 of the three-dimensional visual sensor of the present invention.
It is the figure which showed the example typically.

FIG. 6 is a flowchart showing an example of an outline of processing when three-dimensional measurement of an object to be measured is executed using the three-dimensional visual sensor of the present invention having the system configuration illustrated in FIG.

FIG. 7 is a diagram illustrating a standard form of a spot light beam scanning pattern.

8 is a schematic diagram illustrating a spot light scanning pattern (light band distribution) when a spread coefficient k is applied to the spot light beam scanning pattern standard form (i) shown in FIG.

9 is a schematic diagram for explaining a spot light scanning pattern (light band distribution) when a light band length coefficient 1 is applied to the spot light beam scanning pattern standard form (i) shown in FIG. .

10 is a schematic diagram illustrating a spot light scanning pattern (light band distribution) in the case where a light band number coefficient m is given to the spot light beam scanning pattern standard form (i) shown in FIG. .

11 is a partially enlarged view showing in detail a cutout portion U of the work W shown in FIG.

[Explanation of symbols]

 1 laser light source 2 X-scanner 3 Y-scanner 2M, 3M deflection mirror 4 image plane of video camera (CCD camera) L spot light beam Ga, Gb light band P, Q light band end point Ha, Hb light band center point sequence W , W1, W2 Object to be measured (workpiece) E Edge R Ring area

Claims (1)

[Claims] 1. A video camera apparatus, an image processing apparatus connected to the video camera apparatus, and deflection scanning with a spot light beam within a strip scanning range at a period sufficiently shorter than an imaging scanning period of the video camera apparatus. Spot light beam scanning type three-dimensional visual sensor provided with a light projecting device having means for performing.