DE19536297C2 - Method for the geometric calibration of 3D optical sensors for the three-dimensional measurement of objects and apparatus for this purpose - Google Patents

Description

The invention relates to a method for geometric Ka Librierung of optical 3D sensors for three-dimensional Measurement of objects relative to a reference coordinate Tensystem on the principles of fringe projection and triangulation, using at least one Ka mera, a digitalization and storage device tion of camera sequences, at least one to Ka mera fixed lighting projector, the time after each other light structures of at least one-dimensional Strip generated within a working volume, minde at least one  Lighting device for uniform illumination of re inflecting and / or scattering signal marks a Calibration device and a computer for control and Ver processing the images and a device for carrying out of the procedure.

Method for the optical measurement of objects by means of cameras are known, for. Example from DE 40 07 502 A1 or DE 40 11 407 A1.

Methods for camera calibration are known. It will be with a camera recorded points whose coordinates are known are known, so the x, y and z coordinates in the world are and the images evaluated. The calibration of the camera is then based on the fact that the image coordinates with the Weltkoordi be matched. The calibration a projector is, in contrast, more difficult because of the Projector can not see where he shines. The projector sends a certain strip with the Number x off, but there is no way with the project determine where the strip ends up in the world, this requires additional auxiliary equipment, namely another Kame ras in the system. Watched with a calibrated and camera oriented these strips of a projector in the world and knows the strips that the projector internally (virtually) has sent, you can see the connection between the external (visual) strip of the projector and the internal (virtual len) strip the projector. However, you do not know where to be along an external strip be place. If the strip X is sent, you will find in the World with the camera this stripe X again, but you know not, which x-y point in the projector where the light is generated was, then appears at the point x-y-z in the world. That means,  that not every pixel of the camera is a spatial Coordinate value can be assigned.

They are areal optical measurement methods with a Ma trix camera and a projector for uncoded or ko known strips in which the three-dimensional Coordinates of the surface points from the two image coordinates Dinaten the camera image and the s.der Bildko Ordinate detected strip number are calculated (Reinhard W. Malz: Coded light structures for 3-D measurement Technology and Inspection, Thesis, University of Stuttgart 1992, Series: Reports from the Institute of Technical Op University of Stuttgart). Likewise are procedures with several projectors known.

Furthermore, photogrammetric optical measuring methods with multiple matrix cameras or a matrix camera in meh Other known positions in which the three-dimensional Coordinates of clearly determinable points in space from meh their views are determined numerically. to Generation of uniquely definable points in the object space who the natural object features, for example, regular or stochastic grayscale or color distributions on the object, used as well as artificial features such as glued on or projected brands or regular or stochasti grayscale or color distributions.

Also known are photogrammetric methods, the spatial position of one camera relative to several visible reference points whose absolute value lute coordinates are known (Spatial backward cut). Likewise, photogrammetric methods are known  the multiple spatial positions of a camera relative can determine to each other, provided that of several layers taken pictures several common reference points whose coordinates do not need to be known in advance sen (bundle adjustment). In this procedure become too In addition to the positions of the camera also the coordinates of calculated in several pictures visible points (Punktmes measurement).

Furthermore, photogrammetric calibration be. Be knows how to determine the beam geometry of cameras. Here are by multiple observation of a time invariant calibration body by numerical methods the internal parameters of the camera with regard to focal length, Distortions etc. determined. Photogrammetric potash Bring methods achieve high precision and require no precision positioning devices. At a if project field of z. B. 250 mm × 350 mm and a camera resolution of 500x700 pixels are lateral errors in the object field reachable <10 μm (1σ).

Furthermore, an active optical triangulation ver drive to determine the beam geometry of projectors become known for surface measurement, in which the Projector must be able to project line crosses whose position is determined by a photogrammetric camera ver band is measured (Th. Strutz: An exact active opti nice triangulation method for surface measurement, Thesis, Technical University of Braunschweig, 1993). This Calibration thus requires a special projector, the line crosses is able to project.  

Likewise, numerical calibration methods are known which the through central perspectives of camera and projector as well as through optical aberrations distorted image space [ξ (x, y)] by means of Adapt polynomials to the 3D object space (x, y, z) (B. Breuckmann et al: Precision Calibration of Topometric Sensors, Vor at LASER 95, Munich, June 19-23, 1995). It will a white plate on which several black circular rings are placed are moved, repeatedly shifted exactly in the normal direction and each according to the sensor principle with different Illuminated stripe patterns. The detek within the circles The measured values are used for calibration. adversely is that this calibration is a highly accurate and well-defined Mechanical positioning requires, as the calibration bar must be moved exactly in the normal direction. Furthermore, it is the correction calculation due to the numerous polynomial coefficients tedious. The calibration procedure is not for the Use in conjunction with photogrammetric methods (Navi gulation, bundle rangulation), which are based on compact and supported physically based camera and projector models are.

The invention has for its object to provide a method of the aforementioned type and a device for performing tion of the method with which during a measurement In principle, every pixel of the camera has a spatial coordinator value in a reference coordinate system can, where the coordinate value is arbitrarily reproducible.

Purpose of the invention is, for example, manufacturing machines, like pressing or milling or machine tools, by means of  to control the invention so that the optical Recording a specimen a highly accurate workpiece than Image of the specimen can be made or in terms of software virtually any manipulation on the virtu ellen image material and again as a workpiece can be issued.

The solution of the task according to the invention is that a) the calibration device inside and at the edge of the calibrating work volume in succession in different different positions relative to camera and lighting project tor brought and calibrating each be so be is lit, that the areas of the signal marks of Kali Burning device in the digitized and stored Images the highest possible gray value or color modulation without overloading, b) the calibration of the Ka meras by a standard photogrammetric method he follows to find the parameters of the inner and outer Orientation, c) the projector on a light modulator indicates, the externally visible pattern of any Strip or pixels as picture elements on the calibration projected, d) to calibrate the projector this is considered as an inverse camera by the externe In this way, the picture elements are a real or fictitious x-y project onsmuster defined width but indefinite length in Inside the projector, and for identification the parameter of the inner and outer orientation of Be illumination projector several observations (image sequences) be used, in each of which geometric Zusam between the x-y projection pattern (internal Pro jektorstreifenkoordinaten) and the corresponding externally visible patterns are produced, e) the found ones  and stored calibration parameters in use of the 3D sensor system for measuring objects of one Algorithm be reused, on the one hand the Beam geometries inside the camera and the projector corrected, on the other hand based on the principle of tri angulation the xyz coordinates of the object in the object space calculated.

The advantage of the method consists in that with the same in principle every pixel receiving camera a spatial coordinate value in egg can be assigned to a reference coordinate system, the is arbitrarily reproducible. To do this, there is the 3D sensor order from at least one camera, such as video camera, Still Video Camera, Scanning Camera, and a device for the digitization and storage of image sequences of the Ka mera, at least one fixed to the camera Illumination projector, successively light structures of at least one-dimensional stripes inside generated half of a work volume, at least one Be Lighting device for uniform illumination of reflective and / or scattering signal marks of a Ka libriereinrichtung and from a computer to control and Processing the pictures.

The calibration body (world) is for the purpose of calibration equipped with a black and white striped pattern, which is approximately orthogonal to the sent stripe pattern of the projector. This will cause the stripes of the Pro periodically interrupted and approximately arise square bright spots of light. These spots of light can from a camera locally, in two or three dimensions,  be detected and give a statement about the Location of the strip in the reference coordinate system (world). There are two cameras for calibrating a projector, which form a stereo pair, these cameras look at the Calibration plate with the white squares. This stereo pair can thus make these square white spots the Ka Determine librierplatte in the room.

It is also possible to dispense with the second camera and the location of the white squares on the potash bristle plate with a single camera. This is possible, if with a camera first of the Location of the calibration plate in the room is determined by a so-called forward cut. The condition for this is that the camera, so to speak, the geometry of the calibration plate knows and because of the perspective distortion the Position of the calibration plate relative to the camera calculated, then, in a second step, determine where the white squares lie in the plane of the calibration plate, so that they are determined three-dimensionally. To the projector Calibration will initially be a two-camera system kali burns, d. H. It is the inner of a camera itself Calibration and then the relative assignment To each other, so that a complete calibration absolutely This stereo pair is available. This is suitable Absolutely to measure points in the room, that of both Kame be seen. Will be white projector squares generated on the calibration plate, so this can also from solut in the room to be measured. Will this be in different Positions done the calibration plate, so can the Trace the beam path of the projector strip in the room and through the projection center in the camera through the projector pupil  extrapolate, bringing the entire geometry of the projector.

The relative position and orientation of Camera and Pro jector or cameras and projectors can each other porous, d. H. for the duration of calibration and measurement or permanently, for example by a mechanical rigid Be fixed connection.

For self-calibration of the camera, the calibration can direction advantageously at least four calibrated and one Variety of other signal brands or reference brands aufwei sen, which must be in the field of view of the camera. Dar First, with the help of direct solutions (literature Beyer) approximations for the elements of the outer Orientie the camera relative to the reference coordinate system (World coordinate system). Starting from These approximations can be the exact external orientation of the active-optical system with respect to the object-related Coordinate frames are calculated.

Advantageously, for approximate automa orientation of the calibration device relative to Camera at least three of the calibrated and / or the much number of other signal marks to be encoded, if the Coordinates of the latter at least approximately known are. With the use of coded signal marks and direct Solutions for the determination of the approximations can thus be the Procedure be carried out fully automatically. This king The calculations are both indicators of the accuracy speed, d. H. Standard deviations of the orientation elements,  as well as indicators of the robustness of the procedure output.

With several cameras, these can be the same or under different focus for different distances as well as they have the same or different Image angles can be the same as the cameras or have different spectral sensitivity can. Likewise, the cameras can be in the same or in look at different spatial directions and at the same time or take pictures at different times.

It can by the process signal marks with a high Contrast can be measured with very high accuracy and Although at least between 1/30 to 1/100 pixels.

Will you calibrate such systems, which only from a Ka mera and a project, so is a Zusatzinfor required, it must then be required that the Calibration plate is described exactly, d. h. that these is completely flat and that the black stripes, the cut the projector strip, also precise defi are defined. If that is the case, can from a single Ka In each case, the position of the calibration plate in the room be be true, which in turn determines the location of the white squares is determined on the calibration plate. At the Zweikamerakali The white squares on the potash are burnished brierplatte seen by both cameras, more infomatio NEN of the plate are not required. When Einkameraka librierverfahren must loka to the calibration plate in space To be able to lisieren, also a measurement of the Retromarken or signal marks of the calibration plate. On the  This information is based on an affine transformation performed, from which the position of the calibration plate results. In the affine transformation, this becomes the observation tete image of the retroremarks or signal marks with a stored image of the calibration plate compared to the Plate is then ge by a numerical operation so turns until the image matches the image stored true, which then makes all six degrees of freedom of rotation Translation in space are known.

Preferably can with a camera and a projector calibrated active 3D measuring system and such a measuring system be built, including for the purpose of calibration second camera is provided with the first camera forms a stereo pair as long as the camera is calibrated. After calibration, the second one can be provided Camera are removed, after which the calibrated Pro jector along with the first remaining camera forms active measuring system. This approach with two cameras for calibration can of course also for systems used from the outset and permanently two Ka meras have. In this case, the stripes could be ejected the projector with both the one and the other another camera, so that in a sense gekop two pelte measuring systems are present, the once with the one Ka mera and the projector and the other time with the other Camera and same projector work. It is the same possible that both cameras, as in calibration, too When measuring, make a stereo pair and the projector as additional information for marking the object ver is used.  

It is essential that the calibration plate be in their shape is known, the arrangements of the signal mark in advance vermes sen and that the calibration plate on coded marks features an automatic orientation of the calibration plate in the room allows; at least four grid points on the calibration plate then serve the orientation of the same relative to the camera. It is advantageous if the four calibration points are coded to allow their order can be automatically identified.

Advantageously, a projector is used, the only one projected a periodic gray-scale or color grid, which can be converted into a phase image (lateral Phase shift method), wherein the calibration for required projector strips synthetic from the phases image and the black and white black over each of the synthetic strips is more constant Phase in the phase image.

The algorithm for recognizing the calibration parameters also corrects the ones provided in the projector model and deviating from common error models for cameras Distortion error, for example by a cylindrical Projection optics caused errors; not at first directly observable place along the projector strip becomes for the purpose of distortion correction via an iterative Invoice determined.

If the calibration device is a plane calibration plate, so she can at least four calibrated signal marks on and the black required for calibration White stripe grid known geometry generated synthetically  by using a calculated strip grid with Help the currently determined orientation parameters in the Image plane of the camera transformed and with the image of Calibration plate is logically linked.

Thus, a plane is advantageous as a calibration device Calibration plate containing at least four calibrated signal brands and a black and white striped grid known Has geometry.

The calibration device can also be a non-planar Calibration plate containing at least four calibrated Si gnalmarken and a black and white striped grid known Geometry, wherein at least two cameras for Ein set with which the current topology of Ka Libriereinrichtung is anytime nachmessbar.

If the calibration device is a flat calibration plate is at least four calibrated signal marks on indicates the black required for calibration white stripe grid known geometry synthetic he be convinced by using a calculated strip grid with Help of currently determined orientation parameters in the Image plane of the camera transformed and with the image of Calibration plate is logically linked.

Further advantages of the invention are in particular a cost reduction by waiving mechanical pre precision positioning equipment such as coordinate measuring machines, Precision measuring robot, or additional precision measuring stems, such as theodolite, interferometers, otherwise the Measure length of workpiece or sensor.

Brief description of the drawing, in which:

Fig. 1 is a schematic representation of a camera-projector unit as a sensor, which is directed to a calibration plate

Fig. 2 shows a possible embodiment of a calibration device in the form of a flat calibration plate

Fig. 3 is a schematic representation of the displacement of the calibration plate in the working volume

Fig. 4 is an illuminated calibration plate with retroreflective marks

Fig. 5 shows a typical illumination field of a projector, as it finds application in the process

Fig. 6 shows a calibration plate under illumination with a periodic projector strip pattern

Fig. 1 shows a compact embodiment of a 3D sensor consisting of a camera 10 and a projector 11, where referred to in the rays of the camera 10 with xy-coordinates and the one-dimensional Streifenkoordinate the projector 11 with z. Furthermore, in Fig. 1, a calibration plate 12 is shown with a plurality of signal marks 13 , which is in the field of view of the camera 10 and projector 11 .

Fig. 2 shows a possible embodiment of a Kalibrierein direction as a flat plate 21 with the four corners A, B, C, D and with three different structures: First coded marks 22 with a unique identifiable label, which is arranged in the center of the calibration plate 21 square are. Further uncoded marks 23 continue to be parallel black-and-white stripe patterns 24 , which in the application are then orthogonal to the projector strips and cut piecewise sections from the projector strips.

Fig. 3 shows schematically how a calibration plate 30 having its four corners A, B, C, D is moved in the working volume 31, wherein the working volume 31 is illustrated by a wireframe with the points P1 to P12 in correspondence of the corners A, B, C, D should correspond to points P1 to P12; these points are stored in a table, Table 1.

Table 1

Fig. 4 describes the calibration plate 41 of Fig. 2 and an illumination with retro-reflective marks 42 , only those with visible brightness can be seen.

Fig. 5 shows the typical illumination field of Projek sector, in which a plurality of projector strips are combined to form a strip block, z. B. is in the strip block 51 of the project strip 1 to 8 summarized and switched to white, in the strip block 52 are the Projek torstreifen 9 to 16 combined and switched black etc.

Fig. 6 shows the calibration plate of Fig. 2 under illumination with a periodic projector strip pattern at an angle of 90 degrees; In this type of illumination Be the projector light strips are divided into small qua dratische areas 61 which are accessible to a measurement ei ner photogrammetric detection and measurement by means of camera and image correlation, in accordance with FIG. 3rd

The invention is particularly applicable to a Camera projector unit or a two-camera project without expensive precision calibrators to calibrate facilities to ka librated single-camera projector sensor or two-camera Projector sensor in principle for a subsequent measurement ell every pixel of the camera a spatial coordina value in a reference coordinate system in clear and reproducible manner. Her positioning machines, such as presses or milling or tools machines are controlled so that the optical  Recording a specimen a highly accurate workpiece than Image of the specimen can be made or in terms of software virtually any manipulation on the virtu ellen image material and again as a workpiece can be issued.

Claims (9)

1. A method for geometrical calibration of 3D optical sensors for three-dimensional measurement of objects relative to a reference coordinate system on the prin ciple of fringe projection and triangulation, using at least one camera ( 10 ), preferably video camera, a device for digitizing and storage of image sequences of the camera, at least one fixed to the camera lighting projector ( 11 ), the temporally successive light structures from at least einimensiona len strips generated within a working volume, at least one illumination device for uniform illumination of reflective and / or scattering Si gnalmarken a calibration ( 12 ) and a computer for controlling and processing the images, characterized in that

• a) brought the calibration device within and at the edge of the calibrated work volume sequentially in different positions relative to the camera and lighting projector and the calibration device is illuminated in each case so that the areas of the signal marks of the calibration in the digitized and stored images as high as possible grayscale or Have color modulation without clipping,
• b) the calibration of the cameras by a photogrammetric standard method is carried out to find the parameters of the inner and outer orientation,
• c) the projector has a light modulator which projects externally visible patterns of arbitrary strips or pixels as picture elements onto the calibration device;
• d) for the calibration of the projector this is regarded as an inverse camera by the outer Bildele elements a real or fictional xy-Projektionsmu defined width but indefinite length is assigned to the interior of the projector, and Be mood of the parameters of the inner and outer orientation of the Illumination Projector several observations (image sequences) are used, in each of which geometrical relationships between the xy projection patterns (internal projector greifstor coordinates) and the corresponding externally visible patterns are produced,
• e) the found and stored Kalibrierparame ter when using the 3D sensor system for Vermes measurement of objects are reused by an algorithm that corrects the beam geometries inside the camera and the projector on the one hand, on the other based on the principle of triangulation xyz coordinates of the object in the object space.

2. The method according to claim 1, characterized in that the camera by means of a calibration device with little at least four calibrated and a variety of other Si gnalmarken is self-calibrated. 3. The method according to claim 2, characterized in that for approximate automatic orientation of potash Briereinrichtung relative to the camera at least three of calibrated and / or a variety of others Signal marks are encoded, provided that the coordinates of the latter are at least approximately known. 4. The method according to claim 1, characterized in that a projector is used that only has a single peri odisches grayscale or color grid projected into a Phase image can be converted (lateral phase shift method), with the required for the calibration Projector strips synthetically derived from the phase image tet and the black and white black transitions of syn thetic strips each places constant phase in Pha pose. 5. The method according to claim 1, characterized in that the algorithm for recognizing the calibration parameters the provided in the projector model and the usual Feh models for cameras deviating distortion errors,  For example, by a cylindrical projection optics ver caused errors, also corrected and that the initially not directly observable place along the project ghost strip for the purpose of distortion correction an iterative calculation is determined. 6. The method of claim 1 using a ebe NEN calibration plate as a calibration, thereby ge indicates that the calibration plate at least four ka Has librierte signal marks, and that the Kalibrie tion required black and white strip grid known Geometry is synthetically generated by a calculated Streifengitter with the help of the currently determined Orientie transformed into the image plane of the camera is logically ver and with the image of the calibration plate is knotted.   7. Device for geometric calibration of a 3D optical sensor, which has at least one camera,
for the three-dimensional measurement of objects,
characterized,
that it is a plane calibration plate,
the at least four calibratable signal marks and
and a black and white striped grid of known geometry. 8. Device for geometric calibration of an optical 3D sensor, which has at least two cameras,
for the three-dimensional measurement of objects,
characterized,
that it is a non-planar calibration plate,
the at least four calibratable signal marks and
and a black and white striped grid of known geometry. 9. Apparatus according to claim 7, characterized, that the black and white striped grid of known geometry is synthetically generated, by calculating a calculated strip grid using the currently determined orientation parameter is transformed into the image plane of the camera and with the image of the calibration plate is logically linked.