DE102007028735B4 - Three-dimensional measurement of objects with detection of the object texture - Google Patents

Abstract

Method for shape and texture detection of an object, in which a 3D sensor and a digital camera are used for recording digital photos, characterized in that - the object (4) is surrounded by a housing (5), which - the object ( 4) encloses with a large part of the complete solid angle (4π); - whose inside or outside is illuminated directly or indirectly with one or more light sources (8), so that the inside itself is illuminated so that light from all spatial directions equally bright hits the object (4); - The so bright inside serves as a light source for taking a photo with the digital camera; - The object (4) is brought inside the housing by means of a positioning unit or manually in different positions; A sequence of photographs is taken in different positions of the object, which allows texture measurement from different sides; A sequence of 3D images is taken in the different positions of the object, which enables a 3D measurement from different sides; and - the photos from the digital camera are mathematically projected onto the 3D data of the object (4).

Description

In many applications in technology, virtual reality and monument preservation, 3D sensors are used for the three-dimensional detection of the shape. Exemplary tasks are: acquisition of consumer goods, measurement of statues. Exemplary measurement principles for 3D optical sensors are laser triangulation or coded illumination, e.g. B. by fringe projection.

The quality of the 3D optical measurement depends on the surface of the measurement object. Mirroring, very dark or transparent surfaces can not usually be measured. Here one often helps one by the so-called "preparation" of the object surface. Among these is usually a spraying and thus covering the object surface with a bright, well-scattering material, eg. As chalk powder or titanium dioxide, understood.

The resulting 3D data is usually represented by triangular meshes.

Frequently, in addition to the shape of the object, the texture of the object surface should also be recorded. This is particularly in demand in the area of virtual reality.

It is known to perform a so-called "mapping of a texture" on 3D triangular meshes.

The use of textures obtained by digital photography, for example, explains Yahya Alshawabkeh in "Integration of Laser Scanning and Photogrammetry for Heritage Documentation", Ph.D. Thesis at the Institute of Photogrammetry, University of Stuttgart 2006, pages 66 to 71; accessible under
http://elib.uni-stuttgart.de/opus/volltexte/2006/2899/

For this, the 3D data of the object are first measured with a 3D sensor. In addition, a photographic camera is added to the system, which is equipped with an electronic image sensor that can capture color images. The camera is calibrated relative to the sensor using the method of "camera calibration" known from photogrammetry. The rigid unit of 3D sensor and camera is hereinafter referred to as "acquisition unit".

An optical sensor usually performs the acquisition or measurement only from one viewing direction. A common task is the all-around measurement of the object. In order to measure the object all around, take several shots, between each of which the receiving unit directed to the object is brought to different locations around the object. Alternatively, the object can also be rotated. In this case, the arithmetic and evaluation unit controls with the aid of suitable control electronics both the receiving unit and optionally a positioning unit for rotating the object.

Then, the composition and triangulation of the 3D images and the mapping of the texture, in which the texture data from the various directions in which the photographic camera was located, are mathematically projected onto the 3D data with the aid of a computing and evaluation unit. The parameters of the projection are determined by the camera calibration. The 3D data is usually represented as a triangle mesh. During projection, a triangular section of the photographic image is projected onto each triangle. For visualizations, this triangular section is always treated as if it were fixed on the triangle, ie a triangular section of the colored surface of the object. 1 shows such a photo mapping across two triangles.

The images (3D and photo) are generally taken from several directions. It is basically possible to project each of the photos from its recording direction onto the 3D data. Due to the laws of geometry, the texture information is assembled correctly and creates a completely measured object with texture.

• a) fehlerfreie 3D-Ortsinformation,
• b) fehlerfreie Kamerakalibration, aber auch, wie im nächsten Abschnitt dargestellt,
• c) gleich-bleibende Beleuchtungsbedingungen, unabhängig von der Beleuchtungsrichtung.

Condition is among other things

• a) error-free 3D location information,
• b) error-free camera calibration, but also, as shown in the next section,
• c) uniform lighting conditions, regardless of the direction of illumination.

When implementing such a system, there are problems in the transition between photos taken from different directions. On the one hand, the requirements a) and b) are not exact, but in practice only fulfilled with a certain error. As a result, when viewing the textured grid, 3D image jumps occur at the interfaces between the projections of the various photos. On the other hand, jumps in brightness are observed at the same place. Further, the preparation of the objects is often not possible because the texture can be covered by the preparation and thus can not be photographically recorded.

The invention is intended to solve the problems described in the prior art. Furthermore, the invention should make it possible in transparent objects or objects with a reflective component of the otherwise diffuse reflection the To provide triangular meshes with an approximate texture.

Solutions can be found in claims 1, 5 or 6.

The jumps in the brightness can be explained as follows: During the recording, there is a certain lighting situation in the room, eg. B. the cloudy sky illuminates the object through a window. An object detail scatters the received illumination into the room. At some point in the room is the camera. The scattering is generally not uniformly strong in all spatial directions.

The scattering is almost always strongly dependent on the angle between the direction of illumination and the orientation of the object detail. This dependency ("object function") can be described, for example, for "ideal scattering objects" by Lambert's law. For real scattered objects, the dependence on the illumination angle is to be described by another function. This other object function depends on the surface of the object; for objects, in which the light can partially penetrate, also of the scattering properties of the object below its surface. The human face, for example, is such an object in which the light partially penetrates. The described surface and scattering properties of the skin and tissue of the face vary depending on the location of the face and vary from person to person. But even with other (technical) objects, the description of the surface and scattering properties is complicated. Theoretical models exist, but in practice they are unusable due to ignorance of the parameters.

Although attempts to model the object function improve the result somewhat compared to the direct mapping of the photos, the results are still poor and brightness jumps persist at the junctions between the projections of the various photos. The problem can not be solved without loss of quality with the help of calculations in the computing and evaluation unit, but you have to physically change the meter itself.

According to the invention, the problem can be solved by an illumination which arrives at the object with the same brightness from all spatial directions 4π. An object in an integrating sphere solves the problem. Thus, according to the invention, in a method or a device, the object is surrounded by a housing which is to be white on its inside and to be readily scattering.

The necessary feedthroughs (as openings) to the optical components of the recording unit should require only a small solid angle.

The walls can be semi-transparent and illuminated from the outside over a large area with suitable lighting. However, this illumination would interfere with the 3D measurement and is preferably switched off for the 3D measurement. You can use flashlights to capture the texture that flash in sync with the photo.

In order to ensure the rotation of the object in the housing for the different shots, a turntable can be used. This should then also reflect white diffuse. In general, instead of the turntable can occur any other positioning, z. B. a robot arm.

The approached positions of the turntable or the positioning unit should preferably be known for later evaluation. If they are not known, algorithms can be used which can calculate transformations from the 3D partial views (so-called registration algorithms). Because this is technically possible, embodiments without turntables are possible in which the object is rotated and moved manually in not exactly known manner. Furthermore, a combination of defined turntable steps and twists by hand is possible to capture the object from all sides. In combination, the registration algorithms only need to be used for the movements where twists were done by hand. The remaining transformations are calculated on the basis of the known turntable positions.

In the housing will be multiple scattering on the walls, so that sets an approximately homogeneous illumination evenly for all directions and thus also the turntable receives an approximately equal brightness and even appears evenly bright.

Examples exemplify the claimed invention "as intended" without limiting it.

1 shows a photo mapping across two triangles.

2 extends the hardware required for the measurement

3 outlines an economical solution for the realization of the illumination of the housing

The sketched problem is solved by a lighting, which from all spatial directions 4π equal to the object arrives. An object in an integrating sphere solves the problem. Thus, after 2 in a method or device, the object 4 with a housing 5 surrounded, which on its inside should be white and well-dispersing.

The openings as feedthroughs 6 to the optical components of the recording unit 3 should only claim a small solid angle.

The walls can be partially transparent and from the outside large area with a suitable lighting 8th be illuminated. However, this illumination would interfere with the 3D measurement and is preferably switched off for the 3D measurement. You can use flashlights to capture the texture that flash in sync with the photo.

To ensure the rotation of the object in the housing for the various shots, a turntable 7 be used. This should then also reflect white diffuse. In general, instead of the turntable can occur any other positioning, z. B. a robot arm.

The approached positions of the turntable or the positioning unit should preferably be known for later evaluation. If they are not known, algorithms can be used which can calculate transformations from the 3D partial views (so-called registration algorithms). Because this is technically possible, embodiments without turntables are possible in which the object is rotated and moved manually in not exactly known manner. Furthermore, a combination of defined turntable steps and twists by hand is possible to capture the object from all sides. In combination, the registration algorithms only need to be used for the movements where twists were done by hand. The remaining transformations are calculated on the basis of the known turntable positions.

In the housing will be multiple scattering on the walls, so that sets an approximately homogeneous illumination evenly for all directions and thus also the turntable receives an approximately equal brightness and even appears uniformly bright.

An economical solution for the realization of the lighting of the housing is in 3 outlined:
The walls of the housing 5 are white, scattering but not transparent.

As frame background 10 will that cutout on the in 3 called the right wall of the housing, which corresponds to the field of view of the camera.

Around this picture field background 10 around there is a circumferential gap 14 through which the light of a flashlamp 11 which may occur at a suitable scattering surface, preferably with the aid of a "softbox" available as a photography accessory. 12 , is scattered. Because the gap is just outside the field of view, the light must undergo some scattering processes until it can enter the field of view. This process homogenizes the lighting. Additional foreclosures 9 are necessary to prevent a direct reflection of coming from the circumferential gap bright light on the measured object.

The measurement with the sensor can advantageously from the top in the downward direction 15 take place, because then in combination with the rotation of the turntable, a measurement of the object can be made from above and from the sides.

In order to be able to measure from below, a second sensor can be used, which measures from below in upward direction. Alternatively, in a further embodiment of the invention according to 4 a mirror sharing the image field of the recording unit and as a result of the mirroring the recording unit virtually with a part of its image field from the top downwards inclined direction 15 and with the other part out with the direction inclined from the bottom upwards 16 measure up.

If the homogenization of the illumination is not completely successful, and further cracks in the texture occur during the composition of the images, algorithms can be used which approximate an object function and correct the texture with the knowledge of the object normals from the 3D measurements. An even simpler solution is smearing the jumps over low-pass filters.

The problem that due to errors in the 3D measurement or the camera calibration 3D image jumps occur at the transition points between the projections of the various photos can be resolved by taking the photos in smaller angular steps with correspondingly more photos because the error propagation of 3D measurement errors and errors in the camera calibration is smaller, the smaller the relative rotation angle between the two shots. In general, it is not necessary to increase the number of 3D measurements accordingly, as long as the number of measurements for 3D all-round surveying is sufficient. With the following procedure, a preparation of the object is possible: First, only a series of photos taken at certain angular positions of the turntable.

Then a wall of the housing is opened and the object is sprayed with spin finish. During spraying, the turntable is turned and thus the object is prepared from all sides. The object may neither be taken off the turntable nor moved in any other way on the turntable. This is followed by a series of 3D measurements in turntable positions, which have a known position to the turntable positions of the photo series. Preferably, the same turntable positions are used as in the photo series.

As final procedural steps, the 3D data are evaluated and the 3D object is calculated, then the photo textures are projected onto.

A preparation is necessary for many objects, especially for translucent, specular and black objects. Even if the object can be measured three-dimensionally, dissection improves the accuracy of the measurement results.

Thus, the preparation or rotation of the measurement object can be done by hand, a door can be provided for manual intervention in the wall of the housing.

With the following embodiment of the invention, transparent objects can be visualized. As an example for explanation in this description is intended to serve an empty, round in plan bottle of partially transparent-blue material. The background color for the intended visualization must be known: For example, the bottle should be displayed in a 3D animation against a green background.

To measure must the necessary hardware after 2 be extended. The picture frame background 10 is replaced by a plate of exactly the color in which the visualization is planned later.

Now, if the measurement proceeds according to the described method, the background shines through in transparent object areas and is also mapped to the texture, as if the surface partly had the color of the background. If the 3D visualization takes place later in front of the same color, the display looks real, even if you turn the object virtually.

With the following other embodiment of the invention, the transparency of partially transparent objects can be approximately measured and displayed.

To represent a transparent surface, the R, G, B, α model is often used, where R, G, B are the red, green or blue color values and α is an absorption value. According to the model, α = 0 means a completely transparent point and α = 1 a completely opaque point.

The approximation is made as the object consists of a partially transparent shell represented by partially transparent triangular meshes. The interior of the shell is empty in the calculation model.

As an example for explanation in this description should also serve an empty, round in supervision round bottle of partially transparent-blue material. The background color for the intended visualization does not need to be known. The necessary hardware must also be available for the measurement 2 To be extended: A black and a white plate should be inserted one after the other into the frame background. Each plate records a series of photos.

In places where the images with white background plate appear lighter, the object was not completely opaque, but partially transparent.

The observed absorption of the object results from the local quotient β of the brightness of the pixels of the image with a white background and the image with a black background.

This absorption is then mapped onto the triangles, each with half effect on the projecting direction located in front of the object and the object back, since the corresponding light beam has passed both surfaces.

Correspondingly, the root of β is assigned to the α value of the object front side and the object back side, so that the absorption α is visualized when a virtual light beam passes through both surfaces in the visualization.

The RGB object texture appears on the recordings with the black plate in the background. However, this is too dark due to the black background shining through the factor β. Thus, the RGB values measured with a black plate must be divided by β to obtain the actual object texture.

The two previously illustrated embodiments will have weaknesses in the visualization when the object is not empty inside. The difficulties are compounded when the object is not rotationally symmetric. As an example, serve a rectangular in plan rectangular bottle filled with translucent-blue liquid. A shot of an object orientation in which the object has the longest distance to travel for the light rays will have stronger absorption (in the example: appeared in deeper blue) than a shot with a shorter distance for the light rays. When assembling, cracks in the depth of the dyeing due to absorption distances of different lengths are to be expected at the transition points between the projections of the different photos.

The problem is solved by reducing the angular steps of the turntable while increasing the number of shots, because the smaller the angle step, the smaller the difference in the path lengths described. Thus, the jumps described in the brightnesses are smaller. For carrying out the method, experience has shown that angle steps of 15 ° or less are necessary. Thus, at least 24 photographs are necessary in one turn of the turntable.

The three previously illustrated embodiments will have weaknesses in visualization when the object is not only partially transparent but also scatters the light in its interior. As an example, serve a rectangular in plan rectangular bottle, which is filled with a slightly scattering cloudy liquid. The background color for the visualization is green.

An image in an object orientation in which the object has the longest distance to be traversed by the light rays will have greater scattering (in the example: white-green with high white content appear) than a photograph with a longer distance for the light rays (there would be white green with high green content to be expected). When assembling, cracks in the depth of the dyeing due to absorption distances of different lengths are to be expected at the transition points between the projections of the different photos.

The problem is solved by using a white background in the data acquisition and a white background in the visualization, because the observed (more or less strong) whitening stems from the white housing color outside the field of view of the camera. By this measure, the two color components that mix in different proportions, the same color.

LIST OF REFERENCE NUMBERS

1

Foto des Messobjekts

2

Dreiecksnetz

3

Aufnahmeeinheit

4

Messobjekt

5

weißes Gehäuse

6

Durchführungen zur Aufnahmeeinheit

7

Drehteller

8

Beleuchtung

9

Abschottung

10

Bildfeldhintergrund

11

Blitzgerät

12

Softbox

13

Spiegel

14

umlaufender Spalt

15

Beobachtungsrichtung von oben abwärts geneigt

16

Beobachtungsrichtung von unten aufwärts geneigt

List of terms used in the figures

1

Photo of the measurement object

2

triangle mesh

3

recording unit

4

measurement object

5

white case

6

Feedthroughs to the receiving unit

7

turntable

8th

lighting

9

isolation

10

Field background

11

Speedlight

12

softbox

13

mirror

14

circumferential gap

15

Observation direction inclined from above downwards

16

Observation direction inclined from the bottom upwards

Claims (6)

Method for shape and texture detection of an object, in which a 3D sensor and a digital camera are used for recording digital photos, characterized in that - the object ( 4 ) of a housing ( 5 ), which - the object ( 4 ) encloses a large part of the complete solid angle (4π); - whose inside or outside is illuminated directly or indirectly by one or more light sources ( 8th ), so that the inner side itself shines so that light from all directions is equally bright on the object ( 4 ) meets; - The so bright inside serves as a light source for taking a photo with the digital camera; - the object ( 4 ) is brought inside the housing by means of a positioning unit or manually in different positions; A sequence of photographs is taken in different positions of the object, which allows texture measurement from different sides; A sequence of 3D images is taken in the different positions of the object, which enables a 3D measurement from different sides; and - the photos from the digital camera mathematically on the 3D data of the object ( 4 ) are projected. The method of claim 1, wherein the object is at least partially transparent, the luminous inside of the housing ( 5 ) is white, and wherein - in a process step immediately after recording the photographs in the image field of the camera behind the measurement object, a black photo background is introduced, and - take in a further process step additional photos with the black photo background in the image field of the camera, the that corresponding to the white inside as a background, wherein - an absorption in the object point from the root is calculated from the local quotient (β) of the brightness of the black and white background pixels, and An object color value is calculated from the object color value of the pixels with black background divided by the local quotient (β). A method according to claim 1, wherein in a further method step after the recording of the photographs and before the recording of the 3D images, a preparation of the measurement object takes place. The method of claim 1, 2 or 3, wherein the sequence of photographs comprises at least 24 images. An apparatus for shape and texture detection of an object, comprising a 3D sensor and a digital camera and a housing, wherein - The object is surrounded by the housing, which - The object with a large part of the solid solid angle (4π) encloses and - The inside or outside is illuminated directly or indirectly with at least one light source, so that the inside itself can be brought to shine in such a way that light from all directions meets equally bright on the object, and - The luminous inside serves as a light source for taking a photo with the digital camera; - and with A positioning unit, by means of which the object can be brought into different positions inside the housing; - A control electronics and a computing and evaluation, which can control a sequence of photos and 3D shots in different positions of the object and in this way allows a measurement of different sides. Device for shape and texture detection of an object, consisting of a 3D sensor and a digital camera, wherein - The object is surrounded by a housing, which - The object with a large part of the solid solid angle (4π) encloses and the inside or outside directly or indirectly with at least one light source is illuminated so that the inside lights itself so that light from all directions equally brightly hits the object, or whereby - The luminous inside is capable as a light source for a photo recording with the digital camera; - and with A door in the housing through which, in the opened state, an operator can manually move the measurement object to different positions; - A control electronics and a computing and evaluation, which can control a sequence of photos and 3D shots in different positions of the object and in this way allows a measurement of different sides.

Images