DE19757595C2 - Method and device for recording a three-dimensional distance image - Google Patents

Description

The invention relates to a method and an apparatus for Recording a three-dimensional image of the distance from raumli objects.

Three-dimensional recording and processing sensor systems win for various tasks in the indu strategic technology is becoming increasingly important. Known optical Radar systems, such as laser radar, are either based on the principle of laser pulse transit time measurement or on the Determination of the phase difference of modulated laser light for Derivation of the object distance. To build a three-dimensional imaging systems are additional mechanical scans facilities required. This leads to a relatively expensive ren electronic and mechanical effort, the use such three-dimensional systems on a few special limited applications.

Methods are known which use a CCD camera (Charged Cou pled device), whereby for these semiconductor cameras Television (TV) standard is used. So you can be single achieve relatively long readout times.

From DE 196 19 186 C1 a method is known in which for Generating a spatial image a variety of emitted Light pulses are accumulated to form an overall picture. DE 37 32 347 describes a distance image sensor, in which an image of a CCD camera by means of a Ver is controlled and a distance information by forming a difference from successive ones Signal amplitude values is derived.

DE 41 29 912 C1 describes a method for taking Ent distant images, one of which is upstream of the CCD camera  Locking device by means of a control of their transpa generated modulated reflection images.

The invention has for its object a method for Recording a three-dimensional distance image, as well as a To provide a device with which a quick les and inexpensive method for obtaining a dreidi dimensional distance image for spatial objects without expenditure mechanical equipment is provided.

This problem is solved by the features of Claim 1, 2 and claim 23.  

The invention is based on the finding that under one set of a pixel-resolving and choice freely readable optoelectronic sensor, whose integra point of time is adjustable, an extremely fast Image acquisition of a three-dimensional distance image possible is. To do this, the object with one or more very short Light pulses illuminated, whereupon light pulses of the same Length from the object. Scattered this back th light pulses are transmitted to the optoelectronic chip passed. Due to the differ distance between different object points from the sensor the backscattered light corresponding to the locations pulses arrive at the sensor at different times. For egg ne distance measurement, a time measurement window is opened Duration corresponds to a predeterminable integration time. The integration time is less than or equal to the length of the emitted and thus also the length of the reflected Lich timpulse. This ensures that the first in Sen sor incident backscattered light pulse more or less is fully absorbed. The arriving with a delay Light pulses are cut off at the back, so that due to the different charges in the grid of the optoelectronic Sensors the different transit times in cargo sub different can be implemented. This makes a dreidi calculate the dimensional distance image.

In an advantageous embodiment of the invention a very long integration time all light pulses simultaneously with the measurement or time described first then added with their full length offset. This is used for normalization, so that differences in Re Flexibility behavior of the object recognized and balanced who that can.

Further advantageous refinements can be found in the subclaims Chen be removed.  

The main advantages of the invention are that for example, mechanical shutters are eliminated. Ex extremely short image acquisition times can be realized. The use The optoelectronic sensor is commonly used as a CMOS sensor referred to, whereby this is only the technological designation voltage of the semiconductor device. With such a Minimum integration times of 50 to 30 nsec realize (jitter at less than 0.1%). The techni The development is still progressing with the integration times Ahead.

In the following, diagrammatic figures will be used described examples.

Fig. 1 shows the operating principle for detecting a three-dimensional distance image with a CMOS sensor,

Fig. 2 shows the schematic representation of the temporal displacement of two Lichtimpulsle in the integration window de ren corresponding object points a different distance from the CMOS sensor have,

Fig. 3 shows two variants of the sensor for the simultaneous detection of three-dimensional distance images and Intensi täts- or gray scale images with a CMOS sensor,

Fig. 4 shows the schematic representation of the vehicle interior monitoring with a three-dimensional CMOS sensor.

A method for serial or simultaneous detection or generation of an intensity and a three-dimensional distance image of a spatial object with an optoelectronic sensor under short exposure is described. The method uses the runtime differences of the light impulses scattered back from the three-dimensional objects in the case of pixel-synchronous (pixel-synchronous) detection at the sensor within short integration times. A CMOS sensor is used for this. This sensor has a light sensitivity of 1 mLux, for example. Furthermore, it has a high intensity dynamic of up to 10 ⁷ , an optional access to the individual pixels (pixels) and an adjustable integration time (sample & hold). For measuring the amount of charge Q (t) when exposed to a single pixel.

Compared to processes that use a CCD camera achieve special advantages, such as the pa parallel acquisition of intensity and three-dimensional images , as well as the realization of short image acquisition times, the are significantly lower than the readout times of CCD cameras. Wei Furthermore, the CMOS does not require any complex mechanical shutdown ter and there is no need for powerful laser light sources len are used for the short exposure.

The method is particularly useful for recognizing people and motion sequences in room surveillance, for example Vehicle interior and exterior monitoring of automation from Kra systems and navigation.

The essential functional features are explained with reference to FIG. 1. First, the lighting of the spatial objects to be detected is provided with short light pulses, for example <100 ns. Illumination can take place with laser light, for example with a pulsed laser diode, or with light sources, for example with a pulsed LED diode. The method is independent of the angle of the lighting, which does not necessarily have to be central to the general direction of detection. For example, the use of a ring light is also conceivable for coaxial lighting and detection. The arrangement shown in Fig. 1 only serves to illustrate the functional principle schematically.

A first image acquisition A is connected to the CMOS sensor with a short integration time Δ _A. The back scattered light pulses 3 of the length Δ _L (<100 nsec) from the object points G of the three-dimensional scene are detected at the associated pixels 9 of the CMOS sensor within a set short integration time Δ _A ≦ Δ _L. An electronic trigger pulse establishes a fixed temporal relationship between the emitted light pulse 2 and the opening of the integration time window on the CMOS sensor. Due to the running time of the light, there is a different time shift depending on the object distance R.

between the light pulse emitted and detected at the CMOS sensor. As measured by the pixel within the integration time Δ _A charge Q _A is thereby dependent on the distance R between the sensor and object point G. See Fig. 2.

Q _A ∝ I ₀ . O _R (Δ _L - (2R / v _c - t _D )) (1)

I ₀ intensity of the emitted light pulse
O _R surface reflection coefficient at object point G
t _D Trigger point time delay between emitted light pulse and start of the integration window on the CMOS sensor

For object points G with the same surface reflection coefficient O _R , depending on their distance R, a different charge Q _{A is} measured at the associated pixel of the CMOS sensor. This transforms small transit time differences of the light pulses into changes in charge Q _A. With a CMOS sensor, these can be detected very sensitively and with high dynamics. Usually, the objects of a three-dimensional scene have a different surface reflection. A second image recording Q _B is therefore carried out to standardize the distance image, which is only dependent on the surface reflection of the objects of the three-dimensional scene.

The implementation of a second image recording B with a long integration time Δ _B serves to standardize the surface reflection of the three-dimensional scene, in principle using the conventional intensity or gray value image. For this purpose, the integration time Δ _B , which is very large compared to the length of an illuminating light pulse, is set on the CMOS sensor during a second image recording; Δ _B »Δ _L z. B. 1 microsecond. Now all backscattered light pulses 3 are fully detected on the CMOS sensor regardless of their runtime. The charge Q _B measured at one pixel is admitted

Q _B ∝ I ₀ x O _R Δ _L. (2)

The image obtained is only dependent on the illumination intensity I _0, the surface reflection coefficient O _{R of} the associated object point, and the light pulse length Δ _L.

The two-dimensional distance image Q _{R is} generated by the calculation from the difference and normalization of image acquisition A and B or Q _A and Q _B

Q _R = (Q _A - Q _B ) / Q _B (3).

From equations (1) and (2) follows the equation with t _d = 0

Q _R ∝ -2R / (v _c . Δ _L ) (4).

After reading out and digitizing and for additional scaling, this value can be output directly as a distance image Q _R for all pixels. If the trigger delay time t _{d is} not equal to 0, then a constant offset is added to all points of the distance map Q _R

R _D = t _D / (v _c . Δ _L ). (5).

The simultaneous recording of the intensity and three-dimensional image relates to the execution of a spatially and temporally parallel acquisition of intensity and distance values. For this purpose, a chip architecture and pixel-related integration time is chosen such that directly adjacent pixels A and pixel B according to FIG. 3 on the CMOS sensor the backscattered light pulses 3 of the three-dimensional scene simultaneously with a short integration time Δ _A ≦ Δ _L (for pixel A ) and record with a long integration time ΔB »Δ _L (for pixel B). An integrated electronic circuit on the chip can then directly the two-dimensional distance image

Q _R = (Q _A - Q _B ) / Q _B (6)

of the assigned pixels A and B are calculated and output the.

Fig. 3 illustrates this schematically two possible arrangements on the CMOS sensor for the parallel detection of intensity and a three-dimensional distance image. Other variants are possible. The simultaneous acquisition of intensity and three-dimensional distance images is particularly important for the analysis of moving three-dimensional scenes, for example the acquisition of gestures or the tracking of objects. Further special characteristics of the invention are:

• - If necessary, an additional standardization of the three-dimensional distance image with respect to ambient light can be carried out. For this purpose, the charge of a pixel with a short and long integration time is first detected without illuminating the three-dimensional scene or the object and is subtracted from the charges Q _A and Q _B measured with illumination. The distance image Q _{R is} then calculated.
• - By averaging the signals of several lights in time pulse can increase the sensitivity of the procedure  compared to the noise at low backscattered light intensities can be achieved.
• - The measurement uncertainty for the distance determination depends on Signal / noise behavior of the CMOS sensor. This is expected Differences in transit time between 0.1 ns can still be detected can. This results in a measurement uncertainty of less than 3 cm for the distance determination.

The essential uses of the described method and the described device relate to the monitoring of interior spaces, in particular in vehicles in connection with volumetric evaluation methods. The task of optical interior surveillance in vehicles is the detection of seat occupancy, such as people, child seats, other objects, the detection of the seating position of people and theft protection, ie the inadmissible intrusion into the vehicle interior from the outside. The detection of people and their seating position is of high safety-relevant importance for the gradual deployment of an airbag (smart airbag) and must be carried out very reliably and in short measuring times in the event of a collision. The invention fulfills these requirements by quickly and reliably generating a three-dimensional distance image Q _R in the vehicle interior, using volumetric evaluation methods. In this case, from the distance values R in a solid angle element Ω, the net volume portions occupied by objects 1 in the vehicle interior are determined as the difference to the distance values with the vehicle interior unoccupied (see FIG. 4).

The method and the device provide further significant advantages, such as:

• - Fast, global recording of the current seat occupancy by forming a difference between a three-dimensional distance image from the vehicle interior without objects (three-dimensional reference image Q _RO ) and the currently to be evaluated three-dimensional distance image with a person or another object Q _RP on a seat. The net volume V _P applies here the seat occupancy:
  V _P = ∫Ω R ₀ (Ω). dF - ∫Ω R _P (Ω). dF, (7),
  where R _{0 is} the distance values without a person or other object and R _{P is} the distance values with a person or other object on the seat and dF denotes a differential area.
• - The adaptive determination of the seat occupancy from the comp of the relative changes in distance before and after on getting a person into the vehicle can be done. By using regressive and stochiastic evaluation ver The reliability of the difference determination can still drive be further increased.
• - The size determination of the detected objects and global Un differentiation of objects over volume comparison classes possible.
• - Spatial allocation of occupied volume shares is possible Lich.
• - Determination of the spatial extreme positions (x, y, z) of the be set volume in the interior for controlling the airbag Tripping can be determined.
• - Volumetric tracking of movements in space for sequential images and differences boundary formation. Recognition of people and gestures from the movement analysis.

This integral volume analysis enables a global one Detection of objects and positions in space and is not on the determination of features such as contours edges, corners, edges in the image are directed to object detection. The evaluation times can be for the three-dimensional image acquisition and volumetric evaluation are less than 10 ms.  

A vehicle interior is particularly suitable as an area of application for the method and device described. An object is exposed for three-dimensional image acquisition with LED light pulses of, for example, 50 ns (nanoseconds). The integration times on the CMOS sensor are chosen for the image acquisition Q _A to 50 ns and for the image acquisition Q _B to 0.5 µs. The scene dynamics to be recorded inside the vehicle should be 200: 1. The distance values R should be recorded with a measurement uncertainty <15 cm (corresponding transit time difference of a light pulse = 1 ns) in a measuring range up to 1.5 m (transit time 10 ns).

With these requirements, an intensity dynamic of (10 × 200 =) 2000: 1 is required on the CMOS sensor. The digital acquisition of the three-dimensional distance image Q _R is thus ensured by a 12 bit A / D converter. For a sensor location resolution of 50 × 50 pixels, a maximum of 10 ⁴ readout operations are required for the image recordings A with a short integration time and B with a long integration time, which lead to a total image acquisition time for the three-dimensional distance image of a maximum of 5 ms at readout frequencies of, for example, 2 MHz. The calculation of the difference volume from the 2500 distance values can be carried out with a fast processor, such as a Pentium with 200 MHz, in a further 5 ms without difficulty.

In FIG. 4 a Schemal for an application of the invention is illustrated in vehicle interiors. The arrows with dotted lines represent a vacant seat and those with solid lines represent a seat occupied by a person. For global object recognition and position determination, the enveloping net volume fraction is determined from the three-dimensional distance data for occupied and unoccupied vehicles. The net volume V _{P of} a person or other object on a car seat is calculated according to equation (7).

Reference list

1

object

2nd

Light pulse of length Δ _L

3rd

Light pulse of length Δ _L

, scattered back

4th

sensor

5

Lighting device

6

,

7

Optics

8th

Trigger device

9

Pixel element
G object point
Δ _A

; Δ _B

Integration time

Claims (18)

1. Method for recording a three-dimensional distance image of spatial objects using an image-resolving and optionally readable optoelectronic sensor ( 4 ), the integration time Δ of which can be set pixel by pixel, consisting of the following steps:

• 1. the object ( 1 ) is illuminated with at least one light pulse ( 2 ),
• 2. from object points (G) backscattered light pulses ( 3 ) with a time period Δ _L are detected at associated pixels of the sensor ( 4 ) within a predetermined short integration time Δ _A , with Δ _A ≦ Δ _L , the time for the beginning of Integration time Δ _A before the arrival of the first backscattered light pulse ( 3 ), which corresponds to the closest object point (G),
• 3. Distance values are determined from the different recorded intensities of the backscattered light pulses ( 3 ) resulting from their different running times.

2. The method according to claim 1, wherein for the simultaneous or subsequent normalization of the surface reflection of the object ( 1 ) additionally all backscattered light pulses ( 3 ) with a long integration time Δ _B »Δ _{L are} completely detected. 3. The method according to any one of the preceding claims, wherein the start of an integration time Δ _A ; Δ _{B is associated} with a trigger pulse delay. 4. The method according to any one of the preceding claims, wherein an integration time Δ _{A is} less than 100 ns. 5. The method according to any one of the preceding claims, wherein an integration time Δ _{B is} approximately 1 µs. 6. The method according to any one of the preceding claims, wherein a light pulse length is less than 100 ns. 7. The method according to any one of claims 2-6, wherein for the simultaneous recording of a three-dimensional and a gray-scale image on the sensor ( 4 ), line-by-line different integration times Δ _A and Δ _{B are} set. 8. The method according to any one of claims 2-6, wherein for the simultaneous recording of a three-dimensional and a gray-scale image on the sensor ( 4 ) different integration times Δ _A and Δ _B are set alternately. 9. The method according to any one of the preceding claims, wherein the object ( 1 ) is illuminated with light pulses from a laser or a ge pulsed light emitting diode. 10. The method according to any one of the preceding claims, wherein the sensor ( 4 ) is a CMOS sensor. 11. The method according to any one of the preceding claims, wherein detection of static objects and / or movement is carried out. 12. The method of claim 11, wherein objects are monitored the, such as objects or people in rooms or in vehicles gene. 13. The method of claim 11, wherein vehicles or crane were monitored and / or in what a general Naviga tion is made. 14. The method of claim 11 or 12, wherein a Sitzbele tion and / or a seating position in a vehicle becomes.   15. Device for recording a three-dimensional distance image, consisting of:

• 1. an illumination device ( 5 ), which emits Lichtim pulses ( 2 ) via optics ( 6 ) on an object ( 1 ),
• 2. an optoelectronic sensor ( 4 ) with an optical circuit ( 7 ) which detects the light pulses ( 3 ) scattered back from the object ( 1 ), the sensor ( 4 ) being built up by a large number of pixel elements ( 9 ) to resolve pixels and optionally can be read out and the integration time Δ can be set pixel by pixel,
• 3. a trigger device ( 8 ) for temporal coordination between the lighting device ( 5 ) and the sensor ( 4 ),
• 4. a computing unit for calculating a three-dimensional image from the corresponding charges of the pixels.

16. The apparatus of claim 15, wherein on the pixel elements ( 9 ) of the sensor ( 4 ) rows or columns alternating alternately a short integration time Δ _A and a long integration time Δ _{B is} set. 17. The apparatus of claim 15, wherein on the pixel elements ( 9 ) of the sensor ( 4 ) alternating short and long integration times Δ _A and Δ _{B are} set. 18. Device according to one of claims 15-17, wherein the Re cheneinheit is arranged on the sensor ( 4 ).