DE102014224903A1 - Optical environment sensor for vehicles - Google Patents

Abstract

The invention relates to an optical environment sensor (1) for a vehicle (10) and comprises a first lens system (1.1), a second lens system (1.2) spaced from the first lens system (1.1), wherein the fields of view of the lens systems (1.1, 1.2) are at least partially overlap, and an image sensor (3), wherein the field of view of the first lens system (1.1) and the field of view of the second lens system (1.2) are mapped to disjoint sensor areas (3.1, 3.2). According to the invention, it is provided that the second lens system (1.2) is formed as a wide-angle lens in at least horizontal or vertical direction relative to the first lens system (1.1). The invention further relates to a method for operating the optical environment sensor according to the invention by the image sensor (3) is exposed and read line by line or column by column according to the rolling shutter method.

Description

The invention relates to an optical environment sensor for a vehicle with a first and second lens system and a common image sensor according to the preamble of claim 1. Furthermore, the invention relates to a method for operating such an optical environment sensor according to the invention and a vehicle with such an optical environment sensor according to the invention.

Camera systems, in particular stereo cameras or optical environment sensors with more than two lens systems for improved depth resolution, are used to implement driver assistance systems, such as, for example, a traffic sign recognition system or a lane departure warning system. A stereo camera is understood to mean two cameras, each with a lens optic and an image sensor, which are offset from one another by a predetermined distance, for example, on a windshield of the vehicle. Stereo cameras or optical environment sensors with more than two lens systems offer the advantage that distances can be measured with them.

A generic optical environment sensor for headlight control of a vehicle is from the CA 2 333 596 C with a first and second lens system with a common image sensor known. The viewing windows of the two lens systems, which are arranged in the vertical direction, are imaged on disjoint areas of the image sensor, namely on an upper portion and a lower portion of the image sensor, wherein the two lens systems are each designed for different wavelengths. Thus, a lens system is formed with a red filter, while the other lens system has a filter for the color complementary to the color red. As the opening angle of the field of view of the two lens systems, an angle in the horizontal with 22 ° and an angle in the vertical with 9 ° is specified. It is also proposed to realize such an optical environment sensor with more than two, for example with four lens systems. Thus, with three lens systems with corresponding filters, the colors red, green and blue can be detected, while unfiltered light can be detected with the fourth lens system.

The authors Stephan Neumeier and Georg Färber describe in the article "Sensor eye after the model of nature" (4th Munich Science Days in the Year of Technology, 2004) a vehicle camera with a wide-angle and Teleoptik, with which by means of image processing according to the so-called 4D approach the own lane detected and driving vehicles are detected. However, the detailed structure of this vehicle camera is not described.

Furthermore, from the US Pat. No. 3,612,764 a surveillance camera with a wide-angle and telephoto lens is known, the viewing areas are imaged onto a photosensitive layer of a vidicon image pickup tube.

Based on this prior art, it is an object of the invention to provide an optical environment sensor of the type mentioned, which is inexpensive to implement and simultaneously meets all requirements for the realization of driver assistance systems.

The object is achieved by an optical environment sensor having the features of patent claim 1.

Such an optical environment sensor for a vehicle comprises a first lens system, a second lens system spaced from the first lens system, wherein the fields of view of the lens systems at least partially overlap, and an image sensor, wherein the field of view of the first lens system and the field of view of the second lens system on disjoint sensor areas are imaged according to the invention characterized in that the second lens system is formed with respect to the first lens system in at least horizontal or vertical direction as a wide-angle lens.

With such a video camera is achieved that an object scene can be recorded with at least two mutually different magnifications, wherein the formed as a wide-angle lens second lens system in at least the horizontal or vertical direction leads to a much larger field of view than the first lens system. The wide-angle property of the second lens system also results in having a much smaller resolution than the first lens system. This video camera thus represents a monocamera with at least two lens systems whose partially overlapping viewing areas are disjointly imaged onto a single image sensor.

With such a video camera, the differences in the horizontal or vertical imaging beam path can be advantageously exploited.

The use of only a single image sensor for both the first and second lens system allows the use of the so-called rolling-shutter method, in which the image sensor is exposed and read line by line or column and thus due to the disjoint sensor areas of the two lens systems of the same with the two lens systems different magnification recorded object areas with double frame rate can be detected. This double frame rate can be used to advantage for time-critical functions, since within a 10/1000 second, another image of the same scene can be processed directly, while a conventional image sequence would require a hundredth of a second. Thus, fast object approaches, in particular of pedestrians, cyclists and vehicles coming to the vehicle from the side, can be more accurately detected and defined with respect to a collision course.

Another advantage of using this rolling shutter method is that exposure control is performed to change the integration time during a pass of the rolling shutter through the image. Thus, on the basis of the wide-angle image in a fraction of a second ahead of the telephoto image, the integration time can be adjusted or regulated, as well as the telephoto for the following wide-angle image. The algorithm may be a statistical evaluation made in image areas, such as mean, median or histogram formation of gray scale values, or a more complex evaluation via FFT edge quality algorithms in an image with a highly dynamic image sensor with dark and saturation limits. By means of such an adaptive integration time for the wide-angle image and the telephoto realized on the basis of the rolling shutter method, the different imaging paths realized by the first and second lens systems also complement each other here.

Also, an algorithm for compensating for motion blur can be realized due to the interaction between the images of the different imaging paths realized by the two lens systems.

Finally, it is also possible to perform an evaluation of rain on the windshield, fog on the inner pane of the vehicle and contamination of the optical paths of the two lens systems by the image artifacts caused by rain, for example, on the disjoint sensor areas of the two lens systems with different horizontal or vertical magnification are compared.

According to an advantageous embodiment of the invention, a third lens system is provided, the field of view is imaged on a disjoint to the sensor areas of the first and second lens system sensor area. Preferably, this third lens system is designed as a wide-angle lens, wherein the field of view in the horizontal direction has a smaller opening angle and in the vertical direction has a larger opening angle than the fields of view of the first and second lens system.

Such an imaging path, realized by means of the third lens system, in the extended vertical direction and simultaneous compression of the geometric resolution and restrictions in the field of vision in the horizontal direction makes it possible to detect light sources, such as traffic lights or top-mounted traffic signs and gantries, in the first lens system as well as in significantly better quality Spectral, geometric or higher-resolution appear and can be found, classified and measured there in a small already known by the recognition image area. Thus, a combined evaluation of three image data of these three lens systems achieves an improved system result in terms of performance, computing time and energy consumption compared to, for example, a stereo camera with two complete cameras.

It is particularly advantageous if, according to further development, the first lens system is designed as a telephoto lens. Thus, a high resolution is ensured in the horizontal or vertical direction in relation to the second lens system designed as a wide-angle lens.

According to a further preferred embodiment of the invention, the first and second lens system are arranged in the vertical direction. Preferably, the third lens system is arranged in the horizontal direction adjacent to the first and second lens system.

Finally, according to the invention, it is provided that the first lens system is focused on a long range and the second lens system is focused on a short range. Such a different focussing of the first and second lens systems makes it possible to image an identical object scene with different magnifications and to allow estimation of the object distance by analysis of edge steepnesses or by histogram analysis in the detected object. In particular, in the field of common object image with different magnification accuracies can be achieved, which can be achieved with a monocamera system with a single imaging path only by means of costly components. It is also possible to perform a time tracking with the rolling shutter method for deep calculation.

The optical environment sensor according to the invention can be used for driver assistance systems in vehicles.

The optical environment sensor according to the invention will be explained in more detail with reference to the accompanying figures. Show it:

1 FIG. 2 a schematic sectional view of an optical environment sensor according to the invention with two lens systems of a vehicle, FIG.

2 a schematic representation of an image sensor with disjoint sensor regions of the optical environment sensor according to 1 .

3 a schematic representation of an optical environment sensor with three lens systems,

4 a schematic representation of an image sensor with disjoint sensor regions of the optical environment sensor according to 3 , and

5 a schematic representation of the image sensor according to 4 for displaying an exposure method.

The in 1 represented inventive optical environment sensor 1 (also referred to as a camera-based environment detection sensor) of a vehicle includes a first lens system 1.1 and a second lens system 1.2 with a common image sensor 3 , which together with a diaphragm holder having lens holder 2 on a support structure 4 is arranged. The first lens system 1.1 is as a telephoto lens and the second lens system 1.2 designed as a wide-angle lens. The opening angle +/- α of the field of view of the first lens system 1.1 is much smaller than the field of view of the designed as a wide-angle lens second lens system 1.2 with an opening angle of +/- β. The opening angle α is preferably +/- 30 to 60 °, while the opening angle β is preferably +/-> 90 °. The wide angle property of the second lens system 1.2 is preferably formed in the horizontal direction. Depending on the application, this second lens system 1.2 also have only in the vertical direction a wide-angle property or of course in both directions.

The two viewing areas of the first and second lens system 1.1 and 1.2 become disjoint sensor areas of the image sensor 3 pictured like this in 2 is shown with an exemplary object scene. To separate the two imaging paths is according to 1 a visor body 5 intended.

In particular from 2 it can be seen that the two lens systems 1.1 and 1.2 in the vertical direction, that are arranged one above the other with respect to the vehicle vertical direction of the vehicle. So will according to this 2 the field of view of the first lens system 1.1 on a lower sensor area 3.1 of the image sensor 3 while the field of view of the second lens system 1.2 on an upper sensor area 3.2 of the image sensor 3 is shown.

With the telephoto lens of the first lens system 1.1 an object scene consisting of a person P and a star S is displayed. This object scene is also using the wide-angle lens of the second lens system 1.2 shown, but with a much larger opening angle, so that in addition a traffic light A and two circular objects O1 and O2 in the sensor area 3.2 be imaged.

This is in both sensor areas 3.1 and 3.2 of the image sensor 3 the same object scene, namely the person P and the star S present. By using the rolling shutter method for exposure and readout of the image sensor 3 These two identical object scenes can be captured at twice the frame rate. This will be the lines or columns of the image sensor 3 line by line or column by column exposed and read out. The result is an intermediate frame, which increases the frame rate significantly, because for the two object scenes of the identical image sensor, namely the image sensor 3 is read and no Zwischeninitialisierungen are required. The advantage is a higher temporal resolution, which can be used for object tracking and classification as well as in surveying.

Based on the rolling shutter method, exposure control can also be performed by adjusting or controlling the integration time based on the wide-angle image in a fraction of a second ahead of the telephoto image, as well as the telephoto image for the following wide-angle image. The algorithm may be a statistical evaluation made in image areas, such as mean, median or histogram formation of gray scale values, or a more complex evaluation via FFT edge quality algorithms in an image with a highly dynamic image sensor with dark and saturation limits. By means of such an adaptive integration time for the wide-angle image and the telephoto realized on the basis of the rolling shutter method, the different imaging paths realized by the first and second lens systems also complement each other here.

The first and second lens system 1.1 and 1.2 represent the imaging system of the optical environment sensor 1 and can be realized by means of refractive elements, such as individual lenses or a lens system, reflective elements, such as individual mirrors or a mirror system, but only in combination with the aforementioned elements, and by means of diffractive elements (diffraction structures). The properties of the first lens system 1.1 as a telephoto lens and the property of the second lens system 1.2 as a wide-angle lens are not by rotationally symmetric shaping of said imaging elements implemented in vertical and horizontal imaging property. It is also possible to use the two lens systems 1.1 and 1.2 with respect to the desired wavelength ranges by material selection and shaping to optimize, so that the desired wavelength range in the image on the image sensor 3 is filtered out of the input wavelength range. This can be done with the first and second lens system 1.1 and 1.2 be implemented in different or identical manner.

As an image sensor 3 known types such as CMOS or CCD sensors can be used. Sensors based on organic technology can also be used. The read image data of the image sensor 3 are for further processing of an evaluation, such as, a microcontroller, which may be designed as an FPGA, ASIC or DSP supplied. The processing steps of detection and preclassification may be performed in the imaging path of the second lens system formed as a wide-angle lens 1.2 during the processing steps of identification and measurement in the imaging path of the first lens system formed as a telephoto lens 1.1 or any other suitable imaging path, in order to obtain the target signal of the object scene (ROI) either locally, spectrally or temporally better resolved.

The image sensor 3 can in the different sensor areas 3.1 and 3.2 have different spectral sensitivity settings, z. B. Bayer pattern in the imaging path of the second lens system 1.2 and monochrome or French pattern in the horizontal wide-angle imaging path of the second lens system 1.2 ,

The design of the aperture of the two lens systems 1.1 and 1.2 can be the same or different. In particular, the diaphragm structure may consist of transparent and absorbent material for the working wavelength, in which case in particular the transparent material may have spectrally different properties.

The design of the panel with transparent and absorbent material may also include a non-planar shape in the transparent material, which allows an imaging property and the imaging property of the two lens systems 1.1 and 1.2 as a telephoto lens and wide-angle lens replaced or supplemented. In particular aberrations can be reduced by appropriate choice of material or amplified in the case of exploiting color error for 3D resolution.

LIST OF REFERENCE NUMBERS

1

Optical environment sensor

1.1

first lens system of the optical environment sensor 1

1.2

second lens system of the optical environment sensor 1

1.3

third lens system of the optical environment sensor 1

2

lens holder

3

image sensor

3.1

Sensor area of the image sensor 3

3.2

Sensor area of the image sensor 3

3.3

Sensor area of the image sensor 3

4

support structure

5

visor body

α

Opening angle of the first lens system 1.1

β

Opening angle of the second lens system 1.2

A

traffic light

O1

star

O2

circular object

O3

circular object

P

person

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• CA 2333596 C [0003]
• US 3612764 A [0005]

Cited non-patent literature

• "Sensor eye after the model of nature" (4th Munich Science Days in the Year of Technology, 2004) [0004]

Claims (9)

Optical environment sensor ( 1 ) for a vehicle ( 10 ), comprising - a first lens system ( 1.1 ), - one to the first lens system ( 1.1 ) spaced second lens system ( 1.2 ), whereby the fields of view of the lens systems ( 1.1 . 1.2 ) overlap at least partially, and - an image sensor ( 3 ), wherein the field of view of the first lens system ( 1.1 ) and the field of view of the second lens system ( 1.2 ) on disjoint sensor areas ( 3.1 . 3.2 ), characterized in that - the second lens system ( 1.2 ) compared to the first lens system ( 1.1 ) is formed in at least horizontal or vertical direction as a wide-angle lens. Optical environment sensor ( 1 ) according to claim 1, characterized in that a third lens system ( 1.3 ) is provided whose field of view to one of the sensor areas ( 3.1 . 3.2 ) of the first and second lens systems ( 1.1 . 1.2 ) disjoint sensor area ( 3.3 ) is displayed. Optical environment sensor ( 1 ) according to one of the preceding claims, characterized in that the third lens system ( 1.3 ) is formed as a wide-angle lens, wherein the field of view in the horizontal direction has a smaller opening angle and in the vertical direction a larger opening angle than the fields of view of the first and second lens system ( 1.1 . 1.2 ). Optical environment sensor ( 1 ) according to one of the preceding claims, characterized in that the first lens system ( 1.1 ) is designed as a telephoto lens. Optical environment sensor ( 1 ) according to one of the preceding claims, characterized in that the first and second lens system ( 1.1 . 1.2 ) are arranged in the vertical direction. Optical environment sensor ( 1 ) according to one of the preceding claims, characterized in that the third lens system ( 1.3 ) in the horizontal direction adjacent to the first and second lens system ( 1.1 . 1.2 ) is arranged. Optical environment sensor ( 1 ) according to one of the preceding claims, characterized in that the first lens system ( 1.1 ) to a far area and the second lens system ( 1.2 ) is focused on a near area. Method for operating an optical environment sensor according to one of the preceding claims ( 1 ) by using the rolling shutter method of the image sensor ( 3 ) is exposed line by line or column by column and read out. Use of the optical environment sensor ( 1 ) according to one of claims 1 to 7 for a driver assistance system in a vehicle.

Images