DE102011007464A1 - Method for visualizing scene, involves selecting scene region in three-dimensional image based on distance information, marking selected scene region in two-dimensional image and presenting scene with marked scene region on display unit - Google Patents

Abstract

The method involves recording of a three-dimensional (3D) and two-dimensional (2D) images of a scene using 3D- and 2D-cameras (200, 300), respectively. The cameras are connected with an image processing unit that processes the images. A scene region in the 3D image is selected based on distance information i.e. reduced distance to a monitoring camera (1), and the selected scene region is marked in the 2D image. The scene with the marked scene region is presented on a display unit. The mark is represented as cross. An independent claim is also included for a device for visualizing a scene.

Description

The invention is based on a method and a device for visualizing a scene recorded by a 2D / 3D surveillance camera camera according to the preamble of the independent claim.

From the US 2007/0201859 A1 Already the use of a 3D sensor in combination with a 2D sensor is known. The two sensors and also the light source for the 3D sensor are arranged in a common housing. The incident light through a common lens is split over a beam splitter on the 2D and 3D sensor. The beam splitter is preferably designed so that primarily infrared radiation is deflected in the direction of the 3D sensor.

Particularly suitable as 3D camera systems are so-called light runtime systems in which the transit time information is obtained from the phase shift of an emitted and received radiation. Particularly suitable as 3D cameras are so-called photo-mixing detectors or PMD cameras, as described, inter alia, in the applications DE 196 35 932 . EP 1 777 747 . EP 1 009 984 . US Pat. No. 6,587,186 and also in the DE 197 04 496 described and, for example, by the company "ifm electronic gmbh" as a frame grabber O3D101 / M01594 relate.

From the DE 20 2004 014 778 U1 Furthermore, a camera device for monitoring a blind spot area in front of and at the side of the cab cabin of a utility vehicle is known in which these areas, which are difficult or impossible to see, are monitored with the aid of a 2D camera.

The object of the invention is to provide a method and a compact surveillance camera, in particular for security-relevant applications.

The object is achieved in an advantageous manner by the method according to the invention and the device of the independent claims.

Advantageous is a method for visualizing a with a security camera 1 consisting of a 3D camera and a 2D camera recorded scene, comprising the following method steps: a) taking a 3-dimensional image of a scene with the 3D camera b) taking a 2-dimensional image of the scene with the 2D Camera, c) selecting a scene area in the 3-dimensional image based on distance information, d) marking the selected scene area in the 2-dimensional image, and e) displaying the scene with the marked scene area on a display unit BE.

The measures listed in the dependent claims advantageous refinements and improvements of the independent claim invention are possible.

It is particularly advantageous if the distance information, the smallest distance to the surveillance camera 1 is.

In a preferred embodiment, it is provided that the marking is displayed as a cross.

Advantageously, a device for carrying out the aforementioned method is provided.

The surveillance camera according to the invention advantageously combines a 3D and 2D camera in a common camera housing. The 3D camera is also associated with an active illumination, which together with the camera body forms a common structural unit. Furthermore, each camera has its own lens, so that advantageously the optical beam path can be optimally designed for the respective camera type.

Advantageously, the detection areas of the two cameras are matched to one another such that substantially the same areas are detected. Advantageously, at least one of the two cameras is adapted in this way.

In particular, it is advantageous if the 3D camera and / or the active illumination are configured such that the detection range of the 3D camera covers at least 70% of the illuminated area. Thus, it is advantageously avoided that light is lost in the overexposed areas or has a detrimental effect on the detected area.

Advantageously, the active illumination is configured such that the illuminated area is substantially adapted to the geometry of the detection area. In particular, it is advantageous, for example, in the case of a rectangular TOF sensor array, which also essentially has a rectangular detection area, to also give the illumination area a substantially rectangular geometry.

Expediently, the cameras have a detection aperture angle of greater than 90 °, in particular greater than 120 °, in at least one spatial direction.

In a further advantageous embodiment, the active illumination is constructed from an array of light sources, wherein the light sources together illuminate the entire detection range of the 3D camera.

In a preferred embodiment, at least one light source is designed such that this light source only partially illuminates the detection area. One of the advantages of this approach is that, for example, a selected light source can only be used for illuminating a particularly relevant area.

Furthermore, it is also possible to combine a plurality of light sources to form a lighting module. Such a module can in particular be arranged pivotably in or on the camera housing.

Embodiments of the invention are illustrated in the drawings and explained in more detail in the following description.

Show it

1 a light transit time measuring system,

2 schematically a commercial vehicle with two possible blind spot areas,

3 a surveillance camera according to the invention,

4 schematic a lighting and detection area,

5 a surveillance camera with swiveling lighting modules,

6 a surveillance camera according to the invention with an externally pivotable lighting module,

7 a surveillance camera system with connected screen unit in a schematic representation

1 shows a measurement situation for an optical distance measurement with a TOF camera system, as for example from the DE 197 04 496 is known.

The TOF camera system here comprises a transmission unit or a lighting module 100 with a light source 12 and associated beam shaping optics 50 and a receiving unit or 3D camera 200 with a receiving optics 150 and a photosensor 15 , The photosensor 15 is preferably designed as a pixel array, in particular as a PMD sensor. The receiving optics typically consist of improving the imaging properties of a plurality of optical elements. The beam shaping optics 50 the transmitting unit 100 is preferably formed as a reflector. However, it is also possible to use diffractive elements or combinations of reflective and diffractive elements.

The measurement principle of this arrangement is based essentially on the fact that, based on the phase difference of the emitted and received light, the transit time of the emitted and reflected light can be determined. For this purpose, the light source and the photosensor 15 via a modulator 18 acted upon together with a certain modulation frequency with a first phase position a. The light source sends according to the modulation frequency 12 an amplitude modulated signal with the phase a. This signal or the electromagnetic radiation is in the illustrated case of an object 20 reflects and hits due to the distance traveled correspondingly phase-shifted with a second phase position b on the photosensor 15 , In the photosensor 15 becomes the signal of the first phase a of the modulator 18 mixed with the received signal, which has meanwhile assumed a second phase position b, and determines the phase shift or the object distance from the resulting signal.

2 schematically shows a commercial vehicle with a cab cab. Due to the design of the cab, the areas directly in front of and to the side of the cab are difficult or impossible to see. In 2 characterized as a first, second and third monitoring area 510 . 520 . 530 , Typically, it is intended to monitor these areas with additional mirrors. From the already mentioned DE 20 2004 014 778 U1 Furthermore, systems are known which monitor these areas with a 2D camera. Monitoring these areas with a mirror or a conventional camera has the disadvantage that a danger in this area is only detected if this area is actively monitored by the driver. In particular, persons or obstacles suddenly entering this area can easily be overlooked. Monitoring the surroundings of these close-range areas using a wide-angle video camera, preferably with horizontal opening angles greater than 90 °, alone is not sufficient. Alternatively, for example, is also used on ultrasound or laser scanner. For safety-relevant applications, however, it is disadvantageous to base monitoring only on a physical measured variable or a sensor principle. For example, standard video signals are no longer reliable if too little light illuminates the scene to be monitored. Scanning methods have the disadvantage that they do not cover the environment comprehensively, but only scan one or a few one-dimensional lines and thus can not reliably recognize the presence of an object or obstacle, depending on its position. Furthermore, the frame rate is low due to the scanning process. To overcome the disadvantages and limitations of a single measurement principle, it is advantageous to combine the methods with other methods.

By integration of a 2D camera, preferably based on a CMOS imager chip and a 3D camera, preferably based on the PMD principle in a housing and by designing the lenses of the two cameras and the beam guiding optics of the active illumination at an opening angle greater than 90 In particular, a safe and reliable surveillance camera can be realized for a close range. The security and detection probability of objects is significantly improved over conventional video cameras. The 3D camera monitors the environment by illuminating the scene with modulated infrared light and using it to obtain 3D spatial proximity information based on photo-blending detectors. In contrast, the 2D camera uses unmodulated ambient light, for example natural daylight or headlight light from vehicles, in order to generate an image of the environment without distance information on the basis of CMOS imager chips. In particular for camera systems which are to be certified to SIL, this diversity creates a basis for reliably detecting objects in the vicinity. Furthermore, the failure of a system can be reliably detected by Querplausibilisierung.

3 shows a surveillance camera according to the invention 1 in which the 2D and 3D camera 300 . 200 as well as the active lighting 100 are arranged in a common housing. The 2D and 3D camera 300 . 200 are located in a central area of the housing. The active lighting is in two lighting modules 100 divided, which surround the two cameras sideways. The 2D camera 300 and the 3D camera 200 each have their own lens. The two camera systems are designed such that at least in one spatial direction an opening angle of at least 90 °, preferably even more than 120 °, is detected. The active lighting associated with the 3D camera is adapted accordingly for the illumination of such a detection area. In the illustrated example, the two lighting modules divide the illumination of the detection area in a left and right area. However, it can also be provided that each illumination module illuminates the detection area not only partially, but completely.

The proposed compact design is particularly suitable for outdoor use, and should then meet at least the protection class IP65.

The active illumination of the 3D camera is preferably designed so that the detection range of the 3D camera is accurately illuminated. In 4 If such a case is sketched schematically. The active lighting or the lighting module 100 Here, an illumination range Ω _{S is} clamped, within which the reception range Ω _{E of} the 3D camera 200 as well as the 2D camera 300 (not shown here) is located. In the ideal case, the transmission and reception ranges are congruent so that the emitted beam power can be utilized to the maximum. However, it has been shown that the performance of the system already improves significantly if at least 70% of the illuminated area Ω _{S is} detected by the detection range Ω _{E of} the 3D camera. The improvement in performance is not only due to the fact that more light from the radiation source can be exploited, but also that disturbing and disadvantageous reflections in the overexposed area are suppressed. The geometry of the illumination area can be optimized in various ways. On the one hand, it is possible the beam shaping optics 50 the respective light sources 12 However, it is also possible for a particular light source 12 a uniform beam guidance optics 50 provide and here by pivoting and targeted alignment of the lighting modules 100 to adequately illuminate the detection range Ω _E.

Such an arrangement is exemplary in FIG 5 shown. Unlike the in 3 As shown here arrangement, the lighting modules are pivotally mounted, and thus allow easy adjustment of the lighting areas to the desired detection area. Such an arrangement has the advantage that lighting modules, which consist of a plurality of reflectors or beam-forming optics, can be tilted in certain directions against each other so as to be able to illuminate corresponding detection areas completely.

In 6 a further preferred embodiment is shown in which the 3D and 2D camera 200 . 300 and the active lighting 100 is not arranged in a common housing, but in two housings, namely a camera body 5 and a lighting module housing 105 is divided, which together form a common unit 1 or surveillance camera 1 form.

This modular structure has the advantage that, for example, for a given camera system, the number of lighting modules varies freely can be. In particular, it may also be provided to use more than two lighting modules. By using a variety of lighting modules, it is possible to illuminate the detection area very individually. In particular, it is possible to adapt the geometry of the illumination area Ω _S more precisely to the geometry of the detection area Ω _E , without the need for elaborate optimization of the beam guiding optics of the individual light sources. The modular structure also has the advantage that the electronics of the two cameras can be better protected from the waste heat of the lighting units.

Furthermore, the intensity distribution within the illumination range can be adjusted by correspondingly shaped beam guiding optics or by a corresponding orientation of the illumination modules. Thus, for example, it is possible that in the central area of the detection area, multiple light cones of different illumination modules overlap in order to increase the light intensity in order, for example, to improve the reliability of the object recognition in this area.

In 7 is a surveillance camera system 10 with connected display unit BE shown. The surveillance camera system 10 consists of a surveillance camera 1 , as explained in more detail above. Shown are schematically the 3D camera 200 and the 2D camera 300 both connected to an image processing unit VPU.

In the image processing unit VPU are those of the two cameras 200 . 300 processed images processed and prepared for display on a connected display unit BE.

The method according to the invention is described below 7 explained in more detail. The scene, shown here as a stick figure SM, is recorded both with the 3D camera and with the 2 D camera (method steps a and b). The 3D camera 200 provides a 3-dimensional image of the scene, with the individual objects of the scene (arms, legs head, etc.) being assigned a distance information.

The 2D camera 300 provides "only" a 2-dimensional image of the scene. In general, the 2D camera has a much higher resolution than the 3D camera, ie the individual objects of the scene are displayed much more detailed.

Now, in a method step c, a scene area of interest is selected in the 3-dimensional image. This can be, for example, the scene area with the shortest distance D to the surveillance camera 1 be. In the case shown, this is the hand of the stick figure SM.

Subsequently, the scene area corresponding to the scene area selected in the 3-dimensional image is marked in the 2-dimensional image (method step d) and the entire scene with the marked scene area is displayed on the screen unit BE (method step e).

The observer thus sees on the screen unit BE the stick figure SM with a hand marked by a cross. If the stick figure moves, the mark remains at hand until another part of the stick figure SM of the surveillance camera remains 1 comes closer. The scene area of interest is always visually highlighted on the screen unit BE for the viewer. This makes it much easier for the viewer to capture the relevant screen information. He knows which scene area or which object he needs to focus on.

The method according to the invention can be implemented with a surveillance camera 1 , in which the 2D camera and the 3D camera are integrated in one housing, perform relatively easily. However, it is not necessarily limited to this particular type of camera. The two cameras do not necessarily have to be integrated in one housing.

LIST OF REFERENCE NUMBERS

1

Security camera, common unit

10

Security Camera System

100

Sending unit, active lighting, lighting module

200

Receiving unit, 3D camera

300

2D camera

50

Beam shaping optics

15

photosensor

18

modulator

12

light source

5

camera housing

105

Lighting module housing

510

First surveillance area

520

Second surveillance area

530

Third surveillance area

SM

stick figure

VPU

Image processing unit

BE

monitor unit

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2007/0201859 A1 [0002]
• DE 19635932 [0003]
• EP 1777747 [0003]
• EP 1009984 [0003]
• US 6587186 [0003]
• DE 19704496 [0003, 0029]
• DE 202004014778 U1 [0004, 0032]

Claims (4)

Method of visualizing one with a surveillance camera 1 consisting of a 3D camera and a 2D camera recorded scene, with the following procedural steps a. Take a 3-D image of a scene with the 3D camera b. Taking a 2-dimensional image of the scene with the 2D camera c. Selection of a scene area in the 3-dimensional image due to distance information d. Highlight the selected scene area in the 2-dimensional image e. Representation of the scene with the marked scene area on a screen unit BE A method according to claim 1, characterized in that the distance information, the smallest distance to the surveillance camera 1 is. Method according to one of the preceding claims, characterized in that the marking is displayed as a cross. Apparatus for carrying out the method according to one of the preceding claims.

Images