DE102019212978A1 - Monitoring device for person recognition and method - Google Patents

Abstract

Monitoring device 2 for person recognition in a surveillance area 7, with at least one camera 3 for capturing an image sequence of a section 10 of the surveillance area 7, the image sequence comprising a plurality of surveillance images, with a person recognition module 6, the person recognition module 6 having at least one recognition feature of a person to be found and is designed to search for the person to be recognized based on the surveillance images and the recognition feature, the camera 3 having a fisheye lens 4.

Description

State of the art

A monitoring device for person recognition in a monitoring area is proposed. The monitoring device has at least one camera for recording an image sequence of a section of the monitoring area. The image sequence has a plurality of surveillance images. The monitoring device comprises a person recognition module in order to search for and / or find the person based on a recognition feature of a person to be found and the surveillance images. The invention further relates to a method for monitoring a surveillance area and for locating the person.

The surveillance of public areas and / or other surveillance areas by means of optical methods is widespread. In particular, image-based person recognition plays an increasing role in such areas. The aim is, for example, to recognize a person in an airport or a train station and / or to track them across cameras. For example, lost children or suspicious people can be quickly searched for and found.

For this purpose, video cameras are often used, in the image material of which people are searched, based on personal characteristics. The problem with such methods is that many people in a surveillance area have a high intrapersonal variance and at the same time there is a low interpersonal variance.

The publication EP 2 189 955 B1 , which is probably the closest state of the art, describes a sensor network system for tracking moving objects in a surveillance area. The network system comprises a plurality of network nodes, each of which monitors a sub-area of the monitoring area. In particular, the network nodes are designed as video cameras. The sensor network system has a classifi cator generator.

Disclosure of the invention

According to the invention, a monitoring device for person recognition in a monitoring area is proposed with the features of claim 1. Furthermore, a method for monitoring the surveillance area and for person recognition is proposed with the features of claim 15. Preferred and / or advantageous embodiments of the invention result from the subclaims, the following description and the attached figures.

According to the invention, a monitoring device for person recognition in a monitoring area is proposed. In particular, the monitoring device is designed for optical, image-technical and / or video-technical monitoring of the monitoring area. By means of the monitoring device, a person to be recognized and / or sought can be found in the monitoring area by data evaluation and / or image evaluation. The monitoring area is preferably an interior, alternatively an outside area or a mixed area. For example, the surveillance area is a public building such as an airport, a train station or an authority. Furthermore, the monitoring device can be designed to recognize and / or search for objects, for example pieces of luggage such as suitcases, vehicles such as cars or animals.

The monitoring device has at least one camera. In particular, it can be provided that the monitoring device has exactly one camera, alternatively the monitoring device has more than two, five or ten cameras. If the monitoring device has a plurality of cameras, these can be of identical design. The at least one camera and / or the cameras are designed to record an image sequence of a section of the monitoring area or to record an image sequence of the entire monitoring area. The section of the monitoring area is, for example, the viewing area and / or recording area, which can be imaged and / or recorded by the camera. The camera is preferably designed and / or arranged in a bird's eye view to record the section of the monitoring area and / or the monitoring area. In particular, an optical axis of the camera is oriented essentially downward in the perpendicular direction, such that the camera captures the section of the monitoring area and / or the monitoring area in the nadir direction. The sections of the surveillance area that are monitored by one camera are preferably but not necessarily partially overlapping in the case of several cameras, so that a person can be tracked and / or monitored, for example, without gaps. The camera is preferably a color camera. The camera has, for example, a CCD sensor or a CMOS sensor for data generation.

The image sequence has a plurality of surveillance images. In particular, the image sequence has at least two surveillance images or more than ten surveillance images. The Surveillance images of the image sequence are preferably taken at equidistant time intervals or taken at random in a timely manner. The surveillance images form the section of the surveillance area optically, in particular two-dimensionally, from . In particular, the image sequence can form a video sequence. In particular, the image sequence comprises surveillance images from a single camera. If the monitoring device has a plurality of cameras, then a plurality of image sequences are preferably generated and / or generated.

The monitoring device has a person recognition module. The person recognition module can form a hardware module, for example a computer unit or a processor chip. Alternatively, the person recognition module can form a software module. The person recognition module is preferably encompassed by the camera or alternatively a central person recognition module. The image recognition and / or the surveillance images of the camera and / or the cameras are provided to the person recognition module. The person recognition module has at least one recognition feature of a person who can be found. The recognition feature can be included as a recognition date and / or as a recognition file by the person recognition module. For example, the person recognition module can have several recognition features for one person or for different people to be found. In particular, the recognition feature can be made available and / or stored to the person recognition module. The recognition feature can, for example, be newly stored and / or entered for the person to be found. The recognition feature can include several parameters and / or data. For example, the recognition feature comprises and / or is formed by a size, by a structure, by colors, items of clothing and / or shapes. For example, the recognition feature for a person to be found can be that the person is wearing long brown hair, a yellow sweater and blue trousers. In particular, the person recognition module can have recognition features for the person to be found with regard to different views and / or shooting perspectives, for example different recognition features of the person for shooting perspectives from the front, from behind, from above and / or from the side.

The person recognition module is designed to search for the person to be found and / or found in the surveillance area based on the surveillance images and / or the recognition feature. For this purpose, the surveillance images, in particular the image sequence, are examined by the person recognition module for the presence of the recognition feature. If the recognition feature is present in the surveillance images, all and / or a sufficient number of the recognition features are present, the person recognition module can be designed to conclude that it is the person to be found.

According to the invention, it is provided that the camera has a fisheye lens. In particular, the camera can have a fisheye lens as a fisheye lens and / or part of the fisheye lens. Fisheye objectives are understood to mean, in particular, objectives which do not proportionally depict an objective plane perpendicular to the optical axis. Fisheye lenses form with distortions on an image plane in particular from . For example, straight line pieces that do not run through a center of the image are shown curved. The fisheye lens is specially designed to faithfully represent area ratios and / or radial distances. Furthermore, the fisheye lens and / or the camera preferably has an angle of view of at least 180 ° and in particular of more than 210 °. The fisheye lens can be designed as a circular fisheye lens, as a full-frame fisheye lens or as a cropped circle fisheye lens. The surveillance images of the image sequence and / or the image sequence thus have images of the section of the surveillance area with a fisheye effect.

The invention is based on the consideration that cameras with a fisheye lens can monitor a larger area of a scene than normal cameras, at the same time there is less obscuring of people and fisheye images can also be converted into perspective images. For example, in fisheye lens images, objects are compressed more towards the edge than in the middle. A camera with a fisheye lens is also called a fisheye camera. The camera is preferably designed as a panorama camera with a 360 ° hemispherical recording area. For example, the camera is designed as an omnidirectional camera. In one variant, the fisheye lens or the fisheye camera is designed as a panomorphic lens or as a panomorphic camera. A panomophic lens is a special version of a fisheye lens, in which the optical properties of the lens are changed in predetermined areas of the image in such a way that image processing, in particular object recognition, is facilitated. This is achieved, for example, by means of optical distortions, the field of view is changed such that, for example, marginal areas are shown enlarged. Alternatively or additionally, it is provided that a different number of pixels are provided in different image areas of the camera. In particular, it is provided that the pixel density is higher in edge areas than in the center of the image.

It is particularly preferred that the monitoring device has a perspective determination module. The perspective determination module can form a hardware module or a software module. For example, the person recognition module forms an evaluation module together with the perspective determination module. The perspective determination module is designed to determine a recording perspective in the surveillance images for a person moving in the section of the surveillance area and / or a moving object. The perspective of the photograph is understood to be, for example, the view of the person and / or the object from the front, the view from the rear, a side view or a view from above. For example, when the camera is mounted on the ceiling from a bird's eye view of the surveillance area, people and / or objects are imaged, recorded and / or analyzed from different angles, depending on their distance from the floor level to the optical axis of the camera. People directly under the camera are, for example, imaged and / or recorded in a bird's eye view, while people and / or objects are increasingly depicted from the side with increasing distance from the optical axis. For example, the perspective determination module is designed to be able to infer a recording perspective based on features of the moving person depicted and / or the object. Features can represent, for example, an extent, facial features or color gradients. This embodiment is based on the consideration that cameras with a fisheye lens have a special beam path and are therefore able to record people and / or objects in different views, in particular even if the person and / or the object is moving in a straight path . So far, it has been necessary with normal cameras to let the person take a curved path, for example by structural measures such as barrier tapes, in order to be able to generate multiple shooting perspectives. In particular, the literature has so far made reference to the fact that fisheye lenses are disadvantageous when used in surveillance cameras because of their consuming properties and their use is not recommended.

It is particularly preferred that the perspective determination module is designed to generate at least two useful images based on the surveillance images of a camera, in particular a single camera, and the recording perspectives intended for this purpose. It is provided, for example, that the two useful images are based on the image sequence of a single camera. If several cameras are used in the monitoring device, it is preferably provided that at least two useful images are generated for each camera. The useful images are particularly characterized in that the person is shown in different recording perspectives for the useful images of a camera. For example, the person is shown from the front in a first useful image and the person from behind in a further useful image. In particular, the perspective determination module is designed to generate more than two useful images, these useful images being based on the surveillance images of one of the cameras. This embodiment is referred to in particular as multi-view generation because, based on the recording of the surveillance area with the fisheye lens, useful images of the person and / or an object can be generated by a single camera, which have different recording perspectives on the person and / or the object .

One embodiment of the invention provides that the perspective determination module is designed to determine a useful image triple based on the image sequence. The useful image triple comprises three useful images, in particular based on the image sequence of a single camera. The three useful images of the useful image triple include, for example, a front view of the person and / or the object, a rear view of the person and / or the object and a side view. The perspective determination module can also be designed to generate a useful image tuple, the useful image tuple comprising more than three useful images, for example four useful images comprising a front view, rear view, side view and a view from above. This embodiment is based on the consideration that at least two, three or more shooting perspectives of the person and / or the object can be obtained by means of the fisheye lens based on the monitoring data of a camera, and thus the evaluation stability when a person can be found can be improved. In particular, the interpersonal variance can be increased in this way.

It is optionally provided that the perspective determination module is designed to correct a fisheye effect in the surveillance images and / or the image sequence. In particular, the fisheye effect is corrected using dewarping methods (reshaping). For example, software applications can be used to correct the fisheye effect. Especially it is preferred that the fisheye effect in the surveillance images and / or in the image sequence is corrected before the shooting perspective is determined. For example, all surveillance images of the image sequence are corrected and then the perspective of the image is determined in the corrected images in order to generate the useful images therefrom. This embodiment is based on the consideration that the distorting properties of the fisheye lenses can be corrected and so that further evaluation algorithms can be applied to the corrected images.

An embodiment of the invention provides that the perspective determination module is designed to represent people and / or objects in the useful images in an oriented manner. Oriented, for example, means displaying a person and / or an object upright. Algorithms of person alignment can be used for this purpose, for example. For example, the perspective determination module is designed to project a detected and / or moving person into an image, in particular a useful image, in such a way that it stands upright in a bounding box. In particular, it can be provided that after orienting the person in the useful image, the fisheye effect is corrected by the perspective determination module.

It is particularly preferred that the person recognition module is designed to track the moving person and / or the moving object. For example, the tracking can include determining the path and / or the speed of the person and / or the object. The tracking can in particular be usable by the person recognition module to extrapolate the further path and / or path into the future. In particular, it is provided that the alignment of the person in the useful images is based on a previous track of the person. By knowing the sequence of movements, an orientation of the person and / or the object can be inferred and can thus be shown aligned in the useful images.

In particular, it is provided that the perspective determination module is designed, based on the tracking and / or tracking results, for example the path and / or the speed, to infer which recording perspective is present in the respective surveillance images. If, for example, it is known that the person was imaged from the front at a past point in time, it can be concluded based on the tracking and / or the path that the person must be seen in a rear view or a side view at a later point in time.

An embodiment of the invention provides that the perspective determination module is designed to determine the exposure perspective based on a machine learning algorithm, a computer vision algorithm or a deep learning method. Alternatively or additionally, geometry and the tracked person trajectory can be used. In particular, it is provided that the perspective determination module is based on a neural network, in particular a recurrent and / or coupled neural network.

In particular, it is provided that the monitoring device has a feature determination module. In particular, the feature determination module can be included in the perspective determination module. The feature determination module can form a hardware module or a software module. The feature determination module is designed to determine personal features based on the useful images. In particular, the feature determination module can be designed to determine personal features for each useful image. Person characteristics are, for example, descriptors and / or features of a person and / or an object. For example, personal characteristics are color gradients, clothing, hairstyles, proportions and / or a gender. The personal characteristics are particularly suitable for determining and / or characterizing a personal signature. The person recognition module is designed to search for and / or find the person to be recognized based on the person characteristics and the recognition characteristic. For example, the person recognition module is designed to compare the person characteristics with the recognition feature and, if there is a match, to conclude that the person can be found again. It is particularly preferred that the feature determination module is designed as a machine learning, computer vision or deep learning method and / or happens thereon.

It is particularly preferred that the monitoring device has a fusion module. In particular, the fusion module is part of the feature determination module and / or part of the perspective determination module. The fusion module is designed to combine personal characteristics of a person from different useful images, in particular useful images based on the image sequence of an individual camera, into a master feature. For example, the fusion module can be designed to assign the present personal characteristics to a recording perspective. The person recognition module is specifically designed to search for the person to be recognized in particular based on the master feature and the recognition feature. It is for that For example, provision is made for the recognition feature to be designed analogously to the master feature and for example also to assign a perspective to the recognition features. It is thus possible, for example, that the person recognition module only reports the person to be recognized as having been found when person characteristics have been found in different useful images, in particular in the different perspectives of the person.

Another embodiment of the invention provides that the monitoring device has a person image fusion module. The personal image fusion module is designed to generate a master useful image for a person based on the useful images, a master person characteristic being determined by the master useful image. The person recognition module is designed to search for and / or to find the person to be recognized based on the fusion personal characteristic and / or the fusion personal characteristics and the recognition characteristic.

Different fusion methods are used in particular as methods of the fusion module and / or the personal image fusion module. For example, an early fusion strategy or a late fusion strategy or a hybrid fusion strategy can be used. As an alternative and / or in addition, a deep learning strategy can be used. For an early fusion strategy, surveillance and / or useful images or features from the surveillance and / or useful images are first combined and then a resulting feature vector is generated, which is used to find the person and is compared, for example, with the recognition feature. In the case of a late fusion strategy, the recognition features are compared separately for each surveillance and / or useful image, and only then is the fusion carried out.

Wobei I_Super definierbar ist als $${\text{I}}_{\text{Super}}^{\left(\text{m}\times \text{3}\cdot \text{n}\right)}=\left[\begin{array}{ccc}{\text{I}}_{\text{Front}}^{\left(\text{m}\times \text{n}\right)},& {\text{I}}_{\text{Side}}^{\left(\text{m}\times \text{n}\right)},& {\text{I}}_{\text{Back}}^{\left(\text{m}\times \text{n}\right)}\end{array}\right]$$

Wobei I jeweils für ein Nutzbildet steht, der Index die Perspektive angibt und m, m die Pixelanzahl in Zeile/Spalte angibt.An example of an early fusion strategy is the generation of a super image. For example, the super image is the master usable image. For this purpose, three useful images of the same size, for example the number of pixels, are generated from and / or as a useful image triple. The feature vector for a person in a camera can be described as $\overrightarrow{\text{F}}=\text{f}\left({\text{I.}}_{\text{great}}\right).$

Where I _Super can be defined as $${\text{I.}}_{\text{great}}^{\left(\text{m}\times \text{3rd}\cdot \text{n}\right)}=\left[\begin{array}{ccc}{\text{I.}}_{\text{front}}^{\left(\text{m}\times \text{n}\right)},& {\text{I.}}_{\text{Side}}^{\left(\text{m}\times \text{n}\right)},& {\text{I.}}_{\text{Back}}^{\left(\text{m}\times \text{n}\right)}\end{array}\right]$$

Where I stands for a useful image, the index indicates the perspective and m, m the number of pixels in the row / column.

Ein Beispiel für eine late fusion Strategie ist beispielsweise der inverse rank position Algorithmus (IRPA), alternativ und/oder ergänzend kann als late fusion Strategie inverse score position Algorithmus (ISPA) oder der product rule Algorithmus (PR) verwendet werden. ISPA und IRPA beruhen auf dem mathematischen harmonischen Mittel.Another method of the early fusion strategy can form the feature concentration, for example. In contrast to the strategy described above, the features are first extracted from the useful images, the three features then being concentrated into a main feature vector, the main feature vector with the recognition features being used for recognizing the person. The main recognition vector can be described as $${\overrightarrow{\text{F}}}^{\left(1\times \left(\text{a}+\text{b}+\text{c}\right)\right)}=\left[\begin{array}{ccc}{\overrightarrow{\text{F}}}_{\text{front}}^{\left(1\times \text{a}\right)},& {\overrightarrow{\text{F}}}_{\text{Side}}^{\left(1\times \text{b}\right)},& {\overrightarrow{\text{F}}}_{\text{Back}}^{\left(1\times \text{c}\right)}\end{array}\right]$$

An example of a late fusion strategy is, for example, the inverse rank position algorithm (IRPA); alternatively and / or additionally, the inverse score position algorithm (ISPA) or the product rule algorithm (PR) can be used as the late fusion strategy. ISPA and IRPA are based on the mathematical harmonic mean.

A deep fusion strategy can also be used. A fusion using machine or deep learning is learned with the help of training data.

It is particularly preferred that the camera has a mounting device. The mounting means is designed to arrange and / or attach the camera in the monitoring area, in particular on a ceiling of the monitoring area. The mounting means is preferably designed such that the camera captures the surveillance area from a bird's eye view. In particular, the camera is fastened in the monitoring area by the mounting means such that the optical axis of the camera is perpendicular to the floor level of the monitoring area. This embodiment is based on the consideration that, by arranging the camera in a bird's eye view, several perspectives of a person can be generated in the useful images if the person is moving in space.

One embodiment of the invention provides that the monitoring device has a further optical sensor unit. For example, the further optical sensor unit is a surveillance camera, in particular surveillance camera without a fisheye lens. The further optical sensor unit is designed to monitor the monitoring area and / or the section of the monitoring area using imaging technology and to provide additional image data for monitoring. The person recognition module is preferably designed to additionally use the additional image data to recognize the person based on the surveillance images. This design is based on the consideration, a more robust one To be able to provide monitoring device for the recognition of the person.

Another object of the invention is a method for monitoring a surveillance area and / or for finding a person to be found again in the surveillance area. It is provided to monitor the surveillance area with a camera, the camera having a fisheye lens and thus monitoring and / or imaging the surveillance area. Based on the surveillance images taken with the fisheye lens, the person to be recognized is sought in the surveillance area. In particular, the method can provide that, by monitoring the monitoring area with the camera, based on the recording with the fisheye lens, useful images are generated and / or collected, which shows the person moving in the monitoring area from different perspectives.

The monitoring device for person recognition preferably comprises an unmanned aircraft in a monitoring area, the camera having at least one fisheye lens being arranged on the unmanned aircraft. In addition, it is provided that the person recognition module and / or the perspective determination module and / or the feature determination module and / or the fusion module and / or the person image fusion module is arranged in the unmanned aircraft. In the method for person recognition, the image sequence of the surveillance area is particularly preferably recorded during an overflight of the unmanned aerial vehicle, at least two useful images, in particular three useful images, being generated from the recorded image sequence, the useful images showing the person in different recording perspectives.

An unmanned aircraft is characterized by the fact that there are no persons on board the aircraft for control purposes. Instead, the unmanned aircraft is controlled autonomously by a computer on board and / or by a pilot via remote control. The unmanned aerial vehicle is preferably designed as a helicopter, in particular as a quadrocopter. This monitoring device and this method for person recognition have the advantage that drones only have to fly over crowds of people once and the recorded surveillance images can be used to analyze people from different perspectives. With the fisheye lens, different views of the person can be created during the overflight. The drone only has to fly over the person once and does not have to fly a special trajectory. Advantageously, it is easily possible to generate both a front and a rear view of a person. Front and rear views improve person recognition, as people generally look different in the front than in the back. For example, hair that is tied into a braid is only visible from behind in this form, while a shirt under a jacket is only visible from the front. This monitoring device or the method with a drone for person recognition is particularly suitable for use by the police at large events with a large number of people. By integrating the fisheye camera in the drone, the flight time of the drones is advantageously reduced, since the useful images of the people can be generated more quickly in different shooting perspectives. This helps to reduce energy consumption when using the drones. Patrol-flying drones can therefore work longer until they have to be reloaded. The number of drones can therefore be reduced for large events. The monitoring device with a drone is advantageously used together with stationary recognition systems. For example, if a person has registered in a stationary recognition system in a front view and is to be found again, the search area only has to be flown once, regardless of the direction of movement of the people in the search field, in order to obtain all the necessary views of the person. In a variant, the monitoring device is designed in such a way that, in a first method step, only the same color features are searched for at a greater spatial distance from the person. In a second procedural step, the drone flies closer to people preselected by matching color features in order to provide higher-resolution images of the preselected person. For example, if a person is searched for in green and a person in green was found, the drone can fly closer to the person and provide higher-resolution images without the drone having to perform a second flyover. One side of the person's body is then more highly resolved than the other, but both sides can be used for recognition and early search space reduction.

• 1 ein Ausführungsbeispiel einer Überwachungsvorrichtung;
• 2 Beispiel eines Strahlengangs einer Kamera mit Fischaugenobjektiv bei Verwendung eines Non-Single-Viewpoint-Projektionsmodells;
• 3 Beispiel eines Überwachungsbildes.

Further advantages, effects and configurations result from the attached figures and their description. Show:

• 1 an embodiment of a monitoring device;
• 2nd Example of a beam path of a camera with a fisheye lens when using a non-single viewpoint projection model;
• 3rd Example of a surveillance image.

1 shows a monitoring device 2nd as an embodiment of the invention. The monitoring device 2nd includes a camera 3rd , the camera 3rd a fisheye lens 4th , a perspective determination module 5 and a person recognition module 6 having.

The camera 3rd is for monitoring a surveillance area 7 educated. The surveillance area 7 is an interior, for example an airport. The surveillance area 7 has a blanket 8th and a floor 9 on. The camera 3rd is on the ceiling 8th mounted so that the camera 3rd the surveillance area 7 monitored in bird's eye view. The camera 3rd has a recording angle so that a section 10th as a subset of the surveillance area 7 is imaged technically.

The camera 3rd takes an image sequence from several surveillance images of the surveillance area 7 on. In the surveillance area 7 a person can move freely, the person being shown in the surveillance images. In terms of data technology, the surveillance images and / or the image sequence are the perspective determination module 5 provided. The surveillance images have a fisheye effect.

The perspective determination module 5 is formed from the surveillance images of the image sequence useful images 12a , 12b and 12c to generate. For this purpose, moving objects and / or people are searched in the image sequence and their shooting perspective is determined. The fisheye effect can be corrected before determining the shooting perspective, for example using dewarping methods. It can further be provided that the perspective determination module 5 the orientation of the moving person is oriented, for example that image sections of the surveillance images are rotated so that the person in the useful images 12th is shown upright.

Views are understood as views of the person, in particular whether the person is shown from the front, is shown from the rear or is shown from the side. For example, the perspective determination module 5 determines the shooting perspective for each person and / or each surveillance image of the image sequence. The perspective determination module 5 is then designed to select image sections and / or surveillance images for generating the useful images, which shows the person in different recording perspectives.

The perspective determination module 5 is trained, a useful image triple 11 to determine. The useful image triplet has three useful images 12a , 12b and 12c on, the usable image 12a the person shows the useful image from a front perspective 12b the person in a side perspective and the useful image 12c shows the person from behind in a perspective view. Instead of a useful image triplet, finer granular person views can alternatively be used, for example a useful image quadruple, useful image quintuple, useful image sextuple, useful image hexatuple etc.

The perspective determination module 5 can be based on the surveillance images of a single camera 3rd therefore determine different shooting perspectives for one person, because the fisheye lens 4th an object moving in the surveillance area 13a , 13b and 13c because of the distorted optics from different perspectives. The object is located 13b directly under the camera's optical axis 3rd this is shown in a bird's eye view. The object is located 13a For example, to the left of the optical axis, if this is imaged from the front, the object is located 13c For example, a rear view can be taken to the right of the optical axis.

The person recognition module 6 is the useful image triple 11 provided. Based on the useful image triple 11 and an entered and / or stored recognition feature, for example as a recognition feature data record, a person to be searched can be detected in the surveillance area. The person recognition module 6 compares characteristics of the person in the useful images with the recognition characteristics. If there are enough matches between features and recognition features, the person recognition module can 5 classify the person to be found as found.

2nd shows an example of the beam path of the camera 3rd with the fisheye lens 4th when using a non-single viewpoint projection model. The camera 3rd has an image sensor 14 on which is designed as a CCD or CMOS sensor. Visual rays 15a , 15b , 15c and 15d are through the fisheye lens 4th on the image sensor 14 pictured. The fisheye lens 4th and / or the camera 3rd does not have a point-shaped projection center, as in conventional cameras, but a line-shaped projection center 16 .

The angle α _a , α _b , α _c , α _d the sea rays on the moving object 13a , 13b , 13c , 13d is for that Fisheye lens each different. The further the object 13a , 13b , 13c , 13d from the optical axis or the projection center 16 the more distant the angle is α _a , α _b , α _c , α _d . For example, the angle a _b for the object 13b that is right below the camera 3rd is 90 °. The object 13a left of the projection center 16 has an angle α _a from about 45 ° and shows an imaging perspective from the front. The object 13c to the right of the projection center 16 has an angle α _d from about 20 ° and shows an imaging perspective from behind on the object. This means that you can use the fisheye lens 4th from a moving object different imaging perspectives with a camera 3rd be included.

3rd shows schematically an overlaid image from three surveillance images that are taken at a time interval. The moving object 13 , here is a person who is on a path 17th emotional. At first, this becomes an object 13a pictured, at a second time as an object 13b mapped and at a third point in time as an object 13c pictured. There are also two static objects 18a and 18b pictured. With the objects 18a , 18b are cabinets that are bent and straight in reality. Due to the consuming properties of the fisheye lens 4th they are shown distorted in the picture. The object 18a the image is particularly distorted in the edge area and appears circularly curved. The object 18b the one closer to the center is also shown distorted, but not so strongly curved.

Due to these consuming properties of the fisheye lens 4th is also the person as a moving object 13 mapped in different perspectives after the distance to the center or the projection center 16 changes. Here the person is an object 13a shown from the front, as an object 13b in the center of the image from above and as an object 13c pictured from behind when removed from the center.

In a variant of the exemplary embodiment described, the camera designed as a fisheye camera is arranged on an unmanned aerial vehicle (drone), in particular on a helicopter. The monitoring device for person recognition in a surveillance area is set up to recognize people from a drone perspective. The camera, which is designed as a fisheye camera or omnidirectional camera, is mounted on the drone in Nadir. It is envisaged that the drone will fly over a crowd with the fisheye camera or omnidirectional camera. Due to the fish-eye-like projection of the lens, the person is recorded from different views. It is provided that the person moves through the scene and / or that the drone moves relative to the person. It is further provided that the recorded surveillance images are subjected to a distortion correction and that at least two useful images, preferably three useful images, of the person are cut out of the recorded surveillance images in the form of a full exposure. The three useful images preferably show the person once from the front, once from the side and once from behind. Otherwise, the surveillance device for person recognition in a surveillance area with a drone is at least partially like the one above with reference to FIG 1 to 3rd described monitoring device trained.

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• EP 2189955 B1 [0004]

Claims (17)

Monitoring device (2) for person recognition in a surveillance area (7), with at least one camera (3) for recording an image sequence of a section (10) of the surveillance area (7), the image sequence comprising a plurality of surveillance images, with a person recognition module (6) , The person recognition module (6) has at least one recognition feature of a person to be found and is designed to search for the person to be recognized based on the surveillance images and the recognition feature, characterized in that the camera (3) has a fisheye lens (4). Monitoring device (2) after Claim 1 , characterized by a perspective determination module (5), the perspective determination module (5) being designed to determine, in the surveillance images, a shooting perspective for a person moving in section (10) of the surveillance area (7). Monitoring device (2) after Claim 2 , characterized in that the perspective determination module (5) is designed to generate at least two useful images (12a, b) based on the surveillance images of one of the cameras (3) and the recording perspectives determined therefor, the useful images (12a, b) showing the person in different ways Perspectives shows. Monitoring device (2) after Claim 3 , characterized in that the perspective determination module (5) is designed to determine at least three useful images (12a-c), in particular exactly three useful images (12a-c) or four useful images or more than four useful images, as a useful image triple (11), the three useful images (12a-c) of the useful image triple (11) show the person in a front view, in a rear view and in a side view. Monitoring device (2) according to one of the Claims 2 to 4th , characterized in that the perspective determination module (5) is designed to correct a fisheye effect in the surveillance images, in particular before the determination of the shooting perspective. Monitoring device (2) according to one of the Claims 3 to 5 , characterized in that the perspective determination module (5) is designed to display the person in the useful images (12a-c) in an oriented manner. Monitoring device (2) according to one of the preceding claims, characterized in that the person recognition module (6) is designed to track the moving person in the monitoring images and / or the useful images (12a-c). Monitoring device (2) after Claim 7 , characterized in that the perspective determination module (5) is designed to draw conclusions based on the tracking of the person and / or to extrapolate them. Monitoring device (2) according to one of the Claims 2 to 8th , characterized in that the perspective determination module (5) is designed to determine the admission perspective based on a machine learning, a computer vision and / or deep learning method. Monitoring device (2) according to one of the Claims 3 to 9 , characterized by a feature determination module, the feature determination module being designed to determine personal characteristics based on the useful images (12a-c), the personal recognition module (6) being designed to search for the person to be recognized based on the personal characteristics and the recognition characteristic. Monitoring device (2) after Claim 10 , characterized by a fusion module, the fusion module being designed to combine personal characteristics of a person from different useful images (12a-c) into a master characteristic, the personal recognition module (6) being designed to search for the person to be recognized based on the master characteristic and the recognition characteristic. Monitoring device (2) according to one of the preceding claims, characterized by a person image fusion module, the person image fusion module being designed to generate a master useful image for a person based on the useful images (12a-c) and to determine a fusion personal characteristic based on the master useful image, the person recognition module (6) is designed to search for the person to be recognized based on the fusion person characteristic and the recognition characteristic. Monitoring device (2) according to one of the preceding claims, characterized in that the camera (3) has mounting means for monitoring the monitoring area (7) in a bird's eye view. Monitoring device (2) according to one of the preceding claims, characterized by a further optical sensor unit for image monitoring of the monitoring area (7) and for providing additional image data, the person recognition module (6) being designed to search for the person to be recognized based on the additional image data. Monitoring device (2) according to one of the preceding claims, characterized by an unmanned aerial vehicle, at least the camera (3) having a fisheye lens (4) being arranged on the unmanned aerial vehicle. Method for person recognition in a surveillance area (7) with the surveillance device (2) according to one of the preceding claims, wherein the image sequence of the surveillance area (7) is recorded with the camera (3) comprising the fisheye lens (4). Procedure according to Claim 16 , characterized in that the image sequence of the monitoring area (7) is recorded during an overflight of the unmanned aerial vehicle, at least two useful images (12 a, b), in particular three useful images (12 a - c), being generated from the recorded image sequence, wherein the useful images (12 a, b) show the person in different shooting perspectives.

Images