CZ37555U1 - A contactless multimodal biometric device - Google Patents

Description

Contactless multimodal biometric device

Field of technology

The technical solution relates to a contactless multimodal biometric device for biometric verification and/or biometric identification of persons using a fingerprint and/or facial characteristics. This device is to be used for access controls, border controls and identification/verification of a person by a police officer (mobile version).

Current state of the art

Biometric identification or biometric verification of a person's identity using fingerprints and/or facial characteristics are basically very old methods, based on a very low probability that two people have the same fingerprint and/or facial characteristics. A number of devices and methods are known from the prior art for performing biometric identification/verification using a fingerprint and/or facial characteristic.

Capturing biometric fingerprint samples is possible either (i) contact-wise (i.e. a person places a finger or even the entire palm on a specific surface of a scanning device that takes an image of the finger and/or palm), or (ii) contactless (i.e. without the need for the person was touching parts of the sensing device). A disadvantage of contact methods and devices is the risk of incorrect identification (or verification) due to contamination of the contact surface of the capture device. The second big disadvantage is the possibility of transferring bacteria and viruses from the surface of the device to another user, i.e. the solution is unsanitary. For that reason, contactless methods and devices for biometric identification and/or verification using fingerprints are being developed today. Facial identification/verification devices are inherently contactless by the very nature of the scanning method.

In any case, the captured fingerprint(s) and/or facial image is then processed for comparison with biometric templates stored in the database of registered persons or with a template stored in a biometric identification document.

In the case of contactless biometric identification/verification using fingerprints, a certain set of solutions are known from the state of the art, which mainly deal with obtaining a sufficiently high-quality, sharp image of the finger and/or palm with excellent contrast for subsequent processing and comparison, while the finger and/or palm moves at a certain speed through the sensing space of the capture device.

The applicant's earlier patent application WO2021144171A1 describes a device for capturing a fingerprint while walking and for rapid recognition. With this device, the camera is focused on one area and a photoresistor (or photodiode) is used to initiate the sensing process with a laser beam projected onto the photoresistor through the required area, and after the laser beam is interrupted, one sharp image of a finger or palm is obtained. The disadvantage of the device is that it is not possible to recognize an approaching object and capture it independently of its position in space.

From the file US8340371 (related file EP1987472) a non-contact biodetector is known, which uses the surface of the depth of field of the capturing device for the correct positioning of a person's palm or finger, which is ensured here by the height F above the capturing device. The disadvantage of this solution is mainly the fact that the user has to stop at the biodetector and place his palm or finger in the desired position, which does not require the presence of an operator, but still requires a certain time during which the biodetector is blocked, which reduces the maximum number of verified persons per time unit. The most important difference is that this solution is contact, i.e. touching the device is necessary.

- 1 CZ 37555 U1

US2015/0227774 discloses a non-contact "on-the-go" fingerprint sensor capable of capturing fingerprints on a moving hand of a person and identifying the person during movement around the capturing device or during movement of the hand in a defined sensing area in front of the capturing device. At the same time, an optical gate in the form of a beam interruption sensor is placed in the capture area, whereby the beam interruption sensor (photo-shutter) defines the moment in which the hand passes through the capture area; the hand thereby interrupts the light beam sent through the capture area by the light emitter to the light detector (e.g. photoresistor) on the other side of the capture area. At this point, a capture process is initiated, which is adapted to capture an image focused on a capture area that contains several cameras. As soon as said photo-shutter detects an interruption of the light beam, i.e. that an object (hand) has passed through the capture area, the camera captures an image and, after analyzing the image on the connected computer, decides whether the captured object is a human hand and also detects the location of the fingers on the hand. At least one image of the finger, i.e. a fingerprint image, is then captured by the camera as a high resolution image of at least a portion of any of the detected fingers passing through the capture region. According to this document, a camera that captures fingers (fingerprints) must have a good frame rate, i.e. it must be able to capture a sufficiently high number of individual images per second, which in principle must appropriately correspond to the expected movement speed of the captured hand in the capture area. The frame rate in this solution basically starts at 24 fps (frames per second) and above, while frame rates of 125 fps, 150 fps and even 240 fps are also mentioned. Correspondingly, the fingerprint camera must have a short exposure time. Once the hand-sensing camera detects that a hand is present in the capture area, the finger-sensing camera is triggered even before the fingers of the hand pass through the focal plane in the capture area. The fingerprint camera, which is set to a certain frame rate, will take several pictures of the fingers from that point on. In this way, the finger-scanning camera captures a certain number of images (as the hand moves forward, all fingers pass through the focal plane at different times), while an evaluation algorithm in the attached computer identifies which image is best in focus for each finger. Already at this stage, a large number of high-resolution images must be processed. Individual fingerprints are then extracted from the identified best-focus images and compared to the fingerprint database. The disadvantage of this solution is the need to acquire, process and evaluate a large number of high-resolution images for person identification.

US2014/0253711 discloses an agile non-contact biometric sensor device that includes a sensor monitoring the field of view and further includes an imaging system that captures one or more biometric characteristics in the field of view. The device also includes a tilting pointing device that directs and orients the display system to the position of the object detected by the sensor in the field of view. The disadvantage of this arrangement is the necessity of demanding positioning of the imaging system at a specific position to achieve such a field of view in which the person's palm was mainly detected and used for biometric identification.

US2009/0245591 discloses a contact multi-spectral biometric capture device that focuses on detecting the use of presentation attack tools. This presentation attack detection is based on palm/finger illumination with different wavelengths from the ultraviolet spectrum to the visible radiation to the infrared spectrum. A three-dimensional image is evaluated. The solution in this document describes a layer below the capture surface, so it is expected to be used primarily in touch sensors.

From the document US2016/0070898, an authentication device, a prismatic body for authentication and a method of authentication are known. The solution described in this document is intended for contact fingerprint capture.

The main goal of the presented technical solution is to eliminate or at least minimize the disadvantages of the current state of the art, especially contactless biometric identification/verification of persons.

- 2 CZ 37555 U1

The essence of the technical solution

The disadvantages of the prior art are eliminated by a non-contact multimodal biometric device for biometric verification and/or biometric identification of persons, which according to this technical solution includes:

- control unit,

- a first camera unit which is arranged for sensing the space outside the biometric device or in the open cavity of the biometric device and which includes a first lens and a first image sensor which is connected to the control unit, and

- a first proximity sensor arranged to measure the distance of an object arranged in the space captured by the first camera unit from the first proximity sensor and connected to a control unit for transmitting the measured distance values, the first lens being connected to the control unit for receiving signals for focusing it based on the measured distance values the first proximity sensor.

Preferably, said first lens is a liquid lens.

It is also advantageous if the non-contact multimodal biometric device includes a first illumination unit, preferably in the form of a system of point lights, located with equal angular spacing around said first lens.

In a particularly advantageous embodiment, the non-contact multimodal biometric device includes a second camera unit, which includes a second image sensor and a second lens, connected to the control unit. The second lens is preferably a liquid lens or a lens with a motor refocusing mechanism.

In some advantageous embodiments, the second camera unit is arranged to sense the space outside the biometric device, wherein the biometric device includes a second proximity sensor arranged to measure the distance of an object arranged in the space sensed by the second camera unit from the second proximity sensor and connected to the control unit for transmitting measured distance values, wherein the second lens is connected to the control unit for receiving signals for focusing it based on the distance values measured by the second proximity sensor.

In an advantageous embodiment, the contactless multimodal biometric device has a front side on which the second camera unit is arranged and the first camera unit below it.

In another advantageous embodiment, the non-contact multimodal biometric device has a front side, on which the second camera unit is arranged, and below it, on the front side, there is an entrance to said open cavity, in which a mirror is arranged to reflect the area scanned by the first camera unit in the open cavity.

In a particularly advantageous embodiment, the contactless multimodal biometric device includes a reader for reading biometric identification documents, which is connected to the control unit.

The contactless multimodal biometric device preferably further includes a display for displaying the identification/verification results, connected to the control unit.

- 3 CZ 37555 U1

The contactless multimodal biometric device may also include a 3D camera that is arranged to sense the space outside the biometric device and that is connected to the control unit.

Clarification of drawings

The technical solution will be further described using exemplary embodiments of a non-contact multimodal biometric device, while Fig. 1 shows a diagram of the first exemplary embodiment, Fig. 2 shows a diagram of a second exemplary embodiment, Fig. 3A - 3D show schematic views of four types of front side configuration of a contactless multimodal biometric device according to this technical solution.

Examples of implementing a technical solution

An exemplary embodiment of the non-contact multimodal biometric device 1 shown in Fig. 1 includes a control unit 20 which is designed to receive and process users' biometric data and to control the connected components listed below.

Furthermore, the biometric device 1 includes a first camera unit 4, which includes a first image sensor 4a and a first lens 4b. The camera unit 4 is preferably located in the lower part of the device below the display 2. The first lens 4b is preferably a liquid lens, but it can also be a fixed lens, or lens with motor type refocusing. A liquid lens is a type of lens that allows the focal length to change depending on the distance of the subject from the lens. In order to take a sharp image of an object, it is not necessary for the object to be in one specific position relative to the camera unit 4.

Furthermore, the biometric device 1 is equipped with a first proximity sensor 6, which is designed to detect the distance of the user's palm from the first camera unit 4 and to transmit the measured distance to the control unit 20. Based on this distance, the first camera unit 4 or to the first lens 4b, a signal is issued, on the basis of which automatic focus will occur on the palm, or user's fingers and taking a picture of at least one finger.

The first proximity sensor 6 and the first camera unit 4, respectively. the first lens 4b, in this embodiment are preferably located in one plane so that the measured distance is the actual distance of the hand from the first lens 4b.

In other embodiments not shown, it is possible to place the proximity sensor 6 in a different plane than the first lens 4b is placed. In that case, the control unit 20 is adapted to calculate the difference, i.e. the offset of the proximity sensor 6 with respect to the first lens 4b.

In addition, a first lighting unit 5 is located near the first camera unit 4, e.g. in the form of a system of point lights, distributed in a circle (or according to an embodiment not shown in another geometric shape) with a mutual angular distance around the first lens 4b, while said first lighting unit 5 it is designed to illuminate the user's hand so that a sufficiently bright and clear picture can be taken, or a set of pictures with different depth of field.

When using the biometric device 1, it may be advantageous to instruct the user to first point at the first camera unit 4 his open palm approximately perpendicular to the optical axis of the first camera unit 4, thereby obtaining a first image or a first set of images with different depth of field , and to subsequently rotate the palm around an axis corresponding to the distal direction of the palm, i.e. essentially around the axis of the outstretched middle finger, to one side, obtaining a second image, or a second set of images with different depth of field, and the other side, obtaining a third

- 4 CZ 37555 U1 picture, possibly a third set of pictures with different depth of field. Individual images can be connected to each other to obtain a panoramic image of the surface of the finger / fingers / palm, which corresponds to a rolled fingerprint, or spatial representation of the finger (3D).

The described first embodiment also includes a second camera unit 3, which includes a second image sensor 3a and a second lens 3b. The second lens 3b is preferably a liquid lens similar to the first lens 4b, but it can also be a fixed lens, or lens with motor type refocusing. The design of the second camera unit 3 is therefore analogous to the design of the first camera unit 4.

Furthermore, the biometric device 1 is equipped with a second proximity sensor 10, which is designed to detect the distance of the user's face from the second camera unit 3 and to transmit the measured distance to the control unit 20. Based on this distance, the second camera unit 3 or to the second lens 3b, a signal is issued, on the basis of which the face will be automatically focused and the image of the face will be taken.

The second proximity sensor 10 and the second camera unit 3, respectively. the second lens 3b, are again advantageously located in one plane in this embodiment.

In other non-illustrated embodiments, it is possible to place the proximity sensor 10 in a different plane than the second lens 4a is placed. In that case, the control unit 20 is adapted to calculate the difference, i.e. the offset of the proximity sensor 10 with respect to the second lens 4a.

The second camera unit 3 is intended for facial identification/verification and is therefore placed higher than the first camera unit 4, e.g. between the first camera unit 4 and the display 2.

A second lighting unit 9 can then be placed around the second camera unit 3, e.g. in the form of a system of point lights, distributed in a circle (or according to an embodiment not shown in a different geometric shape, or the second lighting unit 9 can be completely omitted) with a mutual angular distance around the second lenses 3b.

The second camera unit 3 with the corresponding second proximity sensor 10 and the second lighting unit 9 is optional and not necessarily part of the biometric device 1.

The first version also includes a reader for 7 biometric identification documents, especially radio frequency data from the chip on the document. For this purpose, the reader of 7 biometric identification documents includes an RFID/NFC reader. The first camera unit 4 is intended for scanning the MRZ (machine-readable area) of the document. The reader 7 of biometric identification documents is optional and is not necessarily part of the biometric device 1.

The above-mentioned display 2 can be either touch or non-touch. If the display 2 is non-contact, the device 1 additionally includes an assembly of 11 additional proximity sensors. The assembly of 11 additional proximity sensors is then used to ensure the detection of the presence of a finger in a specific space in front of the display 2, while enabling the control of the display 2 from a distance of several centimeters to lower tens of centimeters.

Furthermore, the first exemplary embodiment of the biometric device 1 includes a 3D camera 8, preferably arranged in the upper part of the device 1, the purpose of which is the precise positioning of the hand to capture a fingerprint and/or face for facial identification/verification, including the detection of a suitable position and distance. The 3D camera can also be advantageously used for biometric recognition in 3D, especially the face/head and spatial geometry of the hand. The 3D camera 8 is optional and not necessarily included in the device 1.

Individual components of biometric device 1, namely 3D camera 8, display 2, sensors 6, 10 proximity, assembly 11 of additional proximity sensors, lighting units 5, 9, image sensors

- 5 CZ 37555 U1

3a, 4a and the lenses 3b, 4b of the camera units 3, 4 are directly or indirectly signal-connected to the control unit 20.

In the first exemplary embodiment, all camera units 3, 4, 8 are arranged on the front side of the biometric device 1 and directed in essentially the same direction forward, or at least the first camera 4 and the second camera 3 are mounted such that their optical axes are substantially parallel.

The second embodiment, shown in Fig. 2 differs from the first embodiment in particular in that it additionally includes a mirror 12 and the first camera 4 is facing the mirror so as to capture the image reflected from the mirror 12 which is arranged inside the biometric device 1 and is obliquely facing the opening in the front of the biometric device 1, so that the first camera 4 reflects the image of the space in front of the biometric device 1. This second embodiment is suitable, for example, if the focal plane of the first camera 4 is too far from the device 1, and therefore a mirror 12 is used to extend the optical path. In the illustrated second embodiment, the mirror 12 is positioned so that the image of the hand (finger) is reflected in the biometric device 1 at an angle of 90° upwards.

The method of operation of device 1 is basically the same for both versions. First, the user places his hand in the space in front of the first camera unit 4 so that the palm faces the device 1 or to the first camera unit 4. The first proximity sensor 6 subsequently detects the distance of the hand from the first lens 4b of the first camera unit 4.

The detected distance is then transmitted by the first proximity sensor 6 to the control unit 20. Based on the detected distance, the control unit 20 then controls the first lens 4b so that it selects a suitable focal length for taking a picture. Subsequently, the first camera unit 4 takes a picture of the hand, or finger, under simultaneous illumination by the first illumination unit 5. The image is then sent to the control unit 20, which performs image evaluation and identification/verification. This consists in the extraction of areas of interest in the image of the finger, while the fingerprints are then, for example, compared with templates stored in the database of registered users, or to the data obtained from the biometric identification document.

The result of the identification/verification is shown on display 2, i.e. for example, if the identification is successful, the identity will be displayed, while in the opposite case, the information about the failed identification will be displayed. The result is also sent to device 1 via the API, for example to open a gate.

For the second camera unit 3, the procedure proceeds analogously, while a picture of the face is also taken, under simultaneous illumination by the second lighting unit 9, and thus two biometric data - fingerprint and face images - are evaluated in the control unit 20.

If the device 1 is also equipped with a reader 7 of biometric identification documents, the user places the biometric identification document in front of the first camera unit 4, similar to a hand. The first camera unit 4 then takes a picture of the front side of the biometric identification document. From this image, a machine-readable region containing the key to the publicly accessible data in the chip (i.e. the user's personal data and facial image) is recognized using optical character recognition.

Subsequently, the user attaches the document to the document reader 7, equipped with an RFID/NFC reader, which sends the read data to the control unit 20, which compares this data from the biometric identification document with the biometric data obtained from the images taken in the above-mentioned manner.

If the device 1 is equipped with a 3D camera 8, said 3D camera 8 creates a three-dimensional image of the user's face and/or his hand, and this image is subsequently sent to the control unit 20. The control unit 20 similarly to the images taken by the first camera unit 4 and /or the second camera unit 3, evaluates the image, i.e. user identification/verification.

- 6 CZ 37555 U1

A non-contact multimodal biometric device in accordance with the presented technical solution can use the multispectral principle of detection of presentation attacks on fingerprints using palm/finger illumination with different wavelengths from the ultraviolet spectrum through visible radiation to the infrared spectrum, or by sensing the bloodstream, visible in the infrared spectrum.

Giant. 3A to 3D show various examples of the configuration of individual components on a device that differ in the presence or location of the second camera unit 3. Variant from Fig. 3A does not include the second camera unit 3 - in this variant, facial identification/verification is therefore not included. Giant. 3B shows a front view of the device according to the diagram in Fig. 1. The variant shown in Fig. 3C is based on the variant from Fig. 3B with the difference that the second camera unit 3 is not located above the first camera unit 4, but next to it. The variant from Fig. 3D is also based on the variant from Fig. 3B with the difference that the second camera unit 4 is not located above the first camera unit 4, but below it. In the variant shown in Fig. 3D is thus the first camera unit 4 intended for facial identification, while the second camera unit 3 is intended for capturing fingerprints and points towards the ground.

The first image sensor 4a and/or the second image sensor 3a are preferably of the CCD or CMOS type.

Although particularly preferred exemplary embodiments have been described, it is apparent that one skilled in the art will readily find other possible alternatives to these embodiments. Therefore, the scope of protection is not limited to these exemplary embodiments, but rather is given by the definition of the appended claims for protection.

Claims (11)

1. Contactless multimodal biometric device (1) for biometric verification and/or biometric identification of persons, characterized in that it includes: - control unit (20), - a first camera unit (4) which is arranged for sensing the space outside the biometric device (1) or in the open cavity of the biometric device (1) and which includes a first lens (4b) and a first image sensor (4a) of which the first lens ( 4b) upstream and which is connected to the control unit (20), a - the first proximity sensor (6) arranged to measure the distance of the object arranged in the space scanned by the first camera unit (4) from the first proximity sensor (6) and connected to the control unit (20) for transmitting the measured distance values, the first lens (4b) being connected to the control unit (20) for receiving signals for its focusing based on the distance values measured by the first proximity sensor (6). 2. Contactless multimodal biometric device (1) according to claim 1, characterized in that the first lens (4b) is a liquid lens or a lens with a motor refocusing mechanism. 3. A non-contact multimodal biometric device (1) according to claim 1 or 2, characterized in that it includes a first lighting unit (5), preferably in the form of a system of point lights, located with equal mutual angular spacing around the first lens (4b). 4. Contactless multimodal biometric device (1) according to any one of claims 1 to 3, characterized in that it includes a second camera unit (3) that includes a second image sensor (3a) and a second lens (3b) connected to a control unit ( 20) and upstream of the second image sensor (3a). 5. Contactless multimodal biometric device (1) according to claim 4, characterized in that the second lens (3b) is a liquid lens or a lens with a motorized refocusing mechanism. 6. Contactless multimodal biometric device (1) according to claim 4 or 5, characterized in that the second camera unit (3) is arranged for sensing the space outside the biometric device (1), wherein the biometric device (1) includes a second sensor (10) approach arranged for measuring the distance of an object arranged in the space sensed by the second camera unit (4) from the second proximity sensor (10) and connected to the control unit (20) for transmitting the measured distance values, the second lens (3b) being connected to the control unit (20 ) for receiving signals for its focusing based on the distance values measured by the second proximity sensor (10). 7. Contactless multimodal biometric device (1) according to any one of claims 4 to 6, characterized in that it has a front side on which the second camera unit (3) and the first camera unit (4) are arranged below it. 8. Contactless multimodal biometric device (1) according to any one of claims 4 to 6, characterized in that it has a front side on which the second camera unit (3) is arranged and below it is an entrance to an open cavity in which a mirror (12) is arranged to reflect the space captured by the first camera unit (4) in the open cavity. - 8 CZ 37555 U1 9. Contactless multimodal biometric device (1) according to any one of claims 1 to 8, characterized in that it includes a reader (7) for reading biometric identification documents, which is connected to the control unit (20). 10. Contactless multimodal biometric device (1) according to any one of claims 1 to 9, 5, characterized in that it further includes a display (2) for displaying identification/verification results, connected to the control unit (20). 11. Contactless multimodal biometric device (1) according to any one of claims 1 to 10, characterized in that it includes a 3D camera (8) which is arranged for sensing the space outside the biometric device (1) and which is connected to the control unit (20) ).