CN111898415B - An under-screen fingerprint imaging method, device and electronic equipment - Google Patents

Abstract

The invention discloses an under-screen fingerprint imaging device, which comprises a display screen with a fingerprint identification area, wherein the display screen comprises a contact panel layer and a display screen pixel layer, and a mask layer with a light-transmitting small hole array is arranged below the display screen; the optical image sensor is positioned below the fingerprint identification area of the display screen and is used for acquiring an initial image of the finger fingerprint on the contact panel layer; the system also comprises a processing unit which controls the pixel layer of the display screen to lighten and irradiate the fingerprint contact surface on the contact panel layer, receives the initial image led out by the optical image sensor and calculates the object plane information of the fingerprint contact surface. The invention also discloses an under-screen fingerprint imaging method and electronic equipment. The invention not only can optimize the thickness of the fingerprint detector module, but also can recover the phase distribution of the detection surface to improve the recognition rate of complex scenes such as liquid scenes on the contact surface of the fingerprint.

Description

An under-screen fingerprint imaging method, device and electronic equipment

technical field

The invention relates to optical imaging technology in the field of electronic devices, in particular to an under-screen fingerprint imaging method, device and electronic equipment.

Background technique

Fingerprint identification technology realizes user identification by imaging fingerprint patterns, extracting features and matching them with data in the database. Since fingerprint imaging and recognition are applied to mobile smart devices such as mobile phones, it has brought convenience and great security to the use of mobile phones. In recent years, fingerprint detectors have continued to evolve in principle and method. Its development is mainly reflected in the improvement of resolution and the optimization of detector size. As for the optical fingerprint detection method, compared with the traditional capacitive method in the past, due to its special design, it can be directly imaged under the display screen, which greatly optimizes the structural design of mobile devices and more intuitive fingerprint use logic.

However, there are still some limitations in optical detection methods. For example, the fingerprint identification device disclosed in the patent application document with the publication number CN 111062367 A includes a light-emitting structure, an optical sensing element and at least one optical lens; the display device with the fingerprint identification function provided by the patent application document with the publication number CN 107066976 , including a plurality of identification units arranged in an array, each identification unit is provided with a photosensitive element for photoelectric conversion of incident light, and also includes a collimation filter layer arranged on the light incident side of the photosensitive element, so that The incident light is irradiated to the photosensitive element in parallel. In the existing technology, the use of optical detection will introduce more complex optical structures such as lenses or optical fiber layers or light shielding, resulting in thicker modules, complex structures, and higher costs. This is contrary to the development trend of mobile smart devices becoming thinner and thinner. In addition, since the optical image sensor can only receive light intensity signals, it is difficult to improve the recognition rate for unclear lines caused by liquids attached to the fingerprint area.

Contents of the invention

The invention provides an under-screen fingerprint imaging device, which includes a display screen with a fingerprint identification area, and the display screen includes a touch panel layer distributed from top to bottom, a display screen pixel layer and a light-transmitting small hole array. mask layer;

It also includes an optical image sensor located below the fingerprint identification area of the display screen, used to collect an initial image of the finger fingerprint on the touch panel layer;

It also includes a processing unit, which controls the pixel layer of the display screen to illuminate the fingerprint contact surface on the touch panel layer, receives the initial image derived from the optical image sensor, and calculates the object plane information of the fingerprint contact surface.

The principle of the device is to introduce support domain constraints to the imaging light field by adding a mask layer with a small hole array between the fingerprint contact surface and the optical sensor surface, and compress the size of the solution space, so the image surface receives the light field that introduces constraints information. Then different images are obtained on the image plane by irradiating fingerprints with coherent light sources at different positions, and then the intensity and phase information of the object plane are gradually optimized through algorithm iteration based on these known information. Using this method can well remove the influence of near-field imaging diffraction. At the same time, the iterative algorithm has good parallelism and can quickly optimize high-quality images. This method can restore the phase of the object plane at the same time, which improves the fingerprint recognition rate. Also provides more useful information.

In order to obtain different imaging original images to reconstruct the object plane information, it is necessary to change the position and shape of the illumination source. By time-sharing controlling the self-illuminating screen, more specifically, controlling the touch area display pattern for OLED can freely control the position of the lighting source. Preferably, the processing unit controls the pixel dot matrix of the display screen pixel layer in the fingerprint identification area, controls the lighting of pixels at different positions and uses the optical image sensor to take pictures synchronously to obtain multiple initial images of finger fingerprints. In addition, light sources from different angles can also be used to illuminate the fingerprint area through light guidance, and the illumination light source can be an infrared band light source arranged in an array.

In order to overcome the problems of insufficient pixel brightness and small range of the display screen, preferably, the under-display fingerprint imaging device further includes an auxiliary light source for illuminating the fingerprint contact surface. The auxiliary light source adopts LED or microLED array and is independent of the display screen, which can cooperate with the triggering of the image sensor to control the lighting position in time-sharing.

Preferably, the small holes in the array on the mask layer are light-transmitting regions, and the others are light-impermeable regions. In this application, the light transmittance in the small hole is close to 1, and the light transmittance in other positions is close to 0.

In this application, the aperture of the small hole depends on the pixel density of different screens and the size of the pixel electrode. Preferably, the diameter of the small holes is 5-80 μm, and the distance between two adjacent small holes is larger than the diameter of the holes. In addition, the distance between small holes depends on the distance between screen pixels, and the distance between small hole arrays can be one or more OLED pixels. More preferably, the distance between two adjacent small holes is 20-90 μm.

The mask layer with the small hole array is closely attached to the bottom of the display screen. Preferably, the mask layer is made of chrome-plated film, and the mask layer of the small hole array is coated with a metal film on the back of the pixel light-emitting layer, and the light-transmitting small holes The position is matched with the pixel light-emitting electrode gap, and the optical image sensor includes a pitch of 70-400 μm.

In this application, the principle of the pinhole array used is different from that of some previously proposed pinhole array methods, and the functions of the pinholes are also different. The traditional pinhole array uses the principle of pinhole imaging. Specifically, using the principle of pinhole imaging requires that the opening of the pinhole be as small as possible in structure, usually limited to 5-40 μm, and at the same time, in order to avoid overlapping of sub-images imaged by different pinholes on the image plane, the distance between the pinholes is large enough , generally controlled at 1-1.5mm, the disadvantage of this method is mainly reflected in the fact that the resolution is limited by the PSF of the small hole, so the imaging resolution is limited, and the size of the small hole is too small, which requires high brightness of the lighting source and light guide design. The function of the pinhole array in the present invention is to make the light field in the local area 0, and compress the support domain space in solving the inverse problem so that the algorithm can converge quickly, so there is no need to require that the sub-images in the image space have no overlap, and the pinholes The aperture and spacing can be set according to the pixel electrode arrangement of the display screen that actually uses the fingerprint area, resulting in a simple mechanism design.

In this application, an optical image sensor may include an integrated circuit integrated image detection circuit and the above-mentioned processing unit (ie, an image processing chip).

According to the above-mentioned device, the present invention also provides an under-display fingerprint imaging method, comprising the following steps:

Turn on the pixels at different positions on the pixel layer of the display screen in turn, and illuminate the fingerprints on the touch panel layer;

Collect initial images of finger prints under illumination at different angles through an optical image sensor;

The processing unit calculates the object plane information of the fingerprint by using the initial image.

Preferably, the processing unit obtains the calculated fingerprint surface intensity and phase distribution through a reconstruction and recovery algorithm according to the multiple captured initial images.

The invention can also restore the phase distribution of the fingerprint contact surface. Optical image sensors cannot image phase information. However, for coherent light near-field imaging, the phase can affect the light intensity of the image plane and change with the wavelength or position of the illumination source. The established special imaging system model can calculate the distribution of light intensity and phase information of the fingerprint contact surface consistent with the observed results through iterative optimization.

In addition, the present invention also provides an electronic device, including the above-mentioned under-display fingerprint imaging device.

The invention can not only optimize the thickness of the fingerprint detector module, but also restore the phase distribution of the detection surface and improve the recognition rate of complex scenes such as the scene with liquid on the contact surface of the fingerprint.

Description of drawings

FIG. 1 is a schematic structural view of an under-display lensless imaging device according to an embodiment of the present invention;

FIG. 2 is a schematic block diagram of a fingerprint detection device according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a hole array mask used in an embodiment of the present invention;

Fig. 4 is a schematic diagram of a recovery algorithm according to an embodiment of the present invention.

Detailed ways

In the following description, many specific details are set forth in order to fully understand the present invention, but the present invention can also be implemented in other ways different from those described here, therefore, the present invention is not limited to the specific embodiments disclosed below limit.

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

In the present invention, unless otherwise clearly specified and limited, terms such as "installation", "connection", "connection" and "fixation" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

As shown in FIGS. 1-4 , the under-display fingerprint imaging device of this embodiment includes a display screen 100 with a fingerprint identification area 101 . As shown in Figure 2, it specifically includes a touch panel layer 2, a transparent glass substrate 3, a display screen pixel layer 4, a mask layer 5 with a small hole 301 array, a support layer 6, an optical image sensor 7, an auxiliary light source 8 and image processing. IC. The finger 1 is placed on the touch panel layer 2 .

This embodiment of the device can be divided into three parts: an illumination system for generating different illumination modes, an imaging system and an image restoration system. The functions and cooperation between different components are described in detail below.

In this embodiment, the lighting system is mainly composed of pixel light sources in the pixel layer 3 of the display screen. Specifically, as will be understood by those skilled in the art, a display screen such as an OLED functions as a display screen and a lighting source at the same time. By controlling different patterns displayed on the screen, different lighting modes can be regulated. Specifically, the optical detector can In the corresponding screen area, a 3x3-7x7 pixel point 301 matrix is selected as the illumination source, and different lighting positions are controlled synchronously with the optical image sensor.

In another embodiment, an auxiliary light source 8 may also be used because the pixel illumination mode at a larger angle is insufficient to satisfy a higher signal-to-noise ratio. Specifically, the auxiliary light source 8 may be an infrared LED, because the penetration depth of infrared light is deeper than that of other waveband light sources.

The imaging system is composed of an image sensor 7 and a pixel light source of the pixel layer 3 of the display screen. The positions and different colors of the lighted pixels are controlled to refresh different images on the display screen. Specifically, in order to balance recovery efficiency and quality, the interval of lighted pixels can be adjusted. At the same time, parameters such as exposure time and gain of the camera are adjusted at the imaging control terminal.

As shown in the schematic diagram of the detectable range of fingerprints in FIG. 1 , this embodiment can realize a relatively large fingerprint detection area. Specifically, an ultra-large target surface optical detector can be used, and a CCD/CMOS with a diagonal size of 30-60 mm can be used as a photosensitive device. As mentioned earlier, if the traditional optical imaging method uses a CCD/CMOS with a large target surface, the size of the optical lens it is equipped with will inevitably be very large, which is not advisable for the design of mobile smart terminal equipment. The lensless lens used in this embodiment The imaging method can make full use of the large target image sensor to greatly increase the detectable area of fingerprints, providing a solution for "half-screen fingerprints".

In this embodiment, the imaging system includes a mask layer 5 with an array of small holes 301, the specific details of which will now be described. In the past imaging of similar structures, the pinhole array is based on the principle of pinhole imaging, and its resolution is affected by the size of the pinholes, and the smaller the pinholes, the higher the requirements for the light source of the system.

As shown in FIG. 3 , the small holes 301 arrayed on the mask layer 5 are light-transmitting areas, and the others are light-impermeable areas. The light transmittance in the small holes is close to 1, and the light transmittance in other positions is close to 0. The aperture of the small hole depends on the pixel density of different screens and the size of the pixel point electrode. Specifically, the aperture of the small hole is 5-80 μm. In addition, the distance between the small holes depends on the spacing between the pixels of the screen. The spacing of the small hole array can be It is the distance between one or more OLED pixels, for example, the interval between two adjacent small holes is 20-90 μm.

In another embodiment, an off-screen fingerprint imaging method, implemented according to the above-mentioned off-screen fingerprint imaging device, includes the following steps:

Turn on the pixels at different positions on the pixel layer of the display screen in turn, and illuminate the fingerprints on the touch panel layer;

Collect initial images of finger prints under illumination at different angles through an optical image sensor;

The processing unit calculates the object plane information of the fingerprint by using the initial image.

In this embodiment, after lighting up sequentially, the number of multiple pictures taken in time-sharing can be 9-100.

The processing unit obtains the calculated fingerprint surface intensity and phase distribution through a reconstruction restoration algorithm according to the multiple initial images taken.

In this embodiment, when constructing the imaging model of the entire system, a constraint is added to restore the phase to facilitate more accurate and rapid convergence when solving the inverse problem. Referring to Figure 4, the specific optimization algorithm steps are as follows:

1) Establish system imaging model

Build an imaging model based on the optical properties of the screen and aperture. As shown in Figure 4, the light field is generated by the pixel point light source and the auxiliary light source and propagates to the finger screen contact surface for reflection. The reflected light is modulated by the intensity and phase modulation of the fingerprint contact surface and then freely propagates to the pinhole array surface. Limit to 0, and then spread to the image plane through the gap between the pinhole array and the optical sensor.

2) Gradually restore the intensity and phase of the object plane using the multiple illumination time patterns captured. The initial value of the object surface is given randomly, and its distribution on the image plane is obtained through the light field propagation model. The real part is replaced by the real-shot intensity map and the imaginary part is kept unchanged, that is, the intensity is replaced and the phase is not changed. Then the image plane light field is propagated back to the object plane to obtain updated object plane information, and the above process is repeated with the new object plane information, and so on for several iterations until the algorithm converges to obtain the optimal solution of the object plane.

When a lighting source is turned on, it will be modulated by the object plane to form a light intensity distribution and a phase distribution related to the position of the lighting source. Among them, the phase distribution will have a more significant difference with the position of the light source, and then propagate through the free space to the pinhole mask layer and form a new light intensity and phase distribution through pinhole modulation, and finally reach the optical The sensor captures the distribution of the phase surface, and obtains the light intensity distribution at different lighting positions through the optical sensor. Although the detector only collects the light intensity information, the phase information of the object plane is hidden in it. Using the alternate projection method, the collected light intensity distribution maps corresponding to light sources at different positions are used as input, and the phase information of the object surface is iteratively calculated in combination with the pinhole and imaging model. The iterative process is as follows:

1. First assume a light intensity and phase distribution on the object plane;

2. Calculate the light intensity and phase distribution of the image plane according to the imaging model;

3. Replace the calculated light intensity distribution with the light intensity distribution map collected corresponding to the illumination light at a certain position while keeping the phase distribution unchanged;

4. Then map the updated light intensity and phase distribution on the image plane in step 3 to a new light intensity and phase distribution on the object plane according to the imaging model.

Iterating in this way until the algorithm converges, the light intensity information and phase information of the object surface can be calculated at the same time.

In this embodiment, when there is water on the contact surface of the finger, the phase distribution information is restored, and the imaging quality and recognition rate are improved by using the phase distribution information.

An electronic device includes the above-mentioned under-display fingerprint imaging device. Electronic devices may include various handheld devices with wireless communication functions, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to wireless modems, as well as various forms of user equipment (User Equipment, UE), mobile stations ( Mobile Station, MS), terminal equipment (terminal device), etc.

The above descriptions are only examples of the preferred implementation of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention within.

Claims (7)

1. A fingerprint imaging device under the screen, comprising a display screen with a fingerprint identification area, characterized in that, the display screen includes a touch panel layer and a display screen pixel layer, and an array of light-transmitting apertures is arranged below the display screen mask layer; It also includes an optical image sensor located below the fingerprint identification area of the display screen, used to collect an initial image of the finger fingerprint on the touch panel layer; It also includes a processing unit, which controls the pixel layer of the display screen to illuminate the fingerprint contact surface on the touch panel layer, and receives the initial image derived from the optical image sensor, and calculates the object plane information of the fingerprint contact surface; the processing unit controls the display screen The pixel dot matrix of the pixel layer in the fingerprint identification area controls the lighting of pixels at different positions and uses the optical image sensor to take pictures synchronously to obtain the initial images of multiple finger fingerprints under illumination at different angles, and calculate the fingerprint contact surface according to the initial images. surface information; The diameter of the small holes is 10-80 μm, and the distance between two adjacent small holes is larger than the diameter; the distance between two adjacent small holes is 20-90 μm. 2. The under-display fingerprint imaging device according to claim 1, further comprising an auxiliary light source for illuminating the fingerprint contact surface. 3 . The under-screen fingerprint imaging device according to claim 1 , wherein the small holes arrayed on the mask layer are light-transmitting regions, and the others are light-impermeable regions. 4 . 4 . The under-screen fingerprint imaging device according to claim 1 , wherein the small holes arrayed on the mask layer are light-transmitting regions, and the others are light-impermeable regions. 5. An under-screen fingerprint imaging method, characterized in that it is implemented according to the under-screen fingerprint imaging device according to any one of claims 1-4, comprising the following steps: Turn on the pixels at different positions on the pixel layer of the display screen in turn, and illuminate the fingerprints on the touch panel layer; Collect initial images of finger prints under illumination at different angles through an optical image sensor; The processing unit calculates the object plane information of the fingerprint by using the initial image. 6. The off-screen fingerprint imaging method according to claim 5, wherein the processing unit obtains the calculated fingerprint surface intensity and phase distribution through a reconstruction restoration algorithm based on the multiple initial images taken. 7. An electronic device, characterized in that it comprises the under-display fingerprint imaging device according to any one of claims 1-4.