CN114202778A - Method and system for estimating three-dimensional gesture of finger by planar fingerprint - Google Patents

Abstract

The present application proposes a method and system for estimating the three-dimensional posture of a finger from a planar fingerprint, which belongs to the technical field of human-computer interaction. The plane posture estimation model determines the plane fingerprint posture corresponding to the plane fingerprint image; according to the plane fingerprint posture, the complete three-dimensional posture of the finger matching the object to be measured is determined by parameter learning or statistical modeling. The present application adopting the above solution solves the current problem by pre-training an accurate plane posture estimation model, and determining the complete three-dimensional posture of the finger matching the plane fingerprint posture of the object to be measured through parameter learning or statistical modeling based on this. Fingerprint three-dimensional pose estimation technology is not ideal in terms of function and convenience.

Description

A method and system for estimating three-dimensional gesture of finger from planar fingerprint

technical field

The present application relates to the technical field of human-computer interaction, and in particular, to a method and system for estimating a three-dimensional gesture of a finger from a planar fingerprint.

Background technique

In the field of finger three-dimensional pose estimation of fingerprints, the existing technologies are divided into the following types according to the mode (collection technology) of the input fingerprint:

1) Based on the gesture estimation of capacitive sensing fingerprints, capacitive sensing fingerprints use the difference imaging of the sensor capacitance between the fingerprint touching the screen and the non-contact situation. Because of its relatively simple principle and widely used in touch screen devices, (Xiao et al.) from 42 features were extracted from capacitive sensing fingerprints, and a Gaussian regression model was trained to estimate pitch and yaw angles, and achieved good results on smartphones and smartwatches, but they could not predict the roll angle, so the obtained The fingerprint pose is incomplete, and due to the low resolution of such sensors, the fingerprint ridge and valley lines cannot be distinguished, and the accuracy of pose estimation is limited;

2) Integrate the pose estimation of other modal information, obtain the depth information of the fingerprint by using the externally installed depth camera, and integrate the prior knowledge to constrain the range of the pitch angle to be between 0 and 90 degrees. Set a camera, and calculate the pitch angle and deflection angle by detecting the change of the light intensity of the fingernail. This scheme obtains the complete three-dimensional posture of the fingerprint by using the modal information other than the planar fingerprint, but it needs to introduce additional hardware to provide practical Application scenarios bring obstacles;

3) Pose estimation based on plane-pressed fingerprints. So far, there is only one solution to estimate the three-dimensional posture of fingers directly based on plane-pressed fingerprints. Angle, in the test phase, by finding the library fingerprint that is most similar to the input fingerprint, the three-dimensional attitude of the current input fingerprint is inferred, including the pitch angle, roll angle and yaw angle, but this solution needs to pre-establish the fingerprint database of each angle of each finger and the true value of its angle, which is a tedious and tedious process, and requires the use of technologies (such as optical tracking technology, inertial navigation technology) to obtain the true value of the finger, which brings difficulties to the actual implementation of this technical solution.

To sum up, the existing technical solutions for 3D finger pose estimation of fingerprints either cannot accurately estimate the complete fingerprint pose, or require additional sensor equipment to assist, or require users to perform complex fingerprint registration links, both in terms of functionality and convenience. not ideal. Therefore, there is an urgent need for a technical solution for directly estimating the three-dimensional pose of a fingerprint based on a planar fingerprint image, which will greatly facilitate the interactive application of fingerprints.

SUMMARY OF THE INVENTION

The present application aims to solve one of the technical problems in the related art at least to a certain extent.

Therefore, the first purpose of the present application is to propose a method for estimating the three-dimensional posture of a finger from a planar fingerprint, so as to solve the technical problem that the current fingerprint three-dimensional posture estimation technology is not ideal in terms of function and convenience.

The second purpose of this application is to propose a system for estimating the three-dimensional gesture of a finger from a planar fingerprint.

In order to achieve the above purpose, a method for estimating the three-dimensional posture of a finger from a planar fingerprint proposed by the embodiment of the first aspect of the present application includes:

Collect the planar fingerprint image of the object to be tested;

Use the pre-trained plane pose estimation model to determine the plane fingerprint pose corresponding to the plane fingerprint image;

According to the plane fingerprint posture, the complete three-dimensional posture of the finger that matches the object to be measured is determined by means of parameter learning or statistical modeling.

Optionally, in an embodiment of the present application, the planar fingerprint pose includes a position and a deflection angle of the planar fingerprint image;

The complete three-dimensional gesture of the finger that matches the object to be measured is determined by parameter learning or statistical modeling, including:

Determine the three-dimensional pose mapping function corresponding to the planar fingerprint image according to the parameter learning or statistical modeling;

Determine the roll angle and the pitch angle of the planar fingerprint image according to the three-dimensional attitude mapping function and the position of the planar fingerprint image;

A complete three-dimensional pose matching the object to be measured is determined according to the yaw angle, roll angle and pitch angle of the planar fingerprint image.

Optionally, in an embodiment of the present application, the determining of the three-dimensional pose mapping function corresponding to the planar fingerprint image according to the parameter learning or statistical modeling includes:

Build a first planar fingerprint image database by means of machine learning, wherein the first planar fingerprint image database includes feature descriptors;

determining a plurality of planar fingerprint images matching the feature descriptors from the first planar fingerprint image database, and determining a first mapping parameter corresponding to each of the multiple planar fingerprint images;

Fusing a plurality of the first mapping parameters to obtain second mapping parameters, and determining a three-dimensional attitude mapping function corresponding to the planar fingerprint image according to the second mapping parameters, wherein, by calculating a plurality of the first mappings The average value of the parameters, or the weighted average value, or the average value of the preset number of the most matching first mapping parameters to obtain the second mapping parameter;

Alternatively, at least one probability model describing the finger is determined through the simulation system and the actually collected data set, and the three-dimensional attitude mapping function corresponding to the current planar fingerprint image is determined according to the at least one probability model describing the finger.

Optionally, in an embodiment of the present application, before using a pre-trained plane pose estimation model to determine the plane fingerprint pose corresponding to the plane fingerprint image, the method further includes:

Collecting a plane fingerprint image for training and three-dimensional angle data for training corresponding to the plane fingerprint image for training;

The plane pose estimation model is trained according to the training plane fingerprint image and the training three-dimensional angle data to obtain the pre-trained plane pose estimation model.

Optionally, in an embodiment of the present application, the acquisition of the training plane fingerprint image and the training three-dimensional angle data corresponding to the training plane fingerprint image includes:

Obtain the training plane fingerprint image by using a plane fingerprint collector;

Use two three-axis gyroscopes to obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training, or use an optical tracking measurement system to obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training, or use the The simulation system and the data generator obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training; wherein,

The three-dimensional angle data for training includes the true value of the pitch angle, the true value of the yaw angle and the true value of the roll angle. Not more than 90° and not less than -90°, the range of the true value of the roll angle is not more than 75° and not less than -75°.

Optionally, in an embodiment of the present application, after the acquisition of the training plane fingerprint image and the training three-dimensional angle data corresponding to the training plane fingerprint image, the method further includes:

storing the planar fingerprint image for training in a second planar fingerprint image database;

Acquire a plurality of planar fingerprint images for training from the second planar fingerprint image database according to preset conditions, and a feature descriptor and a first mapping parameter with each of the multiple planar fingerprint images for training;

The plurality of training plane fingerprint images, the first mapping parameter and the feature descriptors of the plurality of training plane fingerprint images are respectively stored in the first plane fingerprint image database.

Optionally, in an embodiment of the present application, the training of the plane pose estimation model according to the plane fingerprint image for training and the three-dimensional angle data for training includes:

The first mapping parameters of each training plane fingerprint image in the first plane fingerprint image database are obtained by means of function fitting, and the first mapping parameters include pitch angle mapping parameters and roll angle mapping parameters.

Optionally, in an embodiment of the present application,

Determine the pitch angle mapping parameter of each training plane fingerprint image in the first plane fingerprint image database by the following formula:

f _pitch (x,y)=b ₁ x ² +b ₂ y ² +b ₃ x+b ₄ y+b ₅ ln(x)+b ₆ ln(y)+b ₇

为平面指纹图像对应的俯仰角映射参数；Among them, (x, y) is the position of the fingerprint, f _pitch (x, y) is the true value of the pitch angle of the fingerprint,

is the pitch angle mapping parameter corresponding to the planar fingerprint image;

Determine the rolling angle mapping parameter of each training plane fingerprint image in the first plane fingerprint image database by the following formula:

f _roll (x,y)=a ₁ x ² +a ₂ y ² +a ₃ x+a ₄ y+a ₅ ln(x)+a ₆ ln(y)+a ₇

为平面指纹图像对应的滚动角映射参数。Among them, (x, y) is the position of the fingerprint, f _roll (x, y) is the true value of the roll angle of the fingerprint,

It is the rolling angle mapping parameter corresponding to the planar fingerprint image.

Optionally, in an embodiment of the present application, the plane pose estimation model is trained by the following model:

L _all =L _pos +λL _yaw

L _yaw = ‖θ-θ′‖ ₂

表示指纹的位置的真值，

表示平面姿态估计模型对指纹的位置的预测值，

表示指纹的偏转角的真值，

表示平面姿态估计模型对指纹的偏转角的预测值。Among them, L _all represents the loss function value, L _pos represents the loss function value of position prediction, L _yaw represents the loss function value of the yaw angle, λ is the weighting coefficient,

the truth value representing the location of the fingerprint,

represents the predicted value of the position of the fingerprint by the plane pose estimation model,

represents the true value of the deflection angle of the fingerprint,

Represents the predicted value of the deflection angle of the fingerprint by the plane pose estimation model.

To sum up, the method proposed by the embodiment of the first aspect of the present application collects the planar fingerprint image of the object to be tested; uses a pre-trained planar pose estimation model to determine the planar fingerprint pose corresponding to the planar fingerprint image; Gesture, the complete three-dimensional gesture of the finger that matches the object to be measured is determined by means of parameter learning or statistical modeling. In the present application, by pre-training an accurate plane posture estimation model, and based on it, the complete three-dimensional posture of the finger matching the plane fingerprint posture of the object to be measured is determined by mapping or fitting, so as to solve the problem of the three-dimensional finger of the current fingerprint. The technical problem that pose estimation techniques are not ideal in terms of functionality and convenience.

In order to achieve the above purpose, a system for estimating the three-dimensional posture of a finger from a planar fingerprint proposed in the second aspect of the present application includes:

The acquisition module is used to acquire the planar fingerprint image of the object to be tested;

a plane pose estimation module for determining a plane fingerprint pose corresponding to the plane fingerprint image by using a pre-trained plane pose estimation model;

The determining module is configured to determine the complete three-dimensional gesture of the finger matched with the object to be measured by means of parameter learning or statistical modeling according to the gesture of the plane fingerprint.

To sum up, the system proposed in the second aspect of the present application collects the planar fingerprint image of the object to be measured through the acquisition module; the planar pose estimation module uses the pre-trained planar pose estimation model to determine the planar fingerprint pose corresponding to the planar fingerprint image. ; The determining module determines the complete three-dimensional gesture of the finger matched with the object to be measured by means of parameter learning or statistical modeling according to the plane fingerprint gesture. In the present application, by pre-training an accurate plane posture estimation model, and based on it, the complete three-dimensional posture of the finger matching the plane fingerprint posture of the object to be measured is determined by mapping or fitting, so as to solve the problem of the three-dimensional finger of the current fingerprint. The technical problem that pose estimation techniques are not ideal in terms of functionality and convenience.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic diagram of three posture angles when a finger is pressed against a screen provided by an embodiment of the present application;

FIG. 2 is a flowchart of a method for estimating a three-dimensional gesture of a finger from a planar fingerprint according to an embodiment of the present application;

3 is a schematic diagram of data collection provided by an embodiment of the present application;

4 is a schematic diagram of a three-dimensional gesture fitting of a finger provided by an embodiment of the present application;

5 is a schematic structural diagram of a deep neural network provided by an embodiment of the present application;

6 is a schematic flowchart of a method for estimating a three-dimensional gesture of a finger from a planar fingerprint according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a system for estimating a three-dimensional gesture of a finger from a planar fingerprint according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, but should not be construed as a limitation on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents falling within the spirit and scope of the appended claims.

In the field of human-computer interaction, touch screen input is favored by a large number of smart mobile devices as a simple and fast interaction method. It is no exaggeration to say that the popularization of smart devices, in addition to the development of technologies such as chip manufacturing and mobile phone systems, is also inseparable from the iterative update of the interactive method of touch screen. Traditional smart devices can sense the user's finger pressing area and give corresponding feedback. For example, in the case of performing operations such as image enlargement, rotation, and zooming, the user needs to use at least two fingers to successfully complete the above operations. If the angle of the finger when touching the touch screen can be accurately estimated as additional input information, it will greatly improve the user experience, simplify the logic of some operations, broaden the application scenarios of fingerprint input, and bring more interesting interactive forms. And if the three-dimensional pose can be estimated directly from the planar fingerprint, it will not bring additional hardware update burden to smart devices such as mobile phones.

The three attitude angles when the finger is pressing the screen are shown in Figure 1. Based on the three attitude angles of roll angle, pitch angle, and yaw angle (offset angle), the user can be asked to tilt toward a specific attitude, and by judging the user feedback Whether the action of the fingerprint recognition meets a certain angle threshold to determine whether the current fingerprint is a fake fingerprint, based on this, the smart device system developer can enhance the security and reliability of the fingerprint recognition system; at the same time, based on the existing interactive information Three additional gesture angles of the fingers are provided to create more combined gestures, enrich the touch screen information input, increase the interaction interest, and make up for the gap between touch screen devices and traditional physical typing devices while maintaining the advantages of simplicity and ease of use.

Example 1

FIG. 2 is a flowchart of a method for estimating a three-dimensional gesture of a finger from a planar fingerprint according to an embodiment of the present application.

As shown in FIG. 2 , a method for estimating a three-dimensional gesture of a finger from a planar fingerprint provided by an embodiment of the present application includes the following steps:

Step 210, collecting a planar fingerprint image of the object to be tested;

Step 220, using the pre-trained plane posture estimation model to determine the plane fingerprint posture corresponding to the plane fingerprint image;

Step 230 , according to the plane fingerprint posture, determine the complete three-dimensional posture of the finger that matches the object to be measured by means of parameter learning or statistical modeling.

In this embodiment of the present application, the planar fingerprint pose includes the position and deflection angle of the planar fingerprint image;

Determine the complete three-dimensional gesture of the finger that matches the object to be tested by means of parameter learning or statistical modeling, including:

Determine the three-dimensional pose mapping function corresponding to the planar fingerprint image according to parameter learning or statistical modeling;

Determine the roll angle and pitch angle of the plane fingerprint image according to the three-dimensional attitude mapping function and the position of the plane fingerprint image;

According to the yaw angle, roll angle and pitch angle of the planar fingerprint image, the complete three-dimensional pose matching the object to be measured is determined.

In the embodiment of the present application, the three-dimensional attitude mapping function corresponding to the planar fingerprint image is determined according to parameter learning or statistical modeling, including:

Build a first planar fingerprint image database by means of machine learning, wherein the first planar fingerprint image database includes feature descriptors;

determining a plurality of planar fingerprint images matching the feature descriptors from the first planar fingerprint image database, and determining a first mapping parameter corresponding to each of the multiple planar fingerprint images;

A plurality of first mapping parameters are fused to obtain a second mapping parameter, and a three-dimensional attitude mapping function corresponding to the planar fingerprint image is determined according to the second mapping parameter. value, or obtain the second mapping parameter by taking the average value of the preset number of the most matching first mapping parameters;

Alternatively, at least one probability model describing the finger is determined through the simulation system and the actually collected data set, and the three-dimensional attitude mapping function corresponding to the current planar fingerprint image is determined according to the at least one probability model describing the finger.

Specifically, when the three-dimensional attitude mapping function corresponding to the planar fingerprint image is determined according to statistical modeling, at least one probability model describing the finger is determined through the simulation system and the actually collected data set, and several possible spatial transformations are applied to the description The probability model of the finger, compare the position of the projected planar fingerprint image and the planar fingerprint posture, take the spatial transformation corresponding to the closest probability model describing the finger, and calculate the three-dimensional corresponding to the current planar fingerprint image according to the closest probability model describing the finger. Attitude mapping function.

In the embodiment of the present application, before using the pre-trained plane posture estimation model to determine the plane fingerprint posture corresponding to the plane fingerprint image, the method further includes:

Collecting a plane fingerprint image for training and three-dimensional angle data for training corresponding to the plane fingerprint image for training;

The plane pose estimation model is trained according to the plane fingerprint image for training and the 3D angle data for training to obtain a pre-trained plane pose estimation model.

In the embodiment of the present application, collecting the planar fingerprint image for training and the three-dimensional angle data for training corresponding to the planar fingerprint image for training, including:

Acquire a plane fingerprint image for training through a plane fingerprint collector;

Use two three-axis gyroscopes to obtain the training 3D angle data corresponding to the training plane fingerprint image, or use the optical tracking measurement system to obtain the training 3D angle data corresponding to the training plane fingerprint image, or use the simulation system and data generation The device obtains the three-dimensional angle data for training corresponding to the planar fingerprint image for training; wherein,

The three-dimensional angle data for training includes the true value of the pitch angle, the true value of the yaw angle and the true value of the roll angle. Less than -90°, the range of the true value of the roll angle is not greater than 75° and not less than -75°.

Specifically, the accuracy of the plane pose estimation model depends on the richness of the collected samples, so data collection is a crucial part of this application, and the following three data collection schemes are specifically adopted:

The first data acquisition scheme uses a plane fingerprint collector to collect plane fingerprint images and two three-axis gyroscopes to read three-dimensional angle data. The data acquisition is shown in Figure 3. When collecting data, the two three-axis gyroscopes are respectively Fixed on the fingerprint sensor and the finger to be collected, the collector rolls the finger with the gyroscope on the fingerprint sensor from one side to the other, and controls the fingerprint collector through the program to collect several planar fingerprint images at a frequency of 50Hz. The three-axis gyroscope Then the angle is read out synchronously, and the three-dimensional angle data corresponding to the planar fingerprint image is obtained by the difference between the values of the two three-axis gyroscopes;

The second data acquisition scheme uses an optical tracking measurement system combined with a fingerprint acquisition instrument to simultaneously acquire three-dimensional angle data and corresponding planar fingerprint images.

The third data collection scheme is to construct a three-dimensional fingerprint database in advance, and use the three-dimensional fingerprint database to synthesize the planar fingerprint image under any three-dimensional angle data.

Further, in order to determine that the number of samples is sufficiently abundant, the collected fingerprint sequences are not less than 500, the collected planar fingerprint images are not less than 40,000, and the three-dimensional angle data corresponding to each planar fingerprint image is recorded.

Specifically, data collection only needs to be performed once, and the method for estimating the three-dimensional posture of a finger from a planar fingerprint provided in this embodiment can be applied to any finger of any person. Therefore, the application does not require finger registration to estimate a finger from a planar fingerprint. 3D pose, which is the essential difference from the method proposed by (Holz and Baudisch).

In the embodiment of the present application, after collecting the plane fingerprint image for training and the three-dimensional angle data for training corresponding to the plane fingerprint image for training, the method further includes:

storing the plane fingerprint image for training in the second plane fingerprint image database;

Obtain a plurality of training plane fingerprint images from the second plane fingerprint image database according to preset conditions, and the feature descriptor and the first mapping parameter with each of the plurality of training plane fingerprint images;

The plurality of training plane fingerprint images, the first mapping parameters and the feature descriptors of the plurality of training plane fingerprint images are respectively stored in the first plane fingerprint image database.

In the embodiment of the present application, the plane pose estimation model is trained according to the plane fingerprint image for training and the three-dimensional angle data for training, including:

The first mapping parameters of each training plane fingerprint image in the first plane fingerprint image database are obtained by means of function fitting, and the first mapping parameters include pitch angle mapping parameters and roll angle mapping parameters.

Specifically, the three-dimensional gesture fitting of the finger is shown in Figure 4, wherein, the contact fingerprint image first obtains its relative position in the rolling fingerprint through the fingerprint registration algorithm, that is, the plane attitude estimation model, and then uses the second plane fingerprint image database, namely, the relative position of the fingerprint image. The three-dimensional surface of the finger is obtained by fitting several fingerprint images of known poses in the training database, and finally the three-dimensional angle of the finger is obtained.

In the embodiment of the present application, the pitch angle mapping parameter of each training plane fingerprint image in the first plane fingerprint image database is determined by the following formula:

f _pitch (x,y)=b ₁ x ² +b ₂ y ² +b ₃ x+b ₄ y+b ₅ ln(x)+b ₆ ln(y)+b ₇

为平面指纹图像对应的俯仰角映射参数；Among them, (x, y) is the position of the fingerprint, f _pitch (x, y) is the true value of the pitch angle of the fingerprint,

is the pitch angle mapping parameter corresponding to the planar fingerprint image;

Determine the rolling angle mapping parameters of each training plane fingerprint image in the first plane fingerprint image database by the following formula:

f _roll (x,y)=a ₁ x ² +a ₂ y ² +a ₃ x+a ₄ y+a ₅ ln(x)+a ₆ ln(y)+a ₇

为平面指纹图像对应的滚动角映射参数。Among them, (x, y) is the position of the fingerprint, f _roll (x, y) is the true value of the roll angle of the fingerprint,

It is the rolling angle mapping parameter corresponding to the planar fingerprint image.

Specifically, the three-dimensional angle data of the finger directly affects the horizontal and vertical positions of the planar fingerprint image, that is to say, the first planar fingerprint image database constructed in this application can infer the mapping relationship between the planar fingerprint position and the three-dimensional posture of the finger. Surface modeling will affect the specific expression of the mapping function, but the general idea and implementation process are the same.

In the embodiment of the present application, the method further includes training the plane pose estimation model through the following model:

L _all =L _pos +λL _yaw

L _yaw = ‖θ-θ′‖ ₂

表示指纹的位置的真值，

表示平面姿态估计模型对指纹的位置的预测值，

表示指纹的偏转角的真值，

表示平面姿态估计模型对指纹的偏转角的预测值。Among them, L _all represents the loss function value, L _pos represents the loss function value of position prediction, L _yaw represents the loss function value of the yaw angle, λ is the weighting coefficient,

the truth value representing the location of the fingerprint,

represents the predicted value of the position of the fingerprint by the plane pose estimation model,

represents the true value of the deflection angle of the fingerprint,

Represents the predicted value of the deflection angle of the fingerprint by the plane pose estimation model.

Specifically, the plane posture estimation model is a deep neural network, and the structure of the deep neural network is shown in Figure 5. In order to make full use of the information in the plane fingerprint image and improve the accuracy of posture prediction, the deep neural network is divided into the following three according to functions. modules:

The feature extraction backbone network is based on the backbone network of the most expressive backbone network framework (Yin et al.), and some modifications have been made to it. At the same time, the skills of multi-scale fusion are fully utilized, and finally the features of fixed dimensions are output. descriptor;

The attention mechanism module, which combines and modifies the latest attention mechanism module in (Yin et al.), uses the output of this module as a mask to enhance the foreground area in the output of the feature extraction module, so that the deep neural network can pay more attention to the foreground area of the fingerprint image, ignoring invalid background information;

The three-dimensional angle prediction module is used to output the predicted value of the position of the fingerprint by the deep neural network and the predicted value of the deflection angle, that is, the offset angle of the fingerprint by the deep neural network.

Further, in order to improve the precision and generalization ability of the deep neural network, a representative planar fingerprint image is selected from the first planar fingerprint image database and placed in the second planar fingerprint image database, and selected from the second planar fingerprint image database. No less than 400 fingerprint sequences and no less than 25,000 plane fingerprint images are used to train the deep neural network. The amount of data used for training is not less than 70% of the data in the second plane fingerprint image database, and at the same time, no less than 25,000 plane fingerprint images are selected. 4000 fingerprint images are used as the validation set to adjust the hyperparameters of the deep neural network training, and further improve the generalization ability and robustness of the deep neural network.

Specifically, FIG. 6 is a schematic flowchart of a method for estimating a three-dimensional gesture of a finger from a planar fingerprint proposed by an embodiment of the present application, which includes the following four steps:

Step 210, collect fingerprints, and put the sampled data into the first plane fingerprint image database;

Step 220, select representative sampling data from the first plane fingerprint image database and put it into the second plane fingerprint image database, use the sampling data in the second plane fingerprint image database to train the deep neural network, and use the loss function at the same time. training deep neural networks;

Step 230, using the position information of the planar fingerprint image obtained by the deep neural network and the sampled roll angle and pitch angle data to obtain the fitting parameters from the planar fingerprint image to the three-dimensional posture of the finger, and storing the fitting parameters. into the first plane fingerprint image database;

Step 240, in the testing phase, use the deep neural network to obtain the depth descriptor, offset angle and position data corresponding to the test image, perform an adjacent search based on the depth descriptor to obtain the mapping parameter corresponding to the test image, and obtain the test image according to the mapping parameter and the position data. Corresponding roll angle and pitch angle data.

To sum up, the method proposed in the embodiment of the present application collects the planar fingerprint image of the object to be tested; uses a pre-trained planar pose estimation model to determine the planar fingerprint pose corresponding to the planar fingerprint image; The way of modeling determines the complete three-dimensional gesture of the finger that matches the object to be measured. The present application solves the current fingerprint three-dimensional gesture estimation by pre-training an accurate plane pose estimation model, and mapping or fitting based on it to determine the complete three-dimensional finger pose matching the plane fingerprint pose of the object to be measured. A technical issue where technology is not ideal in terms of functionality and convenience.

In order to realize the above embodiments, the present application also proposes a system for estimating the three-dimensional gesture of a finger from a planar fingerprint.

FIG. 7 is a schematic structural diagram of a system for estimating a three-dimensional gesture of a finger from a planar fingerprint according to an embodiment of the present application.

As shown in Figure 7, a system for estimating the three-dimensional gesture of a finger from a planar fingerprint includes:

A collection module 710, configured to collect a planar fingerprint image of the object to be measured;

The plane pose estimation module 720 is used to determine the plane fingerprint posture corresponding to the plane fingerprint image by using the pre-trained plane posture estimation model;

The determining module 740 is configured to determine the complete three-dimensional gesture of the finger that matches the object to be measured by means of parameter learning or statistical modeling according to the plane fingerprint gesture.

To sum up, the system proposed in the embodiments of the present application collects the planar fingerprint image of the object to be measured through the acquisition module; the planar pose estimation module uses the pre-trained planar pose estimation model to determine the planar fingerprint pose corresponding to the planar fingerprint image; Fingerprint posture, the complete three-dimensional posture of the finger that matches the object to be measured is determined through parameter learning or statistical modeling. The present application solves the current fingerprint three-dimensional gesture estimation by pre-training an accurate plane pose estimation model, and mapping or fitting based on it to determine the complete three-dimensional finger pose matching the plane fingerprint pose of the object to be measured. A technical issue where technology is not ideal in terms of functionality and convenience.

It should be noted that, in the description of the present application, the terms "first", "second" and the like are only used for the purpose of description, and should not be construed as indicating or implying relative importance. Also, in the description of this application, unless otherwise specified, "plurality" means two or more.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (10)

1. a method for estimating three-dimensional gesture of finger by plane fingerprint, it is characterized in that, described method comprises: Collect the planar fingerprint image of the object to be tested; Determine the plane fingerprint posture corresponding to the plane fingerprint image by using the pre-trained plane posture estimation model; According to the plane fingerprint posture, the complete three-dimensional posture of the finger matching the object to be measured is determined by means of parameter learning or statistical modeling. 2. The method of claim 1, wherein the planar fingerprint posture comprises a position and a deflection angle of the planar fingerprint image; The complete three-dimensional gesture of the finger that matches the object to be measured is determined by parameter learning or statistical modeling, including: Determine the three-dimensional pose mapping function corresponding to the planar fingerprint image according to the parameter learning or statistical modeling; Determine the roll angle and the pitch angle of the planar fingerprint image according to the three-dimensional attitude mapping function and the position of the planar fingerprint image; A complete three-dimensional pose matching the object to be measured is determined according to the yaw angle, roll angle and pitch angle of the planar fingerprint image. 3. The method according to claim 2, wherein, determining the three-dimensional attitude mapping function corresponding to the planar fingerprint image according to the parameter learning or statistical modeling, comprising: Build a first planar fingerprint image database by means of machine learning, wherein the first planar fingerprint image database includes feature descriptors; determining a plurality of planar fingerprint images matching the feature descriptors from the first planar fingerprint image database, and determining a first mapping parameter corresponding to each of the multiple planar fingerprint images; Fusing a plurality of the first mapping parameters to obtain second mapping parameters, and determining a three-dimensional attitude mapping function corresponding to the planar fingerprint image according to the second mapping parameters, wherein, by calculating a plurality of the first mappings The average value of the parameters, or the weighted average value, or the average value of a preset number of the most matching first mapping parameters to obtain the second mapping parameter; Alternatively, at least one probability model describing the finger is determined through the simulation system and the actually collected data set, and the three-dimensional attitude mapping function corresponding to the planar fingerprint image is determined according to the at least one probability model describing the finger. 4. The method of claim 1, wherein before using a pre-trained plane pose estimation model to determine the plane fingerprint pose corresponding to the plane fingerprint image, further comprising: Collecting a plane fingerprint image for training and three-dimensional angle data for training corresponding to the plane fingerprint image for training; The plane pose estimation model is trained according to the training plane fingerprint image and the training three-dimensional angle data to obtain the pre-trained plane pose estimation model. 5. method as claimed in claim 4 is characterized in that, described acquisition training plane fingerprint image and the training three-dimensional angle data corresponding to described training plane fingerprint image, comprise: Obtain the training plane fingerprint image by using a plane fingerprint collector; Use two three-axis gyroscopes to obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training, or use an optical tracking measurement system to obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training, or use the The simulation system and the data generator obtain the three-dimensional angle data for training corresponding to the planar fingerprint image for training; wherein, The three-dimensional angle data for training includes the true value of the pitch angle, the true value of the yaw angle and the true value of the roll angle. Not more than 90° and not less than -90°, the range of the true value of the roll angle is not more than 75° and not less than -75°. 6. The method according to claim 4, characterized in that, after the plane fingerprint image for training and the three-dimensional angle data for training corresponding to the plane fingerprint image for training, further comprising: storing the planar fingerprint image for training in a second planar fingerprint image database; Acquire a plurality of planar fingerprint images for training from the second planar fingerprint image database according to preset conditions, and a feature descriptor and a first mapping parameter with each of the multiple planar fingerprint images for training; The plurality of training plane fingerprint images, the first mapping parameter and the feature descriptors of the plurality of training plane fingerprint images are respectively stored in the first plane fingerprint image database. 7. The method according to claim 6, wherein the training of the plane pose estimation model according to the plane fingerprint image for training and the three-dimensional angle data for training, comprising: The first mapping parameter of each training plane fingerprint image in the first plane fingerprint image database is obtained by means of function fitting, and the first mapping parameter includes a pitch angle mapping parameter and a roll angle mapping parameter. 8. The method of claim 7, wherein Determine the pitch angle mapping parameter of each training plane fingerprint image in the first plane fingerprint image database by the following formula: f _pitch (x,y)=b ₁ x ² +b ₂ y ² +b ₃ x+b ₄ y+b ₅ ln(x)+b ₆ ln(y)+b ₇ Among them, (x, y) is the position of the fingerprint, f _pitch (x, y) is the true value of the pitch angle of the fingerprint,

is the pitch angle mapping parameter corresponding to the planar fingerprint image; Determine the rolling angle mapping parameter of each training plane fingerprint image in the first plane fingerprint image database by the following formula: f _roll (x,y)=a ₁ x ² +a ₂ y ² +a ₃ x+a ₄ y+a ₅ ln(x)+a ₆ ln(y)+a ₇ Among them, (x, y) is the position of the fingerprint, f _roll (x, y) is the true value of the roll angle of the fingerprint,

It is the rolling angle mapping parameter corresponding to the planar fingerprint image. 9. The method of claim 4, wherein the plane pose estimation model is trained by the following model: L _all =L _pos +λL _yaw

L _yaw = ‖θ-θ′‖ ₂ Among them, L _all represents the loss function value, L _pos represents the loss function value of position prediction, L _yaw represents the loss function value of the yaw angle, λ is the weighting coefficient,

the truth value representing the location of the fingerprint,

represents the predicted value of the position of the fingerprint by the plane pose estimation model,

represents the true value of the deflection angle of the fingerprint,

Represents the predicted value of the deflection angle of the fingerprint by the plane pose estimation model. 10. A system for estimating a three-dimensional gesture of a finger from a planar fingerprint, wherein the system comprises: The acquisition module is used to acquire the planar fingerprint image of the object to be tested; a plane pose estimation module for determining a plane fingerprint pose corresponding to the plane fingerprint image by using a pre-trained plane pose estimation model; The determining module is used for determining the complete three-dimensional gesture of the finger matching the object to be measured by means of parameter learning or statistical modeling.

Images