CN102045162A - Personal identification system of permittee with tri-modal biometric characteristic and control method thereof - Google Patents

Abstract

The invention relates to a three-modal biological feature certificate holder identification system and a control method thereof. It includes four parts: terminal equipment hardware device for identity authentication, terminal management software, terminal service software and center service software. Terminal equipment is mainly used to collect personal ID card information, static fingerprint feature images, sound signals and video face images; terminal management software is mainly responsible for terminal management, fingerprint registration, face registration, voice feature registration, and collection of ID card information and data analysis and other functions; the terminal service software is mainly responsible for comparing fingerprints, voice and face features and providing identification results based on three-modal fusion; the central service software is mainly responsible for accessing fingerprint, voice and face feature data, and providing The terminal device provides retrieval services and controls various service requests of the terminal device. The system can enable users to obtain high-quality, high-reliability certificate holder identification services.

Description

A three-mode biometric certificate holder identity authentication system and its control method

technical field

The invention relates to the technical field of biological feature identification, in particular to a three-mode certificate holder identity identification system based on fingerprints, voice and human face and a control method thereof. the

Background technique

With the increasing requirements for the accuracy and reliability of social security and identification, a single biometric identification technology can no longer meet the application needs of society, which hinders the wider application of this field. Since no single biometric system can provide sufficiently accurate and reliable identification, the emergence of multiple biometric systems would be an effective alternative strategy, such as voice and face, voice and fingerprint, or as in this project The combination of voice, face and fingerprint proposed in the paper for comprehensive recognition can greatly improve the accuracy and reliability of the system, as well as the robustness in complex environments. As far as biometric identification technology is concerned, its development trend will be to gradually transition from the stage of identification relying on a single pattern to the stage of comprehensive identification relying on multiple biometric features. the

The rapid development of data fusion technology in recent years has provided a solid theoretical basis for multi-biometric identification, and many scholars in the world have been extensively devoted to the research of multi-biometric identification technology. In 1995, Roberto proposed the method of using multiple biometrics for personal identity authentication. He used the weighted geometric mean and super basis function network fusion recognition for the voice classifier and face image classifier respectively, and achieved good results. Bigun et al. (1997) proposed to use supervised learning combined with Bayes theory to fuse voice and face for identity identification, which achieved a high accuracy rate. Dieckmann et al. used two dynamic features of voice and lip movement information and static features of face image for fusion identity recognition. They used a simple voting algorithm to judge whether the decision of a single classifier was consistent with the other two classifiers. Verlinde et al. proposed in 1997 to use the K-NN method to fuse voiceprint and visual features, and achieved good results. In 1998, Jain et al. proposed to fuse the results of fingerprint and face recognition, and theoretically and quantitatively proved the improvement of the efficiency of the multi-biometric authentication system compared with the single biometric authentication system. In 1999, from the theory It has been proved that the fusion of multiple biometric features can improve the authentication rate, and in 2000, a method to determine the specific parameters of each user was proposed to fuse the recognition results of fingerprints, face images and hand shapes. Kittler et al. proposed a fusion theoretical framework and divided it into three layers, and compared the advantages and disadvantages of algorithms such as addition criterion and multiplication criterion in fusion. For the first time, Maes et al. realized a system combining biometrics (fingerprints) and non-biometrics (passwords). In addition, many other multi-biometric feature identification methods and identification methods combining biological features and non-biological features have been shown in many open literatures, all of which have achieved relatively good identification results. Although the multi-biometric identification technology is still in the initial stage, many research results have been applied to practice by commercial companies. The most famous system is DCSAG's BioID system, which uses three components: face, voice and lip movement. The fusion of two basic biometric features to identify individual identities has also achieved relatively good comprehensive identification results. the

In China, a group of scientific research units represented by the Chinese Academy of Sciences and Tsinghua University have set foot in this research field and achieved a large number of research results. The research team led by Tan Tieniu and Wang Yunhong of the State Key Laboratory of Pattern Recognition of the Institute of Automation, Chinese Academy of Sciences has successfully developed a new technology for identification through various human biological characteristics such as iris, face and voice. The multi-mode biometric identification system "TH-ID" independently developed by Tsinghua University includes two parts: four based on face (TH-FaceID), handwriting (TH-writerlD), signature (TH-SignID) and iris (TH-IrisID) Subsystem of biometric identity authentication (including identification and verification) and a multi-modal biometric fusion identity authentication system using multiple biometric features. This system builds a unified database based on face, handwriting, signature, The multi-mode biometric identification and authentication system of the four iris biometrics can select the fusion mode and perform various possible mode fusions. the

At present, the identity authentication system based on biometric features still mainly stays at the level of a single biometric feature. There are few mature systems based on multiple biometric features. There is almost no public report on the mature application system for identity authentication of the certificate holder at home and abroad. the

Contents of the invention

The technical problem to be solved by the present invention is to comprehensively utilize the fingerprint, voice and face feature information collected by the terminal equipment to identify the real identity of the ID card holder, and accordingly propose a three-modal identity authentication system and its control method. the

The specific solution of the present invention to solve the technical problem is to provide a three-mode biometric identity authentication system for the bearer, including a terminal device hardware device for identity authentication, a terminal management software module, a terminal service software module and a central service software Four parts of the module, characterized in that the hardware device of the terminal equipment is mainly used to collect personal ID card information, fingerprint features, voice and face features; the terminal management software module is mainly responsible for terminal management, fingerprint registration, voice registration, face registration , collecting ID card information and data analysis, etc.; the terminal service software module is mainly responsible for comparing fingerprints, voice and face features and giving fusion-based identification results; the central service software module is mainly responsible for accessing fingerprints, voice and face features. Facial feature data, and control the use of terminal equipment. the

Correspondingly, a control method of a three-modal biometric certificate holder identity authentication system is provided, including four parts: a terminal device hardware device for identity authentication, a terminal management software module, a terminal service software module, and a central service software module , is characterized in that, described control method comprises the step:

(1) The terminal device accesses the system through the network communication interface and connects to the central server. After the legality of the terminal device’s identity is verified, it obtains the corresponding service request authorization, and rejects unauthorized terminal devices accessing the system to request characteristic data from the central server. ;

(2) Collect the ID card information, fingerprint information, voice and video face information of the cardholder, and perform preliminary processing such as pattern classification and noise removal;

(3) Preprocess the fingerprint, voice and face data after the preliminary processing in the above step (2) and extract corresponding features;

(4) The terminal service software retrieves the local database LDB according to the ID card number collected by the ID card reader, and obtains the corresponding fingerprint, voice and face feature template data;

(5) The fingerprint, voice and face feature comparison module retrieves the database according to the matching algorithm currently selected by the system, and records the similarity scores obtained by each comparison, and the values are v1, v2 and v3 respectively;

(6) After obtaining the matching scores of the fingerprint, voice and face features, the decision-level fusion is performed on the values of v1, v2 and v3, and the matching result after fusion is calculated. the

The beneficial effects of the present invention are: through fusion and identification of a variety of single biological features, the problem of system identification conclusion error caused by failure of certain biological features can be effectively prevented, thereby greatly improving the usability and adaptability of the identity identification system and the reliability of the identification results. The identity authentication system and its control method can also provide online/offline, terminal-expandable identity authentication services that can operate independently or on the Internet, enabling users to obtain high-quality services without investing a lot of money and equipment. Quality and reliability of certificate holder identification services. the

Description of drawings

Fig. 1 is a topological structure diagram of the system provided by the present invention. the

Fig. 2 is a structural diagram of functional modules of the present invention. the

Fig. 3 is an outline view of the terminal device of the present invention. the

Fig. 4 is a structural diagram of the independent operation mode of the present invention. the

Fig. 5 is a structural diagram of the networking operation mode of the present invention. the

Fig. 6 is a data flow diagram of identity verification in the present invention. the

Fig. 7 is a diagram of the interface arrangement among the subsystem elements in the system of the present invention. the

Fig. 8 is a flow chart of the terminal device legality verification in the present invention. the

Fig. 9 is a detailed flow chart of identity authentication processing in the present invention. the

Detailed ways

Below in conjunction with accompanying drawing and specific embodiment the present invention is described in further detail:

The technical problem to be solved by the present invention is how to comprehensively utilize the fingerprint, voice and face feature information collected by the terminal equipment to identify the real identity of the ID card holder, and accordingly construct a three-modal identity authentication The system and its control method, the identity authentication system overcomes the defects of the prior art, and can provide identity authentication services that are online/offline, terminal expandable, independent or networked, and enable users to operate without investing a lot of money. Get high-quality and reliable bearer identity authentication services in the case of devices and devices. the

As shown in Fig. 1, Fig. 2, Fig. 3, Fig. 4, Fig. 5, Fig. 6 and Fig. 7, the technical problem proposed by the present invention is solved in this way: a kind of certificate holder identification system is provided, including collecting fingerprint, Voice, face and ID card information terminal equipment (see Figure 3 for details), terminal management software, terminal service software and center service software are four parts (see Figure 7 for details). This solution is characterized in that the terminal equipment It is composed of ID card reader, fingerprint instrument, microphone, video camera device and other components, and is mainly used to read ID card information, fingerprints, voice and facial feature signals; the terminal management software is mainly composed of system management, identity information collection, fingerprint It consists of functional modules such as registration, voice registration, face registration and simulation verification. It is mainly responsible for system parameter configuration, reading ID card information, registering and storing fingerprint features, voice features and face features, and simulated verification of the identity of the cardholder. and system operation data statistical analysis and other functions; the terminal service software is mainly composed of communication module, feature collection module, feature verification module and decision fusion module, etc. It is mainly responsible for collecting fingerprint, voice and face feature data, searching database, verifying fingerprint, Functions such as voice and face features and calculation of decision-level fusion results; the central service software is mainly composed of terminal authority verification, data access and interactive communication modules, and is mainly responsible for network communication between the central server and terminal equipment, terminal equipment access and Validation of the legitimacy of service requests, as well as access to fingerprint and facial feature data and ID card information and other functions. the

Among them, the terminal management software is specifically composed of the following functional modules (see Figure 2 for details):

① System management module: to register legal terminal equipment information, to authorize the operators of the system, to set important operating parameter values in the system, etc. the

②Identity information entry module: manually add, delete, and modify the basic identity information of the entered personnel (according to the content displayed on the ID card), and import large quantities of identity information in EXCEL. the

②Fingerprint registration module: the fingerprint feature information of the registrant, and save the registration information into the database (local database or system database). the

③Voice registration module: register the person's voice characteristic information, and save the registered characteristic information into the database (local database or system database). the

⑤ face registration module: register the face feature information of the person, and save the registration information into the database (local database or system database). the

⑥Simulation verification module: After inputting the identity information on the management workstation, the process of the dual-modal identity fusion identification system based on fingerprint and face features will be simulated. the

⑦Data analysis module: to conduct statistics and analysis on the operation data of the whole system, and to dig out valuable potential information, such as system operation log record data and identity identification record data, etc. the

The terminal service software is specifically composed of the following functional modules:

①Interactive communication module: mainly responsible for exchanging data with the interactive communication module of the "verification service center software" to complete the legality verification of terminal equipment, upload the extracted biometric data, and receive the biometric data downloaded from the central server. the

②Data collection module: mainly complete the automatic collection and manual entry of ID card information based on the ID card reader, the collection of fingerprint features based on the fingerprint instrument, and the collection of face video images based on the camera device, etc. the

③Fingerprint\sound\face feature verification module: responsible for preprocessing the collected fingerprint, voice and face data, extracting feature information, and completing 1:1 verification with the stored template features. the

④Fusion decision-making module: After the face, voice and fingerprint features are individually matched, the decision-level fusion is carried out, and the identity of the certificate holder is identified according to the fused score. the

The central service software is specifically composed of the following functional modules:

①Interactive communication module: mainly responsible for receiving the device legitimacy verification information sent by the terminal device, as shown in Figure 8, to determine whether the access request of the current device is legal, and responsible for receiving the biometric features and auxiliary data uploaded and downloaded by the legal terminal . the

②Data access module: responsible for storing the terminal data received through the communication module (stored in the system database, SDB), and retrieving the biometric data requested by the terminal according to the ID card information and sending it back to the communication module for downloading to the terminal equipment. the

③Terminal device authority verification module: Verify whether the terminal device connected to the service center system is legal according to the received terminal device code and other information, and assign rights to the terminal according to the verification result. the

A kind of cardholder identification system and control method thereof based on fingerprint, sound and facial feature, it is characterized in that, comprises following several steps (see Fig. 6 and Fig. 9 specifically):

(1) The terminal device accesses the system through the network communication interface and connects to the central server. After the legality of the terminal device’s identity is verified, it obtains the corresponding service request authorization, and rejects unauthorized terminal devices accessing the system to request characteristic data from the central server. (Figure 8);

(2) Collect the ID card information, fingerprint information, voice and video face information of the cardholder, and perform preliminary processing such as pattern classification and noise removal;

(3) Preprocess the fingerprint, voice and face data after preliminary processing in step (2) and extract corresponding features. the

Among them, the processing of fingerprints mainly includes the following steps:

①Fingerprint effective area segmentation: Divide the fingerprint image into a series of 16×16 non-intersecting image blocks, each block is marked as B(1,1), B(1,2),…, B(i,j), and then Using the following formula v(i, j)

$\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{\frac{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; .</span><span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; N</span>}}}$

to calculate the pixel gray value variance of each image block, where x _n and x respectively represent the gray value of the pixels in the image block, N represents the number of pixels contained in the block, set the segmentation threshold v _θ = 11.5, respectively divide each The variance value of the block is compared with v _θ , if v(i, j) > v _θ , then the block B(i, j) is judged as a valid fingerprint area, otherwise it is judged as the background area of the fingerprint;

②Fingerprint direction field calculation: Calculate the gradients G _x and G _y of each pixel in the block B(i, j) in the x and y directions respectively, using the formula d(i, j)

$\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\mathrm{<span\; class="notranslate">\; arctan</span>}<span\; class="notranslate">\; (</span>\frac{\underset{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span>}{\underset{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; w</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>$

Calculate the local direction of the block B(i, j) respectively, where (i′ _b , j′ _b ) represents the coordinates of the pixels in the block, w represents the pixel width of the block, and the calculated local direction value of the block is only Taking the closest one of the four component values 0, π/4, π/3, and 3π/4, the final local direction d(i, j) ∈ {0, 4π/4, π, 3π/4 }, correct the calculation result of the direction field by the consistency feature of the direction field, and calculate its consistency value C(i, j) within the 5×5 neighborhood D of the block B(i, j)

$\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; 1</span>}}^{<span\; class="notranslate">\; ,</span>}}{\mathrm{<span\; class="notranslate">\; twenty\; four</span>}}\sqrt{\underset{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; D.</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; ,</span>$

This formula has Here d=mod(d(i′,j′)-d(i,j)+2π)

If C(i, j) < 0.35, adjust the local direction of the block B(i, j) to the direction with the most significant local direction in the neighborhood D, otherwise the local direction of the block B(i, j) remains unchanged Change;

③Use the M-PCNN network to filter the preprocessed image;

④ Feature extraction of fingerprint images, including global feature extraction and detail feature extraction:

The global feature is extracted as a Fourier spectrum feature: firstly, the input fingerprint image is divided into 32×32 image blocks, and two-dimensional discrete Fourier transform is performed on the blocks, the formula is

$\mathrm{<span\; class="notranslate">\; G</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; g</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="j"\; filenames="H2009101679254E00041.png"\; state="\{\{state\}\}">j</figure-callout></span>}\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; mi</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; +</span>\frac{\mathrm{<span\; class="notranslate">\; k\; n</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$

In the formula, (m, n) is the value domain coordinate of the pixel, (i, k) is the frequency domain coordinate corresponding to the pixel, and after each sub-block undergoes Fourier transform, the Fourier spectrum map of the whole image is obtained, which is pressed by The rules are quantized into global feature vectors of fingerprints;

Detailed feature extraction, including core points, bifurcation points and endpoints, the detailed feature extraction templates used are as follows:

P _{3 P2 P1} P ₄ P P ₀ P ₅ P ₆ P ₇

in,

$t=\underset{i=0}{\overset{7}{\mathrm{\&Sigma;}}}{P}_i$ and $m=\left\{\begin{array}{cc}1,& \mathrm{if\; t}=1\\ -1,& \mathrm{if\; t}>2\end{array}\right.$

If m=1, it means that P is an end point, otherwise P is a branch point. the

The ASM model processing of video face images mainly includes the following steps:

① Sampling the grayscale face image to obtain the shape vector and contour point feature information. the

First, building a model requires manual calibration of training images. In the present invention, in order to ensure the accuracy of recognition, 256 face images (including different expressions and postures of multiple people) are selected, and each image is manually marked with 68 contour points as training data. Contour points are generally marked at places that can represent the target contour, and the contour points selected in the present invention are marked on the outer contour of the face and the edges of organs. the

The relevant calibration points are:

S _i =(xi _,1 ,y _i,1 ,xi _,2 ,y _i,2 ,...,xi _,68 ,y _i,68 ) ^T , i=1,2,..., 256

Among them, (xij, yij) represents the coordinates of the jth (1≤j≤68) contour point of the i-th image; Si of each image represents a shape vector. And obtain the feature information near each calibration point (contour point), these features are the main basis for matching using the ASM model. the

②Establish the ASM model of the face. the

Due to the differences in shooting conditions and resolutions of each sample image, the coordinates of the obtained shape vectors have different scales, so the shape vectors of the sample images must be normalized, and they are represented in the same coordinate system by rotating, translating, and scaling when consistent. In the present invention, the following method is used to align two shape vectors: min D=[T(x)-x′]°

The transformation rule T(x) used in it is defined as:

$\mathrm{<span\; class="notranslate">\; T</span>}\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right)<span\; class="notranslate">\; =</span>\left(\begin{array}{cc}\mathrm{<span\; class="notranslate">\; a</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}\\ \mathrm{<span\; class="notranslate">\; b</span>}& \mathrm{<span\; class="notranslate">\; a</span>}\end{array}\right)\left(\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\end{array}\right)<span\; class="notranslate">\; +</span>\left(\begin{array}{c}{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}\\ {\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}\end{array}\right)$

Then transform each shape vector to the mean tangent space (x _t -x)·x _t =0, so that x is exactly magnified by 1/(x.xt) times, where |xt|=1.

After the shape vectors are aligned, follow the steps below to perform ASM modeling on the face shape:

First calculate the mean value x of each shape vector in 256 samples, and then use the following formula to calculate the covariance S of 256 shape vectors, the specific method is as follows:

$\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; the\; s</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}$

Then calculate the eigenvector φi, and the corresponding eigenvalue λi, based on these calculation results, estimate the approximate value of the arbitrary shape vector according to the unified rules, x≈x+Φ _i b where b is a 180-dimensional vector by the formula b=Φ _i ^T (xx) is calculated. In addition, b needs to be reduced to $\left|b_i\right|\&le;3\sqrt{{\mathrm{\&lambda;}}_i}.$ In the present invention, it is assumed that each component of b is independent and obeys Gaussian distribution, so P(b) just satisfies the following equation,

$\mathrm{<span\; class="notranslate">\; log</span>}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 0.5</span>}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; <figure-callout\; id="2"\; label="t\; b\; i"\; filenames="H2009101679254E00041.png"\; state="\{\{state\}\}">t</figure-callout></span><figure-callout\; id="2"\; label="t\; b\; i"\; filenames="H2009101679254E00041.png"\; state="\{\{state\}\}">\; </figure-callout>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{{\mathrm{<span\; class="notranslate">\; b</span>}}_{}^{}}{}\frac{{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{}}{{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; const</span>}$

The constant parameter const is selected as the mean value of the sample, and λi is the corresponding eigenvalue component. the

In addition, the processing of the sound signal mainly includes the following steps:

① Preprocessing of speech signals, including pre-filtering, pre-emphasis and framing. What pre-filtering adopts is the band-pass filter of 60Hz to 7.8Hz. The filtered signal is passed through a pre-emphasis filter, and then framed using a 16ms frame length and 8ms frame shift;

② Speech feature extraction, the speech feature uses 14th-order linear predictive cepstrum (linear predictive coding cepstrum, LPCC) coefficients. In the training phase, the cepstral coefficient features extracted frame by frame from the two-pass training speech (recognized person reads his name) are stored as speech feature templates in the form of vector sequences, and the feature templates are stored in the database. In the confirmation stage, the 14th-order linear predictive cepstral coefficient is extracted from a speech (the person being recognized reads his name) as the input speaker's voice feature;

③ speech feature matching, assuming that the feature vector sequence of the template is X=(X1, X2, ..., XT), the input speaker's voice feature vector sequence is Y=(Y1, Y2, ..., YT), and calculates the average of the two distance, that is

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; \&Center\; Dot;</span><span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; \{</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}\underset{\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; T</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span>$

Among them: d(t) is the matching distance of the tth frame; Q(t) is a monotonic time warping integer function, Q(1)=1, Q(T)=T, the matching algorithm gives Q(t) in other The value of the point minimizes the average matching distance. the

The matching distance between frames is the square of the Euclidean distance:

$\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

Wherein: Xi=(xi, 1, xi, 2,..., xi, N), Yj=(yj, 1, yj, 2,..., yj, N), N feature vector dimension, in this patent, N= 14 (that is, 14th order). The matching distance between input speech Y and template X is calculated as D(X, Y). the

④ Calculate the similarity score, and convert the distance D (X, Y) obtained during the matching process into a similarity score V3, where V3 is converted according to the following rules:

${\mathrm{<span\; class="notranslate">\; V</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; min</span>}\mathrm{<span\; class="notranslate">\; D.</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; min</span>}\mathrm{<span\; class="notranslate">\; D.</span>}}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 100</span>}$

Among them, maxD and minD represent the maximum distance and minimum distance of the vector obtained in the experiment, which are 14 and 1 respectively. the

(4) The terminal service software retrieves the local database (that is, LDB) according to the ID card number collected by the ID card reader, and obtains corresponding fingerprint, voice and face feature template data. the

(5) The fingerprint, voice and face feature comparison module retrieves the database according to the matching algorithm currently selected by the system, and records the similarity scores obtained by each comparison, and the values are V1, V2 and V3 respectively. the

(6) After obtaining the matching scores of the fingerprint, voice and face features, the decision-level fusion of V1, V2, and V3 values is carried out to calculate the matching result after fusion. the

In the present invention, an optimal Bayesian decision fusion method is adopted. First, V1, V2, and V3 are vector-normalized to form a 3-dimensional vector V as the input vector of the Bayesian decision network. According to the output of this classifier, the final decision-level fusion result is obtained. The optimal multimodal fusion rule can be searched adaptively, so that the Bayesian risk value reaches the minimum value, so as to construct an optimal multimodal biometric fusion system. the

The specific steps are as follows:

① Calculate the FAR and FRR of single-mode biometrics (fingerprint, voice and face)

The false acceptance rate and false rejection rate of fingerprint and face single-modal biometric identification methods are obtained by statistical methods, that is, how many illegal individuals are identified as legitimate (FAR) and how many legitimate Instance is Recognized as Illegal (FRR). the

In the matching phase of biometric identification, the identification method compares the biometrics to be identified with the templates in the database, and finds the template most similar to the individual to be verified. Then, further judge whether the individual to be verified exists in the database, if it is an illegal individual, the system can deny the individual's access request. For this reason, corresponding thresholds should be set to classify individual features (legal or illegal). the

Calculate the threshold for biometric identification

Assume that there are M individuals in the template database, and each individual has K feature samples. Therefore, the entire database can be expressed as X={x _mk : m=1, 2, . . . , M; k=1, 2, . . . , K}. Record the degree of similarity between two biometric features x, y as l(x, y). If the identity of individual x needs to be identified, and the feature template with the highest similarity to x has been found The legitimacy of an individual is determined according to the following rules:

$\mathrm{<span\; class="notranslate">\; output</span>}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}& \mathrm{<span\; class="notranslate">\; ifl</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; *</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; T</span>}\\ \mathrm{<span\; class="notranslate">\; reject</span>}& \mathrm{<span\; class="notranslate">\; otherwise</span>}\end{array}\right.$

Some biometric identification systems set two thresholds for each individual: a low threshold L _m and a high threshold U _m (m=1, 2, ..., M), and the identification rules of the system become the following form [i] :

$\mathrm{<span\; class="notranslate">\; output</span>}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}& \mathrm{<span\; class="notranslate">\; ifl</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; *</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}}\\ \mathrm{<span\; class="notranslate">\; reject</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; ifl</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; *</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ></span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}}\\ \mathrm{<span\; class="notranslate">\; applyheuristic</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if</span>}{\mathrm{<span\; class="notranslate">\; L</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}{\mathrm{<span\; class="notranslate">\; k</span>}}^{<span\; class="notranslate">\; *</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{{\mathrm{<span\; class="notranslate">\; m</span>}}^{<span\; class="notranslate">\; *</span>}}\end{array}\right.$

“足够接近”：若两者匹配程度足够高，则鉴定x的身份为m^*。高阈值用于明确排除两个个体之间的相关性，当相似度超过高阈值，则认为是非法个体。当两个个体之间的距离 

介于高低阈值之间时，则需要采用别的方法，如启发式方法判定个体的身份。 In the above rules, the low threshold is used to determine whether the individual feature x to be verified is consistent with the feature template

"Close enough": If the degree of matching between the two is high enough, then identify the identity of x as m ^* . The high threshold is used to explicitly exclude the correlation between two individuals, and when the similarity exceeds the high threshold, it is considered an illegal individual. when the distance between two individuals

When it is between the high and low thresholds, other methods, such as heuristic methods, are required to determine the identity of the individual.

The threshold calculation process is divided into the following steps:

First, the biometric database is divided into two parts: legal individual training set X and illegal individual training set Y. Data set X is used to adjust the recognition system's ability to identify and classify legal individuals; data set Y is used to adjust the system's ability to reject legal individuals. Then divide the training set X into two disjoint parts: X ₁ and X ₂ , assuming that each individual in the training set has K feature samples, then X ₁ and X ₂ are respectively expressed as the following forms: X ₁ ={x _mk ∈ X: m=1, 2, ..., M; k = 1, 2, ..., K/2}X ₂ = {x _mk ∈ X: m = 1, 2, ..., M ;k=K/2+1,...,K}

Next, calculate the intra-class distance of the training set X: For individual m in the class, select a feature x _mk of individual m in X ₁ , and calculate the distance between x _mk and all features of the same type of individual m in X ₂ in turn. Then calculate the average value of these distances, and use this average value as the low threshold L _m of the recognition system for individual m.

Finally, calculate the inter-class distance between the training set X and Y: select a biological feature x _mk in X, and calculate the distance between the feature x _mk and each feature in Y in turn. The minimum of these distances is taken as the high threshold U _m of the recognition system for individual m.

Note that L _m > U _m may occur in actual calculation, and U _m needs to be adjusted, for example, U _m = L _m × 1.2. Generally, it can be adjusted according to actual needs.

③ Calculate the global FAR and FRR of the fusion system

The global error rate FAR _fus and FRR _fus of the fusion system can be calculated by fusion rules and FAR and FRR of each single mode. Assuming that the decision rule is denoted by f, the decision rule of the bimodal biometric fusion system is simplified as follows:

d ₁ d ₂ f 0 0 f ₀ 0 1 f ₁ 1 0 f ₂ 1 1 f ₃

In the above table, d ₁ and d ₂ respectively represent two single-modal recognition methods. The results of each single-modal recognition are either 0 (individual illegal) or 1 (individual legal), and the output f _i (i∈ {0, 1, 2, 3}) is also 0 or 1. The global error rate FAR _fus and FRR _fus of the fusion system can be expressed as a function of f:

${\mathrm{<span\; class="notranslate">\; FRR</span>}}_{\mathrm{<span\; class="notranslate">\; fuss</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}}\mathrm{<span\; class="notranslate">\; \&Psi;</span>}{\mathrm{<span\; class="notranslate">\; RR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; FAR</span>}}_{\mathrm{<span\; class="notranslate">\; fuss</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Pi;</span>}}}\mathrm{<span\; class="notranslate">\; \&Psi;</span>}{\mathrm{<span\; class="notranslate">\; AR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>$

in:

${\mathrm{<span\; class="notranslate">\; \&Psi;AR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; FAR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}& <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; FAR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}& <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}\right.$ ${\mathrm{<span\; class="notranslate">\; \&Psi;RR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; FRR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}& <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; FRR</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}& <span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; d</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\end{array}\right.$

The variable S in the above formula represents the length of the fusion rule. In the fingerprint-face dual-modal biometric fusion system of the present invention, the fixed value of S is 4. the

(7) Display the current certificate holder identity verification result, pass or fail. the

其中Δ(k)符合 According to the control method of fingerprint identification provided by the present invention, it is characterized in that, in step (3), in the fingerprint minutiae feature extraction step, the core point extraction is based on the fingerprint direction field, utilizing the poincare method to extract, the value

where Δ(k) corresponds to

$\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; <</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; if\&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span><span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; \&pi;</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; otherwise</span>}\end{array}\right.<span\; class="notranslate">\; ,</span>$

δ(k)=O'(ψ _x (i'), ψ _y (i'))-O'(ψ _x (i), ψ _y (i)),

i'=(i+1)mod N _ψ ,

After the calculation, judge the value of p(i, j), if P(i, j) = 0.5, it is the core point; the extraction of bifurcation points and endpoints is obtained by sliding template method detection, first define the bifurcation The pixel structure templates of points and endpoints are used to traverse the whole image in order from left to right and top to bottom respectively. Every time the template slides once, the matching degree between the template and the corresponding area of the image is calculated. When the matching value is greater than 0.7, it is determined that the current pixel of the image is consistent with the detailed features corresponding to the used template. A bifurcation point, if it conforms to the endpoint template, the pixel is an endpoint. the

According to fingerprint feature identification method provided by the present invention, it is characterized in that, in step (6), fingerprint feature comparison comprises the following steps:

①Fingerprint alignment: input fingerprint feature point set Vi={v1, v2,...,Vn}, where vi=(x, y, θ), Tj={t1, t2,...tm} is the feature point set of the template fingerprint, First align the core points, after alignment, the coordinates of the Vi point set are transformed as follows:

x' _i = x _i +(x _to -x _io ) and y' _i =y _i +(y _to -y _io ),

Among them (x _to , y _to ) and (x _io , y _io ) are the core points of the template fingerprint and the input fingerprint respectively; and then rotate and align, and the rotate alignment is obtained by calculating a reflection transformation t:

$\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; a</span>}& <span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; sin</span>}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; \&Delta;x</span>}\\ \mathrm{<span\; class="notranslate">\; sin</span>}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; cos</span>}\mathrm{<span\; class="notranslate">\; a</span>}& \mathrm{<span\; class="notranslate">\; \&Delta;y</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right\}\left\{\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; <figure-callout\; id="1"\; label="y"\; filenames="H2009101679254E00041.png"\; state="\{\{state\}\}">y</figure-callout></span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right\}$

In the above transformation, the core points are aligned, Δx and Δy have been determined, a is obtained by making the expression |v _t -t(v _i )| ² <θ established, and finding the maximum logarithm of Vi and Vt matching points, where θ is a set threshold, set to 0.05;

②Similarity calculation: For global feature matching, the features of the input fingerprint and template fingerprint are Fourier spectrum features, and the feature vector is the serialization result of the spectrogram pixels. The similarity between the two is calculated by calculating unequal length vectors The Euclidean distance between them is realized, and the similarity is calculated as follows:

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; dis</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 100</span>}<span\; class="notranslate">\; \%</span>$

In the formula, dis is the Euclidean distance value between the two, and max(v _i , v _t ) means to take the largest vector length between the input vector and the template vector; for detailed feature matching, the template is used as the benchmark, and the input point set Vi After transforming and projecting onto the template point set Vt according to (Δx, Δy, a), calculate the number of all matching point pairs within a distance error of less than 5 pixels. The similarity value at this time is calculated according to the following formula:

$\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; couple</span>}}}{\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 100</span>}<span\; class="notranslate">\; \%</span>$

In the formula, n _couple represents the number of successfully paired points, and max(v _i , v _t ) still represents the maximum number of detail points in the input point set and the template point set;

③Fingerprint verification: After calculating the similarity data S between the input fingerprint and the template fingerprint, compare S with the preset recognition threshold T _s , where T _s is adjusted according to different security requirements, if S ≥T _s means that the two fingerprints are successfully matched, that is, they come from the same finger; otherwise, it is judged that the two fingerprints are not from the same finger.

According to the facial feature identification method provided by the present invention, it is characterized in that, in step (6), the comparison of human face feature comprises the following steps:

① Use the mean Viola Jones face detector to detect the face images in the video sequence, and automatically complete the labeling of 68 feature points. the

② Take 7 points in the normal direction of each feature point in the face image and the center of the feature point (that is, a total of 2K+1 points, K=7), these 15 points form a vector gi, and the face image The vectors gi corresponding to the 68 feature points form a set of vector sequences g1, g2, ..., g68. the

Normalizes the sequence of vectors gi, computes the mean g and covariance Sg. the

Iteratively calculate the Mahalanobis distance value between the input sample and the face feature template corresponding to the ID card number in the database according to the formula f(g _s )=(g _s -g) ^T S _g ^-1 (g _s -g).

If the minimum Mahalanobis distance is less than the given threshold T, the face feature comparison is successful, otherwise the feature comparison fails, that is, the current certificate holder and the feature registrant are not the same person. the

According to the certificate holder identification system provided by the present invention, it is characterized in that the identification system provides two operating modes, i.e. an independent operating mode (see Figure 4 for details) and a networked operating mode (see Figure 5 for details):

① Independent operation mode: The terminal independently completes feature data collection, local storage, data retrieval, and feature data matching verification, and gives the verification conclusion after fusion. the

②Networked operation mode: the terminal is connected to the remote verification service center, and the biometric data is extracted independently. The characteristic data during registration are stored locally and submitted to the central server for remote storage. The service software queries the system data SDB and downloads the fingerprint and face feature data to the terminal device, and then performs feature verification processing. the

According to the certificate holder identification system provided by the present invention, it is characterized in that the system operates in the following software and hardware environment (see Fig. 1 for details):

The terminal management software, terminal service software and center service software all support Windows 2000/XP/vista operating system, and the auxiliary software support environment required is MS SQL SERVER2000 and OFFICE2000; the hardware operating environment of the terminal management software is an ordinary desktop computer, and the center The hardware environment of the service software is a high-performance PC server, and the terminal service software runs on the terminal device, which is composed of a tablet computer (with touch screen), a fingerprint collector, a camera, and an ID card reader. the

The present invention mainly provides an application system for verifying the identity of the holder by using fingerprints, faces and ID card numbers. Its main features are summarized as follows:

①Verify the identity of the certificate holder effectively through fingerprint and face feature information. the

②The working mode of the system can be switched, and it can be configured to work in a single biometric verification mode, or in two biometric fusion verification modes;

③The system can work both stand-alone and networked;

④The terminal equipment and the central server can be connected through the Internet;

⑤ The central server provides the function of verifying the legitimacy of the terminal equipment to avoid unauthorized terminal equipment accessing the system and overloading of the central server. the

⑥The management end of the system is separated from the server end, and the terminal management software independently completes functions such as identity information registration, biometric collection, operation data analysis, monitoring system operating status, and configuration of important system operating parameters, and the terminal service software completes identity Authentication, registration and verification are separated. the

Those skilled in the art will appreciate that the embodiments described herein are to help readers understand the principles of the present invention, and it should be understood that the protection scope of the invention is not limited to such specific statements and embodiments. All possible equivalent replacements or changes made according to the above descriptions are deemed to belong to the protection scope of the claims of the present invention. the

Claims (10)

1. A kind of three-modal biological characteristic bearer identity authentication system, including terminal equipment hardware device, terminal management software module, terminal service software module and central service software module used for identity authentication four major parts, characterized by that, the terminal equipment hardware device is mainly used for gathering the personal identity card information, fingerprint characteristic, pronunciation and facial characteristic; the terminal management software module is mainly responsible for terminal management, fingerprint registration, voice registration, face registration, identity card information acquisition, data analysis and the like; the terminal service software module is mainly responsible for comparing the fingerprint, voice and face characteristics and giving a fusion identity identification result based on fusion; the central service software module is mainly responsible for accessing fingerprint, voice and face feature data and controlling the use of the terminal equipment.

2. A multimodal biometric bearer identity verification system as claimed in claim 1 wherein the terminal management software modules further comprise the following modules:

s21, a system management module: for registering legal terminal device information, authorizing the use operator of the system, setting important operation parameter values in the system, and the like;

s22, an identity information input module: the EXCEL import is used for manually adding, deleting and modifying basic information (content displayed according to an identity card) of the identity of an input person and large-batch identity information;

s23, a fingerprint registration module: registering fingerprint characteristic information of a person, and storing the registered information into a database (a local database or a system database);

s24, a voice registration module: registering voice feature information of a person and saving the registered feature information into a database (a local database or a system database);

s25, a face registration module: registering the face feature information of a person, and storing the registered information into a database (a local database or a system database);

s26, a simulation verification module: after identity information is input on a management workstation, simulating and operating a dual-mode identity fusion authentication system flow based on fingerprints and human face characteristics;

s27, a data analysis module: and (4) counting and analyzing the operation data of the whole system, and mining valuable potential information, such as system operation log record data, identity authentication record data and the like.

3. A multimodal biometric bearer identity verification system as claimed in claim 1, wherein said terminal services software module further comprises:

s31, an interactive communication module: the system is mainly responsible for exchanging data with an interactive communication module of 'verification service center software' so as to finish the validity verification of terminal equipment, upload extracted biological characteristic data and receive the biological characteristic data downloaded from a center server;

s32, a data acquisition module: the method mainly completes automatic acquisition and manual input of identity card information based on an identity card reader, fingerprint characteristic acquisition based on a fingerprint instrument, face video image acquisition based on a camera device and the like;

s33, fingerprint, voice and face feature verification module: the system is responsible for preprocessing the collected fingerprint, voice and face data, extracting characteristic information and finishing 1:1 verification of template characteristics corresponding to a licensee;

s34, a fusion decision module: mainly based on the scores obtained by matching the human face, the voice and the fingerprint features, the decision-making fusion is carried out according to a decision-making fusion method so as to identify whether the identity of the licensee is consistent with the identity of the licensee.

4. A tri-modal biometric bearer identity verification system according to claim 1, wherein the central service software module further comprises:

s41, an interactive communication module: the terminal equipment is mainly responsible for receiving equipment validity verification information sent by the terminal equipment so as to determine whether an access request of the current equipment is legal or not and receiving biological characteristics and auxiliary data thereof uploaded and downloaded by a legal terminal;

s42, a data access module: the system is responsible for storing terminal data (stored in a system database SDB) received by the communication module and retrieving the biological characteristic data requested by the terminal according to the identity card information and sending the biological characteristic data back to the communication module to be downloaded to the terminal equipment;

s43, the terminal equipment authority verification module: and verifying whether the terminal equipment accessed to the service center system is legal or not according to the received information such as the terminal equipment codes and the like, and distributing the authority to the terminal according to the verification result.

5. A control method of a three-mode biological characteristic bearer identity authentication system comprises four parts of a terminal equipment hardware device, a terminal management software module, a terminal service software module and a central service software module, wherein the terminal equipment hardware device, the terminal management software module, the terminal service software module and the central service software module are used for identity authentication, and the control method is characterized by comprising the following steps:

(1) the terminal equipment is accessed to the system through the network communication interface and is connected with the central server, after the terminal equipment identity is verified by validity, corresponding service request authorization is obtained, and the unauthorized terminal equipment access system is refused to request the characteristic data from the central server;

(2) collecting identity card information, fingerprint information, voice and video face information of a licensee, and performing primary processing such as mode classification, noise removal and the like;

(3) respectively preprocessing the fingerprint, voice and face data after the preliminary processing in the step (2) and extracting corresponding features;

(4) the terminal service software retrieves a local database LDB according to the ID card number acquired by the ID card reader, and acquires corresponding fingerprint, voice and face feature template data;

(5) the fingerprint, sound and face feature comparison module searches a database according to a matching algorithm currently selected and matched by the system, and records similarity scores obtained by each comparison, wherein the similarity scores are v1, v2 and v3 respectively;

(6) and after the matching scores of the fingerprint, the voice and the human face characteristics are respectively obtained, performing decision-level fusion on the v1, v2 and v3 values, and calculating a matching result after fusion.

6. A method of controlling a trimodal biometric bearer authentication system according to claim 5, wherein the fingerprint processing includes the steps of:

(11) fingerprint effective area segmentation: the fingerprint image is divided into a series of 16 x 16 non-intersecting image blocks, each block labeled B (1, 1), B (1, 2), …, B (i, j), and then the following formula v (i, j) is used

To calculate the variance of the gray-scale values of the pixels of each image block, where x_nX represents the gray value of the pixel in the image block, N represents the number of the pixels contained in the image block, and a segmentation threshold value v is set_θ11.5, the variance value of each image block is respectively compared with v_θBy comparison, if v (i, j) > v_θIf yes, the block B (i, j) is determined as a valid fingerprint area, otherwise, the block B (i, j) is determined as a background area of the fingerprint;

(12) direction field calculation of fingerprints: the gradients G of each pixel in the block B (i, j) in the x and y directions are calculated separately_xAnd G_yUsing the formula d (i, j)

Local directions of the block B (i, j) are calculated, respectively, where (i'_b，j′_b) Representing the coordinates of the pixels in the block, w representing the pixel width of the block, the calculated local direction value of the block only takes the most approximate value of the four component values 0, pi/4, pi/3 and 3 pi/4, the final local direction D (i, j) of the block belongs to {0, 4 pi/4, pi, 3 pi/4 }, the calculation result of the directional field is corrected by the consistency characteristic of the directional field, and the consistency value C (i, j) is calculated in the range of 5 multiplied by 5 neighborhood D of the block B (i, j)

In this formula are

Where D ═ mod (D (i ', j') -D (i, j) +2 pi) if C (i, j) < 0.35, then the local direction of tile B (i, j) is adjusted to be the direction in the neighborhood D where the local direction is most prominent, otherwise the local direction of tile B (i, j) remains unchanged;

(13) filtering the preprocessed image by using an M-PCNN network;

(14) and (3) carrying out feature extraction on the fingerprint image, wherein the feature extraction comprises global feature extraction and detail feature extraction: the global feature extraction is a Fourier spectrum feature: firstly, an input fingerprint image is divided into 32 multiplied by 32 image blocks, and two-dimensional discrete Fourier transform is carried out on the image blocks, wherein the formula is

In the formula, (m, n) is a value domain coordinate of a pixel, and (i, k) is a frequency domain coordinate corresponding to the pixel, after Fourier transformation is carried out on each sub-image block, a Fourier frequency spectrum of a full image is obtained, and the Fourier frequency spectrum is quantized into a global feature vector of a fingerprint according to rules;

and detail feature extraction, including a core point, a bifurcation point and an end point, wherein the detail feature extraction templates are respectively as follows:

P₃ P₂ P₁ P₄ P P₀ P₅ P₆ P₇

wherein,

and is

If m is 1, the P is an end point, otherwise, the P is a branch point.

7. A method of controlling a trimodal biometric bearer authentication system according to claim 5, wherein the fingerprint processing includes the steps of:

(21) sampling the gray-scale face image to obtain shape vectors and contour point characteristic information;

(22) and establishing an ASM model of the human face.

8. The method as claimed in claim 5, wherein the minutiae extraction step includes extracting a minutiae value by a poincare method based on a fingerprint direction field, and the value is obtained by extracting the minutiae value by a poincare method based on the fingerprint direction field

Wherein Δ (k) is in accordance with

δ(k)＝O′(Ψ_x(i′)，Ψ_y(i′))-O′(Ψ_x(i)，Ψ_y(i))，

i′＝(i+1)mod N_Ψ，

After the calculation is finished, judging the value of P (i, j), and if the value of P (i, j) is 0.5, determining the value as a core point; the extraction of the bifurcation point and the end point is obtained by detecting through a sliding template method, firstly, a pixel structure template of the bifurcation point and the end point is defined, and the template traverses the whole graph from left to right and from top to bottom respectively; and calculating the coincidence degree of the template and the corresponding area of the image every time the template slides once, and judging that the current pixel of the image is consistent with the detail characteristics corresponding to the used template when the coincidence value is greater than 0.7, namely if the coincidence value is consistent with the bifurcation template, the pixel is a bifurcation point, and if the coincidence value is consistent with the endpoint template, the pixel is an endpoint.

9. The method of claim 5, wherein the fingerprint comparison comprises the steps of:

(31) fingerprint alignment: firstly, aligning core points, and after the alignment, transforming the coordinates of a Vi point set as follows:

x′_i＝x_i+(x_to-x_io) And y'_i＝y_i+(y_to-y_io)，

Wherein (x)_to，y_to) And (x)_io，y_io) Respectively template fingerprints and core points of input fingerprints; then, the alignment is rotated, and the rotational alignment is obtained by solving a reflection transformation t:

in the above transformation, the core points are aligned, Δ x and Δ y have been determined, and a is obtained by making the expression | v_t-t(v_i)|²When theta is satisfied, the maximum point number of Vi and Vt matching points is obtained, wherein theta is a set threshold and is set to be 0.05;

(32) and (3) similarity calculation: for global feature matching, the features of the input fingerprint and the template fingerprint are Fourier spectrum features, feature directionsThe quantity is the serialization result of the spectral image pixels, and the similarity between the two is realized by calculating the Euclidean distance between unequal length vectors, wherein the similarity is calculated according to the following formula:

in the formula, dis is the Euclidean distance value of the two, max (v)_i，v_t) Representing the maximum vector length in the input vector and the template vector; for detail feature matching, after an input point set Vi is projected onto a template point set Vt according to (Δ x, Δ y, a) transformation with respect to a template, the number of all matched point pairs is determined within a distance error of less than 5 pixel positions, and a similarity value at this time is calculated according to the following formula:

in the formula, n_coupleIndicates the number of successfully paired points, max (v)_i，v_t) Still represent the maximum number of minutiae in the input point set and template point set;

(33) fingerprint verification: after similarity data S between the input fingerprint and the template fingerprint is calculated, the S and a preset identification threshold value T are calculated_sComparison, here T_sIs adjusted according to different safety requirements, if S is more than or equal to T_sIt is determined that the two fingerprints match successfully, i.e. from the same finger, otherwise it is determined that the two fingerprints are not from the same finger.

10. The method of claim 5, wherein the face feature comparison comprises the steps of:

(41) detecting a face image in a video sequence by using an average value Viola Jones face monitor, and automatically completing the labeling of 68 feature points;

(42) normal direction of each feature point in the face imageTaking 7 points from the center of the feature point (namely 2K +1 points in total, and K is 7), forming a vector gi by the 15 points, forming a group of vector sequences g by the vectors gi corresponding to 68 feature points on the face image₁、g₂、...、g₆₈；

(43) The vector sequence gi is normalized and the mean g and covariance Sg are calculated.

(44) Iteratively according to the formula f (g)_s)＝(g_s-g)^TS_g ^-1(g_s-g) calculating a mahalanobis distance value of the face feature template corresponding to the identity card number in the input sample and the database;

(45) if the minimum Mahalanobis distance is smaller than a given threshold value T, the face feature comparison is successful, otherwise the feature comparison is failed, namely the current licensee is not the same as the feature registrant.

Images