JP6771361B2 - Authentication method, authentication device and learning method - Google Patents

Description

The present invention relates to an authentication method, an authentication device and a learning method.

Recently, user authentication has been performed by various methods such as face recognition and fingerprint recognition. The recognition device used for user authentication has a structure similar to that of a human neural network. The more complicated the internal structure of the recognition device, the better the recognition performance, but the output of the recognition device is delayed. For example, as the number of layers included in the recognition device increases, the output of the recognition device will be delayed.

An object of the present invention is to provide an authentication method, an authentication device, and a learning method.

The authentication method according to one aspect includes a step of providing an input image to a recognition device including a plurality of layers, and at least one feature value of the input image output by at least one intermediate layer in the recognition device. It includes a step of authenticating the input image based on at least one feature value of the registered image corresponding to the intermediate layer.

The step of authenticating the input image may include a step of determining acceptance or rejection of the input image. The plurality of layers can be layers of a deep learning network. The at least one intermediate layer is a plurality of intermediate layers, and the step of authenticating the input image is a feature value of the input image output by each intermediate layer and a feature of the registered image corresponding to each intermediate layer. It may include a step of determining the success or failure of the authentication of the input image based on the value.

The step of authenticating the input image determines the similarity based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer. The step of performing the authentication of the input image based on the similarity and the threshold value may be included. The step of authenticating the input image may include a step of determining acceptance of the input image when the similarity exceeds the similarity corresponding to the false acceptance rate for the intermediate layer.

The step of authenticating the input image may include determining rejection of the input image if the similarity is less than the similarity corresponding to the authentication rate for the intermediate layer. The step of authenticating the input image is next to the recognition device when the similarity exceeds the similarity corresponding to the authentication rate for the intermediate layer and is less than the similarity corresponding to the false acceptance rate for the intermediate layer. It may further include steps to proceed to the layer.

The threshold value may include a first threshold value determined by the false acceptance rate and a second threshold value determined by the authentication rate. The first threshold is larger than the second threshold.

The step of authenticating the input image determines the distance based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer. It may include a step and a step of authenticating the input image based on the distance and the threshold. The step of authenticating the input image may include a step of determining acceptance of the input image when the distance is less than the distance corresponding to the false acceptance rate for the intermediate layer.

The step of authenticating the input image may include determining rejection of the input image if the distance exceeds a distance corresponding to the authentication rate for the intermediate layer. The step of authenticating the input image proceeds to the next layer of the recognition device when the distance exceeds the distance corresponding to the false acceptance rate for the intermediate layer and is less than the distance corresponding to the authentication rate for the intermediate layer. It may include additional steps to do. The threshold value may include a first threshold value determined by the false acceptance rate and a second threshold value determined by the authentication rate. The first threshold is larger than the second threshold.

The step of authenticating the input image is to authenticate the input image based on the feature value of the input image output by the final layer in the recognition device and the feature value of the registered image corresponding to the final layer. It may further include steps to determine success or failure. In the authentication method, a step of providing the registered image to the recognition device, and at least one feature value output by the intermediate layer and at least one feature value output by the final layer are set as feature values of the registered image. It may further include a step to store as. The authentication method may further include the step of receiving the registered image from the camera. The at least one intermediate layer may include a plurality of intermediate layers among the plurality of layers. The plurality of layers may include at least one convolution layer and a pooling layer.

The learning method according to one aspect includes a step of learning a recognition device including a plurality of layers, a step of determining a first threshold value for an intermediate layer in the recognition device based on a predetermined performance index, and the performance. It includes a step of determining a second threshold for the final layer in the recognizer based on the index.

The learning method includes a step of providing verification data to the recognition device, a step of determining a first similarity between intermediate feature values output by the intermediate layer, and a final feature value output by the final layer. Further, a step of determining the second similarity between, a step of determining the first threshold value based on the first similarity degree, and a step of determining the second threshold value based on the second similarity degree. Can include.

The learning method includes a step of providing verification data to the recognition device, a step of determining a first distance between intermediate feature values output by the intermediate layer, and a final feature value output by the final layer. It may further include a step of determining the second distance between, a step of determining the first threshold based on the first distance, and a step of determining the second threshold based on the second distance.

The first threshold and the second threshold may be for acceptance or rejection of the input image.

The step of determining the first threshold value determines the first threshold value based on the similarity between the first intermediate feature value output by the intermediate layer and the second intermediate feature value output by the intermediate layer. May include steps to do.

The step of determining the first threshold value determines the first threshold value based on the distance between the first intermediate feature value output by the intermediate layer and the second intermediate feature value output by the intermediate layer. May include steps.

The step of training the recognition device may include the output of the intermediate layer for the training sample, the output of the final layer for the training sample, and the step of training the recognition device based on the label value of the training sample.

The step of learning the recognition device includes an intermediate feature value output by the first network to which the output of the intermediate layer is input, a final feature value output by the second network to which the output of the final layer is input, and a final feature value. It may include a step of training the recognition device based on the label of the training sample.

The authentication device according to one aspect includes a recognition device that outputs at least one feature value of an input image by at least one intermediate layer among a plurality of layers, and at least one of the input images output by the at least one intermediate layer. It includes an authentication processing unit that authenticates the input image based on one feature value and at least one feature value of the registered image corresponding to the intermediate layer.

The authentication processing unit may decide to accept or reject the input image. The plurality of layers can be layers of a deep learning network. The at least one intermediate layer can be a plurality of intermediate layers.

The authentication processing unit can determine the success or failure of authentication of the input image based on the feature value of the input image output by each intermediate layer and the feature value of the registered image corresponding to each intermediate layer. The authentication processing unit determines the degree of similarity based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer, and determines the similarity. The input image can be authenticated based on the degree and the threshold value.

The authentication processing unit determines the distance based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer, and determines the distance and the distance. The input image can be authenticated based on the threshold value. The authentication processing unit provides the registered image to the recognition device, and uses at least one feature value output by the intermediate layer and at least one feature value output by the final layer as feature values of the registered image. Can be stored.

According to the present invention, it is possible to provide an authentication method, an authentication device, and a learning method.

It is a block diagram which shows the authentication apparatus which concerns on one Embodiment. It is a block diagram which shows the authentication processing part which concerns on one Embodiment. It is a block diagram which shows the recognition apparatus which includes a plurality of layers which concerns on one Embodiment. It is a block diagram which shows the generation process of the feature value of the registered image which concerns on one Embodiment. It is a figure explaining a plurality of intermediate layers which concerns on one Embodiment. It is a figure explaining the threshold value by the performance index which concerns on one Embodiment. It is a figure explaining the threshold value by the performance index which concerns on one Embodiment. It is a figure explaining the relationship between the form of the graph which concerns on one Embodiment, and the recognition device performance. It is a flowchart which shows the operation of the authentication processing part which concerns on one Embodiment. It is a block diagram which shows the learning apparatus and recognition apparatus which concerns on one Embodiment. It is a block diagram which shows the determination process of the threshold value which concerns on one Embodiment. It is a block diagram which shows the learning process of the recognition device which concerns on one Embodiment. It is a block diagram which shows the learning process of the recognition device which concerns on another embodiment. It is a block diagram which shows the learning process of a plurality of recognition devices which concerns on one Embodiment. It is a block diagram which shows the learning process of the recognition device which concerns on further embodiment. It is a flowchart which shows the determination process of the threshold value which concerns on one Embodiment. It is a flowchart which shows the authentication process which concerns on one Embodiment. It is a block diagram which shows the internal structure and the authentication process of the recognition device which concerns on one Embodiment. It is a block diagram which shows the internal structure of the recognition device which concerns on embodiment, and the learning process of a recognition device. It is a block diagram which shows the electronic device which concerns on one Embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The same reference numerals presented in each drawing indicate the same elements. The embodiments described below may be used in the user interface, for example, for personal authentication through a camera.

FIG. 1 is a block diagram showing an authentication device according to an embodiment. Referring to FIG. 1, the authentication device 100 includes a recognition device 110 and an authentication processing unit 120. The recognition device 110 and the authentication processing unit 120 can be embodied by one or more hardware modules, one or more software modules, or various combinations thereof.

The recognition device 110 receives an input image (for example, a user's face image or a fingerprint image) and outputs a feature value corresponding to the input image. The recognition device 110 includes a plurality of layers for extracting feature values from an input image. The plurality of layers may be layers of a deep learning (DL) network structure, and the input image may be received from a camera, a communication module, a memory, and the like.

The authentication processing unit 120 outputs an authentication result based on the feature value output by the recognition device 110. For example, the authentication processing unit 120 determines whether or not the face of the input image is the face of the registered user based on the feature value output by the recognition device 110, and determines the authentication result based on the determination result. Can be output. The authentication result includes, for example, acceptance indicating authentication success and rejection indicating authentication failure as information indicating whether or not the input image matches the registered image. The authentication result is transmitted to an encryption application or the like, and is used for, for example, online payment and unlocking of a terminal.

As will be described in detail later, the authentication device 100 performs early authentication on the input image. The early authentication is executed based on the feature value output by the first layer or the intermediate layer among the plurality of layers included in the recognition device 110, and includes an early acceptance or an early rejection. Here, the first layer and the intermediate layer include the remaining layers other than the final layer among the plurality of layers. In the following, the intermediate layer means the remaining layers of the plurality of layers excluding the final layer. Therefore, if early authentication by the last intermediate layer fails, the final layer can be used for authentication.

Faster authentication can improve authentication speed. For example, in the case of the recognition device 110 including a plurality of layers, the output value of the intermediate layer is generated first and can be used to generate the output value of the final layer. Therefore, by appropriately using the output value of the intermediate layer, the authentication speed is improved as compared with the case where only the output value of the final layer is used. Depending on the embodiment, a specified number of intermediate layers may be utilized for early authentication. For example, odd or even layers can be used for fast authentication. The specified number of layers is based on the learning process or network structure.

The recognition device 110 is pre-learned by the training sample, and after learning, a threshold value for early authentication is determined. The specific learning process and the threshold determination process for quick authentication will be described later.

FIG. 2 is a block diagram showing an authentication processing unit according to an embodiment. With reference to FIG. 2, the authentication processing unit 120 determines the success or failure of authentication for the input image based on the distance between the input image and the registered image. The registered image means an image that serves as a reference for authentication. For example, the authentication processing unit 120 receives the feature values of the plurality of input images and the feature values of the registered images, and calculates the distance between the feature values of the input images and the feature values of the registered images. The more similar the input image and the registered image are, the smaller the distance between the feature value of the input image and the feature value of the registered image is calculated. The more different the input image and the registered image are, the smaller the feature value of the input image and the registered image are. The distance from the feature value of the image is calculated to be large.

The authentication processing unit 120 can determine the success or failure of authentication for the input image by comparing the calculated distance with the threshold value, and output the authentication result. For example, when the calculated distance is smaller than the threshold value, the authentication processing unit 120 determines that the input image is sufficiently similar to the registered image to authenticate the user, and outputs an acceptance indicating authentication success. On the other hand, when the calculated distance is larger than the threshold value, the authentication processing unit 120 outputs a rejection indicating an authentication failure.

Before explaining the fast authentication operation, the feature value of the input image and the feature value of the registered image will be described. The feature value of the input image is transmitted from the recognition device 110 shown in FIG. 1, and the feature value of the registered image is transmitted from the storage unit (not shown). The feature value of the registered image is stored in advance in the storage unit in the registration step. Depending on the embodiment, the feature value of the registered image may also be transmitted from the recognition device. In this case, the feature value of the registered image is extracted in real time from the registered image using the recognition device.

The feature value of the input image and the feature value of the registered image can be classified by the layer of the recognition device. For example, when an input image is input to the recognition device, the feature value output by the intermediate layer of the recognition device is called the intermediate feature value of the input image, and the feature value output by the final layer of the recognition device is the final of the input image. It is called a feature value. Similarly, when the registered image is input to the recognition device, the feature value output by the intermediate layer is called the intermediate feature value of the registered image, and the feature value output by the final layer is called the final feature value of the registered image. To. Further, the distance corresponding to the intermediate layer is referred to as an intermediate distance, and the distance corresponding to the final layer is referred to as the final distance. The intermediate distance is determined based on the intermediate feature value of the input image and the intermediate feature value of the registered image, and the final distance is determined based on the final feature value of the input image and the final feature value of the registered image. Intermediate distances can be used for fast authentication.

The threshold used for authenticating the input image is determined based on a predetermined performance index. The performance index includes an index related to the false acceptance rate (FAR) and an index related to the certification rate (VR). FAR indicates the ratio of erroneously recognizing that the object of the registered image and the object of the input image are the same even if the object of the registered image and the object of the input image are different from each other. VR indicates the ratio of correctly recognizing that the object of the registered image and the object of the input image are the same when the object of the registered image and the object of the input image are the same. For example, when the threshold value is determined based on FAR, the authentication processing unit 120 determines whether the distance between the input image and the registered image is less than the threshold value and accepts the input image. When the threshold value is determined based on VR, the authentication processing unit 120 determines whether the distance between the input image and the registered image exceeds the threshold value and rejects the input image.

The threshold is determined for each of the plurality of layers of the recognition device. In other words, each of the plurality of layers has a threshold value for determining the success or failure of authentication of the input image. For example, an intermediate threshold (distance of the intermediate threshold) is determined for the intermediate layer, and a final threshold (distance of the final threshold) is determined for the final layer. The authentication processing unit 120 determines the success or failure of authentication of the input image based on the intermediate distance and the intermediate threshold value (distance of the intermediate threshold value). For example, the authentication processing unit 120 determines the success or failure of authentication of the input image by comparing the intermediate distance with the intermediate threshold value (distance of the intermediate threshold value). As mentioned above, the intermediate distance can be used for early authentication to determine early acceptance or early rejection of the input image. Hereinafter, the performance index for the intermediate layer is referred to as FAR1 and VR1, and the performance index for the final layer is referred to as FAR2 and VR2. FAR1 and FAR2 and VR1 and VR2 can be determined independently of each other.

According to one aspect, the threshold determined based on FAR1 is determined as the distance corresponding to FAR1 and can be used for early acceptance. For example, the authentication processing unit 120 determines early acceptance of the input image when the intermediate distance is less than the distance corresponding to FAR1. Also, the threshold determined based on VR1 is determined as the distance corresponding to VR1 and can be used for early rejection. For example, the authentication processing unit 120 determines an early rejection of the input image when the intermediate distance exceeds the distance corresponding to VR1. If the input image satisfies the conditions for such fast authentication, the authentication procedure can be completed without further additional processing on the input image, so that the authentication speed can be improved.

If the authentication processing unit 120 fails in early authentication, the authentication processing unit 120 can subsequently improve the authentication. For example, if the intermediate distance exceeds the distance corresponding to FAR1 and is less than the distance corresponding to VR1, the authentication processing unit 120 may fail in early authentication. In this case, the authentication processing unit 120 can improve the authentication for the input image based on the final distance. The authentication processing unit 120 determines the success or failure of authentication of the input image by comparing the final distance with the final threshold value (distance of the final threshold value). For example, the authentication processing unit 120 determines acceptance of the input image when the final distance is less than the distance corresponding to FAR2. Further, the authentication processing unit 120 determines to reject the input image when the final distance exceeds the distance corresponding to VR2.

If authentication based on the final distance fails, a policy-based decision proceeds. For example, authentication can be terminated due to authentication failure, authentication can be improved based on a new input image, or authentication can be attempted again based on a conventional input image.

FIG. 3 is a block diagram showing a recognition device including a plurality of layers according to an embodiment. Referring to FIG. 3, the recognition device 110 according to one embodiment includes an input layer 111, an intermediate layer 112, and a final layer 113. The input layer 111, the intermediate layer 112, and the final layer 113 are embodied by one or more hardware modules, one or more software modules, or a combination thereof. The input layer 111, the intermediate layer 112, and the final layer 113 are pre-learned using the training sample. The specific learning process will be described later.

The input layer 111 receives the input image, extracts the feature value from the input image, and transmits the extracted feature value to the intermediate layer 112. The intermediate layer 112 extracts and outputs the intermediate feature value of the input image based on the feature value received from the input layer 111. Although the intermediate layer 112 is shown in one block in FIG. 3, the intermediate layer 112 may include a plurality of layers. The intermediate layer 112 transmits the intermediate feature value of the input image to the final layer 113. The final layer 113 extracts and outputs the final feature value of the input image based on the feature value received from the intermediate layer 112. The final layer 113 and the other intermediate layers can be embodied so as not to perform additional authentication operations if the fast authentication is successful by either the input layer 111 or the intermediate layer 112.

Although the input layer 111 and the intermediate layer 112 are shown separately in FIG. 3 for convenience of explanation, the intermediate layer 112 may include the input layer 111. In this case, the feature values output by the input layer 111 can also be used for fast authentication.

FIG. 4 is a block diagram showing a process of generating feature values of the registered image according to the embodiment. Referring to FIG. 4, the recognition device 110 according to the embodiment outputs the feature value of the registered image based on the registered image. The registered image (for example, the user's face image for authentication) is received from a camera, a communication module, a memory, and the like. For example, the feature value of the registered image is stored in the memory 130 in advance before the authentication procedure is executed.

As described above, the feature value of the registered image is generated corresponding to a plurality of layers like the feature value of the input image. For example, the intermediate layer generates the intermediate feature value of the registered image, and the final layer generates the final feature value of the registered image. The memory 130 stores the intermediate feature value of the registered image corresponding to the intermediate layer, and stores the final feature value of the registered image corresponding to the final layer. Further, the memory 130 stores the intermediate threshold value corresponding to the intermediate layer and stores the final threshold value corresponding to the final layer. According to one aspect, the memory 130 stores the feature value of the registered image and transmits the feature value of the registered image to the authentication processing unit in response to the execution of the authentication. According to another aspect, the feature value of the registered image can be extracted in real time by the recognition device 110 and transmitted to the authentication processing unit without being stored in the memory 130 in advance. In this case, the memory 130 can also store the registered image itself instead of the feature value of the registered image.

FIG. 5 is a diagram illustrating a plurality of intermediate layers according to an embodiment. Referring to FIG. 5, the recognition device 115 according to one embodiment includes an input layer 116, an intermediate layer 117, and a final layer 118, and the intermediate layer 117 includes a first intermediate layer and a second intermediate layer to an nth intermediate layer. Including. Although not shown in FIG. 5, the input layer 116 outputs a feature value depending on the embodiment, and the feature value can be compared with the feature value of the registered image stored in the memory 130.

The input layer 116 extracts feature values from the input image and transmits the extracted feature values to the intermediate layer 117. The intermediate layer 117 includes a plurality of layers, and each of the plurality of layers extracts and outputs intermediate feature values. Specifically, the first intermediate layer extracts and outputs the first intermediate feature value of the input image based on the feature value received from the input layer 116. The second intermediate layer extracts and outputs the second intermediate feature value of the input image based on the first intermediate feature value of the input image. The nth intermediate layer extracts and outputs the nth intermediate feature value of the input image based on the (n-1) th feature value. A plurality of layers may be included between the second intermediate layer and the nth intermediate layer. The final layer 118 extracts and outputs the final feature value of the input image based on the nth intermediate feature value of the input image.

The memory 130 stores in advance the feature values of the registered images corresponding to the plurality of layers of the recognition device 115. The feature value of the registered image can be generated in advance by the recognition device 115 and stored in the memory 130. For example, in the memory 130, the first intermediate feature value of the registered image corresponding to the first intermediate layer, the second intermediate feature value of the registered image corresponding to the second intermediate layer, and the second registered image corresponding to the nth intermediate layer. n Intermediate feature values and final feature values of the registered image corresponding to the final layer 118 can be stored in advance.

The feature values of the input images output from the intermediate layer 117 and the final layer 118 are sequentially compared with the feature values of the registered image, and the feature values of the input image are compared according to the distance between the feature values of the input image and the feature values of the registered image. The success or failure of early or final certification can be determined.

Although FIG. 5 shows the input layer 116 and the intermediate layer 117 separately for convenience of explanation, the intermediate layer 117 may include the input layer 116. In this case, the input layer 116 operates as the first intermediate layer.

6A and 6B are diagrams for explaining the threshold value according to the performance index according to the embodiment. With reference to FIG. 6A, a graph for the same object and a graph for non-identical objects are shown. The recognition device learned in advance is provided with verification data for the same object to acquire a graph for the same object, and the recognition device is provided with verification data for non-identical objects to acquire a graph for non-identical objects. Here, the x-axis indicates the distance between feature values, and the y-axis indicates the number of sample pairs.

For example, if the distance between the feature value extracted from the first sample for object A and the feature value extracted from the second sample for object A is calculated, the number of sample pairs of the corresponding distance in the graph for the same object is calculated. Will increase. Further, if the distance between the feature value extracted from the first sample for the object A and the feature value extracted from the first sample for the object B is calculated, the sample pair of the corresponding distance is shown in the graph for the non-identical object. The number increases.

Graphs for the same object and graphs for non-identical objects all have the form of a normal distribution. Authentication of the input image is accepted when the distance between the input image and the registered image is included in the distribution for the same object, and input when the distance between the input image and the registered image is included in the distribution for non-identical objects. Image authentication is rejected. However, if the distance between the input image and the registered image is included in the distribution for the same object and the distribution for non-identical objects, an appropriate judgment is required.

The threshold can be determined based on the graph shown in FIG. 6A and a predetermined performance index. Performance indicators include indicators related to false acceptance rate (FAR) and indicators related to certification rate (VR). For example, FAR is set to 1% and VR is set to 100%. In this case, the distance α corresponding to 1% FAR and the distance β corresponding to 100% VR are the threshold values. Therefore, if the distance between the feature value of the input image and the feature value of the registered image is less than the distance α, the input image is accepted, and the distance between the feature value of the input image and the feature value of the registered image is the distance. If it exceeds β, the input image is rejected.

With reference to FIG. 6B, similarity may be used instead of distance. The distance between feature values and the degree of similarity between feature values are inversely proportional. Even if similarity is used instead of distance, the threshold can be determined based on the index for FAR and the index for VR. In this case, if the similarity between the feature value of the input image and the feature value of the registered image exceeds the threshold similarity with respect to FAR, the input image is accepted and between the feature value of the input image and the feature value of the registered image. The input image is rejected if the similarity of is less than the threshold similarity for VR. According to one embodiment, FAR is set to 1% and VR is set to 100%.

Hereinafter, embodiments using distances for consistency of description will be described, but embodiments described below may be modified or extended to utilize similarity instead of distances.

FIG. 7 is a diagram for explaining the relationship between the form of the graph and the performance of the recognition device according to the embodiment. With reference to FIG. 7, a graph 10 corresponding to a lower layer and a graph 20 corresponding to a higher layer are illustrated. The higher the layer, the farther away the center of the graph for the same object and the center of the graph for non-identical objects. In other words, the overlapping area between graphs will be reduced. The reduction of overlapping areas means an increase in authentication accuracy.

As explained above, faster authentication increases authentication speed. The graph 10 corresponding to the lower layer has lower authentication accuracy than the graph 20 corresponding to the higher layer, but early acceptance and early rejection are determined based on the threshold value of the lower layer. For example, if the distance between the feature value of the input image and the feature value of the registered image is less than the threshold value α1 by FAR, early acceptance is determined, and the distance between the feature value of the input image and the feature value of the registered image. Early rejection can be determined if is above the VR threshold β1. In this case, the authentication speed is increased because it is not necessary to execute the calculation of the higher layer. If the distance between the feature value of the input image and the feature value of the registered image is between the threshold value α1 by FAR and the threshold value β1 by VR, the high layer calculation can be performed to execute the authentication.

FIG. 8 is a flowchart showing the operation of the authentication processing unit according to the embodiment. Referring to FIG. 8, in step S510, the authentication processing unit receives the intermediate feature value. The intermediate feature value includes the intermediate feature value of the input image and the intermediate feature value of the registered image. In step S520, the authentication processing unit determines the distance X ₁ of the intermediate feature value. Distance X ₁ of intermediate feature value is calculated as the distance between the intermediate characteristic value of the intermediate characteristic value and the registered image of the input image.

In step S530, the authentication processing unit compares the distance _{X 1} threshold α1 and .beta.1. The threshold α1 can be determined by FAR1 and the threshold β1 can be determined by VR1. When the distance X ₁ is outside the range of the threshold values α1 and β1, the authentication processing unit performs step S540. In step S540, the authentication processing unit determines the success or failure of the authentication, in other words, whether the input image is authenticated or rejected. The authentication in step S540 means the early authentication described above. The success or failure of the authentication is determined in step S540, and the authentication speed is increased by completing the authentication procedure.

When the distance X ₁ is within the thresholds α1 and β1, the authentication processing unit performs step S550. In step S550, the authentication processing unit receives the final feature value. The final feature value includes the final feature value of the input image and the final feature value of the registered image. In step S560, the authentication processing unit determines the distance X ₂ of the final feature value. The distance X ₂ of the final feature value is calculated as the distance between the final feature value of the input image and the final feature value of the registered image.

In step S570, the authentication processing unit compares the distance _{X 2} threshold α2 and .beta.2. The threshold α2 can be determined by FAR2 and the threshold β2 can be determined by VR2. The FAR2 and VR2 that determine the thresholds α2 and β2 of the final layer are different from the FAR1 and VR1 that determine the thresholds α1 and β1 of the intermediate layer, but the same FAR and VR may be used regardless of the layer. In addition, the FAR value and the VR value can be increased or decreased as the number of layers increases. When the distance X ₂ is outside the range of the threshold values α2 and β2, in step S580, the authentication processing unit determines the success or failure of the authentication. The authentication processing unit determines acceptance or rejection of the input image. When the distance X ₂ is within the thresholds α2 and β2, the authentication processing unit determines the authentication failure in step S590. Although not shown in the drawing, after the authentication failure is determined, step S510 may be executed again for the conventional input image or step S510 may be executed for the new input image according to the decision based on the policy. , The certification procedure ends as it is.

FIG. 9 is a block diagram showing a learning device and a recognition device according to an embodiment. Referring to FIG. 9, the learning device 300 according to one embodiment includes a learning device 310 and a threshold value determining unit 320. The learner 310, the threshold value determination unit 320, and the recognition device 400 are realized by one or more hardware modules, one or more software modules, or a combination thereof.

The learning device 310 trains the recognition device 400 based on the learning sample. The learner 310 adjusts the parameters of the recognition device 400 based on the output value of the recognition device 400 for the training sample. The threshold value determination unit 320 determines the threshold value using the learned recognition device 400. The threshold value determination unit 320 can acquire a graph as shown in FIG. 7 based on the output value of the recognition device 400 for the verification data. The threshold value determination unit 320 determines the threshold value for each layer.

FIG. 10 is a block diagram showing a threshold value determination process according to an embodiment. Referring to FIG. 10, the threshold value determination unit 325 according to the embodiment determines the threshold value based on the feature values received from the learned recognition devices 420 and 430. Although the recognition device 420 and the recognition device 430 are shown independently of each other for convenience of explanation, the recognition device 420 and the recognition device 430 may be the same recognition device.

The recognition device 420 includes an input layer 421, an intermediate layer 422, and a final layer 423, and the recognition device 430 includes an input layer 431, an intermediate layer 432, and a final layer 433. Based on the verification data, the intermediate layer 422 outputs the intermediate feature value, and the final layer 423 outputs the final feature value. Further, based on the verification data, the intermediate layer 432 outputs the intermediate feature value, and the final layer 433 outputs the final feature value. The validation data includes data on the same object and data on non-identical objects.

The threshold determination unit 325 acquires the graph 326 of the first distribution by comparing the intermediate feature values. Further, the threshold value determination unit 325 acquires the graph 327 of the second distribution by comparing the final feature values. As for graphs 326 and 327, the contents described with reference to FIGS. 6 and 7 can be applied.

The threshold value determination unit 325 determines the threshold value based on a predetermined performance index. The threshold determination unit 325 determines the intermediate threshold for the intermediate layers 422 and 432 based on the graph 326 of the first distribution, and determines the final threshold for the final layers 423 and 433 based on the graph 327 of the second distribution. To do. For example, the intermediate threshold includes a threshold α1 determined by FAR1 and a threshold β1 determined by VR1. The final threshold includes the threshold α2 determined by FAR2 and the threshold β2 determined by VR2.

FIG. 11 is a block diagram showing a learning process of the recognition device according to the embodiment. Referring to FIG. 11, the recognition device 410 according to one embodiment includes an input layer 411, an intermediate layer 412, and a final layer 413. The recognition device 410 outputs the feature value for the training sample. The learner provides a learning sample to the recognition device 410, and the recognition device 410 can be trained based on the feature value output from the recognition device 410. The training sample may consist of images for authentication. For example, the training sample may consist of a face image and may include a label that identifies the face. The learner can repeatedly train the recognition device 410 so that the feature value output in the final layer 413 of the recognition device 410 corresponds to the label of the learning sample.

FIG. 12 is a block diagram showing a learning process of the recognition device according to another embodiment. Referring to FIG. 12, the recognition device 440 according to another embodiment includes an input layer 441, an intermediate layer 442, and a final layer 443. The intermediate layer 442 outputs the intermediate feature value, and the final layer 443 outputs the final feature value. The recognition device 440 is learned based on the intermediate feature value and the final feature value. For example, the learner can train the recognition device 440 so that not only the final feature value but also the intermediate feature value corresponds to the label of the training sample. By learning the recognition device 440 in consideration of the intermediate feature value, the probability or frequency of early authentication increases.

FIG. 13 is a block diagram showing a learning process of a plurality of recognition devices according to an embodiment. According to one aspect, the learner may train a plurality of recognition devices. The plurality of recognition devices include a first recognition device 420 and a second recognition device 430. The first recognition device 420 includes an input layer 421, an intermediate layer 422, and a final layer 423, and the second recognition device 430 includes an input layer 431, an intermediate layer 432, and a final layer 433. The learner may train the first recognition device 420 and the second recognition device 430 based on the learning sample for the same object and the learning sample for the non-identical object. The learner first performs the learning sample for the same object so that the distance between the first feature value output from the first recognition device 420 and the second feature value output from the second recognition device 430 is small. The recognition device 420 and the second recognition device 430 can be trained. Further, the learner increases the distance between the first feature value output from the first recognition device 420 and the second feature value output from the second recognition device 430 for the learning sample for non-identical objects. , The first recognition device 420 and the second recognition device 430 can be trained.

FIG. 14 is a block diagram showing a learning process of the recognition device according to a further embodiment. Referring to FIG. 14, the recognition device 450 according to a further embodiment includes an input layer 451, an intermediate layer 452, a final layer 454, a first network 453, and a second network 455. The input layer 451, the intermediate layer 452, the final layer 454, the first network 453, and the second network 455 are realized by one or more hardware modules, one or more software modules, or a combination thereof.

The recognition device 450 outputs an intermediate feature value and a final feature value based on the training sample. The output of the intermediate layer 452 is input to the first network 453, and the first network 453 outputs the intermediate feature value. Further, the output of the final layer 453 is input to the second network 455, and the second network 455 outputs the final feature value. The first network 453 and the second network 455 may have a structure that increases the possibility of early authentication. For example, the first network 453 and the second network 455 include a convolution layer, a fully connected network, and the like, respectively.

The learner may train the intermediate layer 452 so that the intermediate feature values correspond to the labels of the training sample. Further, the learner can train the intermediate layer 452 and the final layer 453 so that the final feature value corresponds to the label of the training sample. Further, the learner can learn the first network 453 and the second network 455 together. The recognition device 450, which includes the first network 453 and the second network 455 and is learned based on the intermediate feature value and the final feature value, can increase the possibility of early authentication.

FIG. 15 is a flowchart showing a threshold value determination process according to an embodiment. With reference to FIG. 15, in step S710, a recognition device including a plurality of layers is learned. Step S710 can be performed by the learner described above. In step S720, the verification data is provided to the recognition device. In step S730, the distance to the intermediate feature value output by the intermediate layer is determined. In step S740, the distance from the final feature value output by the final layer is determined. In step S750, an intermediate threshold for the intermediate layer in the recognition device is determined based on a predetermined performance index. In step S760, the final threshold for the final layer in the recognition device is determined based on the performance index. The performance index may be set differently depending on the intermediate layer and the final layer, or may be set the same regardless of the layer. Steps S720 to S760 can be performed by the threshold determination unit described above.

FIG. 16 is a flowchart showing an authentication process according to an embodiment. Referring to FIG. 16, in step S910, the input image is provided to a recognition device that includes a plurality of layers. Step S910 can be performed by the recognition device described above. In step S920, the distance is determined based on the feature value of the input image output by the intermediate layer in the recognition device and the feature value of the registered image corresponding to the intermediate layer. In step S930, the success or failure of authentication of the input image is determined by comparing the distance with the threshold value of the intermediate layer. Step S920 and step S930 can be executed by the authentication processing unit described above.

FIG. 17 is a block diagram showing an internal structure and an authentication process of the recognition device according to the embodiment. Referring to FIG. 17, recognition device 140 includes a plurality of layers 141-145. Each of the plurality of layers 141 to 145 includes a convolution layer and a pooling layer. Each layer can be repeatedly configured according to the purpose. For example, the output of the convolution layer 11 may be transmitted to the convolution layer 12, and the output of the convolution layer 12 may be transmitted to the pooling layer 1. The above-mentioned layer configuration is merely an example, and the layer configuration according to the embodiment can be changed in various ways. The registered image is pooled into the registered pooling 1 feature to the registered pooling 5 feature. The registered pooling 1 feature to the registered pooling 5 feature are provided to the authentication processing unit 150. The authentication processing unit 150 can determine the success or failure of the authentication by sequentially comparing the features received from the plurality of layers 141 to 145 and the registered pooling 1 feature to the registered pooling 5 feature. For example, the authentication processing unit 150 can determine the success or failure of early authentication by comparing the distance between the output of layer 141 and the registered pooling 1 feature with the first threshold value.

FIG. 18 is a block diagram showing the internal structure of the recognition device and the learning process of the recognition device according to the embodiment. Referring to FIG. 18, the recognition device 470 includes a plurality of layers 471, 473, 475, 477 and 479. Each of the plurality of layers 471, 473, 475, 477 and 479 includes a convolution layer and a pooling layer. Also, at least a portion of the plurality of layers 471, 473, 475, 477 and 479 is connected to the additional networks 472, 474, 476 and 478. Additional networks 472, 474, 476 and 478 include a convolution layer (Conv. K in FIG. 18) and a fully connected network (FCk in FIG. 18), respectively. The additional networks 472, 474, 476 and 478 output feature values based on the outputs of the plurality of layers 471, 473, 475 and 477. The recognition device 470 is trained based on the feature value and the label of the training data. Each layer can be repeatedly configured according to the purpose. For example, the output of the pooling layer 1 may be transmitted to the convolution layer 1, and the output of the convolution layer 1 may be transmitted to the fully connected network 1. The above-mentioned layer configuration is merely an example, and the layer configuration according to the embodiment can be changed in various ways.

FIG. 19 is a block diagram showing an electronic device according to an embodiment. Referring to FIG. 19, the electronic device 1100 according to one embodiment includes a camera 1110, a communication module 1120, a memory 1130, and an authentication unit 1140. The electronic device 1100 is installed in smartphones, smart TVs, desktop computers, notebook computers, tablet PCs, and the like.

As described above, the authentication unit 1140 outputs the authentication result based on the image. The authentication unit 1140 includes an authentication processing unit that outputs an authentication result based on a recognition device learned in advance based on a learning sample and a threshold value. The authentication unit 1140 outputs an authentication result from images received from the camera 1110, the communication module 1120, and the memory 1130. The image may include the input image and the registered image described above. For example, the input image may be received from the camera 1110 and the registered image may be received from the memory 1130. The registered image includes a user's face image.

The camera 1110 acquires an input image based on the image sensor. The camera 1110 provides the input image to the authentication unit 1140. The communication module 1120 communicates with an external device by various communication techniques. The communication module 1120 provides the image received from the external device to the authentication unit 1140. The memory 1130 stores various data. For example, the memory 1130 may store threshold values corresponding to a plurality of layers and feature values of registered images corresponding to the plurality of layers. The memory 1130 provides the stored data to the authentication unit 1140.

As described above, the above-described embodiment is embodied by a hardware component, a software component, or a combination of the hardware component and the software component. For example, the apparatus and components described in this embodiment include, for example, a processor, a controller, an ALU (arithmetic logic unit), a digital signal processor, a microcomputer, an FPA (field program array), a PLU (programmable log unit), and a microprocessor. , Or, such as different devices that execute and respond to instructions, are embodied using one or more general purpose computers or special purpose computers. The processing device executes an operating system (OS) and one or more software applications running on the OS. The processing device also accesses the data in response to the execution of the software and stores, manipulates, processes, and generates the data. For convenience of understanding, one processing device may be described as being used, but if the person has ordinary knowledge in the technical field, the processing device may have a plurality of processing elements and a plurality of processing elements. / Or you will find that it contains multiple types of processing elements. For example, the processing device includes a plurality of processors or one processor and one controller. In addition, other processing configurations such as a parallel processor are also possible.

The software includes computer programs, codes, instructions, or a combination of one or more of these, configures the processor to operate as desired, and commands the processor independently or collectively. The software and / or data is interpreted by the processing device and is any kind of machine, component, physical device, virtual device, computer storage medium or device, or signal wave transmitted to provide instructions or data to the processing device. It is embodied permanently or temporarily through. The software may be distributed on a networked computer system, may be stored in a distributed form, or may be executed. The software and data are stored on one or more computer-readable recording media.

The method according to this embodiment is embodied in the form of program instructions implemented via various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, etc. alone or in combination. The recording medium and program instructions may be specially designed and configured for the purposes of the present invention, or may be known and usable by those skilled in the art of computer software. Examples of computer-readable recording media include hard disks, magnetic media such as floppy (registered trademark) disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and optomagnetic media such as floptic disks. And include hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine language code as generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the present invention is not limited to the above-described embodiments, and various modifications are made without departing from the technical scope of the present invention. It is possible to do.

100 Authentication device 110, 115, 140 Recognition device 400, 410, 420, 430, 440, 450, 470 Recognition device 120, 150 Authentication processing unit 111, 116, 411, 421, 431, 441, 451 Input layer 112, 117, 412, 422, 432, 442, 452 Intermediate layers 113, 118, 413, 423, 433, 443, 454 Final layer 130 Memory 300 Learning device 310 Learner 320, 325 Threshold determination unit 453 First network 455 Second network 141, 142, 143, 144, 145 Layers 471, 473, 475, 477, 479 Layers 472, 474, 476, 478 Additional Networks

Claims (29)

A step of providing an input image to a recognition device containing multiple layers,
A step of authenticating the input image based on at least one feature value of the input image output by at least one intermediate layer in the recognition device and at least one feature value of the registered image corresponding to the intermediate layer. and, only including,
The step of authenticating the input image is
The input image is based on the first intermediate feature value of the input image output by the first intermediate layer of the at least one intermediate layer and the first intermediate feature value of the registered image corresponding to the first intermediate layer. Steps to perform primary authentication and
The input image is based on the second intermediate feature value of the input image output by the second intermediate layer of the at least one intermediate layer and the second intermediate feature value of the registered image corresponding to the second intermediate layer. Including steps to perform secondary authentication,
Authentication method. The authentication method according to claim 1, wherein the step of authenticating the input image includes a step of determining acceptance or rejection of the input image. The authentication method according to claim 1 or 2, wherein the plurality of layers are layers of a deep learning network. The at least one intermediate layer is a plurality of intermediate layers.
The step of authenticating the input image determines the success or failure of the authentication of the input image based on the feature value of the input image output by each intermediate layer and the feature value of the registered image corresponding to each intermediate layer. The authentication method according to any one of claims 1 to 3, which includes the step of performing. The step of authenticating the input image is
A step of determining the similarity based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer.
A step of authenticating the input image based on the similarity and the threshold value,
The authentication method according to any one of claims 1 to 4, which comprises. The authentication method according to claim 5, wherein the step of authenticating the input image includes a step of determining acceptance of the input image when the similarity exceeds the similarity corresponding to the false acceptance rate for the intermediate layer. .. The authentication method according to claim 6, wherein the step of authenticating the input image includes a step of determining rejection of the input image when the similarity is less than the similarity corresponding to the authentication rate for the intermediate layer. .. The step of authenticating the input image is next to the recognition device when the similarity exceeds the similarity corresponding to the authentication rate for the intermediate layer and is less than the similarity corresponding to the false acceptance rate for the intermediate layer. The authentication method according to claim 7, further comprising steps to proceed to layers. The authentication method according to claim 5, wherein the threshold value includes a first threshold value determined by the false acceptance rate and a second threshold value determined by the authentication rate. The authentication method according to claim 9, wherein the first threshold value is larger than the second threshold value. The step of authenticating the input image is
A step of determining a distance based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer.
A step of authenticating the input image based on the distance and the threshold value, and
The authentication method according to any one of claims 1 to 4, which comprises. The authentication method according to claim 11, wherein the step of authenticating the input image includes a step of determining acceptance of the input image when the distance is less than a distance corresponding to the false acceptance rate for the intermediate layer. The authentication method according to claim 12, wherein the step of authenticating the input image includes a step of determining rejection of the input image when the distance exceeds a distance corresponding to the authentication rate for the intermediate layer. The step of authenticating the input image proceeds to the next layer of the recognition device when the distance exceeds the distance corresponding to the false acceptance rate for the intermediate layer and is less than the distance corresponding to the authentication rate for the intermediate layer. The authentication method according to claim 13, further comprising the step of performing. The authentication method according to claim 11, wherein the threshold value includes a first threshold value determined by the false acceptance rate and a second threshold value determined by the authentication rate. The authentication method according to claim 15, wherein the first threshold value is larger than the second threshold value. The step of authenticating the input image is to authenticate the input image based on the feature value of the input image output by the final layer in the recognition device and the feature value of the registered image corresponding to the final layer. The authentication method according to any one of claims 1 to 16, further comprising a step of determining success or failure. The step of providing the registered image to the recognition device,
A step of storing at least one feature value output by the intermediate layer and at least one feature value output by the final layer as feature values of the registered image, and
The authentication method according to any one of claims 1 to 17, further comprising. The authentication method according to any one of claims 1 to 18, further comprising the step of receiving the registered image from the camera. The authentication method according to any one of claims 1 to 19, wherein the at least one intermediate layer includes a plurality of intermediate layers among the plurality of layers. The authentication method according to any one of claims 1 to 20, wherein the plurality of layers include at least one convolution layer and a pooling layer. A recognition device that outputs at least one feature value of an input image by at least one intermediate layer among a plurality of layers.
An authentication processing unit that authenticates the input image based on at least one feature value of the input image output by the at least one intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer. only including,
The authentication processing unit
The input image is based on the first intermediate feature value of the input image output by the first intermediate layer of the at least one intermediate layer and the first intermediate feature value of the registered image corresponding to the first intermediate layer. Perform primary authentication,
The input image is based on the second intermediate feature value of the input image output by the second intermediate layer of the at least one intermediate layer and the second intermediate feature value of the registered image corresponding to the second intermediate layer. Perform secondary authentication,
Authentication device. The authentication device according to claim 22 , wherein the authentication processing unit determines acceptance or rejection of the input image. The authentication device according to claim 22 or 23 , wherein the plurality of layers are layers of a deep learning network. The authentication device according to any one of claims 22 to 24 , wherein the at least one intermediate layer is a plurality of intermediate layers. The authentication processing unit determines the success or failure of authentication of the input image based on the feature value of the input image output by each intermediate layer and the feature value of the registered image corresponding to each intermediate layer. The authentication device according to any one of 22 to 25 . The authentication processing unit determines the degree of similarity based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer, and determines the similarity. The authentication device according to any one of claims 22 to 26 , which authenticates the input image based on the degree and the threshold value. The authentication processing unit determines the distance based on at least one feature value of the input image output by the intermediate layer and at least one feature value of the registered image corresponding to the intermediate layer, and determines the distance and the distance. The authentication device according to any one of claims 22 to 26 , which authenticates the input image based on a threshold value. The authentication processing unit provides the registered image to the recognition device, and uses at least one feature value output by the intermediate layer and at least one feature value output by the final layer as feature values of the registered image. The authentication device according to any one of claims 22 to 28 , which is stored.

Images