JP2011159030A - Subject authentication apparatus, subject authentication method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce a state in which a subject can not be correctly authenticated due to temporary variation in similarity. <P>SOLUTION: A face detection part 113 detects a first subject area from image data. A face tracking processing part 114 tracks changes with the lapse of time from past image data about a second subject area in which the subject has been authenticated on the basis of the image data. A feature information extraction processing part 115 extracts first feature information of the subject from the first subject area. A similarity calculation processing part 117 compares the first feature information with the second feature information stored in a feature information storage part 116 and calculates similarity between the subject corresponding to the first feature information and the subject corresponding to the second feature information. A threshold determination part 119 sets a different threshold according to whether or not the second subject area corresponding to the first subject area is tracked. An authentication determination part 120 authenticates coincidence between the subject corresponding to the first feature information and the subject corresponding to the second feature information according to a compared result of the calculated similarity with the threshold. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for authenticating a subject included in a subject region in image data.

  In an imaging device having an authentication processing function, a subject image region including a specific subject region is detected from image data obtained by photographing with the imaging device. Next, the imaging apparatus extracts feature information for identifying the subject area as the authentication target from the image data of the subject image area, and extracts the extracted feature information and the registration target image registered in the database in advance. Compare with feature information. Next, the imaging apparatus determines the similarity between the authentication target and the registration target from the comparison result, and based on the similarity, authenticates which registration target is the authentication target or there is no corresponding registration target I do. As an example of such a technique, a face of a person is detected as a specific subject area from image data obtained by photographing, and feature information of the face is extracted, and personal authentication is performed as to who this face is. System exists.

JP 2008-271310 A

  However, when performing the personal authentication process as described above, the degree of similarity varies depending on factors such as the size of the face in the image data, the orientation of the face, the brightness of the face, and the like. Then, there is a case of false authentication. In addition, when tracking a face that has been authenticated using face detection or subject tracking in a frame where authentication is not performed, it may be possible to track the face incorrectly, so it is desirable to periodically authenticate a once authenticated face.

  Accordingly, an object of the present invention is to reduce a situation in which a subject is not correctly authenticated due to a temporary change in similarity.

  The subject authentication device according to the present invention is configured to detect a temporal change from image data that is earlier than the image data of a detection unit that detects a first subject region from image data, and a second subject region for which the subject has been authenticated. Tracking means for tracking, extraction means for extracting first feature information of the subject from the first subject area, and second feature information stored in advance in the first feature information and storage means extracted by the extraction means First calculation means for comparing the feature information of the subject and the degree of similarity between the subject corresponding to the first feature information and the subject corresponding to the second feature information; and Setting means for setting different thresholds depending on whether or not the corresponding second subject area is being tracked by the tracking means, and the similarity calculated by the first calculation means and the setting means are set by the setting means. Depending on the result of comparison between the threshold value, and having an authentication means for authenticating the consistency of the object corresponding to the object and the second characteristic information corresponding to the first characteristic information.

  According to the present invention, when the subject area of a subject that has already been authenticated is tracked and the subject in the subject area seems to be the same as the subject being authenticated, the subject is not correctly authenticated due to a temporary change in similarity. This can be reduced.

It is a block diagram which shows the structure of the imaging device provided with the personal authentication function which concerns on embodiment of this invention. It is a flowchart which shows the personal authentication process by the imaging device which concerns on embodiment of this invention. It is a figure for demonstrating the correlation value calculation method by the imaging device which concerns on embodiment of this invention. It is a figure for demonstrating the threshold value setting method by the imaging device which concerns on embodiment of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram illustrating a configuration of an imaging apparatus having a personal authentication function according to an embodiment of the present invention. The image pickup apparatus shown in FIG. 1 detects a person's face as a specific subject area from image data obtained by photographing, extracts the feature information of the face, and identifies who the face is Has a function to perform authentication. Note that the imaging apparatus according to the present embodiment has a configuration that is an application example of the subject authentication apparatus of the present invention.

  In FIG. 1, reference numeral 101 denotes an imaging lens. Reference numeral 102 denotes a focus lens. The imaging apparatus according to the present embodiment is configured to be able to control the focus state of the subject to be imaged by controlling the focus lens 102. Reference numeral 103 denotes an image sensor such as a CCD or CMOS that photoelectrically converts the light flux of the subject that has passed through the imaging lens 101 and converts it into an electrical signal. Reference numeral 104 denotes an A / D conversion unit that performs clamp processing, gain processing, and the like on the analog image signal output from the image sensor 103 and converts the analog image signal into image data that is a digital signal. Reference numeral 105 denotes a digital signal processing unit that executes predetermined pixel interpolation processing and color conversion processing on the image data from the A / D conversion unit 104. The image data signal-processed by the digital signal processing unit 105 is supplied to the recording / reproducing unit 106, and is recorded on a recording medium 107 that can be removed according to various user requirements, such as an optical disk recording medium and a semiconductor recording medium. Further, the image data is sent to the display control unit 108, and the display of the image data on the display device 109 is controlled.

  Reference numeral 110 denotes a system control unit. The system control unit 110 basically controls the operation of the entire imaging apparatus, and usually includes a CPU inside. For example, the system control unit 110 executes predetermined calculation processing using image data obtained by imaging, and controls the focus control unit 111 and the exposure control unit 112 based on the calculation result. With this control, the focus control unit 111 performs automatic focus control (AF) on the focus lens 102, and the exposure control unit 112 controls the aperture and shutter in the imaging lens 101 and further the electronic shutter of the image sensor 103. Perform automatic exposure control (AE).

  Reference numeral 113 denotes a face detection unit. The face detection unit 113 detects edges of eyes, mouths, and the like from subjects included in the image data to be captured to detect a feature portion of a person's face. That is, the face detection unit 113 detects face image data as face image data from the image data area (face position and face size) occupied by the face and the likelihood of the face (face reliability). Perform processing and identification processing. Further, the face detection unit 113 detects the number of detected face image data at the same time. Since the face detection process does not need to store reference data for each individual as in the case of personal authentication, the process can be completed sufficiently earlier than the personal authentication.

  Reference numeral 114 denotes a face tracking processing unit, which uses information that can be extracted from the face image data detected by the face detecting unit 113 to track changes in the face of the same person over time between image frames having different timings. Details of the function of the face tracking processing unit 114 will be described later.

  A feature information extraction processing unit 115 extracts feature information used in personal authentication from the face image data detected by the face detection unit 113 and the face image data of a person registered for personal authentication. . A feature information storage unit 116 stores the feature information extracted by the feature information extraction processing unit 115 from the face image data of a person registered for personal authentication. The feature information extracted by the feature information extraction processing unit 115 is an example of the first feature information of the present invention, and the feature information stored in the feature information storage unit 116 is the second feature information of the present invention. It is an example.

  A similarity calculation processing unit 117 compares the feature information of at least one face image data stored in the feature information storage unit 116 with the feature information of the face image data extracted by the feature information extraction processing unit 115. Then, the similarity that is the comparison result is calculated.

  Note that, as the feature information extracted by the feature information extraction processing unit 115, for example, physical shape features such as eyes, nose, mouth, and facial contours, and information indicating their relative positions can be used. The similarity calculation processing unit 117 performs pattern matching using these feature information, and the face image data in which the feature information is stored in the feature information storage unit 116 and the face from which the feature information is extracted by the feature information extraction processing unit 115 A value (similarity) representing how much the image data is similar is calculated.

  A correlation value calculation processing unit 118 calculates a correlation value indicating the correlation of the face image data using information that can be extracted from the face image data detected by the face detection unit 113. Reference numeral 119 denotes a threshold value determination unit that determines a threshold value according to the correlation value calculated by the correlation value calculation processing unit 118. An authentication determination unit 120 compares the similarity calculated by the similarity calculation processing unit 117 with the threshold determined by the threshold determination unit 119 according to the correlation value calculated by the correlation value calculation processing unit 118. . When the similarity is equal to or greater than the threshold, the authentication determination unit 120 determines whether the two face data are of the same person by determining the matching of the faces of the two face image data used in the similarity calculation. Perform some authentication. Details of the correlation value calculation processing unit 118 and the threshold value determination unit 119 will be described later.

  The configuration of the imaging apparatus illustrated in FIG. 1 is merely an example, and the configuration of the imaging apparatus to which the present invention can be applied is not limited to the configuration illustrated in FIG. 1 as long as the operations described below can be performed. Absent. Moreover, in FIG. 1, the face detection part 113 is a structure used as the application example of the detection means of this invention. The face tracking processing unit 114 is configured as an application example of the tracking unit of the present invention. The feature information extraction processing unit 115 is configured as an application example of the extraction means of the present invention. The similarity calculation processing unit 117 is a configuration serving as an application example of the first calculation unit of the present invention. The threshold value determination unit 119 has a configuration as an application example of the setting unit of the present invention. The authentication determination unit 120 is configured as an application example of the authentication unit of the present invention. The correlation value calculation processing unit 118 is configured as an application example of the second calculation unit of the present invention.

  Next, the personal authentication processing by the imaging apparatus according to the embodiment of the present invention will be described in detail with reference to the flowchart shown in FIG. In FIG. 2, in step S <b> 201 after the start of processing, the face detection unit 113 executes face detection processing on the image data of the current image frame t + k. In the subsequent step S202, the face tracking processing unit 114 executes a tracking process for a face that has been successfully authenticated in a past image frame. That is, in step S202, the face tracking processing unit 114 detects the face detected in step S201 when there is a face that is being tracked up to the image frame t + k−1 or when there is a face that has been successfully authenticated in the image frame t + k−1. A face that can be determined to be the same person as the face that is being tracked or the face that has been successfully authenticated in the image frame t + k−1 is detected as a tracking result. As an example of the tracking process of the face tracking processing unit 114, a method of tracking the image with the closest distance between the face position coordinates detected by the face detecting unit 113 in the image frames with different timings is regarded as the same person. is there. Further, as another example of the tracking process, a region including the face detected by the face detection unit 113 in an image frame having different timing is considered to be the same person with the highest similarity in luminance and color information. It is also possible to perform this.

  In subsequent step S203, the feature information extraction processing unit 115 extracts feature information used for authentication from the face image data of the face F [t + k] to be authenticated among the faces detected in step S201. In subsequent step S204, the similarity calculation processing unit 117 compares the feature information of the face F [t + k] extracted in step S203 with the registrant's feature information stored in the feature information storage unit 116, and compares the comparison information. The similarity that is the result is calculated. That is, the similarity between the face F [t + k] extracted in step S203 and the face corresponding to the registrant whose feature information is stored in the feature information storage unit 116 is calculated. In the subsequent step S205, the face tracking processing unit 114 determines whether or not the authentication has succeeded in the past image frame (for example, the image frame t) and the successfully authenticated face can be tracked up to the current image frame t + k. . If the face that has been successfully authenticated in the past image frame has been traced to the current image frame t + k, the process proceeds to step S206. In step S206, matching is executed between the face F [t + k] that is the face to be authenticated and the face that has been successfully authenticated in the past image frame t in the current image frame t + k. That is, in step S206, the similarity calculation processing unit 117 determines whether the face tracking the face F [t + k] and the face F [t] by the face tracking processing unit 114 is the same face. In this case, when the position coordinates of the face F [t + k] and the position coordinates of the face obtained as a result of tracking the face F [t] are equal, it is determined that they are the same face. When it is determined that the face F [t + k] and the face F [t] tracked by the face tracking processing unit 114 are the same face, the process proceeds to step S207, where the face F [t + k] and the face F [ The correlation of t] is examined. The face image data of the face F [t + k] is an example of the first subject area in the present invention, and the face image data of the face F [t] is an example of the second subject area in the present invention.

  In step S207, the correlation value calculation processing unit 118 calculates a correlation value between the face F [t + k] and the face F [t]. Here, the correlation value between the two faces will be described in detail with reference to FIG. FIG. 3 is a diagram for explaining a correlation value calculation method by the imaging apparatus according to the present embodiment. In the example of FIG. 3, the case where the distance that the face has moved is used as the correlation value will be described. As shown in FIG. 3, the center position coordinates (x [t + k], y [t + k]) of the face F [t + k] from the face image data of the image frame t + k, and the face F [t] of the face image data of the image frame t. The center position coordinates (x [t], y [t]) can be acquired. The correlation value C is calculated by the correlation value calculation processing unit 118. For example, the correlation value C is calculated from the distance (movement amount) from the face F [t] to the face F [t + k] as shown in Equation 1.

  Note that the above calculation formula is only an example, and what is important here is to obtain a correlation value C between two face image data. Therefore, as long as the correlation value C between the two face image data can be obtained, the parameters and calculation formulas other than those described above can be used. Naturally, parameters and calculation formulas may differ depending on how the correlation value is defined. That is, as shown in FIG. 3, from the face image of the face F [t + k], the face size FS [t + k], the face area average pixel value I [t + k], the face area average luminance value Y [t + k], etc. Information can be acquired. Similarly, information such as the face size FS [t], the average pixel value I [t] of the face area, and the average luminance value Y [t] of the face area can be acquired from the face image of the face F [t]. . Using these, for example, the correlation value C can be calculated from the difference in the size of the face as shown in Equation 2.

  Further, for example, the correlation value C can be calculated from the number of elapsed frames (elapsed time) k as in Expression 3.

  Further, for example, the correlation value C can be calculated from the difference between the average pixel values of the face area as in Expression 4.

  Further, for example, the correlation value C can be calculated from the difference in the average luminance value of the face area as shown in Equation 5. It is also possible to calculate a final correlation value C by weighting and adding some correlation values C obtained by the equations 1 to 5 or all the correlation values C.

  In step S208, the authentication determination unit 120 sets a threshold using the correlation value C calculated in step S207. Here, the threshold value will be described in detail with reference to FIG. FIG. 4 is a diagram for explaining a threshold setting method by the imaging apparatus according to the present embodiment. The threshold value TH is a value used for authentication determination. In the authentication, when the similarity calculated by the similarity calculation processing unit 117 is equal to or greater than the threshold, it is determined that the authentication is successful. The threshold value is set in advance by the authentication determination unit 120 to a predetermined value TH = th1, but as shown in FIG. 4, according to the value of the correlation value C calculated in step S207, if C> c2, TH = The threshold can be set low as in th2. That is, when the correlation between the face F [t] and the face F [t + k] is high and it is determined that the face to be authenticated is the same face as the face to be authenticated, the threshold value is set lower than the normal value. A correction is made so that the authentication is easy to succeed. As a result, it is possible to reduce a situation where the subject is not correctly authenticated due to a temporary change in similarity.

  If it is determined in step S205 that the authentication has succeeded in the past image frame t and the face tracked by the face tracking processing unit 114 up to the current image frame t + k does not exist in the image frame t + k, the process proceeds to step S205. The process proceeds to S209. In step S205, it is determined that the authentication has succeeded in the past image frame t and the face tracked by the face tracking processing unit 114 up to the current image frame t + k exists in the image frame t + k, and step S206. If it is determined that the face F [t] and the face F [t + k] tracked by the face tracking processing unit 114 are not the same face, the process proceeds to step S209. In step S209, the authentication determination unit 120 resets the threshold by setting the threshold to a predetermined value th1, and the process proceeds to step S210.

  In step S210, the authentication determination unit 120 compares the similarity calculated by the similarity calculation processing unit 117 in step S204 with the threshold value TH determined in step S208. If the similarity is greater than or equal to the threshold value TH, the process proceeds to step S211. In step S211, the system control unit 110 displays, for example, a face frame indicating that personal authentication is successful on the display device 109 as a result of successful authentication. In this case, the face F [t + k] is tracked by the face tracking processing unit 114 in subsequent frames. On the other hand, if the similarity is less than the threshold value TH, the process proceeds to step S212. In step S212, the system control unit 110 displays, for example, a face frame different from that at the time of successful personal authentication on the display device 109 as a result of the authentication failure.

  Through the processing described above, in this embodiment, when an already authenticated face is tracked and the face seems to be the same person as the face being authenticated, the subject is correctly authenticated with a temporary change in similarity. It is possible to reduce the loss of being performed.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  110: system control unit, 113: face detection unit, 114: face tracking processing unit, 115: feature information extraction processing unit, 116: feature information storage unit, 117: similarity calculation processing unit, 118: correlation value calculation processing unit, 119: Threshold determination unit, 120: Authentication determination unit

Claims (10)

Detecting means for detecting a first subject region from the image data;
Tracking means for tracking a change over time in the second subject area where the subject has been authenticated from past image data than the image data;
Extraction means for extracting first feature information of a subject from the first subject region;
The first feature information extracted by the extraction unit and the second feature information stored in advance in the storage unit are compared, and the subject corresponding to the first feature information and the second feature information are supported. First calculating means for calculating the similarity to the subject to be
Setting means for setting different thresholds depending on whether or not the second subject area corresponding to the first subject area is being tracked by the tracking means;
A subject corresponding to the first feature information and a subject corresponding to the second feature information according to a comparison result between the similarity calculated by the first calculation unit and the threshold set by the setting unit And an authentication unit for authenticating the coincidence with the subject authentication apparatus. A second calculating means for calculating a correlation value between the first subject region and the second subject region;
The subject authentication apparatus according to claim 1, wherein the setting unit sets a threshold value when the second subject region is tracked by the tracking unit according to the correlation value.   The second calculating means calculates the correlation value based on a movement amount between the position of the first subject area in the image data and the position of the second subject area in the past image data. The subject authentication device according to claim 2.   3. The subject according to claim 2, wherein the second calculation unit calculates the correlation value based on a difference between a size of the first subject region and a size of the second subject region. Authentication device.   The subject authentication apparatus according to claim 2, wherein the second calculation unit calculates the correlation value based on an elapsed time from the past image data to the image data.   3. The subject according to claim 2, wherein the second calculation unit calculates the correlation value based on a difference between a color of the first subject region and a color of the second subject region. Authentication device.   3. The subject according to claim 2, wherein the second calculation unit calculates the correlation value based on a difference between a luminance of the first subject region and a luminance of the second subject region. Authentication device.   The subject authentication apparatus according to claim 1, wherein the subject region is a region including a face. A subject authentication method by a subject authentication device,
A detection step of detecting a first subject region from the image data;
A tracking step of tracking a change over time in the second subject area where the subject has been authenticated from image data past the image data;
An extraction step of extracting first feature information of the subject from the first subject region;
The first feature information extracted in the extraction step is compared with the second feature information stored in advance in the storage means, and the subject corresponding to the first feature information and the second feature information are supported. A calculating step for calculating the similarity to the subject to be
A setting step for setting a different threshold depending on whether or not the second subject region corresponding to the first subject region is being tracked by the tracking step;
The subject corresponding to the first feature information matches the subject corresponding to the second feature information according to the comparison result between the similarity calculated in the calculation step and the threshold set in the setting step. An authentication step for authenticating the subject. A detection step of detecting a first subject region from the image data;
A tracking step of tracking a change over time in the second subject area where the subject has been authenticated from image data past the image data;
An extraction step of extracting first feature information of the subject from the first subject region;
The first feature information extracted in the extraction step is compared with the second feature information stored in advance in the storage means, and the subject corresponding to the first feature information and the second feature information are supported. A calculating step for calculating the similarity to the subject to be
A setting step for setting a different threshold depending on whether or not the second subject region corresponding to the first subject region is being tracked by the tracking step;
The subject corresponding to the first feature information matches the subject corresponding to the second feature information according to the comparison result between the similarity calculated in the calculation step and the threshold set in the setting step. A program for causing a computer to execute an authentication step for authenticating sex.

Images