WO2018179723A1

WO2018179723A1 - Facial authentication processing apparatus, facial authentication processing method, and facial authentication processing system

Info

Publication number: WO2018179723A1
Application number: PCT/JP2018/001863
Authority: WO
Inventors: 郁雄渕上; 孝治堀内; 中村　剛
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2017-03-30
Filing date: 2018-01-23
Publication date: 2018-10-04
Also published as: JP2018169943A

Abstract

The present invention is provided with: a facial recognition unit (21) that acquires a captured image obtained by capturing a target person and detects the face of the target person in the input image of the captured image; a frame detection unit (22) that detects a linear frame in the input image; a disposition determination unit (23) that determines whether a frame surrounding the face of the target person exists by using face position information acquired through detection of the face and frame position information acquired through detection of the frame; and a validity determination unit (25) that determines the validity of the recognition result of the face information of the target person on the basis of a frame disposition determination result.

Description

Face authentication processing device, face authentication processing method, and face authentication processing system

The present disclosure relates to a face authentication processing device, a face authentication processing method, and a face authentication processing system that perform authentication using a face image of a target person.

In order to authenticate a target person, a face authentication technique is known in which a face image of the target person is acquired and verified against the face image of the registrant to determine whether the person is the person. In this type of face authentication technology, various technologies for detecting a so-called “spoofing” act of improper use on behalf of a regular registrant have been proposed.

For example, in Patent Document 1, a face image sequence for a predetermined time is input, and a face included in the face image sequence is impersonated based on temporal changes in predetermined color information extracted from the face image sequence. An information processing apparatus for determining whether or not is disclosed. According to this conventional example, it is possible to accurately detect impersonation when a fake face is presented during face authentication.

International Publication No. 2015/122183

In Patent Document 1 described above, a mask for disguise is obtained by acquiring information derived from changes in blood flow measured from a face image based on temporal changes in predetermined color information extracted from the face image sequence. It is possible to detect impersonation of a face image with little change, such as when wearing. However, in the conventional face authentication device, it is sometimes difficult to detect fraud against impersonation using a moving image of a face image.

The present disclosure is devised in view of the above-described conventional circumstances, and an object thereof is to provide a face authentication processing device, a face authentication processing method, and a face authentication processing system that can accurately detect impersonation during authentication. .

The present disclosure acquires a captured image obtained by capturing a target person, detects a face of the target person in an input image of the captured image, and a frame detection unit detects a linear frame in the input image. An arrangement determining unit that determines whether a frame surrounding the face of the target person exists using the face position information acquired by detecting the face and the frame position information acquired by detecting the frame; There is provided a face authentication processing device comprising: an authentication determination unit that determines the validity of an authentication result of face information of the target person based on a frame arrangement determination result.

Further, the present disclosure is a face authentication processing method in a face authentication processing device that performs face authentication of a target person, obtains a captured image obtained by capturing the target person, and includes the face of the target person in an input image of the captured image , Detecting a straight frame in the input image, and surrounding the face of the target person using the face position information acquired by detecting the face and the frame position information acquired by detecting the frame Provided is a face authentication processing method for determining whether or not a frame exists and determining the validity of the authentication result of the face information of the target person based on the frame arrangement determination result.

The present disclosure also includes an imaging unit that captures a target person, a face detection unit that acquires a captured image of the target person and detects the face of the target person in an input image of the captured image, and a straight line in the input image Whether there is a frame surrounding the face of the target person using the frame detection unit for detecting a frame, the face position information acquired by the face detection, and the frame position information acquired by the frame detection An arrangement determination unit for determining whether or not, a face verification unit for comparing face information of the captured image of the target person with registered face information, a face verification result of the face information, and an arrangement determination result of the frame An authentication determination unit that determines the validity of the authentication result of the face information of the target person, an authentication result output unit that outputs the authentication result to a control target, the imaging unit, the face detection unit, and the frame detection unit , The arrangement determination unit, the authentication determination unit Chino provides face recognition processing system comprising a display unit, a displaying at least one processing result.

According to the present disclosure, impersonation at the time of authentication can be detected with high accuracy.

FIG. 1 is a diagram illustrating a first example of the configuration of the face authentication system according to the present embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the first embodiment. FIG. 3 is a diagram for explaining frame detection processing in the present embodiment. FIG. 4 is a diagram for explaining the background removal processing in the present embodiment. FIG. 5 is a diagram illustrating a first example of a functional configuration of the frame detection unit in the present embodiment. FIG. 6 is a diagram illustrating a second example of a functional configuration of the frame detection unit in the present embodiment. FIG. 7 is a diagram illustrating the arrangement determination process in the present embodiment. FIG. 8 is a flowchart illustrating a procedure of a first example of face authentication processing according to the first embodiment. FIG. 9 is a flowchart illustrating the procedure of a second example of the face authentication process in the first embodiment. FIG. 10 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the second embodiment. FIG. 11 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the third embodiment. FIG. 12 is a flowchart illustrating a procedure of a first example of face authentication processing according to the third embodiment. FIG. 13 is a flowchart illustrating a procedure of a second example of the face authentication process in the third embodiment. FIG. 14 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the fourth embodiment. FIG. 15 is a diagram illustrating a second example of the configuration of the face authentication system according to the present embodiment. FIG. 16 is a diagram illustrating a third example of the configuration of the face authentication system according to the present embodiment.

Hereinafter, an embodiment that specifically discloses a face authentication processing device, a face authentication processing method, and a face authentication processing system according to the present disclosure (hereinafter referred to as “the present embodiment”) will be described in detail with reference to the drawings as appropriate. . However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the claimed subject matter.

(System configuration of face authentication system, first example)
FIG. 1 is a diagram illustrating a first example of the configuration of the face authentication system according to the present embodiment. In the present embodiment, an example of a face authentication device and a face authentication system that performs authentication by matching a face image of a registrant using a face image of a moving image obtained by capturing the target person when performing face authentication of the target person is shown.

By using a moving image of the face image of the target person as the image information used for face authentication, biometric information such as pulsation in the person to be authenticated can be acquired, and fraud such as impersonation using a photograph can be detected. However, for example, when a non-regular person uses a portable display terminal such as a tablet terminal to reproduce a moving image showing the face of a regular registrant and impersonates, the face using the moving image Even if authentication is performed, fraud cannot be detected, and it may happen that the authentication result is approved. In view of the above circumstances, the present embodiment shows a configuration example capable of detecting fraud against impersonation or the like due to reproduction of a moving image of a regular registrant using a portable display terminal.

The face authentication system 100A is configured to include the face authentication device 10A. The face authentication system 100A images the target person 30, performs face authentication, and outputs an authentication result to the control target 40. As an example of the control target 40, a service use management device such as car sharing, an entrance / exit management device in a security area, and the like are assumed. When applied to a car sharing usage management device, based on the authentication result of the user's face authentication, if the user is a registered regular user (if the authentication result is approved), the vehicle is allowed to be used. If the user is not a user (if the authentication result is denied), the car is disabled. Control relating to permission to use the vehicle can be realized, for example, by inputting an authentication result to the control computer of the automobile and enabling or disabling the operation of the vehicle according to the authentication result. When applied to an entrance / exit management device, based on the authentication result of the face authentication of the entrance / exit person, if it is an authorized person (when the authentication result is approval), it is allowed to pass, and if it is not an authorized person (authentication) When the result is denial), traffic is blocked, intruder detection is notified, etc.

The face authentication device 10 A includes an imaging unit 11, a display unit 12, an ID reading unit 13, and a calculation unit 20. The imaging unit 11 is configured by a camera such as a surveillance camera, for example. The imaging unit 11 includes an imaging lens, an imaging device, an image signal processing circuit, a communication interface, and the like. The imaging unit 11 captures a moving image of an imaging region including the target person 30 and outputs the captured image to the calculation unit 20. The ID reading unit 13 is configured by an information reading device such as a card reader, for example, reads ID information of the ID card 35 such as a membership card, identification card, driver's license, etc. possessed by the target person 30 and outputs the ID information to the calculation unit 20. To do.

The calculation unit 20 is configured by a computer such as a PC. The calculation unit 20 includes a processor, a memory, a communication interface, and the like. Various calculation processes related to face authentication using the image information of the captured image acquired by the imaging unit 11 and the ID information acquired by the ID reading unit 13. Is executed and the authentication result is output. The display unit 12 is configured by a display device including a display device such as a liquid crystal display, and includes various types of information at the time of execution of processing in the calculation unit 20 such as an operation guidance screen, an input image of a captured image, a face detection result, a frame detection result, The verification result and the authentication result of face authentication are displayed. Processing in the face authentication apparatus of this embodiment such as frame detection will be described in detail later. The display unit 12 is disposed, for example, in a monitoring room of the face authentication system 100A, and is provided so as to be visible to an administrator or operator of the face authentication system 100A. The display unit 12 may be provided in the vicinity of the imaging unit 11 for guidance display to the target person 30.

(First Embodiment of Face Authentication Device)
FIG. 2 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the first embodiment. The calculation unit 20A of the face authentication device includes a face recognition unit 21, a frame detection unit 22, an arrangement determination unit 23, a face collation unit 24, and an validity determination unit 25 as functional components. The arithmetic unit 20A implements the functions of the respective units by executing a predetermined program by the processor. In this embodiment, even when there is spoofing by moving image reproduction of a regular registrant using a portable display terminal, the outline of the display terminal is recognized by performing frame detection, and the moving image of the face image is used. Detect fraud that has occurred.

The face recognition unit 21 has a function of a face detection unit. The face recognition unit 21 detects the face of the target person in the input image by face recognition processing using the input image of the captured image including the face image, and acquires face feature information and position information (face arrangement information). The frame detection unit 22 uses the input image of the captured image and the background image to detect a linear frame in the input image from which the background image has been removed. If the captured image includes a frame such as a display terminal, the frame detection unit 22 Position information (frame layout information) is acquired. The arrangement determination unit 23 uses the face arrangement information and the frame arrangement information to perform arrangement determination as to whether or not the face is located within the frame of the display terminal or the like. By this arrangement determination, it is detected that the display terminal exists in the captured image of the input image and the face image is displayed by the display terminal within the frame of the display terminal.

The face collation unit 24 collates the face in the captured image detected by the face recognition unit 21 using the facial feature information acquired from the captured image and the collation data of the registrant's face information, thereby registering the registrant. It is determined whether or not the face matches. As the collation data, image data of the registrant's face image or feature data including feature information of the registrant's face is used. For example, the collation data is recorded as one of the ID information on the ID card possessed by the target person, and the collation data of the ID information read from the ID card is input to the face collation unit 24 of the arithmetic unit 20A. The validity determination unit 25 has functions of an authentication determination unit and an authentication result output unit. The validity determination unit 25 approves whether the face of the target person in the input image is valid based on the arrangement determination result by the arrangement determination unit 23 and the face comparison result by the face matching unit 24, that is, the validity of the authentication result of face authentication. It is determined whether or not, and the authentication result is output.

Here, an example of the functional configuration and operation of the frame detection unit 22 will be described in detail. FIG. 3 is a diagram for explaining frame detection processing in the present embodiment. In the illustrated example, an input image 51 of a captured image is a case where an image obtained by capturing a situation where a moving image in which a regular registrant's face is captured is reproduced on a portable display terminal is captured. It is shown. In this case, in the captured image, the face of the registrant displayed on the display terminal is captured as a subject, and the input image 51 includes a housing 53 of the display terminal and a person 54 on the display image in the display unit of the display terminal. It is included. The frame detection unit 22 performs a known contour extraction (edge detection) process on the input image 51 to generate an edge image 52. The edge image 52 includes the outline 55 of the housing of the display terminal and the outline 56 of the person in the display unit of the display terminal. Then, the frame detection unit 22 performs a straight line detection process on the edge image 52 and extracts a straight line portion (line segment) 57 in the image. As the straight line detection process, for example, straight line detection using Hough transform may be used. Next, the frame detection unit 22 determines whether there are two parallel straight lines and a straight line perpendicular thereto in the straight line portion 57 in the extracted image, extracts a frame 58 surrounded by four sides, and extracts frame coordinates. To get. In addition, even if it is a set of square line segments whose four sides are not closed or a set of line segments that are orthogonal to three sides or two sides, it is determined as a frame surrounded by the line segments based on a predetermined condition. 58 may be extracted.

Further, the frame detection unit 22 may perform straight line detection and frame extraction after performing background removal that removes the background image from the captured image in the frame detection processing. The accuracy of frame detection can be improved by removing the background portion from the captured image. FIG. 4 is a diagram for explaining the background removal processing in the present embodiment. The illustrated example shows a case in which impersonation using the display terminal is not performed as the input image 61 of the captured image, and an image obtained by directly capturing the face of the target person is input. In this case, the input image 61 includes a background 65 and a person 66. As the background image 62, an image including the background 65 in a state where no person is present in the same imaging area as the input image 61 is used. The background image 62 is acquired by, for example, an image obtained by capturing only the background at a timing when there is no person, or an image obtained by extracting a fixed point that does not change in position (the amount of movement is not detected) in the captured image. The frame detection unit 22 removes the background 65 in the input image 61 using the input image 61 and the background image 62, and generates an image 63 after the background is removed. The image 63 after the background removal is an image of only a subject that has no background portion and moves, such as a person 66. When impersonation is performed using a display terminal, the image after the background removal includes the display terminal and a person.

FIG. 5 is a diagram illustrating a first example of a functional configuration of the frame detection unit in the present embodiment. The frame detection unit 22A of the first example includes

contour extraction units

221, 222, a background removal unit 223, a straight line detection unit 224, and a frame extraction unit 225 as functional components. The first example is an example in which a contour is extracted for each of the input image and the background image, the contour of the background portion is removed from the contour of the captured image, and frame detection is performed on the edge image from which the background is removed. The contour extraction unit 221 performs edge detection on the input image of the captured image and extracts the contour of the captured image. The contour extracting unit 222 performs edge detection on the background image to extract the contour of the background image. The background removing unit 223 removes the contour of the background image from the contour of the captured image, and generates an edge image with a contour that does not include the background. The straight line detection unit 224 performs straight line detection processing such as Hough transform on the edge image, and extracts a straight line portion in the image. The frame extraction unit 225 extracts a frame surrounded by four line segments in a straight line portion in the extracted image, and outputs it as frame coordinates. In this way, by removing the contour of the background image after extracting the contour of the input image and the contour of the background image, the background removal efficiency can be improved and frame detection can be performed with high accuracy.

FIG. 6 is a diagram illustrating a second example of the functional configuration of the frame detection unit in the present embodiment. The frame detection unit 22B of the second example includes a background removal unit 226, a contour extraction unit 227, a straight line detection unit 224, and a frame extraction unit 225 as functional components. The second example is an example in which the background image is removed from the input image, the contour is extracted from the image after the background removal, and frame detection is performed on the edge image from which the background has been removed. The background removal unit 226 removes the background image from the captured image and generates a captured image that does not include the background. The contour extraction unit 227 performs edge detection on the captured image after background removal, and extracts the contour of the captured image. The straight line detection unit 224 performs straight line detection processing such as Hough transform on the edge image, and extracts a straight line portion in the image. The frame extraction unit 225 extracts a frame surrounded by four line segments in a straight line portion in the extracted image, and outputs it as frame coordinates. As described above, when the conditions such as the brightness of the image are approximated between the captured image and the background image, the outline can be extracted from the image after the background is removed, and the frame detection can be executed with high accuracy.

FIG. 7 is a diagram for explaining the arrangement determination processing in the present embodiment. In the illustrated example, as in FIG. 3, an image obtained by capturing a situation in which a moving image in which a regular registrant's face is reflected is reproduced on a portable display terminal is input as a captured image. Is shown. The arrangement determination unit 23 acquires face arrangement information of the target person obtained as one of the face detection results by the face recognition unit 21. The face arrangement information includes a face frame 68 indicating the face contour position in the input image of the captured image. In addition, the arrangement determination unit 23 acquires frame arrangement information of the display terminal obtained as one of the frame detection results by the frame detection unit 22. As shown in FIG. 3, the frame arrangement information includes a frame 58 indicating the contour position of the display terminal detected in the captured image. As the placement determination process, the placement determination unit 23 compares the positions of both the face frame 68 and the frame 58 and determines whether the face frame 68 exists in the area inside the frame 58, that is, the target person within the frame of the display terminal. It is determined whether or not the face is located. When there is a face frame 68 inside the frame 58 as shown in FIG. 7, the arrangement determination unit 23 outputs an arrangement determination result that there is a face image in the frame. The validity determination unit 25 considers that the face image detected in the captured image is a face image displayed by the display terminal when the face image exists in the frame based on the result of the placement determination by the placement determination unit 23, and the face The collation result is determined to be invalid. Thus, when the determination result by the validity determination part 25 becomes invalid determination, the impersonation generation using the display terminal is detected. In the case of invalidity determination, the validity determination unit 25 outputs denial as the authentication result of face authentication.

FIG. 8 is a flowchart showing the procedure of a first example of face authentication processing in the first embodiment. The calculation unit 20A of the face authentication apparatus captures the input image in the face recognition unit 21 (S11), and performs face recognition processing of the target person in the input image (S12). The computing unit 20A determines whether or not the face recognition unit 21 has detected the face of the target person (S13), and continues the face recognition process until a face is detected in the input image. When the face of the target person is detected (S13: Yes), the calculation unit 20A performs face matching of the detected target person in the face matching unit 24 (S14). The calculation unit 20A determines whether or not the facial feature information of the target person matches the matching data of the registrant's face in the face matching performed by the face matching unit 24 (S15), and if they match (S15: Yes), Continue face recognition processing. On the other hand, when the feature information of the face of the target person does not match the collation data of the registrant's face (S15: No), the calculation unit 20A performs an invalid determination in the validity determination unit 25 and rejects it as a face authentication authentication result. Is output.

In addition, the calculation unit 20A performs frame detection in the input image at the frame detection unit 22 (S16), and the layout determination unit 23 performs layout determination between the face position by face detection and the frame position by frame detection ( S17). In the arrangement determination by the arrangement determination unit 23, the calculation unit 20A detects a frame in the input image and determines whether or not a face exists inside the frame, that is, whether or not a frame surrounding the outside of the target person's face exists. Determine (S18). When there is a frame surrounding the outside of the target person's face (S18: Yes), the calculation unit 20A determines invalidity in the validity determination unit 25, and outputs denial as the authentication result of face authentication. On the other hand, when there is no frame surrounding the outside of the target person's face (S18: No), the calculation unit 20A performs the validity determination in the validity determination unit 25 and outputs an approval as the authentication result of the face authentication.

In the face authentication process, the face detection and the frame detection may be executed in parallel, or one process is executed in series and then the other process is executed. The execution timing and order of the processing are not limited. Face matching and placement determination may be performed in parallel with each other, or one of the processes may be performed in series and the other may be performed. Implementation timing and order are not limited.

FIG. 9 is a flowchart showing the procedure of a second example of the face authentication process in the first embodiment. The second example of face authentication processing shows a processing example in the case where there are a plurality of faces in the input image of the captured image. The images captured for face authentication include, for example, a plurality of face images including a plurality of face images of a person, a plurality of face images of a spoofed face image and a face image of the person or another person. May be detected.

The computing unit 20A of the face authentication apparatus captures the input image in the face recognition unit 21 (S21), and performs face recognition processing of the target person in the input image (S22). At this time, the calculation unit 20A acquires the face detection number (N) as the number of face images detected by the face recognition unit 21. The calculation unit 20A determines whether the face detection number (N) is greater than 0, that is, whether face detection including the target person has been performed (S23), and continues face recognition processing until a face is detected in the input image. To do. When one or more faces are detected and the face detection number (N) is 1 or more (S23: Yes), the calculation unit 20A performs frame detection in the input image in the frame detection unit 22 (S24). Further, the calculation unit 20A starts from the initial value k = 1 with the face authentication process counter set to k, and the face matching unit 24 performs face matching on the k-th face among the detected faces (S25). The calculation unit 20A determines whether or not the facial feature information of the processing target matches the collation data of the registrant's face in the face matching performed by the face matching unit 24 (S26).

Here, when the feature information of the face to be processed does not match the collation data of the registrant's face (S26: No), the calculation unit 20A increments the face authentication processing counter to k = k + 1, and the unprocessed face Is determined, for example, whether the number of face processing does not exceed the number of face detections (whether k ≦ N) (S27). When there is an unprocessed face (S27: Yes), the arithmetic unit 20A performs face collation for the next k-th face (S25, S26), and repeats the same processing until there is no unprocessed face. When the face collation result is inconsistent and there is no unprocessed face (S26: No, S27: No), the calculation unit 20A performs invalidity determination in the validity determination unit 25, and outputs denial as the authentication result of face authentication To do.

On the other hand, when the feature information of the face to be processed matches the collation data of the registrant's face (S26: Yes), the calculation unit 20A determines the position of the face by the face detection for the kth face in the placement determination unit 23. An arrangement determination with the position of the frame by frame detection is performed (S28). In the placement determination by the placement determination unit 23, the calculation unit 20A detects a frame in the input image and determines whether a face exists inside the frame, that is, whether there is a frame surrounding the outside of the face to be processed. Determine (S29). When there is a frame surrounding the outside of the face to be processed (S29: Yes), the arithmetic unit 20A increments the face authentication processing counter to k = k + 1, and determines whether there is an unprocessed face ( S27). When there is an unprocessed face (S27: Yes), the arithmetic unit 20A performs face matching and arrangement determination on the next kth face in the same manner (S25, S26, S28, S29). When there is no unprocessed face (S27: No), the calculation unit 20A determines invalidity in the validity determination unit 25, and outputs denial as the authentication result of face authentication. On the other hand, when there is no frame surrounding the outside of the face to be processed (S29: No), the calculation unit 20A performs the validity determination in the validity determination unit 25, and outputs an approval as the authentication result of the face authentication.

In the face authentication process, the face detection and the frame detection may be executed in parallel, or one process is executed in series and then the other process is executed. The execution timing and order of the processing are not limited. Face collation and placement determination may be performed in parallel for each of a plurality of detected faces, or one process is performed in series and the other is performed in parallel. Processing may be executed, and the execution timing and order of the processing are not limited.

Further, in the frame detection process, a plurality of frames of a moving image may be used as an input image, frame detection may be performed on each frame, and a frame detection result obtained by integrating detection results on the plurality of frames may be acquired. Good. For example, there may be a case where a frame cannot be detected from an image of one frame, for example, when a person's hand is on the display unit of a portable display terminal. Thus, even if a frame detection result is missing in one frame, for example, when a frame is detected in an image of 3 frames out of 5 frames, the detection results in multiple frames are integrated. Therefore, the frame detection can be executed with high accuracy, and the stability of the frame detection can be improved.

As described above, according to this embodiment, fraud such as impersonation using a portable display terminal can be detected by frame detection, and impersonation at the time of authentication can be detected with high accuracy. Thereby, it is possible to realize a highly accurate face authentication device and face authentication system.

It should be noted that in the face authentication device, the facial feature information and the biological information may be combined in the face matching using the biological information acquired from the captured image of the moving image captured by the imaging unit. The reliability of face authentication can be improved by combining biometric information and face image information.

(Second Embodiment of Face Authentication Device)
FIG. 10 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the second embodiment. The calculation unit 20B of the face authentication device according to the second embodiment includes, as functional components, a face recognition unit 21, a frame detection unit 22, an arrangement determination unit 23, a face collation unit 24, an validity determination unit 25a, and an ID authentication unit. 26. The functions and operations of the face recognition unit 21, the frame detection unit 22, the arrangement determination unit 23, and the face collation unit 24 are the same as those in the first embodiment shown in FIG. The explanation will focus on the part.

The ID authentication unit 26 uses the ID information acquired by the ID reading unit 13 to authenticate ID information recorded on an ID card such as an identification card, a membership card, or a driver's license possessed by the target person. Note that the ID authentication unit 26 does not use the ID information acquired from the ID card, but input information such as a password or an identification code input by a user who is the target person, or for each person such as an iris or fingerprint of the target person. The identification information may be authenticated using the unique identification information as the ID information.

The validity determination unit 25a has functions of an authentication determination unit and an authentication result output unit. The validity determination unit 25a determines whether the face of the target person in the input image is valid based on the arrangement determination result by the arrangement determination unit 23, the face verification result by the face verification unit 24, and the ID authentication result by the ID authentication unit 26, that is, the face It is determined whether the authentication result is approval or not, and the authentication result is output.

As described above, according to this embodiment, fraud such as impersonation using a portable display terminal can be detected by frame detection, and impersonation at the time of authentication can be detected with high accuracy. Thereby, it is possible to realize a highly accurate face authentication device and face authentication system. Further, the reliability of face authentication can be improved by combining ID information and face image information.

(Third Embodiment of Face Authentication Device)
FIG. 11 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the third embodiment. The calculation unit 20C of the face authentication device according to the third embodiment includes a face recognition unit 21, a frame detection unit 22, an arrangement determination unit 23, and a face matching unit 24a as functional components. The functions and operations of the face recognition unit 21, the frame detection unit 22, and the arrangement determination unit 23 are the same as those in the first embodiment shown in FIG. To do.

The face matching unit 24a has functions of an authentication determination unit and an authentication result output unit. The face collating unit 24a determines whether the frame is not detected in the captured image based on the arrangement determination result by the arrangement determining unit 23 or when the frame is detected but does not surround the outside of the face. The face in the captured image detected in this way is collated to determine whether or not it matches the registrant's face. The face matching unit 24a determines whether the authentication result of face authentication is approval or not based on the result of face matching, and outputs the authentication result.

FIG. 12 is a flowchart showing a procedure of a first example of face authentication processing in the third embodiment. The calculation unit 20C of the face authentication apparatus captures the input image in the face recognition unit 21 (S31), and performs face recognition processing of the target person in the input image (S32). The calculation unit 20C determines whether or not the face recognition unit 21 has detected the face of the target person (S33), and continues the face recognition processing until a face is detected in the input image. When the face of the target person is detected (S33: Yes), the calculation unit 20C performs frame detection in the input image at the frame detection unit 22 (S34), and the placement determination unit 23 determines the position of the face by face detection. Arrangement determination with the position of the frame by frame detection is performed (S35). In the placement determination by the placement determination unit 23, the calculation unit 20C detects a frame in the input image and determines whether a face exists inside the frame, that is, whether there is a frame surrounding the outside of the target person's face. Determination is made (S36). When there is a frame surrounding the outside of the target person's face (S36: Yes), the calculation unit 20C determines invalidity in the face matching unit 24a, and outputs a denial as the authentication result of the face authentication.

On the other hand, when there is no frame surrounding the outside of the face of the target person (S36: No), the calculation unit 20C performs face matching of the detected target person in the face matching unit 24a (S37). The calculation unit 20C determines whether or not the facial feature information of the target person matches the matching data of the registrant's face in the face matching by the face matching unit 24a (S38), and if it matches (S38: Yes), Approval is output as the authentication result of face authentication. On the other hand, when the feature information of the face of the target person does not match the face collation data of the registrant (S38: No), the calculation unit 20C determines invalidity in the face collation unit 24a and rejects it as the authentication result of face authentication. Is output.

In the face authentication process, the face detection and the frame detection may be executed in parallel, or one process is executed in series and then the other process is executed. The execution timing and order of the processing are not limited.

FIG. 13 is a flowchart showing the procedure of the second example of the face authentication process in the third embodiment. The second example of face authentication processing shows a processing example in the case where there are a plurality of faces in the input image of the captured image.

The computing unit 20C of the face authentication apparatus captures the input image in the face recognition unit 21 (S41), and performs face recognition processing of the target person in the input image (S42). At this time, the calculation unit 20C acquires the face detection number (N) as the number of face images detected by the face recognition unit 21. The calculation unit 20C determines whether the face detection number (N) is greater than 0, that is, whether face detection including the target person has been performed (S43), and continues face recognition processing until a face is detected in the input image. To do. When one or more faces are detected and the face detection number (N) is 1 or more (S43: Yes), the calculation unit 20C performs frame detection in the input image in the frame detection unit 22 (S44). Further, the calculation unit 20C starts from the initial value k = 1 with the face authentication processing counter set to k, and the placement determination unit 23 detects the face position and frame detection by face detection for the kth face among the detected faces. The frame is determined to be positioned with respect to (S45). In the placement determination by the placement determination unit 23, the calculation unit 20C detects a frame in the input image and determines whether a face exists inside the frame, that is, whether a frame surrounding the outside of the processing target face exists. Determine (S46).

Here, when there is a frame surrounding the outside of the face to be processed (S46: Yes), the arithmetic unit 20C increments the face authentication processing counter to k = k + 1, and determines whether or not an unprocessed face exists. For example, it is determined whether the number of face processes exceeds the number of detected faces (whether k ≦ N) (S47). When there is an unprocessed face (S47: Yes), the calculation unit 20C performs an arrangement determination for the next k-th face (S45, S46), and repeats the same process until there is no unprocessed face. When the placement determination result has a frame and there is no unprocessed face (S46: Yes, S47: No), the calculation unit 20C performs an invalid determination in the face matching unit 24a and rejects the authentication result of face authentication. Output.

On the other hand, when there is no frame surrounding the outside of the face to be processed (S46: No), the calculation unit 20C performs face matching on the kth face in the face matching unit 24a (S48). The calculation unit 20C determines whether or not the feature information of the face to be processed matches the collation data of the registrant's face in the face collation performed by the face collation unit 24a (S49). When the feature information of the face to be processed does not match the collation data of the registrant's face (S49: No), the calculation unit 20C increments the face authentication process counter to k = k + 1, and there is an unprocessed face. Whether or not (S47). When there is an unprocessed face (S47: Yes), the calculation unit 20C performs the same arrangement determination and face matching for the next k-th face (S45, S46, S48, S49). When there is no unprocessed face (S47: No), the calculation unit 20C determines invalidity in the face matching unit 24a, and outputs denial as the authentication result of face authentication. On the other hand, when the feature information of the face to be processed matches the collation data of the registrant's face (S49: Yes), the calculation unit 20C determines that the face collation unit 24a is valid and approves it as the authentication result of face authentication. Is output.

(Fourth Embodiment of Face Authentication Device)
FIG. 14 is a block diagram illustrating a functional configuration of the face authentication apparatus according to the fourth embodiment. The calculation unit 20D of the face authentication device according to the fourth embodiment includes, as functional components, a face recognition unit 21, a frame detection unit 22, an arrangement determination unit 23, a face collation unit 24a, an validity determination unit 25b, and an ID authentication unit. 26. The functions and operations of the face recognition unit 21, the frame detection unit 22, and the arrangement determination unit 23 are the same as those in the first embodiment shown in FIG. 2, and the functions and operations of the face matching unit 24a are 11 is the same as that of the third embodiment shown in FIG.

The validity determination unit 25b has functions of an authentication determination unit and an authentication result output unit. The validity determination unit 25b determines whether the face of the target person in the input image is valid based on the face collation result by the face collation unit 24a and the ID authentication result by the ID authentication unit 26, that is, whether the authentication result of face authentication is approval or disapproval. And the authentication result is output.

(Face authentication system configuration, second example)
FIG. 15 is a diagram illustrating a second example of the configuration of the face authentication system according to the present embodiment. The face authentication system 100 B includes a face authentication device 10 B, captures an image of the target person 30, performs face authentication, and outputs an authentication result to the control target 40.

The face authentication device 10 B includes an imaging unit 11, a display unit 12, an input device 14, a storage device 15, and a calculation unit 20. Note that the functions and operations of the imaging unit 11, the display unit 12, and the calculation unit 20 are the same as those in the first example shown in FIG. 1, and description thereof will be omitted here, and different parts will be mainly described. The input device 14 has a function of an ID input unit for inputting ID information. The input device 14 includes input devices such as a touch panel, a keyboard, and a mouse. Based on an input operation by the target person 30 or another operator, an ID such as a member number, an identification number, and a driver's license number of the target person 30 Enter information. The storage device 15 has a function of a storage unit that stores collation data. The storage device 15 includes a storage device such as a memory and a hard disk. The storage device 15 stores and accumulates registrant face collation data (face image image data, face feature information, etc.), and the ID of the target person 30. The collation data corresponding to the information is read and output to the calculation unit 20. The calculation unit 20 executes various calculation processes related to the face authentication described above and outputs an authentication result.

It should be noted that the target person 30 can input the password from the input device 14 and perform password authentication, or face authentication combining face verification and password authentication may be performed.

Also in the face authentication system 100B of the second example like this, the face detection and the frame detection are executed as in the face authentication system 100A of the first example described above, and the portable display terminal is impersonated by the frame detection. It is possible to detect fraud such as, and the accuracy of face authentication of the target person 30 can be improved.

(Face authentication system configuration, third example)
FIG. 16 is a diagram illustrating a third example of the configuration of the face authentication system according to the present embodiment. The face authentication system 100 C includes a face authentication device 10 C and a server device 80. The face authentication system 100 C images the target person 30 to perform face authentication, and outputs an authentication result to the control target 40. The face authentication device 10 C and the server device 80 are connected via a network 70 so that various types of information can be transmitted and received.

The face authentication device 10 C includes an imaging unit 11, a display unit 12, an ID reading unit 13, and a control unit 16. Note that the functions and operations of the imaging unit 11, the display unit 12, and the ID reading unit 13 are the same as those in the first example shown in FIG. 1, and description thereof will be omitted here, and different parts will be mainly described. The control unit 16 is configured by a computer such as a PC and has a communication unit having a communication function. The control unit 16 encodes information used for authentication, such as image information of a captured image acquired by the imaging unit 11 and ID information acquired by the ID reading unit 13, and communicates with the server device 80. Further, the control unit 16 inputs processing information such as an authentication result of face authentication transmitted from the server device 80, and transmits it to the display unit 12 for display together with an operation guidance screen and an input image of a captured image.

The server device 80 includes an arithmetic device 81 and a storage device 82. The calculation device 81 is configured by a computer including a processor, a memory, a communication interface, and the like, and executes various calculation processes related to face authentication, like the calculation unit 20 of the face authentication device 10A of the first example. The storage device 82 has a function of a storage unit that stores collation data. The storage device 82 includes a storage device such as a memory and a hard disk. The storage device 82 stores and accumulates registrant's face collation data (face image image data, face feature information, etc.), and the ID of the target person 30. The collation data corresponding to the information is read and output to the arithmetic unit 81. The arithmetic device 81 executes various arithmetic processes related to the face authentication described above and outputs an authentication result.

Various calculation processes related to face authentication may be performed in the control unit 16 of the face authentication device 10C in the same manner as the calculation unit 20 of the face authentication device 10A of the first example, or with the control unit 16 of the face authentication device 10C. You may carry out by distributing with the arithmetic unit 81 of the server apparatus 80. FIG. When performing the face authentication process in the control unit 16, the face authentication apparatus 10C acquires the collation data from the server device 80 via the network 70, and executes the face authentication process. In addition, the authentication result may be transmitted via the network 70 and notified not only to the control target 40 and the server device 80 but also to a management device at another location.

Also in the face authentication system 100C of the third example like this, the face detection and the frame detection are executed as in the case of the face authentication system 100A of the first example, and the portable display terminal is impersonated by the frame detection. It is possible to detect fraud such as, and the accuracy of face authentication of the target person 30 can be improved. Further, by using the collation data accumulated in the storage device 82 of the server device 80, face authentication using a large number of collation data becomes possible.

As described above, the

face authentication devices

10A, 10B, and 10C as examples of the face authentication processing device according to the present embodiment acquire a captured image obtained by capturing a target person, and detect the face of the target person in an input image of the captured image. A face recognition unit 21 as a face detection unit, a frame detection unit 22 that detects a linear frame in the input image, face position information acquired by face detection, and frame position information acquired by frame detection. And using an arrangement determination unit 23 that determines whether or not a frame surrounding the face of the target person exists, and an authentication determination unit that determines the validity of the authentication result of the target person's face information based on the frame arrangement determination result. The validity determining unit 25 or the face matching unit 24a.

With the above configuration, the frame of the captured image is detected, the presence / absence of the frame in the captured image and the arrangement of the frame with respect to the face are determined. Based on the determination result of the frame, fraud such as impersonation using a portable display terminal is detected. Detection is possible, and impersonation at the time of authentication can be detected with high accuracy. Thereby, the reliability and stability of face authentication can be improved, and a highly accurate face authentication processing device and face authentication processing system can be realized.

The computing unit 20A of the face authentication apparatus 10A includes a face matching unit 24 that matches face information of a captured image of the target person with registered face information, and the validity determining unit 25 includes a face matching result of the face information. The validity of the authentication result of the face information of the target person is determined based on the frame arrangement determination result. Thereby, fraud such as impersonation using a portable display terminal can be detected, and impersonation at the time of authentication can be accurately detected.

The computing unit 20C of the face authentication device 10A includes a face matching unit 24a that matches face information of a captured image of the target person with registered face information, and the face matching unit 24a includes a target in the frame placement determination result. When there is no frame surrounding the person's face, face information is collated, and the validity of the authentication result of the target person's face information is determined based on the face collation result of the face information. Thereby, fraud such as impersonation using a portable display terminal can be detected, and impersonation at the time of authentication can be accurately detected.

Further, the

calculation units

20B and 20D of the face authentication device 10A include an ID authentication unit 26 that acquires and authenticates the ID information of the target person, and the

validity determination units

25a and 25b further use the authentication result of the ID information, The validity of the authentication result of the target person's face information is determined. Thereby, the reliability of face authentication can be improved by combining ID information and face image information.

Also, the

frame detection units

22A and 22B of the face authentication device 10A have

background removal units

223 and 226 that remove the background in the input image. As a result, by removing the background portion from the captured image, frame detection can be performed from an image of only a subject such as a person, and frame detection accuracy can be improved.

Further, the

background removal units

223 and 226 of the face authentication device 10A remove the background of the input image using the background when the target person is imaged as the background image. Alternatively, the

background removal units

223 and 226 use the background when the target person does not exist as the background image, and removes the background of the input image. As a result, frame detection can be executed from an image of only a subject having a background and no movement, such as a person, and frame detection accuracy can be improved.

Further, the

frame detection units

22A and 22B of the face authentication apparatus 10A use a plurality of frames of moving images as input images, and acquire a detection result of a frame obtained by integrating the detection results of the plurality of frames. As a result, even if a frame cannot be detected in an image of one frame and frame detection results are missing, frame detection can be performed with high accuracy by integrating detection results in a plurality of frames. Stability can be improved.

In addition, the face authentication systems 100A, 100B, and 100C as examples of the face authentication processing system according to the present embodiment acquire the captured image of the target person and the captured image of the target person, and in the input image of the captured image By face recognition unit 21 as a face detection unit for detecting the face of the target person, frame detection unit 22 for detecting a linear frame in the input image, face position information obtained by face detection, and frame detection Using the acquired frame position information, the face determination unit 23 that determines whether there is a frame surrounding the face of the target person, and the face that matches the face information of the captured image of the target person with the registered face information Based on the collation unit 24, the face collation result of the face information, and the frame arrangement determination result, an authentication determination unit that determines the validity of the authentication result of the face information of the target person, and authentication that outputs the authentication result to the control target Result output It includes a validity determination section 25 as an imaging unit 11, the face recognition unit 21, the frame detector 22, arranged determination unit 23, a display unit 12 for displaying at least one of the processing result of the validity determination section 25, the.

With the above configuration, the frame of the captured image is detected, and when the frame is detected, the arrangement of the frame with respect to the face is determined. Based on the determination result of the arrangement of the frame, fraud such as impersonation using a portable display terminal is detected. It can be detected. Thereby, the impersonation at the time of authentication can be detected with high accuracy, and the reliability and stability of face authentication can be improved.

The

face authentication systems

100A, 100B, and 100C include an ID reading unit 13 that reads ID information of a target person, or an input device 14 as an ID input unit that inputs ID information, and an ID authentication unit that authenticates ID information. 26, the validity determination unit 25 further determines the validity of the authentication result of the face information of the target person using the authentication result of the ID information. Thereby, the reliability of face authentication can be improved by combining ID information and face image information.

Further, in the

face authentication systems

100A and 100C, the ID information includes the target person face collation data, and the face collation unit 24 collates the face information of the captured image of the target person using the ID information collation data. Thereby, face authentication is possible using the collation data included in the ID information, and the reliability of face authentication can be improved by combining the frame detection result, the face detection result, and the ID information.

In addition, the

face authentication systems

100B and 100C include

storage devices

15 and 82 as storage units for storing the verification data of the target person's face, and the face verification unit 24 acquires the verification data from the

storage devices

15 and 82. The face information of the captured image of the target person is collated. Thereby, face collation can be performed using collation data stored in a storage device such as its own device or an external server device, and face authentication using a large number of collation data is possible.

Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are of course within the technical scope of the present disclosure. Is done. In addition, the constituent elements in the above-described embodiment may be arbitrarily combined without departing from the spirit of the invention.

The present disclosure is useful as a face authentication processing device, a face authentication processing method, and a face authentication processing system that can accurately detect impersonation during authentication.

10A, 10B, 10C Face authentication device 11 Imaging unit 12 Display unit 13 ID reading unit 14 Input device 15 Storage device 16

Control unit

20, 20A, 20B, 20C, 20D Calculation unit 21

Face recognition unit

22, 22A, 22B Frame detection unit DESCRIPTION OF SYMBOLS 23 Arrangement | positioning

determination part

24,24a

Face collation part

25,25a, 25b Validity determination part 26 ID authentication part 30 Target person 35 ID card 40 Control object 70 Network 80 Server apparatus 81 Arithmetic apparatus 82

Storage apparatus

100A, 100B, 100C

Face authentication system

221, 222, 227

Outline extraction unit

223, 226 Background removal unit 224 Straight line detection unit 225 Frame extraction unit

Claims

A face detection unit that acquires a captured image obtained by capturing the target person and detects the face of the target person in the input image of the captured image;
A frame detection unit for detecting a linear frame in the input image;
An arrangement determining unit that determines whether a frame surrounding the face of the target person exists using the face position information acquired by detecting the face and the frame position information acquired by detecting the frame;
An authentication determination unit that determines the validity of the authentication result of the target person's face information based on the frame placement determination result;
A face authentication processing apparatus.
The face authentication processing device according to claim 1,
A face collation unit that collates face information of a captured image of the target person and registered face information;
The authentication determination unit determines the validity of the authentication result of the target person's face information based on the face matching result of the face information and the arrangement determination result of the frame;
Face authentication processing device.
The face authentication processing device according to claim 1,
A face collation unit that collates face information of a captured image of the target person and registered face information;
The face collation unit performs collation of the face information when there is no frame surrounding the face of the target person in the frame arrangement determination result;
The authentication determination unit determines the validity of the authentication result of the face information of the target person based on the face matching result of the face information.
Face authentication processing device.
The face authentication processing device according to any one of claims 1 to 3,
An ID authentication unit that acquires and authenticates the ID information of the target person,
The authentication determination unit further determines the validity of the authentication result of the face information of the target person using the authentication result of the ID information;
Face authentication processing device.
The face authentication processing device according to claim 1,
The frame detection unit includes a background removal unit that removes a background in the input image.
Face authentication processing device.
The face authentication processing device according to claim 5,
The background removing unit uses a background when the target person is imaged as a background image, and removes the background of the input image.
Face authentication processing device.
The face authentication processing device according to claim 5,
The background removal unit uses a background when the target person does not exist as a background image, and removes the background of the input image.
Face authentication processing device.
The face authentication processing device according to claim 1,
The frame detection unit uses a plurality of frames of moving images as the input image, and acquires a detection result of a frame obtained by integrating detection results in a plurality of frames.
Face authentication processing device.
A face authentication processing method in a face authentication processing apparatus for performing face authentication of a target person,
Obtaining a captured image of the target person, detecting the face of the target person in the input image of the captured image,
Detecting a linear frame in the input image;
Using the face position information acquired by detecting the face and the frame position information acquired by detecting the frame, it is determined whether there is a frame surrounding the target person's face,
Determining the validity of the authentication result of the target person's face information based on the frame placement determination result;
Face authentication processing method.
An imaging unit for imaging a target person;
A face detection unit that acquires a captured image of the target person and detects a face of the target person in an input image of the captured image;
A frame detection unit for detecting a linear frame in the input image;
An arrangement determining unit that determines whether a frame surrounding the face of the target person exists using the face position information acquired by detecting the face and the frame position information acquired by detecting the frame;
A face matching unit that matches face information of a captured image of the target person with registered face information;
An authentication determination unit that determines the validity of the authentication result of the face information of the target person based on the face comparison result of the face information and the frame placement determination result;
An authentication result output unit for outputting the authentication result to a control target;
A display unit that displays a processing result of at least one of the imaging unit, the face detection unit, the frame detection unit, the arrangement determination unit, and the authentication determination unit;
A face authentication processing system.
The face authentication processing system according to claim 10,
An ID reading unit that reads ID information of the target person, or an ID input unit that inputs the ID information,
An ID authentication unit for authenticating the ID information;
The authentication determination unit further determines the validity of the authentication result of the face information of the target person using the authentication result of the ID information;
Face recognition processing system.
The face authentication processing system according to claim 11,
The ID information includes collation data of the target person's face,
The face collation unit collates face information of a captured image of the target person using collation data of the ID information;
Face recognition processing system.
The face authentication processing system according to claim 11,
A storage unit for storing collation data of the target person's face;
The face collation unit obtains collation data from the storage unit and collates face information of a captured image of the target person;
Face recognition processing system.