JP6666266B2 - Enrollment and authentication methods and systems - Google Patents

Description

1. TECHNICAL FIELD The field of this disclosure relates to identity verification using camera and display based methods of user enrollment and authentication.

2. 2. Description of the Related Art In order to prevent fraudulent activities in e-commerce, many resources have been invested in developing reliable schemes for identity verification. Existing methods for identity verification can rely on items owned by the user, such as smart cards, or data known to the user, such as passwords, personal identification numbers (PINs) or other information.

  Considerable effort has also been put into identity verification using biometrics based on physiological and / or behavioral characteristics. Features used for biometric authentication may include fingerprint, retinal or iris scanning, voice, signature, and / or face.

  U.S. Pat. No. 6,077,086 discloses biometric authentication using a prominent line of the hand when scanned by a scanning device such as a scanner with a planar platen.

US Patent No. 7,327,858

  Various computerized biometric methods for user enrollment and authentication are provided herein. During the authentication process, the target candidate is presented with the previously stored enrollment image on the display.

  It will be appreciated that the use of body part terms in the present invention is for illustrative purposes only and should be referred to as "observed object" as defined below.

  The user is instructed to present the body part to the camera while the user holds the body part in free space without supporting the camera. Also, the user can use the portable camera to capture any stationary object / scene, and will then be required to align the camera with the object. Using the camera, candidate images of the body part are captured and presented along with previously stored enrollment images. The target candidate aligns the candidate image with a previously stored enrollment image. Once registered, the candidate image is verified as a true image of the target candidate, and the target candidate is authenticated as a previously enrolled target. Previously stored enrollment images can be generated independently of the subject. The same enrollment image can be displayed for the enrollment of multiple users that have not been previously enrolled.

  In the enrollment process, the user may hold the camera in free space without supporting the body part. Alternatively, the user may hold the camera in free space without support for still scenes / objects. Using a camera, it is possible to capture an enrollment image of a body part and its background. This authentication may include capturing a time series of candidate images of the candidate body part and its background.

  In the process of capturing, the time series of the candidate image group is superimposed on the enrollment image together with the respective backgrounds and presented on the display.

  The observation object may be part of an item such as the entire face, hands, ears, eyes, and photographs or documents. The target candidate can align the features of the observation object in the candidate image with the corresponding features in the enrollment image. This feature and corresponding features may be lines, bumps, moles, scars, bruises, different skin pigmentation, nevus, and wrinkles. Further, a feature and corresponding feature may be a point, an edge, a unique visual feature, a color or texture of an area, or any item having contrast distinguishable from other portions of the captured image of the target portion. It can be.

  Verification can be performed by comparing the enrollment image with the authentication image. This alignment may include alignment of horizontal and vertical image coordinates, angular orientation in image space, and real space distance to the camera.

  Before presenting a previously stored enrollment image, the enrollment image can be subjected to a process of performing a horizontal mirror flip of the enrollment image.

  The target candidate can be instructed to align features of the observed object in the candidate image with corresponding features in the enrollment image. When the target candidate aligns features of the observed object in the candidate image with corresponding features in the enrollment image, one or more candidate images are verified as authentic images of the target candidate. Thereby, one or more candidate images can be authenticated as a genuine image of the target candidate. The target candidate is then authenticated as a previously enrolled user by comparing features in the candidate image with corresponding features in the enrollment image.

  The enrollment pre-image of the observation object can be captured while the user holds the observation object of the user in free space without supporting the camera. This pre-enrollment image may be presented to the user on a display. When the enrollment pre-image is presented, the user can superimpose the symbols on the imaged features of the observation object presented on the display. When the symbol is overlaid on the imaging feature of the observation object, the enrollment image is stored with the position and orientation of each of the symbol and the group of symbols in image space in the pre-enrollment image.

  These symbols can be from a predetermined group of symbols, such as symbols or characters generated by typing on a keyboard. Alternatively, these symbols may be manually defined, such as handwritten symbols, shapes, or any other symbols defined by the target candidate.

  During the registration process, the user can define and set a sequence of steps, where each step applies different / identical symbols on the screen without capturing the object of interest. Become. The detailed definition of these symbols is referred to as one or more password symbols. Furthermore, the above-mentioned sequence steps can be timed so that only the target candidate can repeat the synchronized and registered sequence steps known to him during the authentication process.

  Further, the enrollment and authentication may include presenting the observation object in a specific time frame predefined during the enrollment process.

  The method of time synchronization can be implemented by any indication of time progress, such as a timer, progress bar, vibration, or any other form of visual and audio indication of the length of time measured It is.

  Another implementation may be any keyboard tap during a time interval, where the visual symbol is a keyboard by the subject candidate, including gaps between the step sequences defined herein, set during the enrollment. Indicates the length of time of the press.

  The step sequence defined here is effective only when the target candidate can reproduce or synchronize the same step sequence as at the time of registration in the authentication phase. The invention also describes on-screen guidance to assist the subject candidate in reproducing the same registration step in the authentication phase.

  Prior to superposition, the symbols may be rearranged, resized, or rotated. Presentation of the enrollment image to the target candidate on the display can include at least a portion of the stored symbol and the image of the observed object. Alternatively, presenting the enrollment image on the display to the target candidate may include presenting the stored symbols without any portion of the image of the observation object.

  A plurality of previously defined enrollment scales may be presented to the user on the display. The user can select one or more scales presented on the display to generate a selected enrollment scale. The user can overlay and / or reposition one of the enrollment images using the selected scale. The selected scale can be stored as an enrollment image. Authentication includes presenting on a display a timeline of candidate images superimposed on a selected enrollment scale. The target candidate can also position the candidate image using the selected enrollment scale.

  In the authentication process, the presentation of the enrollment scale to the target candidate may include a plurality of enrollment scales, and the target candidate may select the enrollment scale.

  Various portable computer systems are provided herein that enable implementation of the computerized biometric methods disclosed herein, and the camera and display are part of the portable computer system.

  Various computer-readable media are provided herein that encode processing instructions for causing a processor to perform the methods disclosed herein.

  The foregoing and / or other aspects will become apparent from the following detailed description when considered in conjunction with the accompanying drawings.

  The present invention is described herein, by way of example and not by way of limitation, with reference to the accompanying drawings.

FIG. 2 illustrates a simplified block diagram of a portable computer system for operating various features of the present invention. 4 illustrates a flowchart of a method, according to certain aspects of the present invention. FIG. 4 illustrates an enrollment image presented on a display, also with a captured background, in accordance with aspects of the invention. FIG. 3 shows images of enrollment and authentication superimposed with respective backgrounds according to the method shown in FIG. 2. Fig. 4 illustrates another method of biometric enrollment and authentication according to features of the present invention. FIG. 5 illustrates an example of a previously stored enrollment image on a display, in accordance with an aspect of the present invention. FIG. 5 illustrates an example of a previously stored enrollment image on a display, in accordance with an aspect of the present invention. 4 illustrates a flow diagram of a method that includes various features of the present invention. 4 illustrates a flow diagram of a method in accordance with various aspects of the present invention. 4 illustrates an enrollment image according to different features of the present invention. 4 illustrates an enrollment image according to different features of the present invention. 10 shows an example of an enrollment and authentication method using a timed sequence of the set steps according to the method shown in FIGS. 5 and 9. 12 shows an example of scaled symbols presented on a screen according to the method shown in FIG. 12 illustrates an example of a possible superposition of an object by a user using the scaled symbols of FIG. 12, according to the method illustrated in FIG. 12 illustrates an example of a possible superposition of an object by a user using the scaled symbols of FIG. 12, according to the method illustrated in FIG. 10 shows an example of an enrollment and authentication method using the setting of the timed sequence of steps / second according to the method shown in FIGS. 5 and 9. FIG. 7 shows an example of registration of a handwritten symbol on a screen and another auxiliary symbol for masking and / or hiding the registered symbol.

  Reference will now be made in detail to embodiments of the invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout the views. Embodiments will be described below with reference to these drawings to explain the present invention.

  As used herein, the term “registration” refers to the spatial registration of image features of different images captured at different locations and / or times of similar or identical objects in different image frames. To tell.

  As used herein, the term "identification" is used to uniquely identify the initial information, name, identification number, or target candidate that the enrolled user presents before or during the enrollment. Say receiving useful other information.

  As used herein, the term "enroll" or "enrollment" refers to the enrollment of a user in a biometric system, in which biometric information is received and stored for later identity verification or authentication. The term “enrollment image” refers to image capture during the enrollment process.

  The term "authentication" as used below refers to biometric verification that the candidate is who he claims to be during the process of identification.

  As used below, the term "verification" refers to authenticating or denying authentication as one of a previously enrolled user or a previously identified subject based on characteristics of a biometric image of the subject candidate. .

  In the context of presenting two or more images simultaneously on a display, the term “overlap” as used herein refers to (i) combining two or more images using image processing techniques and presenting the combined images And / or (ii) alternatively presenting two or more images to the target candidate such that the target candidate sees the combined overlay image.

  The term "body part" as used herein, as used in biometric enrollment and authentication, may be a hand, eye, ear, nose, face, or other part of a user.

  The term "viewable image" as defined herein includes any reproducible visual scene captured by a camera or any other imaging device. This scene can be captured with a camera and can be stationary (doors, building backgrounds, etc.) or any other form of body part, document, barcode, QR code or visual code, photograph, any kind of It can be any combination of tangible objects that are movable, such as any combination of cards or the like.

  The term "password symbol" refers to a signature, image, any shape, character, any combination of connected dots or pixels, and any other form of symbol that can be used for identification, or any combination of these features Say any of them. The handwritten symbols in the authentication will be authenticated as a biometric signature, as is well known to those skilled in image processing and biometrics technology.

  As used herein, the term "target area" refers to any part of the visual recognition image that is selected by the user and marked on the screen during registration. In authenticating the viewed image, the hot spot portion must be correlated independently of the viewed image. This correlation process is well known to those skilled in image processing.

  The term "observed object" is visually reproducible, can be captured during the registration process, and is repeatedly superimposed on a partial / deformed image on the screen with the same object and / or candidate image of the scene. The present invention relates to a viewable image that captures an object or a scene that can be matched.

  In the context of body part alignment, the term "feature" as used herein excludes the "contour" or outer edge of the body part.

  As used herein, the term "user" relates to any common designation of an end user, a system user, and / or a customer, or a subject, associated with an enrollment phase.

  As used herein, the term “subject candidate” relates to an end user, a system user, and / or a customer associated with the authentication phase.

  As used herein, the term "time synchronization method" refers to any indication of the progress of time, such as a timer, progress bar, vibration, or any other form of visual or audio indication of the length of time measured. For time measurement by a user or target candidate.

  The term "timeframe" as used herein relates to any measured time course that can be measured and reproduced by a user.

  As used herein, the term "timed sequence step" relates to steps in which the subject candidate must follow the same order and / or time synchronization as recorded and presented by the user in the enrollment phase. These sequence steps can consist of a group of candidate images of the same object or scene superimposed on the screen. Further, these sequence steps can be any combination of symbols, on-screen arrangements, drawings, keyboard taps, or any other combination of the like. The sequence step is capable of presenting the observation object during the measured time lapse. This timed step can be any combination of a given time lapse during the application of one step or a time lapse between successive steps.

  The article “a (a, an)” used in this specification such as “a certain candidate image”, “a certain enrollment image”, and “a certain background” means “one or more”, that is, “one or more”. Have the meanings of “candidate image”, “one or more enrollment images”, and “one or more backgrounds”.

  The object should include sufficient contrast, perimeter, and be reproducible so that the authentication phase can be successful.

  By way of introduction, various embodiments of the present invention use several different biometric methods to identify a user and enroll into the system, and then authenticate the subject candidate as one of the previously enrolled users. be able to. Embodiments of the present invention are intended for use with cameras and two-dimensional or three-dimensional displays available in portable computer systems, such as, for example, smartphones. In some embodiments of the present invention, an enrollment image is captured by a user presenting an observation object, such as a hand, in free space in the course of enrollment. Thereafter, when the user wishes to make an online transaction, for example, presents the same observation object, for example a hand, and similarly captures the authenticated image using a free-standing hand in the free space between the camera and the hand. It is. If the authentication image can be processed to be of the same hand as used for the enrollment image, authentication is achieved, otherwise authentication is denied.

  When the enrollment image is used again for authentication, the enrollment image when presented for authentication purposes, the impostor reconstructs the authentication image from the enrollment image when presented during the authentication process Can be blurred, grayscale or color manipulated, reduced in resolution, partially displayed, or otherwise processed separately.

  In various embodiments of the invention that will provide further details, authentication can be performed using a different computer system at a different location and / or time or date than the enrollment, and May be suitable for biometric enrollment and authentication.

  The process of authenticating and verifying identity may be similar to the processing techniques disclosed in U.S. Patent No. 7,327,858, such as by comparing certain features between an enrollment image and an authentication image. A feature used in U.S. Patent No. 7,327,858 is a prominent line of the hand. However, according to the prior art method, image registration is performed before the actual authentication process can be performed, and in the process, each image of the same target feature is included in both the enrollment image and the authentication image. Be placed.

  Various embodiments of the present invention are intended to avoid or facilitate image registration of common imaging features between the enrollment image and the authentication image. Avoiding or facilitating image registration reduces the computational load on the processor being used and / or reduces the required computation time.

  Thus, in order to enable authentication using an image capture by an observation object held in free space or a mobile device held unsupported to the object, the enrollment image and the authentication image are interposed between the enrollment image and the authentication image. It would be beneficial to have a method and system that would be useful to avoid and / or facilitate image registration at

  In some embodiments of the present invention, a portable computer system is used for enrollment and authentication, but in other embodiments, non-portable computer systems and other specialized computer systems may also be used together or instead. It can be used for

  The registration and authentication process described herein shall be combined from the set of sequence steps described above. The sequence steps presented herein are groups of steps that a subject candidate must follow in the same order and / or time synchronization. These sequence steps can consist of a group of candidate images of the same object or scene superimposed on the screen. In addition, these sequence steps can be any combination of symbols, figures, keyboard keystrokes, or any other combination of the like arranged on the screen.

  Before describing the embodiments of the present invention in detail, it will be understood that the present invention is not limited in its application to the details of design and the arrangement of components set forth in the following description or illustrated in the drawings. The invention may take other embodiments or be practiced or carried out in various ways. Also, of course, the terms and terms used herein are for the purpose of description and should not be considered limiting.

  Referring now to the drawings, FIG. 1 illustrates a simplified block diagram of a portable computer system 100 according to features of the present invention. The portable computer system 100 is connectable to a server 108 via a network 106. The portable computer system 100 can be connected to a cellular network 121 via a cellular base station transceiver 119. Portable computer system 100 includes a processor 110C connected to local data storage 102. The data communication module 104C operatively connects the processor 110C to the data network 106. The cellular communication module operatively connects the processor 110C to the cellular network 121. The portable computer system 100 includes a display 109, a global positioning system 107, a camera 105, a microphone 111, a speaker 113, a vibrator 115, and a processor such as an accelerometer / gravity sensor, Bluetooth ™, an IR sensor (not shown). A peripheral accessory device operably connected to 110C may be included.

  Reference is now made to FIG. 2, which illustrates a method of biometric enrollment and authentication 301 according to features of the present invention. The method 301 includes steps 303 and 305 for enrollment of the user, and steps 307 to 317 for authentication / non-authentication or verification of the target candidate. At step 303, an image of an observation object, such as a candidate hand, is captured using, for example, the camera 105 of the portable computer system 100 to capture an enrollment image of the body part.

  Also, here, for example, a user holding a visual recognition image of a hand in free space with respect to the camera 105 and / or the display 109 (of IPAD (trademark) used as an example of the portable computer system 100) Reference is also made to FIG. The display 109 shows the user's key, palm, wrist and arm, and the background enrollment image 20 including the user's face and torso. The camera 105 and / or the display 109 may be mounted on a stand so that the user's hands are free. FIG. 2 b shows, by way of example only, another observation object such as a key, card, document, ear, or eye that can be captured to provide an enrollment image 20.

  Reference is now made to FIG. 3, which shows an enrollment image 20 that includes only the keys presented on the display 109 of the portable computer system 100, for example. Certain features according to embodiments of the present invention include avoiding the process of detecting the contours of an enrollment image 20 of an object of interest, such as a key. Thus, in FIG. 3, the image 20 of the key of the observation object is presented together with the background captured in the enrollment process. This background is shown schematically with the busy image 22 in FIG.

  Referring back to FIG. 2, at step 305, the enrollment image 20 is associated with the user. This association of the user with the enrollment image 20 can include the user's name, birthday, identification (ID) number, or national insurance number or social security number.

  Step 307 initiates the authentication / non-authentication process of the previously enrolled subject candidate. The authentication may be at a different location and / or time or date from the enrollment steps 303-307, or may use a different computer system.

  Reference is now made to FIG. 4, which shows the display 109 during the authentication process. At step 307, the enrollment image 20 is presented on the display 109 along with the background image 22.

  Referring back to FIG. 2, in step 309, the target candidate is instructed to present the observation target object such as a key to the camera 105, and the time series of the key candidate image 40 of the target candidate such as a video is It is captured. The candidate images 40 can include images of candidate observation targets of the target candidate and respective background images 42. According to a feature of the present invention, as in the enrollment image 20, to extract and present an image of an observation target, such as a key, without a background, the contour of the image of the observation target, such as a key, is detected. , The candidate images 40 are presented with their respective backgrounds. In FIG. 4, the background of the hand candidate image 40 is schematically shown together with the house image 42.

  Still referring to FIG. 2, in step 311, during capture (step 309), a time series of candidate images 40 is displayed on display 109 along with enrollment image 20 previously captured in step 303. Thereby, in step 311, one or more candidate images 40 are superimposed on the enrollment image 20 with the respective backgrounds 42 and 22.

  At step 313, the subject candidate aligns or positions and repositions the observation object, eg, a hand, in front of camera 105 such that one or more candidate images 40 match enrollment image 20 on display 109. I do.

  Before the enrollment image 20 and / or the candidate image 40 after capturing (step 303) are displayed on the display 109 (step 311), horizontal mirror inversion or the like is used to enable the alignment in step 313. Can be processed.

  The base for the alignment in step 313 may be the feature of the observation target object such as a key in the candidate image 40 and the corresponding feature in the enrollment image 20. This feature and the corresponding feature may be a group of lines, bumps, lines, scars, areas of different skin pigmentation, nevus or wrinkles, etc. of the body part and candidate body part, respectively. In addition, these features may be contrasting features, such as edges, lines, points, or any visually distinct areas located in the object of the object image. The alignment step 313 includes alignment of the horizontal and vertical image coordinates, the angular direction in the image space, and the real space distance to the camera 105.

  At decision block 315, when one or more candidate images 40 match the enrollment image 20 on the display 109, a verification step 317 can be performed. During the verification process (step 317), the target candidate is identified by comparing the previously stored enrollment image 20 with the authenticated image by any technique known in the image processing arts, steps 303-305. Can be verified as a previously enrolled user and / or a previously identified user.

  If, at decision block 315, one or more candidate images 40 have not yet matched the enrollment image 20 on the display 109, alignment with the target candidate continues at step 313.

  Reference is now made to FIG. 5, which illustrates another method 501 of biometric enrollment and authentication according to features of the present invention. Biometric method 501 includes steps for user enrollment and authentication. In step 503, the observation object of the user is captured using the camera 105, and an enrollment image 20 of the observation object such as a key is generated. At step 305, the enrollment image 20 is associated with the user. The authentication phase begins at step 507, where the enrollment image 20 is presented on the display 109 with or without the background 22. In step 509, the target candidate is instructed to present a key to the camera 105, and the time series of the candidate image 40 of the observation target object such as a key of the target candidate is captured. The candidate image 40 includes an image of the candidate observation object of the target candidate presented with or without the background image 42.

  At step 511, also during capture (step 509), the time series of candidate images is displayed on display 109 along with the display of enrollment image 20 previously captured (with or without background 22) at step 503. Displayed above. In step 511, the candidate image 40 is superimposed on the enrollment image 20.

  At step 513, the subject candidate aligns, positions and repositions his / her hand and / or key in front of camera 105 such that candidate image 40 is superimposed on enrollment image 20 on display 109. Can be.

  To facilitate alignment, the enrolled image 20 and / or the candidate image 40 after capture (step 503) can be processed using horizontal mirroring before being displayed on the display 109. .

  The basis for the alignment in step 513 may be an image of a body part or key feature in the candidate image 40 and an image of the corresponding feature in the enrollment image 20. This feature and the corresponding feature may be a ridge, line, scar, area of different skin pigmentation, nevus or wrinkle, or key, respectively, on the body part or major candidate body part. The alignment in step 513 can be executed without aligning the outer contour of the image of the observation target object such as a key. The outline of the image with respect to the candidate image 40 and / or the enrollment image 20 is not always visible on the display 109. Image feature alignment step 513 includes horizontal and vertical image coordinates, angular orientation in image space, and real space distance alignment to camera 105.

  At decision block 315, if one or more candidate images 40 are matched to the enrollment image 20, authentication / non-authentication can be performed. Authenticating the target candidate as a previously enrolled user can be done by comparing the enrollment image 20 with the authentication image. This authentication is verified by associating the candidate image as an authentication image of the target candidate (step 317).

  If the time series of the candidate image 40 does not completely match the enrollment image 20 on the display 109 at the decision block 315, the alignment between the candidate image 40 and the enrollment image 20 by the target candidate is performed in a step. Continue at 513.

  Reference is now made to FIG. 6, which illustrates a method 801 according to an aspect of the invention. At step 807, the enrollment image previously stored for the enrollment is shown to the user. 7 and 8 showing examples of previously stored enrollment images 65 and 75 on the display 109, respectively. The enrollment image 65 is an ellipse in which the user positions the image of his or her hand 33 within the ellipse 65. The dotted line 37 shown through the ellipse 65 can be used to align the middle finger and wrist images. Image 75 includes dashed lines for the target candidate to align their wrists and respective images of hand 33.

  Referring back to FIG. 6, in step 809, the target candidate is instructed to present an observation target, such as a key, or a body part such as a hand 33, for example, to the camera 105. In step 809, the candidate candidate image 40 is captured while the target candidate is attempting to align the candidate image 40 with a previously stored enrollment image displayed on the display 109. . With respect to image 65 and step 809, the subject candidate aligns the image of each fingertip within the exact outer periphery of ellipse 65, with the middle finger symmetrically aligned about dashed line 37 passing through ellipse 65. With respect to image 75 and step 809, the subject candidate attempts to ensure that each finger of his hand 33 is centered and parallel to the dotted line in image 75. At decision block 815, if the registration at step 809 is achieved, verification is performed by confirming that the candidate image 40 is a genuine image of the enrolled user (step 317). Otherwise, alignment with the target candidate continues at step 809.

  If a previously stored enrollment image 65/75 is to be used, the enrolling user can, for example, store the enrollment image 65/75 in the memory 102 of the portable computer system 100, e.g., for use in performing the enrollment capture step 503. From a bank of ~ 100 stored images, one or more previously stored enrollment images 65/75 can be selected. This selection of a previously stored enrollment image 65/75 provides additional security and, in this example, allows the user to select a suitable enrollment image 65/75 for his hand 33. I do.

  Reference is made to FIGS. 9 and 10, which represent a method 901 and an enrollment image 95 shown on a display 109, respectively, according to a feature of the present invention. At step 903, an enrollment image 95 of the body part of the user is captured using the camera 105. The body part shown in the enrollment image 95 (FIG. 10a) is the hand of the user. Next, the enrollment image 95 captured in step 903 is presented to the user on the display 109 (step 905). The user can overlay a symbol 101, such as a rectangle that covers the features of the body part, on the enrollment image, for example, as presented on the display 109 (step 907). The hand features shown in FIG. 10a may include bumps, lines, wrinkles, scars, or curved sections that connect the fingers. The superposition (step 907) may include the user relocating, resizing, and / or rotating the symbols that cover the features.

  If the symbol group 101 is successfully superimposed on the hand feature on the display 109 in the decision 909, the symbol 101 is displayed along with the position, orientation, and feature of the hand and / or symbol 101 in image space, along with the enrollment image. 95 (step 911). If the symbol 101 on the display 109 has not yet overlapped with the hand feature, the overlay continues at step 907.

  Referring back to FIG. 9, authentication of the subject candidate then continues to steps 913-921. In step 913, the enrollment image 95 stored in step 911 is presented on the display 109. In step 913, the portable computer system 100 instructs the target candidate to present the candidate body part to the camera. For example, if the candidate body part is a hand, the camera 105 is used to capture a time series of candidate hand images. Also during the capturing step 913, the time series of the candidate image is superimposed on the enrollment image 95 (step 915) and displayed on the display 109. At step 917, at least one of the candidate images can be aligned with enrollment image 95 such that corresponding features of enrollment image 95 and one or more candidate images match. In registration (step 917), the candidate places his or her hand in front of camera 105 such that features known only to the enrolled user, such as lines, wrinkles, etc., are registered with symbol 101. It is possible to move with. If the proper alignment of the features with symbols is successful in decision 919, verification 317 can be performed, for example, based on the teachings of US Pat. No. 7,327,858. Otherwise, due to misalignment or rejection of authentication, the subject candidate would be again instructed to present his hand in front of camera 105 in step 913. The subject candidate can use the front camera, rear camera, or any other camera of the mobile device simultaneously or in a specific sequence.

  Reference is now made to FIG. 10b, which shows another example of using a symbol 101 represented as a software peg 101 on a display 109. Software peg 101b can be positioned by a user such that software peg 101 indicates, for example, the location of a curved connection between fingers.

  Reference is made to FIG. 11, which includes a flowchart illustrating a method 1101, in accordance with an aspect of the present invention. At step 1103, a plurality of previously defined enrollment scales are presented to the user on display 109. Examples of previously defined enrollment scales shown on the display 109 are shown in FIG. 12 as scales or grid frames 1201a-1201c with respective grid frame lines 1203a-1203c.

  Referring back to the method 1101 in FIG. 11, at step 1105, an enrollment image of the user's observation target is captured using the camera 105. At step 1107, the user's enrollment image is presented to the user on display 109. The user can simultaneously select the grid frame scales 1201a to 1201c while superimposing the enrollment image on the selected grid frame or scales 1201a to 1201c (step 1109). Using the grid frame 1201b as the selected scale, the user can see that the grid frame line 1203b is located under the women's shoes, as in FIG. 14, while the top of the door is at line 1203c, as shown in FIG. Can be re-positioned or rotated so that the selected grid frame scale 1201b is located along. Alternatively, the user can rotate and position the camera so that the grid frame line 1203b is located on the line 1205b, as shown in FIG. Referring back to method 1101 in FIG. 11, at step 1111 the enrollment image 20 of the user's observation object is stored with the selected grid frame scale 1201b and associated with the enrolled user.

  Authentication steps 1113 to 1123 are steps for verifying whether or not the target candidate is the user previously enrolled in steps 1101 to 1111. At step 1113, an identification (ID) number of the subject candidate can be received by the portable computer system 100.

  The target candidate is instructed by the system 100 to present the observation target to the camera 105 to capture the candidate image 40 of the candidate observation target (step 1115).

  The captured candidate image is presented (step 1117) and superimposed on the selected enrollment scale 1201. At step 1119, the candidate aligns one of the candidate images with the selected enrollment scale 1201. If there is a match between the candidate image 40 and the selected scale 1201 at decision block 1121, then the candidate image 40 is verified at step 1123 as a genuine image of the target candidate as a previously enrolled user. Or no verification. Otherwise, registration by the target candidate will continue from decision 1121 to step 1119, or after some number of registration failures, the target candidate will fail authentication of these images. Is notified.

  Further, steps 1115 and 1117 involve security features. In order to verify the candidate image 40 as a genuine image (step 317), the candidate observation objects and features need to match the observation objects and features used in the enrollment process. Furthermore, in order to achieve the verification that the candidate image is authentic, the scale used in the enrollment process must be selected as the selection scale 1201 by the target candidate.

   In the course of the enrollment process shown in FIGS. 13 and 14, the portable computer system 100 may not have knowledge of the details (size, shape, etc.) of the observation object of the enrolled user. Therefore, during the enrollment stage, several grid frame scales 1201 each having a grid frame line 1203 can be displayed on the display 109, and the user aligns the observation object with each scale 1201. The observation object can be aligned with the scale 1201, and the entire observation object needs to be placed inside the rectangular box of the scale 1201.

  Referring to FIG. 13, during the enrollment process, when the user aligns the door of the observation target on the display 109 with each of the scales 1201, as a result, each of the scales 1201 is effectively observed. The object may be located at different distances from the camera 105. The portable computer system 100 can select the best scale 1201 for the target candidate, and the features extracted from the enrollment image 20 associated with the corresponding scale 1201 will be the most robust and clear. In this regard, the best-selected scale 1201 may be used for the user and the enrollment image stored and used in the verification process.

  For one of the many validation processes described above, the same enrollment mechanism can be used for any one of the validation processes described above, and the subject candidate selects the required scale 1201 and selects his or her own scale. It is necessary to position one's own observation target object according to the selection of the target object arrangement. If the candidate has too many verification attempts, the candidate will instead align their subject exactly as per the enrollment phase without presenting any information about their identity It is possible. Thus, the number of verification trials of the target candidate can be limited to only the related enroll target of the same scale and rotation, and the target candidate can be identified.

  The user uses the grid frame 1201b as the selected scale and re-selects the selected grid frame scale 1201b such that the grid frame line 1203b is located on the top line 1205b of the user's door as shown in FIG. Can be positioned or rotated. Alternatively, as shown in FIG. 15, the user rotates and positions his / her hand so that the grid frame line 1203b is positioned on the line 1205b of the top line of the user's door. The options presented in FIGS. 14 and 15 can apply to both the enrollment and verification processes, or just the enrollment process. In the verification process, the scale 1201 may be predefined and cannot be changed on the display 109.

  The verification process can be repeated in a particular way. For example, during the enrollment process, the user selects one of the scales 1201a, 1201b, and 1201c, and aligns his / her observation target 1205a, a woman's shoe, with the line 1203b of the scale 1201b. If the verification is successful, the subject candidate proceeds to a second verification step using a different scale, and so on. As a more secure option, the user can combine the scales 1201 into a sequential verification step during the enrollment, thereby generating a password with an ordered sequence of scales 1201.

  In another embodiment, a candidate user can capture a card and use a sequence of card placement and arrangement for various scales or symbols on the screen. The steps of card alignment with the scale and the timing of the card alignment are registered in the authentication process in such a way that the impostor cannot guess the correct sequence and timing. According to this method, the impostor would need the card, a mobile device with a particular scale, the correct sequence of card placement, and the timing of each step. For example, the subject candidate aligns the card to any first scale for 2 seconds, and then aligns the card to two additional scales for 0.5 second. The system will verify the authenticity of the card and the correct alignment with the registered sequence of scaled symbols and the timing of the card presentation.

  The use of different cards of different sizes, such as driver's licenses and passports, may require different symbol sizes to ensure that the cards are at the proper focal length from the camera where the cards are in best focus. Need. For example, a driver's license would have a small bounding box on the screen, while a passport would require a much larger box on the display. If the target candidate wants to authenticate the card, the card must be aligned with a matching bounding square on the screen. In this way, the card can be read at the best focal length, and this card reading would not require an auxiliary device.

  If an auxiliary device is needed, the portable device is set so that the reflector reflects off the inner palm of the hand. In this implementation, the mobile camera can authenticate by capturing the palm of the target candidate. The specular surface allows the subject candidate to view the display display of the mobile device (depending on the orientation of the mobile device) and interact with the mobile device and its front camera. Further, the user can capture the back portion of the hand and then position the hand on the lower portion of the device. This allows interaction with the display of the mobile device and at the same time identifies the target candidate using his hand. In this particular implementation, the subject candidate places the mobile device on the auxiliary device and uses a mirror surface to place the hand in a fixed position and a fixed distance from the rear camera of the mobile device, in a planar or peg-shaped manner. You can take a picture of the hand placed on the device. In this way, the target candidate can place an arbitrary portable camera and generate the most robust countermeasures. If the upper part of the hand is taken, no mirror is needed and the hand is located below the portable device. Depending on the location of the mobile device in this implementation, the subject candidate interacts with the display of the mobile device for access control, point of sale (POS), and any other applications that require authentication. It becomes possible.

  Reference is now made to FIG. 15, which shows the sequence steps for authentication and registration. Based on the registration phase, the target candidate is instructed to capture some QR codes from a grid of nine squares containing various QR codes. On the display 109 of the device, the symbol display box displays a visual image. The symbol display boxes 1301, 1301a, 1301b, and 1301c display images captured in each of the four steps of authentication and registration. The timer symbol 1303 indicates either the time from the start of the entire sequence or the length of the capture time required for each step defined in the registration. The display boxes of the captured images in the rectangular frames 1305, 1307, 1309, and 1311 are matched with the time stamps in 1303 for each step. In Phase 1, the target candidate captures only the QR code in the rectangular frame 1305 and the adjacent rectangular frame on the left (which is a part of 1307) on the display box 1301a. In the second step performed in the box display 1301b, the target candidate captures the QR codes in the square frames 1309 and 1311 displayed in 1303. In a third step, the target candidate captures four QR squares from box 1307 on box 1301C for a length of three seconds. If all the QR capture steps displayed in box 1303 have been performed and properly aligned in the correct order and within the required timing window, the subject candidate is authenticated.

  Alternatively, the subject candidate can capture and align grid squares of photos and symbols on any of the cards, walls, signatures, and any other available surfaces. During the registration process, the user captures a sequence of grid pictures, each arranged in grids or cells on the screen in any desired combination. At the time of authentication, the subject candidate holds the mobile device and aligns the grid symbol on the screen with the grid photo, as in the registration step sequence. The authentication step may be synchronized using a timer or any other display symbol measured on the screen.

  The target candidate may align the mobile camera with the observation target or align the card with the camera. In addition, any other combined stationary objects and tangible and movable objects such as cards can be part of the desired sequence of authentication.

  Another example is that in a case where nine rectangular frames of the same size are arranged as in the iPhone (registered trademark) password screen, in the verification process, the target candidate places one finger within a specific rectangular frame. The line may then be presented in a different specific square frame with another finger line, and so on. The combination of steps is saved in the enrollment phase, and in the course of the verification, the candidate has to repeat said combination of steps. Thus, the impostor cannot predict the combination of finger placement and square frame selection in the order performed during the initial enrollment process.

  The symbols on the screen that are aligned with the observation object may be predefined or manually input in any order and arrangement. These symbols can be resized, reordered, moved, or handwritten by the user in a manner that allows the user to align the on-screen symbols to the viewing scene, as was done during the registration process. The target candidate must select a reproducible scene for successful authentication.

  For example, if a user selects a tree as part of a scene image, the tree may change shape and the image may not be reproducible and is not suitable for use.

  Furthermore, the sequence of steps defined during the enrollment process and for subsequent authentication can be combined with any form of password, such as a keyboard type. This type of password can be used alone or in combination with the presented steps as described in the present invention within the registration step. For example, the subject candidate may be able to enter the password of each password at different times, at different times, or at different times of the gap between each character, or at any other combination, in a specific timing sequence. Can be typed. During the authentication process, a timer display, a progress bar, or any other equalizer-like thing, to indicate the desired character press measurement for each character, or to indicate the desired measured character press on each character A visual time indication will be provided, including any other time indication announcements, and any other combination of symbols to time the sequence of keyboard character presses and the gap between character presses. Thus, even if the impostor can know the password, he cannot use it because he does not know the timing of the correct sequence of character pressing. In this way, instead of using multiple passwords, the target candidate can use the same sequence of characters as the password, which is an endless password like permutation, so it cannot be broken. Will be difficult.

  The timing of this keyboard sequence can be timely combined with any other sequence of visual image registration and handwritten symbols, or any other method of authentication presented, in any combination.

  If users could use the same easy-to-remember password while maintaining the highest level of security, it would change the way people authenticate themselves.

  An impostor who has obtained a password cannot use it without knowing the correct step sequence, since there are many possible combinations. The steps below are easily reproduced and intuitive, so the candidate should generate the level of complexity he or she needs, without being limited to a particular implementation, as long as the sequence can be reproduced. Can be.

  Next, reference is made to FIG. 16, which shows a symbol of a handwritten target candidate, such as signature biometrics. In the registration process, in a first step, the target candidate draws a registered symbol or signature as indicated by 1401 on the screen device 109. Subsequently, during the second step, the target candidate arranges any kind of auxiliary symbol or marker with the registered symbol or signature of the 1401 display symbol of step 1. Further, in the registration process, the target candidate erases a part of the signature using the display box 1403 in the second phase. Auxiliary symbols may include registered symbols or any symbols arranged with the signature, removal of portions of the registered signature, any type of variation of the signature, and any other symbol marking portions of the signature.

  During the course of the verification, the display may include the auxiliary markers / hints of phase 2 of the registration 1403 display. The foregoing process allows for a high level of signature biometric authentication. The signature transformation of step 2 includes blurring, geometric transformation, or any other image manipulation to prevent re-creation of the enrollment image as in 1401.

  Another possible implementation includes the following steps.

  In step 1, the candidate applies the signature or symbol in the symbol box 1401 on the display 109. In step 2, the signature on the display box symbol 1401 is displayed on the screen 109, and the target candidate is given a random symbol, a scrambled line, and a hidden line to hide the original signature, as in the symbol box display 1403. Apply any other symbols. Upon authentication, the subject candidate looks at the scrambled line with the hidden signature in the display symbol 1403. The target candidate utilizes the signature or symbol by following the line of the hidden signature or symbol in registration phase 1 1401. In this implementation, the impostor will have difficulty distinguishing the signature from other concealing symbols of phase 2 of registration 1403. This method of authentication is difficult to break within a limited short time frame. This handwritten mark or symbol can be combined with any other form of target candidate authentication.

  During the course of authentication, the subject candidate is presented with one or more randomly selected enrollment images in a continuous manner using unknown combinations.

  In addition, the enrolled biometric method can include a defined sequence of registered images or a registered video sequence, as described herein. Upon authentication, the subject candidate must superimpose the observation object on the registered video sequence or registered image sequence on the screen of the on-screen moving object or symbol presented during registration. .

  For example, at the time of registration, the target candidate can continuously overlay the selected moving line on the screen with the left edge of the driver's license. Alternatively, the subject candidate may overlay the driver's license on a moving line at a particular synchronized position of the line on the screen.

  In another example, the subject candidate may determine a set of symbols, such as lines or symbols, that move on the screen. During the registration process, the two lines on the screen are rotating in different directions, and the subject candidate superimposes the observation object on one or more moving lines at a particular synchronization and line position on the screen. Match. Upon authentication, these two lines will be displayed on the screen and the subject candidate will follow the registered synchronized location, time for the observed object. Only the subject candidate knows the correct sequence of lines and the synchronization between the lines and the observed object and will be authenticated.

  Alternatively, the target candidate could capture the viewable image and, as an additional sequence step in the authentication method, align or apply the symbol at a particular position and time on the viewable image.

  In another example of a time display symbol, the display may include a clock-like display with moving hands that can be used by the subject candidate as the time display synchronization method described herein. Further, the subject candidate may press or swipe the clock display once the moving hand of the clock reaches a specific position on the clock or captures a specific visual image.

  Also, the display symbol may consist of any kind of arbitrarily scaled symbol or any other quantity measurement oriented symbol. For example, the target candidate may use a display bar having a number range of 1 to 1000. During the authentication process, the subject candidate selects the desired number as an additional sequence step by sliding his finger over the bar on the screen. This bar may be of any type, number of ranges, and precision as needed. Another symbol option may be any symbol having a different part of color and texture that can be selected as an additional sequence step by the subject candidate in registration and authentication.

  Embodiments of the present invention can include general-purpose or special-purpose computer systems, including various computer hardware components, which are described in further detail below. Embodiments within the scope of the invention also include computer-readable media that carry or have computer-executable instructions, computer-readable instructions, or stored data structures. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer system. By way of example, and not limitation, such computer readable media includes RAM, ROM, EPROM, flash disk, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device, or computer-executable instructions, computer-readable instructions Or non-transitory physical storage media such as any other media that can be used to carry or store the desired program code means in the form of data structures and that can be accessed by a general purpose or special purpose computer system. obtain.

  In this specification and the appended claims, a "computer system" is defined as one or more software modules, one or more hardware modules, or a combination thereof, that cooperate to perform operations on electronic data. You. For example, this definition of a computer system includes hardware components of a personal computer and software modules such as a personal computer operating system. The physical layout of these modules is not important. A computer system may include one or more computers connected via a computer network. Similarly, a computer system may include a single physical device (such as a telephone or personal digital assistant "PDA") with which internal modules (such as memory and a processor) cooperate to perform operations on electronic data. May be included. Although any computer system may be portable, the term "portable computer system" refers, in particular, to laptop computers, notebook computers, mobile phones, smart phones, wireless phones, personal digital assistants, portable computers with touch-sensitive screens. Including.

  In this specification and the appended claims, a "network" is defined as any architecture that allows two or more computer systems to exchange data. The term “network” refers to a wide area network, an Internet local area network, an intranet, a wireless network such as “Wi-Fi ™”, a virtual private network, a mobile access network using an access point name (APN), And the Internet. The data exchanged may be in the form of electrical signals meaningful to two or more computer systems. If the data is transferred or provided to a computer system or device over a network or another communication connection (either wired, wireless, or a combination of wired and wireless), the connection is made to a transitory computer readable medium. Is reasonably considered. Thus, any such connection is properly termed a transitory computer-readable medium. Also, the above combinations should be included within the scope of transitory and non-transitory computer readable media. Computer-executable instructions include, for example, instructions and data which cause a general purpose or special purpose computer system to perform a certain function or group of functions.

  The term "server," as used herein, generally refers to a computer system that includes a processor, data storage, and a network adapter configured to provide services over a computer network. Computer systems that receive services provided by a server may be known as "client" computer systems.

While selected embodiments of the present invention have been shown and described, it will be appreciated that the invention is not limited to the described embodiments. On the contrary, it is to be understood that modifications can be made to these embodiments without departing from the principles of the invention, the scope of which is defined by the appended claims and equivalents thereof. You.

Claims (20)

A computerized biometric method including enrollment and authentication of a user, the method comprising:
The enrollment is
Capturing the enrollment pre-image of the observation target while the user holds the observation target viewed from the camera ;
Presenting the enrollment pre-image to the user on a display;
Once presenting the enrollment pre-image, allowing the user to superimpose a symbol on the imaging feature of the observation object presented on the display;
Storing the symbol and the position and orientation of the symbol in image space in the enrollment pre-image as an enrollment image if the symbol on the display is superimposed on the imaging feature of the observation object; ,
Including
The authentication is
Presenting at least one portion of a previously stored enrollment image to a target candidate on a display, wherein the previously stored enrollment image includes the symbol;
Instructing the target candidate to provide the observation target to the camera while the observation target is positioned as viewed from the camera;
Using the camera, at least one of the candidate images of the observed object, including the symbol, to a previously stored enrollment image or portion thereof, while allowing alignment by the target candidate, Capturing the time-series candidate images of the observation target,
When at least one of the candidate images is aligned with the symbol in the enrollment image, the candidate image is verified as a genuine image of the observed object, thereby enrolling the target candidate previously. Authenticating as said user performed,
And a computerized biometric method.   The observation object positioned as viewed from the camera is held by the object candidate, and the alignment is performed by spatial and angular alignment in image space, and the actual distance between the observation object and the display. The computerized biometric method according to claim 1, comprising a spatial distance.   2. The computerized biometric method of claim 1, further comprising: selecting, by the user, the previously stored enrollment image from a plurality of previously stored enrollment images during the enrollment. . The enrollment is
While the observation target object is positioned as viewed from the camera,
Using the camera to capture the enrollment image of the observation object and its background;
Including
The authentication is
Capturing the time-series candidate images of the candidate observation target and its background,
Presenting a time-series candidate image superimposed on the enrollment image, on the display, along with the respective background, during the capturing;
The computerized biometric method of claim 1, comprising:   The method according to claim 1, wherein the alignment is performed by instructing the target candidate to align a feature of the observation target object in the candidate image with a corresponding feature in the enrollment image. Computerized biometric method.   2. The computerized biometric method of claim 1, further comprising performing the verification by comparing the enrollment image with an authentication image.   The computerized biometric method of claim 1, wherein the registration includes registration in horizontal and vertical image coordinates, angular orientation in image space, and real space distance to a camera.   Prior to the step of presenting at least one portion of the previously stored enrollment image, an image transformation and operating procedure is performed to prevent the re-formation of the enrollment image by performing the enrollment image 2. The computerized biometric method of claim 1, further comprising the step of: Instructing the target candidate to align features of the observed object in the candidate image with corresponding features in the enrollment image;
If the target candidate has aligned the feature of the observation object in the candidate image with the corresponding feature in the enrollment image, the feature of the observation object and the enrollment image Verifying at least one of the candidate images as a genuine image of the observation object by comparing the corresponding candidate in the candidate feature with the previously enrolled user. Authenticating;
The computerized biometric method of claim 1, further comprising: Before the superposition,
Rearranging the symbols;
Hand-writing the symbol;
Resizing the symbol;
Rotating the symbol;
2. The computerized biometric method of claim 1, further comprising at least one of the following.   The presenting the enrollment image on the display to the target candidate comprises presenting at least one portion of the stored symbol and the image of the observation object on the display. The computerized biometric method of claim 1. The enrollment is
Presenting the user with a plurality of previously defined enrollment scales on a display;
Selecting at least one of the enrollment scales presented on the display, thereby allowing a selected enrollment scale to be generated, while at least one of the enrollment images is Allowing it to be overlaid and repositioned on the selected enrollment scale;
Storing the selected enrollment scale as at least one enrollment image;
Including
The authentication is
Presenting, on a display, the time series of the candidate image superimposed on the selected enrollment scale;
Allowing the target candidate to align at least one of the candidate images with the selected enrollment scale;
The computerized biometric method of claim 1, comprising: In the course of the authentication, the step of presenting the enrollment scale to the target candidate includes the step of presenting a plurality of enrollment scales, and the method comprises:
Selecting the selected enrollment scale by the target candidate;
13. The computerized biometric method of claim 12, further comprising: Previously stored enrollment images can be retrieved electronically from previously stored media, or
The previously stored enrollment image is visually obtained using an imaging device from a photo selected from the group consisting of a passport, an ID card, a driver's license, and photos from other identification documents. The computerized biometric method of claim 1.   2. The computerization of claim 1, wherein the previously enrolled image comprises a partial / whole handwritten password symbol on the screen, with any random symbol unrelated to the registered symbol. Biometric method.   16. The computerized biometric method according to claim 15, wherein upon authentication, the target candidate handwrites the same handwritten password symbol as written during enrollment.   17. The computerized biometric method of claim 16, wherein the enrollment image includes the handwritten symbols and previously defined symbols entered using a keyboard or typing device.   Further comprising the step of allowing a timed sequence of symbols to be applied by the user on the display during enrollment, wherein during the authentication the subject candidate The computerized biometric method of claim 1, wherein the method is instructed to repeat the sequence.   A portable computer system adapted to perform the computerized biometric method of claim 1, wherein the camera and the display are part of the portable computer system.   A computer-readable medium encoded with processing instructions for causing a processor to perform the method of claim 1.

Images