JP4589224B2 - 3D pattern matching device - Google Patents

Description

  The present invention captures a three-dimensional object such as a human face as a detection target, collates a three-dimensional pattern of the captured three-dimensional object, and a registered three-dimensional pattern of a three-dimensional object registered in advance. And a three-dimensional pattern matching device for matching a matching three-dimensional pattern with a registered three-dimensional pattern.

  In recent years, in fields that require personal authentication, such as access control to computer rooms and important machine rooms, and access control to computer terminals and financial terminals of banks, instead of conventional passwords and ID cards, Devices for performing personal authentication based on three-dimensional biological information (biometrics) such as a face and a vein have been developed.

  For example, a detection target (three-dimensional object) is imaged with two cameras from different positions, and a three-dimensional pattern is created by measuring the unevenness of each part of the three-dimensional object imaged based on the stereo image, and obtained at the time of registration. There has been proposed a three-dimensional pattern matching apparatus that performs matching between the obtained “registered three-dimensional pattern” and the “matching three-dimensional pattern” obtained at the time of matching (see Patent Document 1).

  In this three-dimensional pattern collation apparatus, the collation three-dimensional pattern and the registered three-dimensional pattern are used to increase the collation accuracy, and the positional deviation (X, Y, Z-axis direction deviation) or rotational deviation (X It is important to accurately obtain the deviations of the Y, Z, and Z axes and match the positions and orientations of the matching three-dimensional pattern and the registered three-dimensional pattern.

  Conventionally, there is an ICP (Iterative Closest Point) algorithm as a method for matching the position and orientation of a three-dimensional pattern (see Patent Document 2). In this ICP algorithm, when the registered three-dimensional pattern data is M and the collation three-dimensional pattern data is M ′, the positions and orientations of the collation three-dimensional pattern and the registered three-dimensional pattern are the following (1) to (4 ).

(1) For all the points mi included in M, the closest point mi ′ is searched from M ′, and the set of obtained mi and mi ′ is set as a set of corresponding points.
(2) Based on the current correspondence relationship, an optimal motion pattern (rotation R, translation t) between the two three-dimensional pattern data M and M ′ is estimated by the least square method.
(3) M ′ is coordinate-transformed using R and t obtained in (2) above.
(4) The above (1) to (3) are repeated until the change in R and t becomes sufficiently small and converges with respect to M and M ′ after coordinate transformation.

JP-A-9-259271 JP-A-9-277184

However, in the verification method of a conventional three-dimensional pattern described above, for example, when the detection target and the biological information such as a human face, Individuals vary is remarkable, the position and posture of the characteristic portion in the registration three-dimensional pattern Is not always a fixed position. For example, the head of the nose may be low or the nasal muscles may be bent. In such a case, the matching three-dimensional pattern and the registered three-dimensional pattern may be misaligned with each other at the corresponding points, so that the positions and orientations of the matching three-dimensional pattern and the registered three-dimensional pattern are correctly aligned. There was something that could not be done.

  Further, in the conventional three-dimensional pattern matching method described above, one is used when 1: N matching is performed, that is, when one matching three-dimensional pattern and N registered three-dimensional patterns are sequentially compared. The registered 3D pattern must be read out one by one and alignment with the collated 3D pattern must be performed by the ICP algorithm, resulting in a problem that collation speed decreases. In particular, since the ICP algorithm repeats the operation many times until the operation converges, it takes a very long time, and the processing time is unavoidable.

The present invention has been made to solve such a problem, and the object of the present invention is to perform matching by matching the position and orientation of the matching three-dimensional pattern and the registered three-dimensional pattern correctly and reliably. It is to provide a three-dimensional pattern matching device capable of
It is another object of the present invention to provide a three-dimensional pattern matching apparatus capable of performing 1: N matching at high speed by matching the position and orientation of the matching three-dimensional pattern and registered three-dimensional pattern correctly and reliably.

In order to achieve such an object, the first invention (the invention according to claim 1) images a three-dimensional object as a detection target, and pre-registers a three-dimensional pattern of the imaged three-dimensional object as a collation three-dimensional pattern. A three-dimensional pattern of a three-dimensional object that is defined as a detection target in a three-dimensional pattern matching apparatus that uses a registered three-dimensional pattern as a registered three-dimensional pattern and collates the matching three-dimensional pattern with the registered three-dimensional pattern. A reference template storage means for storing a three-dimensional pattern having a reference position and a reference posture as a reference template, and a matching three-dimensional pattern position for correcting the position and posture of the matching three-dimensional pattern by aligning with the reference template Registration that corrects the position and posture of the registered 3D pattern by aligning the posture correction means with the reference template. The three-dimensional pattern position / posture correction means, the matching three-dimensional pattern whose position and posture are corrected by the matching three-dimensional pattern position / posture correction means, and the registered three-dimensional pattern position / posture correction means correct the position and posture. 3D pattern matching means for matching the registered 3D pattern, and the reference template is obtained by extracting the 3D pattern of the target portion extracted from the average 3D pattern of the three-dimensional object to be detected. It is a three-dimensional pattern having a reference position and a reference posture. In the present invention, “alignment” means matching both “position” and “posture”.

In the present invention, for example, when a detection target is a human face, a captured three-dimensional pattern of a face (collation target three-dimensional face pattern) and a pre-registered three-dimensional pattern (registration target three-dimensional face pattern) Is matched. In this case, the registration target 3D face pattern and the reference template (for example, the head of the nose is used as the reference portion extracted from the average 3D pattern (average 3D face pattern) of the human face to be detected). The position and the position of the 3D face pattern to be registered are corrected by the ICP algorithm or the like. Further, the position and orientation of the verification target 3D face pattern are corrected by performing alignment between the verification target 3D face pattern and the reference template using an ICP algorithm or the like. The verification target 3D face pattern and the registration target 3D face pattern, whose position and orientation are corrected, are verified.

  In the reference template, attention portions such as “nose head” and “nasal muscle” are set as a reference position and a reference posture, and are standard with no individual differences. Therefore, the alignment between the verification target 3D face pattern and the reference template is performed so as to correspond to the target portion without individual differences, and the registration target 3D face pattern and the reference template are aligned with individual differences. This is performed in such a way as to correspond to a non-attention part, and there is less possibility that registration will be performed by mistaken corresponding points, and the position and orientation of the verification target 3D face pattern and the registration target 3D face pattern are reduced. It is possible to collate in a state where the two are correctly and reliably combined.

  In the present invention, when 1: N matching is performed, a plurality of registered three-dimensional patterns whose positions and postures have been corrected by the registered three-dimensional pattern position / posture correction means are stored, and the matching three-dimensional pattern position / posture correction is performed. The collation three-dimensional pattern whose position and orientation are corrected by the means and the plurality of registered three-dimensional patterns stored are sequentially collated (second invention (invention according to claim 2)). In this case, since registration between the registered 3D pattern and the reference template has already been performed, alignment at the time of collation is only once with the collation 3D pattern and the reference template, greatly reducing processing time, It is possible to perform 1: N verification at high speed.

  In the present invention (first and second inventions), for example, the detection target is a human face, the reference template includes “hairstyle”, the “hairstyle” of the reference template, and the “hairstyle” of the imaged face. May be aligned with the “hairstyle” instead of the attention portion such as “nose head” and “nasal muscle”. Therefore, in the third invention (the invention according to claim 3), the reference template is a three-dimensional pattern of the attention area including the attention portion. For example, by using a three-dimensional pattern of a limited area including “head of nose” and “nose muscle” as a reference template, and excluding extra portions such as “hairstyle” from the reference template, the matching three-dimensional pattern and registration 3 Increase the accuracy of alignment between the dimensional pattern and the reference template.

  Further, in the present invention (first and second inventions), before collation between the collation three-dimensional pattern and the registered three-dimensional pattern, each of the attention areas including the collation three-dimensional pattern and the attention portion of the registered three-dimensional pattern The pattern data may be weighted (fourth invention (invention according to claim 4)). In this way, for example, it is possible to enhance the collation accuracy by emphasizing individual pattern data of the area of interest including “head of nose” and “nose muscle” more than pattern data of other areas.

According to the first aspect of the present invention, the reference template (the target portion extracted from the average three-dimensional pattern of the three-dimensional object to be detected is set as the reference position and the reference posture with respect to the detection target. A three-dimensional pattern using a three-dimensional pattern as a reference position and a reference posture ), aligning with a reference template to correct the position and posture of the collation three-dimensional pattern, aligning with the reference template, and registering 3 The position and orientation of the three-dimensional pattern are corrected, and the collation 3D pattern with the corrected position and orientation is collated with the registered three-dimensional pattern. The collation 3D pattern and the registered 3D pattern can be collated in a state where the position and orientation are matched correctly and reliably. The ability.

  According to the second invention, a plurality of registered three-dimensional patterns whose positions and postures are corrected by the registered three-dimensional pattern position / posture correction means are stored, and the positions and postures are corrected by the collation three-dimensional pattern position / posture correction means. Since the collation 3D pattern whose posture has been corrected and a plurality of stored registered 3D patterns are sequentially collated, the alignment at the time of collation is performed once as the collation 3D pattern and the reference template. Time can be greatly shortened and 1: N verification can be performed at high speed.

According to the third invention, in the first and second inventions, the three-dimensional pattern of the attention area including the attention portion is used as the reference template. Therefore, the reference template includes only the attention portion such as “nose” and “nasal muscle”. By using the 3D pattern of the area as a reference template and excluding unnecessary parts such as “hairstyle” from the reference template, it is possible to improve the accuracy of alignment between the matching 3D pattern and registered 3D pattern and the reference template It becomes.
According to the fourth invention, in each of the first and second inventions, each of the attention areas including the target portion of the matching three-dimensional pattern and the registered three-dimensional pattern before the matching of the matching three-dimensional pattern and the registered three-dimensional pattern. Since the pattern data is weighted, for example, the individual pattern data of the attention area including “head of nose” and “nose muscle” is emphasized more than the pattern data of other areas, thereby improving the matching accuracy. It becomes possible.

Hereinafter, the present invention will be described in detail with reference to the drawings.
[Embodiment 1: 1: 1 verification]
FIG. 1 is a block diagram showing a main part of an embodiment of a three-dimensional pattern matching apparatus according to the present invention.

  In FIG. 1, reference numeral 1 denotes a human face as a detection target, which is imaged by two cameras 1-1 and 1-2. A three-dimensional pattern creating unit 2 for creating a three-dimensional pattern (a matching target three-dimensional face pattern) PX and a registration target three-dimensional pattern (a registration target three-dimensional face pattern) PY is defined for a detection target (human face) The reference template storage unit stores the reference template PR.

  In this embodiment, the reference template PR creates, for example, an average three-dimensional pattern (average three-dimensional face pattern) of a human face that is a detection target, and “nose” as a target portion from the average three-dimensional face pattern. 3D pattern of the attention area including “head of nose” and “nose muscle” is extracted, and “nose head (A)” of the extracted three-dimensional pattern of the attention area is set as X = 0, Y = 0, Z = 0. The reference position is set, and the "nose muscle (B)" is positioned on the Y-axis to obtain the reference posture. FIG. 3 shows the relationship between the attention area AR in the human face and the attention portion TA included in the attention area AR. In the present embodiment, the attention area AR is defined as an area centering on the nose and not including eyes, mouth and hair.

  If the three-dimensional average is simply obtained by combining the position of the head of the nose and the nose muscle line, an average three-dimensional face pattern (average face) can be obtained. Also, the average face may be created by normalizing and averaging the sizes of the positions of the two eyes and the nose.

  In FIG. 1, reference numeral 3-1 denotes a first position / posture for aligning the comparison target 3D face pattern PX sent from the 3D pattern creation unit 1 with the reference template PR read from the reference template storage unit 2. An alignment unit 3-2 is a second position / posture alignment that aligns the registration target 3D face pattern PY sent from the 3D pattern creation unit 1 and the reference template PR read from the reference template storage unit 2 , 4 is a database that stores the registered 3D face pattern PY registered by the second position / posture matching unit 3-2 as a registered 3D face pattern PY ′, and 5 is the first position / posture. The matching three-dimensional face pattern PX ′ that has been aligned by the matching unit 3-1 and the registered three-dimensional face pattern PY ′ stored in the database 4. A matching unit that performs matching.

  In this embodiment, the first position / posture matching unit 3-1 performs matching between the target 3D face pattern PX and the reference template PR using the ICP algorithm, and corrects the position and posture. A three-dimensional face pattern PX ′ is generated. The second position / posture matching unit 3-2 performs registration between the registration target three-dimensional face pattern PY and the reference template PR using an ICP algorithm, and obtains a registered three-dimensional face pattern PY ′ whose position and posture are corrected. Generate.

  In this case, the position / posture matching unit 3-1 sets the data of the reference template PR as M, the data of the verification target 3D face pattern PX as M ′, the position of the verification target 3D face pattern PX and the reference template PR, and The posture is adjusted as follows (1) to (4). Similarly, the position / posture matching unit 3-2 sets the data of the reference template PR as M, the data of the registration target 3D face pattern PY as M ', and the positions and orientations of the registration target 3D face pattern PY and the reference template PR. Are combined as in the following (1) to (4).

(1) For all the points mi included in M, the closest point mi ′ is searched from M ′, and the set of obtained mi and mi ′ is set as a set of corresponding points.
(2) Based on the current correspondence relationship, an optimal motion pattern (rotation R, translation t) between the two three-dimensional pattern data M and M ′ is estimated by the least square method.
(3) M ′ is coordinate-transformed using R and t obtained in (2) above.
(4) The above (1) to (3) are repeated until the change in R and t becomes sufficiently small and converges with respect to M and M ′ after coordinate transformation.

  In this embodiment, in order to speed up the calculation, in the initial stage of the ICP algorithm, rough alignment is performed using a small number of corresponding points, and the number of corresponding points is gradually increased as the processing proceeds. Increase and adjust the position.

  In the reference template PR, “head of nose (A)” and “nose muscle (B)” are set as reference positions and reference postures, and are standard with no individual differences. Therefore, the alignment of the comparison target 3D face pattern PX and the reference template PR is performed so as to correspond to the “nasal head (A)” and “nasal muscle (B)” in the reference template PR without individual differences. Done. Similarly, the registration of the registration target 3D face pattern PY and the reference template PR also corresponds to the “nasal head (A)” and “nasal muscle (B)” in the reference template PR without individual differences. Is done.

  When the collation 3D face pattern PX ′ is sent from the position / posture matching unit 3-1, the collation unit 5 reads the registered 3D face pattern PY ′ from the database 4 and registers it as the collation 3D face pattern PX ′. Collation with the three-dimensional face pattern PY ′ is performed. In this collation, the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′ correspond to the “nasal head (A)” and “nasal muscle (B)” of the reference template PR, so that the reference The alignment with the template PR is performed correctly and reliably. Therefore, the collation three-dimensional face pattern PX ′ and the registered three-dimensional face pattern PY ′ are collated with their positions and postures correctly matched.

  In the present embodiment, the three-dimensional pattern of a limited area including “nose head (A)” and “nose muscle (B)” is used as the reference template PR, that is, from the reference template PR to “hairstyle” and the like. Since unnecessary portions are excluded, there is no fear that alignment is performed so as to correspond to the “hairstyle”, and the comparison target 3D face pattern PX and the registration target 3D face pattern PY and the reference The accuracy of alignment with the template PR is increased.

  In this embodiment, the position and posture of the registration target three-dimensional face pattern PY are corrected, and the registered three-dimensional face pattern PY ′ having the corrected position and posture is stored in the database 4. The target three-dimensional face pattern PY may be stored, the position and orientation of the registration target three-dimensional face pattern PY may be corrected at the time of matching, and matching with the matching three-dimensional face pattern PX ′ may be performed. In this case, as the alignment at the time of collation, in addition to the alignment between the verification target 3D face pattern PX and the reference template PR, the registration target 3D face pattern PY and the reference template PR need to be aligned. In the case of 1 collation, the alignment speed is sufficient, so the collation speed does not drop so much. Of course, it is needless to say that the registered three-dimensional face pattern PY ′ is stored in the database 4 and collation is performed.

[Embodiment 2: 1: 1 N verification]
The ICP algorithm is very time consuming because the operation is repeated many times until the operation converges. When 1: N collation is performed, alignment by the ICP algorithm N + 1 times is necessary. If this is performed at the time of collation, the processing time becomes enormous and the collation speed decreases. Therefore, in 1: N collation, as shown in FIG. 2, in the position / posture matching unit 3-2, the registration target 3D face patterns PY1, PY2,. Registered three-dimensional face patterns PY1 ′, PY2 ′,... PYN ′ corrected by the algorithm and corrected in position and orientation are stored in the database 4.

  When the collation unit 5 receives the collation three-dimensional face pattern PX ′, the collation unit 5 sequentially reads the registered three-dimensional face patterns PY1 ′, PY2 ′,... PYN ′ from the database 4, and performs the collation three-dimensional face pattern PX ′. Is checked. At the time of this collation, the registered three-dimensional face patterns PY1 ', PY2',... PYN 'have already been registered by the ICP algorithm at the time of registration. Therefore, the processing time can be significantly reduced and 1: N matching can be performed at a high speed, compared with the case where alignment is performed by the ICP algorithm each time at the time of matching.

  In the above-described embodiment, the first position / posture matching unit 3-1 is provided for verification, and the second position / posture matching unit 3-2 is provided for registration. The matching unit 3-1 may be used for both verification and registration, and the second position / posture matching unit 3-2 may be omitted. In addition, the collation by the collation unit 5 may adopt any collation method such as a feature point method or a correlation method.

  Further, before collation between the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′, the “nose head” and “nasal muscle” of the collation 3D face pattern PX ′ and the registered 3D face pattern PY ′. It is also possible to weight each pattern data of the attention area including As a result, individual pattern data of the area of interest including “head of nose” and “nose muscle” can be emphasized more than pattern data of other areas, and matching accuracy can be improved.

  In addition, as an average three-dimensional pattern (average three-dimensional face pattern) of a human face when determining the reference template PR, for example, a registered average three-dimensional face pattern that is an average of the registered three-dimensional face patterns PY1 to PYN is created. Then, the reference template PR may be created by extracting a target part from the created registered average three-dimensional face pattern. In this way, the user's face information is reflected in the reference template PR, so that the alignment accuracy is increased and the matching accuracy is further improved.

  In addition, the reference template PR may include not only “nose” but also “eyes”, “mouth”, and the like as attention parts. Also, instead of creating an average three-dimensional pattern (average face) of a human face, a reference portion PR is created by extracting a target portion from a three-dimensional pattern of a human face that seems to be an average. Also good. Further, when viewed globally, it is considered that the average three-dimensional pattern of a human face differs depending on the country or region, and a representative reference template PR may be created for each country or region. Further, the target portion to be included in the reference template PR may be manually designated.

In addition, the detection target is not limited to a human face, and if it is a three-dimensional object that always has a high part and a low part as human features, as in the case of a human face, It is possible to detect the position and orientation of a three-dimensional object.
Further, in the above-described embodiment, the alignment with the reference template PR is performed by the ICP algorithm, but it is not always necessary to use the ICP algorithm.

It is a block diagram which shows the principal part of one Embodiment (Embodiment 1) of the three-dimensional pattern matching apparatus which concerns on this invention. It is a block diagram which shows the principal part of the three-dimensional pattern collation apparatus (Embodiment 2) in the case of performing 1: N collation. It is a figure which shows the relationship between the attention area in a human face, and the attention part contained in this attention area.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Three-dimensional pattern creation part, 1-1, 1-2 ... Camera, 2 ... Reference | standard template memory | storage part, 3-1, 3-2 ... Position and attitude | position alignment part, 4 ... Database, 5 ... Collation part, PR ... Reference template, PX ... 3D face pattern to be verified, PY (PY1 to PYN) ... 3D face pattern to be registered, PX '... 3D face pattern to be verified, PY' (PY1 'to PYN') ... 3D face pattern to be registered , AR ... attention area, TA ... attention area.

Claims (4)

A three-dimensional object is imaged as a detection target, the three-dimensional pattern of the picked three-dimensional object is a collation three-dimensional pattern, and a three-dimensional pattern of a three-dimensional object registered in advance is a registered three-dimensional pattern. In a three-dimensional pattern matching device that matches a registered three-dimensional pattern,
A reference template storage means for storing, as a reference template, a three-dimensional pattern that is defined for the detection target and in which the three-dimensional pattern of the target portion is a reference position and a reference posture;
Collation 3D pattern position / posture correction means for correcting the position and posture of the collation 3D pattern by aligning with the reference template;
Registered three-dimensional pattern position / posture correction means for correcting the position and posture of the registered three-dimensional pattern by performing alignment with the reference template;
The collation 3D pattern whose position and orientation have been corrected by the collation 3D pattern position / posture correction means and the registered 3D pattern whose position and orientation have been corrected by the registered 3D pattern position / posture correction means Three-dimensional pattern matching means for
The reference template is
A three-dimensional pattern matching apparatus , wherein the three-dimensional pattern is a three-dimensional pattern having a reference position and a reference posture as a three-dimensional pattern of the target portion extracted from an average three-dimensional pattern of the three-dimensional object to be detected . In the three-dimensional pattern matching apparatus according to claim 1,
A registered three-dimensional pattern storage unit that stores a registered three-dimensional pattern whose position and posture are corrected by the registered three-dimensional pattern position / posture correcting unit;
The registered three-dimensional pattern storage means stores a plurality of registered three-dimensional patterns whose positions and postures are corrected by the registered three-dimensional pattern position / posture correcting means,
The three-dimensional pattern matching means includes a matching three-dimensional pattern whose position and orientation are corrected by the matching three-dimensional pattern position / posture correction means, and a plurality of registered three-dimensional patterns stored in the registered three-dimensional pattern storage means. Is a three-dimensional pattern matching device. In the three-dimensional pattern matching device according to claim 1 or 2,
The reference template storage unit stores a three-dimensional pattern of an attention area including the attention portion as a reference template. In the three-dimensional pattern matching device according to claim 1 or 2,
Weighting for weighting individual pattern data of the attention area including the attention portion of the matching three-dimensional pattern and the registered three-dimensional pattern before matching the matching three-dimensional pattern with the registered three-dimensional pattern by the three-dimensional pattern matching means A three-dimensional pattern matching apparatus comprising: means.

Images