JP4011426B2 - Face detection device, face detection method, and face detection program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a face detection apparatus, a face detection method, and a face detection program for detecting a face image in an input image, and in particular, quickly and reliably absorb variations in the positional relationship of face parts, thereby having high reliability and speed. The present invention relates to a face detection apparatus, a face detection method, and a face detection program capable of performing face detection.
[0002]
[Prior art]
In the surveillance camera system, since it is necessary to automatically recognize whether or not a person is reflected in an image captured by the surveillance camera, a technique using a face detection technique called a subspace method is known.
[0003]
For example, Matthew A. Turk, Alex Pentland, “Face Recognition Using Eigefaces”, Proc. CVPR, pp. 586-591, 1991, discloses a technique for constructing a partial space by collecting data of face images of a large number of people. In this prior art, a low-dimensional subspace that represents the entire face is prepared, and the face position is determined by examining which position in the input image best fits the prepared subspace. Detected.
[0004]
Specifically, in a model expressing a pattern distribution assuming a normal distribution with parameters of the mean and covariance matrix obtained from the face image data set prepared for learning, an eigenvector associated with the upper eigenvalue of the covariance matrix The pattern is limited to the subspace to be stretched, and the distance between the input pattern and the subspace is measured by regarding the subspace as a face space.
[0005]
By the way, when performing face detection using such a subspace method, it is difficult to absorb this variation because the variation in position of face parts (eyes, nose, mouth, etc.) is large among individuals. That is, in face detection, it is necessary to normalize the size of the face image. For example, if the reference is “interval between eyes”, the distance from the midpoint of both eyes to other face parts (eg, “mouth”) The distance is greatly different between a face with a face length and a face with a round face.
[0006]
For this reason, Japanese Patent Laid-Open No. 9-91429 (Prior Art 1) discloses an invention in which matching with a partial space is performed while changing the aspect ratio. Japanese Patent Application Laid-Open No. 9-44676 (Prior Art 2) describes an invention in which after detecting a plurality of face parts, the position of the face is specified based on the positional relationship of each face part. By utilizing the inventions disclosed in the related art 1 and the related art 2, it is possible to absorb the variation in the position of the face parts to some extent.
[0007]
[Problems to be solved by the invention]
However, when the above prior art 1 is used, since the number of matching calculations is proportional to the number of times the aspect ratio is changed, the calculation time becomes enormous, and as a result face detection cannot be performed quickly. There's a problem.
[0008]
In addition, when using the above-described conventional technique 2, although the variation in the position of the face part can be absorbed to some extent, the pattern of each face part is simple compared to the entire face, so the reliability of face part detection is low, As a result, there is a problem that the reliability of face detection is lowered.
[0009]
The present invention has been made to solve the above-described problems, and is capable of quickly and reliably absorbing variations in the positional relationship of face parts, and thus performing face detection with high reliability and quickness. An object is to provide an apparatus, a face detection method, and a face detection program.
[0010]
[Means for Solving the Problems]
The present invention has been made to achieve the above object, and the face detection device according to claim 1 is a face detection device that detects a face image present in an input image, and is based on each face part. Storage means for storing the first existence probability of the reference point of the face and the prior information of each face part including the partial space of each face part, and each face part existing in the input image based on the partial space And calculating a reference point of the face in the input image based on each recognized face part and a first existence probability of a face reference point based on the face part. Detecting means for detecting a face image from the input image based on the input information, and the detecting means includes parts information calculating means for calculating the likelihood of each face part as part information using the partial space, and the parts information calculating means. Calculated parts The second existence probability of the reference point of the face based on each face part based on the information and the first existence probability of the reference point of the face based on each face part which is one of the prior information. An existence probability calculating means for calculating, and weighting the importance of each face part to the second existence probability of the face reference point based on each face part calculated by the existence probability calculating means, A sum total calculating means for calculating the sum of the second existence probabilities of the reference points of the face based on each face part obtained, and obtaining a maximum value of the sum of the second existence probabilities calculated by the sum calculating means; Reference point specifying means for specifying the position where the value is taken as the reference point of the face is provided .
[0012]
The face detection device according to claim 2 is the face detection device according to claim 1 , wherein the existence probability calculation means is a part information calculated by the parts information calculation means and each face as one of the prior information. The second existence probability of the face reference point based on each face part is calculated by convolution with the first existence probability of the face reference point based on the part.
[0013]
According to a third aspect of the present invention, there is provided a face detection method for detecting a face image existing in an input image, wherein the first presence probability of a face reference point based on each face part and each face A storing step of storing in advance a front part information of each face part including a partial space of the parts in the storage unit, and recognizing each facial part existing in the input image based on the partial space, and recognizing each recognized facial part and the face Detection of calculating a reference point of a face in the input image based on a first existence probability of a reference point of a face with reference to a part, and detecting a face image from the input image based on the calculated reference point of the face a step seen including, the detecting step includes a part information calculation step of calculating a respective face part likeness as part information using the subspace, the part information calculating part information calculated by steps and the prior information Each face part that is one Based on the first existence probability of the reference point of the face as a reference, the existence probability calculation step of calculating the second existence probability of the face reference point based on each face part, and the calculation by the existence probability calculation step The second existence probability of the face reference point based on each face part is weighted according to the importance of each face part, and the second of the face reference points based on each face part obtained from the weight result A sum total calculation step for calculating the sum of existence probabilities, and a reference point specification for obtaining a maximum value of the second sum of existence probabilities calculated by the sum calculation step and specifying a position where the maximum value is obtained as a reference point of the face And a process .
[0015]
According to a fourth aspect of the present invention, in the invention of the third aspect , the presence probability calculating step includes each piece of face information that is one of the part information calculated in the part information calculating step and the prior information. The second existence probability of the face reference point based on each face part is calculated by convolution with the first existence probability of the face reference point based on the part.
[0016]
According to a fifth aspect of the present invention, there is provided a face detection program for detecting a face image existing in an input image, wherein the first presence probability of a face reference point based on each face part and each face A storing procedure for storing in advance a front part information of each face part including a partial space of the parts in the storage unit, and recognizing each facial part existing in the input image based on the partial space, and recognizing each recognized facial part and the face Detection of calculating a reference point of a face in the input image based on a first existence probability of a reference point of a face with reference to a part, and detecting a face image from the input image based on the calculated reference point of the face The detection procedure includes a part information calculation procedure for calculating the likelihood of each facial part as part information using the partial space, a part information calculated by the part information calculation procedure, and one of the prior information. One An existence probability calculation procedure for calculating a second existence probability of a face reference point based on each face part based on the first existence probability of a face reference point based on each face part; and the existence probability Weighting according to the importance of each face part to the second existence probability of the face reference point based on each face part calculated by the calculation procedure, and the face reference point based on each face part obtained from the weighting result The sum total calculation procedure for calculating the sum of the second existence probabilities of the first and the maximum value of the second sum of the second existence probabilities calculated by the sum calculation procedure, and specifying the position where this maximum value is taken as the reference point of the face The computer is caused to execute each procedure including a reference point specifying procedure to be performed.
[0018]
The face detection program according to claim 6 is the face detection program according to claim 5 , wherein the existence probability calculation procedure is a part information calculated by the parts information calculation procedure and each face as one of the prior information. The second existence probability of the face reference point based on each face part is calculated by convolution with the first existence probability of the face reference point based on the part.
[0019]
DETAILED DESCRIPTION OF THE INVENTION
Exemplary embodiments of a face detection device, a face detection method, and a face detection program according to the present invention will be explained below in detail with reference to the accompanying drawings.
[0020]
FIG. 1 is a block diagram showing a schematic configuration of a face detection apparatus according to an embodiment of the present invention. As shown in FIG. 1, the face detection device 1 recognizes facial parts existing in an input image based on the imaging unit 2 and the prior information of each facial part stored in the storage unit 4. The detection unit 3 that calculates the reference point of the face in the input image based on the face part and detects the face image from the input image based on the reference point of the face, and the presence of the reference point based on each face part And a storage unit 4 for storing prior information of each face part including the probability. Prior information stored in the storage unit 4 will be described later. Here, the facial parts refer to facial components such as the right eye, left eye, nose and mouth.
[0021]
In the case of this example, the imaging unit 2 employs a conventionally known camera device using a CCD or the like. The detection unit 3 uses a computer device (CPU). Further, the storage unit 4 employs a hard disk attached to the computer device forming the detection unit 3. The storage unit 4 may employ a computer device different from the computer device.
[0022]
As shown in FIG. 1, the detection unit 3 includes a parts information calculation unit 3a, an existence probability calculation unit 3b, a sum calculation unit 3c, and a reference point specifying unit 3d. The parts information calculation unit 3 a is a processing unit that determines the likelihood of each face part using the partial space stored in the storage unit 4 for each position of the input image input by the imaging unit 2.
[0023]
In addition, the existence probability calculation unit 3b is based on each face part obtained by the part information calculation unit 3a and the existence probability of the reference point of the face based on each face part which is one of the prior information. It is a processing part which calculates the existence probability of the reference point of the face based on this face part. Here, the existence probability of the reference points of the face based on the respective face parts, the convolution of the existence probability relative to the reference point of the likeness each face part, relative to the respective face parts stored in the storage unit 4 faces (Convolution).
[0024]
Also, the sum total calculation unit 3c first weights the existence probabilities of the face reference points based on the respective face parts obtained by the existence probability calculation unit 3b according to the importance of each face part, and obtains the weight by performing this weighting. It is a processing part which calculates | requires the sum total of the existence probability of the reference point of the face based on each face part obtained.
[0025]
The reference point specifying unit 3d is a processing unit that obtains the maximum value of the sum of the existence probabilities obtained by the total calculation unit 3c and uses the maximum value as the reference point of the face.
[0026]
The procedure for detecting a face image in an input image obtained from the imaging unit 2 using the face detection apparatus 1 having the above configuration is as shown in the flowchart of FIG. However, prior to the face detection, it is necessary to perform the learning process of the flowchart shown in FIG. 2 and store various pre-information obtained by this learning process in the storage unit 4. In the following, after describing the learning process, the face detection process will be described.
[0027]
FIG. 2 is a flowchart showing a learning processing procedure performed prior to face detection. As shown in the figure, a number of face images are input in advance and stored in the storage unit 4 (step S201). Such a face image can be input by capturing a face image for learning using the imaging unit 2. Further, another input device (not shown) such as a scanner may be used.
[0028]
When this input operation is completed, the process proceeds to step S202, where the face reference point and the position of each face part are designated for each face image input in step S201, and the position of each face part is used as a reference. A face part image having a predetermined size is extracted. The face reference point and the position of each face part are appropriately designated by the operator.
[0029]
Next, the process proceeds to step S203, and the relative position of the face reference point determined in step S202 with respect to each face part position is obtained. Thereafter, the process proceeds to step S204, and the average and covariance of the plurality of relative positions obtained in step S203 are obtained. Further, based on these averages and covariances, the existence probability Qi (x) of the face reference point for each face part is obtained using a normal distribution.
[0030]
Specifically, when the two-dimensional average column vector is m and the inverse matrix of the covariance matrix is V,
Qi (x) = Cexp {-<m, Vm>}
Asking. C is a constant, and <m, Vm> represents an inner product of m and Vm.
[0031]
Finally, the process proceeds to step S205, where each face part image is learned using the subspace method described above. That is, an average image and a unique image of each face part are created and stored in the storage unit 4. The average image and unique image of each face part, and the presence probability Qi (x) of the face reference point for each face part correspond to the prior information described in the claims.
[0032]
When learning as described above is completed, a face image can be detected from the input image captured from the imaging unit 2. Next, the face detection procedure will be described with reference to FIG.
[0033]
As shown in step S301 of FIG. 3, if the detection target input image acquired by the imaging unit 2 is input to the detection unit 3, the partial image centered on that position with respect to all the positions in the input image. , And applying the subspace method to each face part i, each distance of each face part i at the position x of each face part i is di (x). The facial part likelihood Pi (x) = exp (−di (x) / d0) is calculated. This is handled by the part information calculation unit 3a of the detection unit 3.
[0034]
Specifically, regarding face part i, if i = 1 is the right eye, i = 2 is the left eye, i = 3 is the nose, and i = 4 is the mouth, these face parts are the right eye, left eye, nose, mouth. By applying the above subspace method, the right eye-likeness P1 (x), the left eye-likeness P2 (x), the nasal-likeness P3 (x), and the mouth-likeness P4 (x) are respectively expressed in the above Pi (x). Calculate based on
[0035]
Next, the process proceeds to step 303, where the existence probability Ri (x) of the reference point of the face using each face part i is obtained. The existence probability Ri (x) is obtained as Pi (x) * Qi (x) from the above Pi (x) and the existence probability Qi (x) of the face reference point for each face part. The arithmetic symbol “*” represents convolution (convolution). Note that such Ri does not satisfy the probability condition ΣRi = 1, but here, the larger Ri (x), the larger the probability that x is the reference point of the face, and here it is referred to as a probability. The work relating to step S303 is handled by the existence probability calculation unit 3b of the detection unit 3.
[0036]
FIG. 4 shows the variation of the face reference point for each face part. For example, FIG. 4A shows that when the position K1 of the right eye is determined, the probability that the face reference point is within the range of the ellipse Q1 is high. Similarly, FIG. 5B shows that when the position K2 of the left eye is determined, the probability that the face reference point is within the range of the ellipse Q2 is high. FIG. When the position K3 is determined, the probability that the face reference point is within the range of the ellipse Q3 is high. FIG. 4D shows the face reference point when the mouth position K4 is determined. Represents a high probability of being within the range of the ellipse Q4.
[0037]
Next, the process proceeds to step S304, for each existence probability Ri (x) in each face part i obtained in step S303, a weight corresponding to the importance is added, and each existence probability Wi · Find the sum of Ri (x). That is, R (x) = ΣWi · Ri (x) is obtained. The operation relating to step S304 is handled by the sum total calculation unit 3c of the detection unit 3.
[0038]
When the sum R (x) of the respective existence probabilities after adding weights is obtained in this way, the process proceeds to step S305, and the maximum value Rmax (x) of R (x) is obtained. Further, a position x satisfying this Rmax (x) is set as a reference point of the face. That is, based on the existence probability Ri (x) of each face part i, the reference point of each face for each face part i is obtained as shown in FIG. By obtaining the maximum value Rmax (x) of R (x) for each of these reference points, the position x in the state where the face parts i are integrated is obtained as shown in FIG. The reference point specifying unit 3d of the detection unit 3 is in charge of the work related to step S305.
[0039]
As described above, the face detection device according to the present invention obtains the likelihood of each face part, and then integrates the results in consideration of the position variation. For this reason, there is no large variation in individual facial parts among individuals as in the conventional method. That is, such variation can be effectively absorbed. As a result, reliable face detection can be performed, and the reliability of face detection is improved.
[0040]
In this embodiment, the case where a CCD camera is used as the imaging unit 2 has been described. However, the present invention is not limited to this, and for example, a device that detects a face image in a photograph using a scanner. It can also be applied.
[0041]
1 comprises a single computer device. In this case, the computer device includes a parts information calculation unit 3a, an existence probability calculation unit 3b, a sum calculation unit 3c, and a reference point specifying unit 3d. Each program has a corresponding program.
[0042]
【The invention's effect】
As described above, according to the face detection device, the face detection method, and the face detection program according to the present invention, after obtaining the likelihood of each face part, the results are integrated in consideration of the variation in position. For this reason, there is no large variation in individual facial parts among individuals as in the conventional method. That is, such variation can be effectively absorbed. As a result, reliable face detection can be performed, and the reliability of face detection is improved. In addition, since the present invention can be incorporated and applied to various conventionally known devices, for example, the present invention is incorporated into a monitoring device, it can contribute to industrial development.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of a face detection apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a learning procedure performed prior to face detection.
FIG. 3 is a flowchart illustrating a face detection procedure.
FIG. 4 is a diagram showing variations in facial reference points for each facial part.
FIGS. 5A and 5B are schematic diagrams for explaining an action in obtaining a final face reference point, where FIG. 5A is a schematic diagram showing before and after integration of each facial part; FIG. .
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Face detection apparatus 2 Imaging part 3 Detection part 3a Parts information calculation part 3b Existence probability calculation part 3c Sum total calculation part 3d Reference point specific | specification part 4 Memory | storage part

Claims (6)

A face detection device for detecting a face image present in an input image,
Storage means for storing prior information of each face part including a first existence probability of a reference point of the face relative to each face part and a partial space of each face part;
Recognizing each face part present in the input image based on the partial space, and in the input image based on a first existence probability of each recognized face part and a reference point of a face based on the face part. Detecting means for detecting a face image from the input image based on the calculated face reference point ,
The detection means includes
Parts information calculation means for calculating the likelihood of each face part as part information using the partial space, and the parts information calculated by the parts information calculation means and each face part as one of the prior information A presence probability calculating means for calculating a second existence probability of the face reference point based on each face part based on the first existence probability of the face reference point; and each of the existence probability calculating means calculated by the existence probability calculating means The second existence probability of the face reference point based on the face part is weighted according to the importance of each face part, and the second existence probability of the face reference point based on each face part obtained from the weighting result Sum total calculating means for calculating the sum, and a reference point specifying means for determining a maximum value of the total sum of the second existence probabilities calculated by the sum calculating means and specifying a position where the maximum value is obtained as a reference point of the face Face detection apparatus characterized by comprising and. The existence probability calculation means is a convolution of the part information calculated by the parts information calculation means and a first existence probability of a face reference point based on each face part which is one of the prior information. The face detection apparatus according to claim 1 , wherein a second existence probability of a face reference point based on each face part is calculated. A face detection method for detecting a face image present in an input image,
A storing step of storing in the storage unit prior information of each face part including a first existence probability of a reference point of the face relative to each face part and a partial space of each face part;
Recognizing each face part present in the input image based on the partial space, and in the input image based on a first existence probability of each recognized face part and a reference point of a face based on the face part. of calculating the reference points of the face, seen including a detection step of detecting a face image from the input image based on the reference point of the calculated face,
The detection step includes
A part information calculation step for calculating the likelihood of each facial part as part information using the partial space, and part information calculated by the part information calculation step and each face part as one of the prior information. A presence probability calculating step of calculating a second presence probability of the face reference point based on each face part based on the first presence probability of the face reference point; The second existence probability of the face reference point based on the face part is weighted according to the importance of each face part, and the second existence probability of the face reference point based on each face part obtained from the weighting result A total point calculation process for calculating the total sum and a reference point specifying process for determining the maximum value of the total sum of the second existence probabilities calculated by the total calculation step and specifying a position where the maximum value is obtained as a reference point of the face Face detection method characterized by including and. The existence probability calculating step is a convolution of the part information calculated by the parts information calculating step and the first existence probability of the reference point of the face based on each face part which is one of the prior information. The face detection method according to claim 3 , wherein a second existence probability of a face reference point based on each face part is calculated. A face detection program for detecting a face image present in an input image,
A storage procedure for storing in advance a first information on each face part including a first existence probability of a reference point of a face relative to each face part and a partial space of each face part in a storage unit;
Recognizing each face part present in the input image based on the partial space, and in the input image based on a first existence probability of each recognized face part and a reference point of a face based on the face part. And a detection procedure for detecting a face image from the input image based on the calculated face reference point .
The detection procedure includes:
A part information calculation procedure for calculating the likelihood of each face part as part information using the partial space, and part information calculated by the part information calculation procedure and each face part as one of the prior information. An existence probability calculation procedure for calculating a second existence probability of a face reference point based on each face part based on the first existence probability of the face reference point; and each of the existence probability calculation procedures calculated by the existence probability calculation procedure The second existence probability of the face reference point based on the face part is weighted according to the importance of each face part, and the second existence probability of the face reference point based on each face part obtained from the weighting result Sum total calculation procedure for calculating the sum, and a reference point specifying procedure for obtaining the maximum value of the total sum of the second existence probabilities calculated by the total calculation procedure and specifying the position where the maximum value is taken as the reference point of the face Including
A face detection program for causing a computer to execute each procedure . The existence probability calculation procedure is a convolution of the part information calculated by the part information calculation procedure and the first existence probability of the reference point of the face with each face part as one of the prior information. The face detection program according to claim 5 , wherein a second existence probability of a face reference point based on each face part is calculated.

Images