JP2006236216A - Person recognition system, person recognizing method and person recognition program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a highly accurate person recognition result by correcting a recognition result even if one and the same person is erroneously recognized as a plurality of persons different from one another. <P>SOLUTION: This person recognition system for recognizing persons included in a plurality of images is provided with: a person recognizing part for recognizing a plurality of persons included in the plurality of images about each of the plurality of images; an intimacy degree calculating part for calculating the degree of intimacy among respective persons included in at least one portion of images among the plurality of images on the basis of a recognition result; a determining part for determining whether a difference in the degree of intimacy to one person about each of the plurality of persons recognized as persons different from one another is not higher than a predetermined reference value; and a correcting part for determining that the plurality of persons are one and the same person when the difference in the degree of intimacy to one person in each of the plurality of persons is determined not to be higher than the predetermined reference value and correcting the recognition result by the person recognizing part. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a person recognition system, a person recognition method, and a person recognition program. In particular, the present invention relates to a person recognition system, a person recognition method, and a person recognition program for recognizing a person included in each of a plurality of images.

2. Description of the Related Art Conventionally, a person recognition system that recognizes a person by recognizing the face image of the person is known (see, for example, Patent Document 1). In these person recognition systems, by performing image processing on a person's face image, the feature amount is extracted from the face image, and each person is recognized based on the difference of the extracted feature amount. Yes.
Japanese Patent Laid-Open No. 2001-27396

  However, since the face image used for person recognition changes variously depending on the captured angle, the imaging environment, the facial expression of the person who is the subject, etc., it is easy to always maintain a high recognition rate in the conventional person recognition system. is not. For this reason, there are not a few cases where each of a plurality of face images representing the same person is erroneously recognized as being face images of different persons, and a person recognition system capable of obtaining a person recognition result with higher accuracy. Was long-awaited.

  Accordingly, an object of the present invention is to provide a person recognition system, a person recognition method, and a person recognition program that can solve the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  In order to solve the above problems, in the first embodiment of the present invention, there is provided a person recognition system for recognizing a person included in each of a plurality of images, wherein each of the plurality of images includes a plurality of A person recognition unit for recognizing a person and a closeness calculation unit for calculating a closeness between each of the persons included in at least some of the images based on the recognition result by the person recognition unit A determination unit that determines whether or not a difference in intimacy with respect to one person of each of a plurality of persons recognized as different persons by the person recognition unit is equal to or less than a predetermined reference value; When it is determined that the difference in intimacy with respect to one person among a plurality of persons recognized as different persons is equal to or less than a predetermined reference value, the plurality of persons are the same. Determines that the person, and a correction unit for correcting the recognition result by the person recognition unit.

  The closeness calculating unit calculates the closeness between the first person and the second person included in at least some of the plurality of images, and the first person among the plurality of images. And the second person may be calculated as a higher familiarity when the number of images including both the second person and the second person is larger. The intimacy calculating unit calculates an in-image intimacy that is an intimacy between persons included in the image based on the number of persons recognized by the person recognizing unit for each of the plurality of images. Based on the in-image intimacy calculated for each of the plurality of images and the calculation unit, an overall intimacy that is an intimacy between each of the persons included in at least some of the images is determined. An overall closeness calculation unit for calculating, and the determination unit has a predetermined difference in overall closeness with respect to one person of each of a plurality of persons recognized as different persons by the person recognition unit. It may be determined whether or not it is below the reference value. The in-image closeness calculation unit may calculate the in-image closeness between the persons included in the image higher as the number of persons recognized by the person recognition unit for each of the plurality of images is smaller.

  The intimacy calculation unit, based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy in the image that is the intimacy between the persons included in the image, and the image indicating each of the persons The image intimacy calculation unit that calculates a higher value as the distance between the images decreases, and at least some of the plurality of images based on the image intimacy calculated for each of the plurality of images. And an overall closeness calculating unit that calculates an overall closeness that is a closeness between each of the persons included in the person, and the determination unit is configured to each of a plurality of persons recognized as different persons by the person recognition unit. It may be determined whether or not the difference in overall familiarity with respect to one person is equal to or less than a predetermined reference value. For each of the plurality of images, the in-image intimacy calculation unit indicates the in-image intimacy between the persons included in the image, and the smaller the distance between the face images included in the image indicating each of the persons, It may be calculated as a higher value.

  The determination unit selects one person from among persons included in at least some of the plurality of images, and selects from among persons included in at least some of the plurality of images. It may be determined whether or not a plurality of different persons whose difference in intimacy with respect to one person is equal to or less than a predetermined reference value can be selected. The determination unit selects a plurality of different persons from the persons included in at least some of the images, and from the persons included in at least some of the images. It may be determined whether or not one person whose difference in closeness with respect to each of the plurality of selected persons is equal to or less than a predetermined reference value can be selected. The determination unit determines whether or not a plurality of selected persons can be selected as a single person with higher priority with respect to a larger number of included persons. Also good.

  The determination unit determines whether the difference in intimacy of each of the plurality of persons recognized as different persons by the person recognition unit with respect to another person different from the one person is equal to or less than a predetermined reference value. The correction unit determines whether the difference in familiarity between each of the plurality of persons recognized as different persons is equal to or less than a predetermined reference value for one person and the other person. If it is determined that the plurality of persons are the same person, the recognition result by the person recognition unit may be corrected. The determination unit further determines whether or not the face images of a plurality of persons recognized as different persons by the person recognition unit are similar with a similarity equal to or higher than a predetermined reference value, and a correction unit Is a difference in intimacy with respect to one person among a plurality of persons recognized as different persons is equal to or less than a predetermined reference value, and each of face images in the plurality of persons is If it is determined that they are similar with a degree of similarity equal to or greater than a predetermined reference value, the plurality of persons may be determined to be the same person, and the recognition result by the person recognition unit may be corrected.

  For each of a plurality of persons, the person recognizing unit stores a feature quantity database that stores the feature quantity of the person's face image, and for each of the plurality of images, a feature quantity extracted from the person image included in the image. Is similar to the feature quantity indicating the face image stored in the feature quantity database with a similarity equal to or higher than a predetermined reference value, the person is the person corresponding to the face image. A recognizing processing unit that recognizes that a plurality of persons recognized as different persons by the recognizing processing unit are the same person. In order for the persons to be recognized as the same person, the similarity reference value in the recognition processing unit may be modified to be lower.

  Further, in the second aspect of the present invention, there is provided a person recognition method for recognizing a person included in each of a plurality of images, wherein the person recognizing a plurality of persons included in the image for each of the plurality of images. In a recognition step, a closeness calculation step of calculating a closeness between each of the persons included in at least some of the images based on the recognition result in the human recognition step, and in the human recognition step A determination step of determining whether or not a difference in intimacy with respect to one person among a plurality of persons recognized as different persons is equal to or less than a predetermined reference value; The plurality of persons are the same person when it is determined that the closeness difference of each of the plurality of persons with respect to one person is equal to or less than a predetermined reference value. Determined, and a modified step of modifying the recognition result in the person recognition stage.

  According to a third aspect of the present invention, there is provided a person recognition program that causes a computer to function as a person recognition system for recognizing a person included in each of a plurality of images. Based on the recognition result of the person recognition unit that recognizes multiple persons included in the image and the person recognition unit, the intimacy between each of the persons included in at least some of the images is calculated. The degree of familiarity of a plurality of persons recognized as different persons by the familiarity calculating unit and the person recognizing unit determines whether or not the difference in familiarity with respect to one person is equal to or less than a predetermined reference value. The difference in intimacy with respect to one person of each of a plurality of persons recognized as different persons from the determination unit that is different from a predetermined reference value If it is determined that, the plurality of persons is determined to be the same person, to function as a human object recognition system comprising a correction unit for correcting the recognition result by the person recognition unit.

  The above summary of the invention does not enumerate all the necessary features of the present invention, and sub-combinations of these feature groups can also be the invention.

  According to the present invention, even when the same person is mistakenly recognized as a plurality of different persons, the recognition result can be corrected and a highly accurate recognition result of the person can be provided to the user.

  Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the scope of claims, and all combinations of features described in the embodiments are included. It is not necessarily essential for the solution of the invention.

  FIG. 1 is a block diagram illustrating an example of a functional configuration of a person recognition system 10 according to an embodiment of the present invention. The person recognition system 10 includes an image storage unit 100, a person recognition unit 110, a closeness calculation unit 120, a determination unit 130, and a correction unit 140. The person recognition system 10 recognizes a person included in each of a plurality of images stored in the image storage unit 100. For example, when a plurality of images included in the electronic album are stored in the image storage unit 100, the person recognition system 10 recognizes a person included in each of the plurality of images and also includes a person included in the images. May be recorded in an image file indicating the image as tag information based on an Exif (Exchangeable image file) format or the like. Thus, the user can easily view the plurality of images as a slide show categorized for each person or organize them for each person. The person recognition system 10 may be a personal computer, for example, or may be a digital still camera, a mobile phone terminal provided with a digital still camera, or the like.

  The person recognition system 10 according to the embodiment of the present invention corrects the recognition result of the person included in each of the plurality of images based on the intimacy between the persons calculated based on the plurality of images. It aims to provide users with high recognition results.

  The image storage unit 100 may be a storage device using, for example, a magnetic recording medium or a semiconductor recording medium, and stores a plurality of images. The person recognition unit 110 recognizes a plurality of persons included in the image for each of the plurality of images stored in the image storage unit 100 by using a known image recognition technique. The person recognition unit 110 includes a feature amount database 112 and a recognition processing unit 114. The feature amount database 112 stores the feature amount of the face image of each person for each of a plurality of persons. For each of a plurality of images stored in the image storage unit 100, the recognition processing unit 114 indicates the face image stored in the feature amount database 112, with the feature amount extracted from the person image included in the image. When the feature amount is similar to the feature value equal to or higher than a predetermined reference value, the person is recognized as a person corresponding to the face image. Then, the recognition processing unit 114 outputs a person recognition result for each image to the closeness calculation unit 120 and the correction unit 140.

  The closeness calculation unit 120 receives at least a part of the plurality of images based on the person recognition result for each of the plurality of images stored in the image storage unit 100 received from the recognition processing unit 114. The closeness between each person included in the image is calculated. Then, the familiarity calculation unit 120 outputs the calculated familiarity to the determination unit 130. The determination unit 130 determines whether or not the difference in familiarity with respect to one person among a plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. . When the determination unit 130 determines that the difference in intimacy of each of the plurality of persons with respect to the one person is equal to or less than a predetermined reference value, the determination unit 130 determines the information indicating the plurality of persons and the determination. The result is output to the correction unit 140. The correction unit 140 is determined by the determination unit 130 that the difference in closeness of each of a plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. If it is determined that the plurality of persons are the same person, the recognition result by the recognition processing unit 114 is corrected.

  According to the person recognition system 10 according to the embodiment of the present invention, when the closeness of each of a plurality of persons is similar to one person, the plurality of persons are actually the same person. The recognition result of the person can be corrected. As a result, in the person recognition process using the image recognition technology, even when a person image is erroneously recognized as a plurality of different person images, the plurality of persons are recognized as the same person. The result can be corrected and a highly accurate person recognition result can be provided to the user.

  FIG. 2 shows an example of the image storage unit 100 according to the embodiment of the present invention. The image storage unit 100 stores, for each of a plurality of images, an image identifier for identifying the image and image data indicating the image in association with each other. Here, the image identifier may be a value determined to be unique for each image when the image storage unit 100 stores the image. In this figure, the image identifier is shown as a numerical value. Alternatively, the image identifier may be a character string such as a file name assigned to a file indicating image data.

  FIG. 3 shows an example of the feature quantity database 112 according to the embodiment of the present invention. The feature quantity database 112 stores, for each of a plurality of persons, a person identifier for identifying the person and a feature quantity of the face image of the person in association with each other. Here, the person identifier may be a value determined to be unique for each person when the feature quantity of the person's face image is stored in the feature quantity database 112, for example. Further, the feature quantity database 112 may store a plurality of feature quantities (for example, A, B,...) For each person. Here, the feature amount of the face image of a person is, for example, the face image itself, the face contour shape extracted from the face image using a known image processing technique, or the eyes, nose, mouth, The information may be various information such as the shape of a characteristic part of the face, the position of each part in the entire face, or the positional relationship between the parts.

  The feature amount database 112 may receive the feature amount of the face image for each person from the recognition processing unit 114 and store it. Specifically, when the recognition processing unit 114 recognizes a person included in the image stored in the image storage unit 100, the feature amount extracted from the face image of the person included in the image is the feature amount database. If the feature amount of any person stored in 112 does not show a similarity equal to or higher than a predetermined reference value, the feature amount extracted from the face image is used as the new face image of the person. The feature amount may be stored in the feature amount database 112 as a feature amount. Thereby, the feature amount database 112 can store the feature amount of the face image of the person included in at least some of the plurality of images stored in the image storage unit 100.

  FIG. 4 shows an example of a person recognition result by the recognition processing unit 114 according to the embodiment of the present invention. For example, the recognition processing unit 114 performs well-known image processing such as contour extraction processing and color distribution analysis processing on each of the plurality of images stored in the image storage unit 100, so that the plurality of images included in the image are included. Detect the subject. Then, the recognition processing unit 114 detects a characteristic amount specific to the face of the person from the detected images of the subjects, and when the subject is a person, the recognition processing unit 114 detects the face of the person from the images of the subject. The partial image which shows is detected. Then, the recognition processing unit 114 extracts the feature amount from the partial image indicating the detected person's face, and the extracted feature amount is stored in the feature amount database 112 illustrated in FIG. It compares with the feature-value of the face image about each. Then, when the extracted feature amount is similar to the feature amount of the face image stored in the feature amount database 112 with a similarity equal to or higher than a predetermined reference value, The subject detected from the image is recognized as a person associated with the feature quantity of the face image in the feature quantity database 112, and a person identifier indicating the person is detected. In this way, the recognition processing unit 114 recognizes each of the plurality of persons included in the image for each of the plurality of images stored in the image storage unit 100 as illustrated in FIG. A plurality of person identifiers indicating the person is detected.

  FIG. 5 shows an example of the calculation result of the familiarity by the familiarity calculation unit 120 according to the embodiment of the present invention. The familiarity calculating unit 120 calculates the familiarity between each of the persons included in at least some of the plurality of images stored in the image storage unit 100. For example, as shown in the figure, the familiarity calculation unit 120 calculates the familiarity between a person whose person identifier is 0 and a person whose person identifier is 1 as 10. Here, the familiarity calculating unit 120 is a familiarity between the first person and the second person included in at least some of the plurality of images stored in the image storage unit 100. May be calculated as a higher familiarity when the number of images including both the first person and the second person among the plurality of images is larger. For example, the familiarity calculation unit 120 may calculate the number of images including both the first person and the second person as the familiarity. Thereby, since it is possible to detect persons who are frequently included in the image as persons having a more intimate relationship, it is possible to accurately calculate the intimacy between the persons.

  Then, the determination unit 130 determines whether or not the difference in familiarity with respect to one person among a plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. To do. Here, the determination unit 130 may use, for example, one of the two methods described below in making the determination. In the first method, the determination unit 130 selects one person from among persons included in at least a part of the plurality of images stored in the image storage unit 100, and Determining whether or not a plurality of different persons can be selected from among persons included in at least some of the images, the difference in intimacy with respect to the selected one being equal to or less than a predetermined reference value To do. In the second method, the determination unit 130 selects a plurality of different persons from at least some of the plurality of images stored in the image storage unit 100, and Whether or not one person whose difference in intimacy with respect to each of a plurality of selected persons is equal to or less than a predetermined reference value can be selected from persons included in at least some of the images. to decide. Regardless of which method is used, the determination unit 130 can efficiently determine whether or not the correction unit 140 should correct the recognition result of the person by the recognition processing unit 114.

  Then, the correction unit 140 includes, for example, information indicating a person recognition result by the recognition processing unit 114 for each of a plurality of images stored in the image storage unit 100, for example, a person identifier included in the image, The tag information based on the Exif format or the like is recorded in an image file indicating the image. Here, the correction unit 140 determines that the difference in familiarity with respect to one person among the plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. When the determination is made, the recognition result corrected so that the plurality of persons indicate the same person may be recorded in each image file.

  In addition, the person recognition system 10 may use the person recognition result by the recognition processing unit 114 corrected by the correction unit 140 by various methods. For example, the correction unit 140 may correct the recognition result of the person by the recognition processing unit 114 based on the familiarity, and then display the corrected recognition result on, for example, a display device provided in the person recognition system 10. . Further, the person recognition system 10 may record not only the person recognition result but also the familiarity calculated by the familiarity calculating unit 120 as tag information in an image file indicating each image.

  In addition, when the correction unit 140 determines that a plurality of persons recognized as different persons by the recognition processing unit 114 are the same person, the correction unit 140 not only corrects the recognition result, but also uses the recognition processing unit 114 to In order for the plurality of persons to be recognized as the same person, the reference value of the similarity of the feature amount in the recognition processing unit 114 may be corrected to be lower. Here, the reference value corrected to be lower may be, for example, the maximum value among the reference values by which the recognition processing unit 114 recognizes the plurality of persons as the same person.

  For example, the person recognition system 10 may initially set a reference value for the similarity of feature amounts in the recognition processing unit 114 higher than a typical value predetermined by a user or the like. In this case, when the recognition processing unit 114 recognizes a person included in each of a plurality of images stored in the image storage unit 100 as a plurality of different persons even though they are the same person, Will increase. However, in the person recognition system 10, when it is determined that the plurality of persons are the same person based on the familiarity between the persons, the similarity reference value is decreased, and an appropriate reference for the person is obtained. A value can be set. Thereby, the recognition process part 114 can perform a highly accurate recognition process after that.

  Further, when the correction unit 140 determines that a plurality of persons recognized as different persons by the recognition processing unit 114 are the same person, the correction unit 140 further stores each of the plurality of persons in the feature amount database 112. The feature amount may be corrected. Specifically, in this case, the correction unit 140 sets each feature amount stored in the feature amount database 112 for each of the plurality of persons as a feature amount in the face image of one person. May be merged. Thereby, the recognition process part 114 can perform a highly accurate recognition process after that.

  Note that the person recognition system 10 determines whether or not to correct the person recognition result by the recognition processing unit 114 with higher accuracy by performing other processes in addition to the processes described above. Also good. For example, the determination unit 130 determines whether the face images of a plurality of persons recognized as different persons by the recognition processing unit 114 are similar with a similarity equal to or higher than a reference value determined in advance by the user or the like. It may be further judged. Then, the correcting unit 140 has a difference in closeness with respect to one person among a plurality of persons recognized as different persons being equal to or less than a predetermined reference value, and the face images of the plurality of persons Are determined to be similar to each other with a predetermined reference value or more, and when the determination unit 130 determines that the plurality of persons are the same person, the recognition processing unit 114 The recognition result may be corrected. As a result, there are a plurality of different persons with similar tendencies of factors that determine intimacy, such as when the number of images stored in the image storage unit 100 is small or a combination of persons included in the images. Even in this case, whether or not the recognition result of the person by the recognition processing unit 114 should be corrected can be determined with high accuracy.

  In addition, for example, the determination unit 130 determines a difference in intimacy of each of a plurality of persons recognized as different persons by the recognition processing unit 114 with respect to another person different from the one person by a predetermined criterion. You may further judge whether it is below a value. Then, the correction unit 140 determines that the closeness difference between each of the plurality of persons recognized as different persons is equal to or less than a predetermined reference value for both the one person and the other person. If it is determined that there is, it may be determined that the plurality of persons are the same person, and the recognition result of the person by the recognition processing unit 114 may be corrected. In this way, when determining whether or not a plurality of persons recognized as different persons are the same person based on the difference in intimacy with respect to a certain person, only based on intimacy with respect to one person By determining based on the familiarity with respect to each of a plurality of persons instead of determining, it can be determined with higher accuracy whether or not the recognition result of the person by the recognition processing unit 114 should be corrected.

  FIG. 6 is a flowchart showing a first example of a process flow in the person recognition method using the person recognition system according to the embodiment of the present invention. In this example, a case where the determination unit 130 uses the first method described in FIG. 5 will be described. First, the person recognizing unit 110 recognizes a plurality of persons included in each of the plurality of images stored in the image storage unit 100 (S1000). Subsequently, the intimacy calculation unit 120 calculates the intimacy between each of the persons included in at least some of the plurality of images based on the recognition result of the persons for each of the plurality of images. Calculate (S1010).

  Subsequently, the determination unit 130 selects at least some of the persons included in at least some of the images, and each of the selected persons (hereinafter referred to as one person). The following processing is repeated for (S1020). Here, the determination unit 130 may select a person included in more images among a plurality of images included in at least some of the images with higher priority. In general, in the correlation between the familiarity with respect to a certain person and the number of persons indicating the familiarity, the higher the familiarity, the smaller the number of persons indicating the familiarity. Therefore, based on the familiarity with respect to the selected one person, whether or not a plurality of persons included in at least some of the plurality of images are misrecognized as a plurality of different persons. In order to make a determination with high accuracy, it is preferable to make the determination based on a higher familiarity. In other words, the determination unit 130 does not select all of the persons included in at least a part of the plurality of images, but the person included in more images, that is, more persons. By selecting only a person showing high intimacy and performing the following processing, the determination can be performed with high accuracy and efficiency.

  First, the determination unit 130 selects a plurality of persons whose intimacy differences with respect to the selected one person are equal to or less than a predetermined reference value from among persons included in at least some of the plurality of images. (S1030). For example, the determination unit 130 extracts, for each person included in at least some of the plurality of images, the familiarity of the person with respect to the selected one person, and is different from the person. In the case where the familiarity of the selected person with respect to the selected one person shows a difference equal to or less than a predetermined reference value with respect to the extracted familiarity, the person and the other person are described above. You may select as several persons. Then, the determination unit 130 determines whether or not a plurality of persons can be selected (S1040).

  When a plurality of persons can be selected (S1040: Yes), the determination unit 130 further determines whether each of the face images of the plurality of selected persons is similar with a similarity equal to or higher than a predetermined reference value. Is determined (S1050). Here, the determination unit 130 extracts a feature amount from each of the face images of the plurality of persons by using a known image processing technique, and compares each of the extracted feature amounts with each other. It may be determined whether or not each of the face images in the person is similar. Then, when it is determined that the face images of the plurality of selected persons are similar with a similarity equal to or higher than the reference value (S1050: Yes), the correction unit 140 determines that the plurality of persons are the same person. It is determined that there is, and the recognition result for the plurality of persons by the person recognition unit 110 is corrected (S1060). Then, the determination unit 130 repeats the above processing for each selected person (S1070).

  FIG. 7 is a flowchart showing a second example of the flow of processing in the person recognition method using the person recognition system according to the embodiment of the present invention. In this example, the case where the determination unit 130 uses the second method described in FIG. 5 will be described. First, the person recognizing unit 110 recognizes a plurality of persons included in each of a plurality of images stored in the image storage unit 100 (S1100). Subsequently, the intimacy calculation unit 120 calculates the intimacy between each of the persons included in at least some of the plurality of images based on the recognition result of the persons for each of the plurality of images. Calculate (S1110).

  Subsequently, the determination unit 130 selects a plurality of different combinations of persons from at least some of the plurality of images and repeats the following processing for each selected combination ( S1120). Here, when selecting a combination of a plurality of persons, the determination unit 130 is similar to each other in facial images among the persons included in the at least some images with a similarity equal to or higher than a predetermined reference value. A plurality of persons may be selected. Accordingly, it is possible to efficiently determine whether or not to correct the person recognition result by the person recognition unit 110 without repeating the following process for all combinations of a plurality of persons.

  First, the determination unit 130 is configured such that a difference in intimacy with respect to each of a plurality of selected persons from persons included in at least some of the plurality of images is equal to or less than a predetermined reference value. A person is selected (S1130). Here, whether or not the determination unit 130 can select a single person with a higher priority with respect to a larger number of persons included in an image including any of the selected plurality of persons. You may try. In general, in the correlation between the familiarity with respect to a certain person and the number of persons indicating the familiarity, the higher the familiarity, the smaller the number of persons indicating the familiarity. Therefore, in order to determine with high accuracy based on the familiarity whether or not the selected multiple persons are persons misrecognized as a plurality of different persons, the determination should be performed at a higher familiarity. Preferably based on That is, the determination unit 130 does not attempt to select all the persons included in at least some of the plurality of images as one person, but with each of the selected persons. Performing the determination with high accuracy and efficiency by trying whether or not only a person with a high frequency in the image, that is, a person with high intimacy with respect to the plurality of persons, can be selected as one person. Can do.

  Then, the determination unit 130 determines whether one person has been selected (S1140). When one person can be selected (S1140: Yes), the correction unit 140 determines that the plurality of selected persons are the same person, and corrects the recognition result of the plurality of persons by the person recognition unit 110. (S1150). Then, the determination unit 130 repeats the above processing for each selected combination of a plurality of persons (S1160).

  FIG. 8 is a block diagram illustrating an example of a functional configuration of the familiarity calculation unit 120 according to the first modification example of the embodiment of the present invention. In the first modification of the embodiment of the present invention, the familiarity calculation unit 120 includes an in-image familiarity calculation unit 122 and an overall familiarity calculation unit 124. The intimacy in image calculation unit 122 is based on the number of persons recognized by the recognition processing unit 114 for each of a plurality of images stored in the image storage unit 100, and the intimacy between persons included in the image The intimacy in the image is calculated. Then, the in-image closeness calculation unit 122 outputs the calculated in-image closeness in each image to the overall closeness calculation unit 124. The overall closeness calculation unit 124, based on the intimacy within the image calculated by the intimacy within image calculation unit 122 for each of the plurality of images stored in the image storage unit 100, The total closeness, which is the closeness between each person included in at least some of the images, is calculated. Then, the overall familiarity calculation unit 124 outputs the calculated overall familiarity to the determination unit 130. Then, the determination unit 130 illustrated in FIG. 1 is configured such that a difference in overall familiarity with respect to one person among a plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. It is determined whether or not.

  According to the person recognition system 10 according to the first modification example of the embodiment of the present invention, based on the number of persons included in an image, an in-image closeness that is an intimacy between persons included in the image is calculated. In addition, based on the in-image intimacy in each of the plurality of images stored in the image storage unit 100, it is possible to calculate the intimacy between the persons in the entire plurality of images. it can. Thus, even when an image of one person is erroneously recognized as an image of a plurality of different persons, the plurality of persons are determined to be the same person based on the calculated overall familiarity. Thus, the recognition result can be corrected, and a highly accurate person recognition result can be provided to the user.

  FIG. 9 shows an example of the calculation result of the in-image familiarity by the in-image familiarity calculation unit 122 according to the first modification of the embodiment of the present invention. The in-image closeness calculation unit 122 calculates an in-image closeness that is a closeness between persons included in the image for each of the plurality of images stored in the image storage unit 100. For example, the image intimacy calculation unit 122 calculates the image intimacy between the persons included in the image as the number of persons recognized by the recognition processing unit 114 for each of the plurality of images is smaller. You can. Specifically, the in-image closeness calculating unit 122 may calculate the in-image closeness between persons included in each image as the reciprocal of the number of persons recognized for the image. Note that the in-image closeness calculation unit 122 may not calculate the in-image closeness in the image when the number of persons recognized for the image is 1 or less. The smaller the number of persons included in an image, the more private the relationship between the persons included in the image and the more often each person is more intimate. Therefore, the smaller the number of persons included in the image, the higher the intimacy within the image, so that the intimacy between persons in the image can be calculated with high accuracy.

  FIG. 10 shows an example of the calculation result of the total familiarity by the total familiarity calculation unit 124 according to the first modification of the embodiment of the present invention. The total closeness calculation unit 124 is included in at least some of the plurality of images based on the closeness in images calculated for each of the multiple images stored in the image storage unit 100. The total familiarity that is the familiarity between each person is calculated. For example, as shown in the figure, the total closeness calculation unit 124 calculates the total closeness between the person whose person identifier is 0 and the person whose person identifier is 1 as 9.8. Here, the total closeness calculating unit 124 calculates the total closeness between the first person and the second person included in at least a part of the plurality of images among the plurality of images. The higher the number of images including both the first person and the second person, the higher the calculation may be. For example, the total closeness calculation unit 124 calculates the total closeness between the first person and the second person, and includes both the first person and the second person among a plurality of images. May be calculated by summing up the intimacy in the image between the first person and the second person, calculated for each of the images including.

  The more frequently each of the two persons is included in the same image, the closer the relationship between the two persons is. Therefore, by calculating the total intimacy by adding up the intimacy within the image, it is possible to calculate the intimacy between persons included in a plurality of images stored in the image storage unit 100 with high accuracy. .

  FIG. 11 is a block diagram illustrating an example of a functional configuration of the familiarity calculation unit 120 according to the second modification example of the embodiment of the present invention. In the second modification of the embodiment of the present invention, the familiarity calculation unit 120 includes an in-image familiarity calculation unit 126 and an overall familiarity calculation unit 128. The intimacy calculation unit 126 within the image is based on the recognition result by the recognition processing unit 114 for each of the plurality of images stored in the image storage unit 100, and between the images indicating each person included in the image. Calculate the distance. Then, the in-image familiarity calculation unit 126 determines, for each of the plurality of images, the in-image familiarity that is the intimacy between persons included in the image based on the distance between images indicating the persons. To calculate. Specifically, the in-image familiarity calculation unit 126 calculates the in-image familiarity between persons included in each image as a higher value as the distance between images representing the persons is smaller. To do. Then, the in-image closeness calculating unit 126 outputs the in-image closeness between each person in each calculated image to the overall closeness calculating unit 128.

  The overall closeness calculating unit 128 is based on the closeness in image calculated by the closeness in image calculation unit 126 for each of the plurality of images stored in the image storage unit 100. The total closeness, which is the closeness between each person included in at least some of the images, is calculated. Then, the overall familiarity calculation unit 128 outputs the calculated overall familiarity to the determination unit 130. Then, the determination unit 130 illustrated in FIG. 1 is configured such that a difference in overall familiarity with respect to one person among a plurality of persons recognized as different persons by the recognition processing unit 114 is equal to or less than a predetermined reference value. It is determined whether or not.

  When two persons are very intimate, an image is often captured with the two persons in close proximity. On the other hand, when the two persons are not very close, images are not often captured in the state where the two persons are close to each other. Thus, it can be said that the distance between persons when an image is captured is related to the intimacy between the persons. Here, according to the person recognition system 10 according to the second modification of the embodiment of the present invention, based on the distance between the persons included in the image, the intimacy between the persons included in the image The intimacy can be calculated, and based on the in-image intimacy in each of the plurality of images stored in the image storage unit 100, the intimacy between persons in the whole of the plurality of images is determined. Can be calculated. Thus, even when an image of one person is erroneously recognized as an image of a plurality of different persons, the plurality of persons are determined to be the same person based on the calculated overall familiarity. Thus, the recognition result can be corrected, and a highly accurate person recognition result can be provided to the user.

  FIG. 12 shows an example of a person recognition result by the recognition processing unit 114 according to the second modification of the embodiment of the present invention. For example, the recognition processing unit 114 performs well-known image processing such as contour extraction processing and color distribution analysis processing on each of the plurality of images stored in the image storage unit 100, so that the plurality of images included in the image are included. Detect the subject. Then, the recognition processing unit 114 detects a characteristic amount specific to the face of the person from the detected images of the subjects, and when the subject is a person, the recognition processing unit 114 detects the face of the person from the images of the subject. The partial image which shows is detected. Then, the recognition processing unit 114 extracts the feature amount from the partial image indicating the detected person's face, and the extracted feature amount is stored in the feature amount database 112 illustrated in FIG. It compares with the feature-value of the face image about each. Then, when the extracted feature amount is similar to the feature amount of the face image stored in the feature amount database 112 with a similarity equal to or higher than a predetermined reference value, The subject detected from the image is recognized as a person associated with the feature quantity of the face image in the feature quantity database 112, and a person identifier indicating the person is detected. In this way, the recognition processing unit 114 recognizes each of the plurality of persons included in the image for each of the plurality of images stored in the image storage unit 100 as illustrated in FIG. A plurality of person identifiers indicating the person is detected.

  In addition, the recognition processing unit 114 uses, for each of the plurality of images, position information indicating the position in the image of the partial image indicating the person detected in the image or the face of the person, the intimacy calculation unit 126 in the image. Output to. Here, the position information may include, for example, the coordinate values of the pixels at the upper left corner and the lower right corner of the partial image indicating the person or the face of the person in each image.

  FIG. 13 shows an example of the calculation result of the in-image familiarity by the in-image familiarity calculation unit 126 according to the second modification of the embodiment of the present invention. The in-image closeness calculation unit 126 calculates an in-image closeness that is a closeness between persons included in the image for each of the plurality of images stored in the image storage unit 100. Specifically, the image intimacy calculation unit 126 calculates the image intimacy between persons included in the image for each of the plurality of images, and the smaller the distance between the images indicating the persons. It may be calculated as a higher value. For example, the in-image closeness calculation unit 126 calculates the in-image closeness so that the in-image closeness between persons included in each image is inversely proportional to the distance between images representing the respective persons. It's okay. In addition, the image intimacy calculation unit 126 calculates, for each of the plurality of images, the image intimacy between the persons included in the image, and the distance between the face images included in the images indicating the persons. A smaller value may be calculated as a higher value. The in-image familiarity calculation unit 126 receives a distance between images indicating each person or a distance between face images included in an image indicating each person from the recognition processing unit 114. You may calculate based on the positional information which shows the position in the said image, and the area information which shows the magnitude | size in the said image of the partial image which shows the person in each of the image or the said person's face.

  When two people's faces are close to each other and an image is captured, the two people are often very close. On the other hand, when the two persons are not very close, images are not often captured with the faces of the two persons close to each other. Here, according to the person recognition system 10 according to the second modified example of the embodiment of the present invention, the intimacy in the image can be calculated based on the distance between the face images of the persons included in the image. Thus, it is possible to calculate the overall familiarity, which is the familiarity between persons in the entire image stored in the image storage unit 100, with high accuracy.

  In addition, the in-image familiarity calculation unit 126 has a distance between images indicating each person included in the image for each of the plurality of images stored in the image storage unit 100 equal to or greater than a predetermined reference value. In this case, the closeness in the image between the persons may be calculated as a predetermined standard value. Here, the predetermined reference value and the predetermined standard value may be values specified in advance by a user or the like, for example.

  Although the distance between two persons and the intimacy between the two persons are related, at any distance, the closer the distance, the more intimate is not always. For example, there may be cases where there is no difference in the intimacy between the two persons when the two persons are 10 meters apart and when the two persons are 15 meters away. Here, according to the person recognition system 10 according to the second modified example of the embodiment of the present invention, when the distance between persons is equal to or greater than a predetermined reference value, Since the familiarity can be calculated as a predetermined standard familiarity, it is possible to calculate the overall familiarity that is the familiarity between persons in the entire image stored in the image storage unit 100 with high accuracy. it can.

  FIG. 14 shows an example of the calculation result of the total familiarity by the total familiarity calculating unit 128 according to the second modification of the embodiment of the present invention. The total closeness calculating unit 128 calculates the plurality of images based on the intimacy within the image between each person included in the image calculated for each of the plurality of images stored in the image storage unit 100. An overall closeness that is a closeness between each person included in at least some of the images is calculated. For example, as shown in the figure, the total closeness calculating unit 128 calculates the total closeness between a person whose person identifier is 0 and a person whose person identifier is 1 as 120. Here, the total closeness calculating unit 128 calculates the total closeness between the first person and the second person included in at least some of the plurality of images, among the plurality of images. The higher the number of images including both the first person and the second person, the higher the calculation may be. For example, the total closeness calculation unit 128 calculates the total closeness between the first person and the second person, and sets both the first person and the second person among the plurality of images. May be calculated by summing up the intimacy in the image between the first person and the second person, calculated for each of the images including.

  The more frequently each of the two persons is included in the same image, the closer the relationship between the two persons is. Therefore, by calculating the total intimacy by summing up the intimacy within the image, the intimacy between persons included in the image stored in the image storage unit 100 can be calculated with high accuracy.

  FIG. 15 is a block diagram showing an example of a hardware configuration of a computer 1500 according to the embodiment of the present invention. A computer 1500 according to an embodiment of the present invention includes a CPU peripheral unit having a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 connected to each other by a host controller 1582, and a host controller 1582 by an input / output controller 1584. I / O unit having a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 connected to the I / O controller, and a legacy input having a ROM 1510, a flexible disk drive 1550, and an I / O chip 1570 connected to the I / O controller 1584. And an output unit.

  The host controller 1582 connects the RAM 1520 to the CPU 1505 and the graphic controller 1575 that access the RAM 1520 at a high transfer rate. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 and the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the communication interface 1530, the hard disk drive 1540, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The communication interface 1530 communicates with other devices via a network. The hard disk drive 1540 stores programs and data used by the CPU 1505 in the computer 1500. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program executed when the computer 1500 is started up, a program depending on the hardware of the computer 1500, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 and, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

  The person recognition program provided to the hard disk drive 1540 via the RAM 1520 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The person recognition program is read from the recording medium, installed in the hard disk drive 1540 in the computer 1500 via the RAM 1520, and executed by the CPU 1505. The person recognition program installed and executed on the computer 1500 works on the CPU 1505 or the like to cause the computer 1500 to function as the person recognition system 10 described with reference to FIGS.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 1500 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

It is a block diagram showing an example of functional composition of person recognition system 10 concerning an embodiment of the present invention. It is a figure which shows an example of the image storage part 100 which concerns on embodiment of this invention. It is a figure which shows an example of the feature-value database 112 which concerns on embodiment of this invention. It is a figure which shows an example of the recognition result of the person by the recognition process part 114 which concerns on embodiment of this invention. It is a figure which shows an example of the calculation result of the familiarity by the familiarity calculation part 120 which concerns on embodiment of this invention. It is a flowchart which shows the 1st example of the flow of a process in the person recognition method using the person recognition system which concerns on embodiment of this invention. It is a flowchart which shows the 2nd example of the flow of a process in the person recognition method using the person recognition system which concerns on embodiment of this invention. It is a block diagram which shows an example of a function structure of the familiarity calculation part 120 which concerns on the 1st modification in embodiment of this invention. It is a figure which shows an example of the calculation result of the in-image closeness by the in-image closeness calculation part 122 which concerns on the 1st modification in embodiment of this invention. It is a figure which shows an example of the calculation result of the total closeness by the total closeness calculation part 124 which concerns on the 1st modification in embodiment of this invention. It is a block diagram which shows an example of a function structure of the familiarity calculation part 120 which concerns on the 2nd modification in embodiment of this invention. It is a figure which shows an example of the recognition result of the person by the recognition process part 114 which concerns on the 2nd modification in embodiment of this invention. It is a figure which shows an example of the calculation result of the image familiarity by the image familiarity calculation part 126 which concerns on the 2nd modification in embodiment of this invention. It is a figure which shows an example of the calculation result of the total closeness by the total closeness calculation part 128 which concerns on the 2nd modification in embodiment of this invention. It is a block diagram which shows an example of the hardware constitutions of the computer 1500 concerning embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Person recognition system, 100 Image storage part, 110 Person recognition part, 112 Feature-value database, 114 Recognition processing part, 120 Intimacy calculation part, 122 In-image intimacy calculation part, 124 Whole intimacy calculation part, 126 In-image parent Density calculation unit, 128 Total familiarity calculation unit, 130 Judgment unit, 140 Correction unit

Claims (14)

A person recognition system for recognizing a person included in each of a plurality of images,
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit, a closeness calculation unit that calculates a closeness between each of the persons included in at least some of the plurality of images,
A determination unit that determines whether or not a difference in the intimacy with respect to one person of each of a plurality of persons recognized as different persons by the person recognition unit is equal to or less than a predetermined reference value;
When it is determined that a difference in intimacy of each of a plurality of persons recognized as different persons with respect to the one person is equal to or less than a predetermined reference value, the plurality of persons are the same person. A person recognition system comprising: a correction unit that determines that there is a correction and corrects a recognition result by the person recognition unit.   The intimacy calculating unit calculates the intimacy between the first person and the second person included in at least a part of the plurality of images, among the plurality of images. The person recognition system according to claim 1, wherein when the number of images including both of the first person and the second person is larger, the person recognition system calculates the higher intimacy. The intimacy calculation unit
An in-image intimacy calculating unit that calculates in-image intimacy that is the intimacy between the persons included in the image based on the number of persons recognized by the person recognizing unit for each of the plurality of images; ,
Based on the intimacy within the image calculated for each of the plurality of images, an overall familiarity that is the intimacy between each of the persons included in at least some of the plurality of images. An overall familiarity calculating unit for calculating,
The determination unit determines whether a difference in the overall familiarity with respect to the one person among a plurality of persons recognized as different persons by the person recognition unit is equal to or less than a predetermined reference value. The person recognition system according to claim 1.   The in-image closeness calculation unit calculates the in-image closeness between the persons included in the image as the number of persons recognized by the person recognition unit for each of the plurality of images is smaller. The person recognition system according to claim 3. The intimacy calculation unit
Based on the recognition result by the person recognition unit for each of the plurality of images, the intimacy within the image, which is the intimacy between the persons included in the image, and the distance between the images indicating each of the persons The smaller the is, the in-image familiarity calculating unit that calculates a higher value;
Based on the intimacy within the image calculated for each of the plurality of images, an overall familiarity that is the intimacy between each of the persons included in at least some of the plurality of images. An overall familiarity calculating unit for calculating,
The determination unit determines whether a difference in the overall familiarity with respect to the one person among a plurality of persons recognized as different persons by the person recognition unit is equal to or less than a predetermined reference value. The person recognition system according to claim 1.   For each of the plurality of images, the in-image intimacy calculation unit calculates the in-image intimacy between persons included in the image, and the distance between face images included in the images indicating the persons. The person recognition system according to claim 5, wherein a smaller value is calculated as a higher value.   The determination unit selects the one person from persons included in at least some of the plurality of images, and includes persons included in at least some of the plurality of images. 2. The person recognition system according to claim 1, wherein it is determined whether or not a plurality of different persons whose difference in intimacy with respect to the selected one person is equal to or less than a predetermined reference value can be selected.   The determination unit selects a plurality of different persons from those included in at least some of the plurality of images, and is included in at least some of the plurality of images. The person recognition system according to claim 1, wherein it is determined whether or not the one person whose difference in intimacy with respect to each of the selected plurality of persons is equal to or less than a predetermined reference value can be selected.   The determination unit determines whether or not a plurality of selected persons can be selected as the one person with higher priority with respect to a larger number of persons included in the image. The person recognition system according to claim 8. In the determination unit, a difference in the intimacy of each of a plurality of persons recognized as different persons by the person recognition unit with respect to another person different from the one person is equal to or less than a predetermined reference value. Further determine whether or not
In the correction unit, the difference in intimacy between each of a plurality of persons recognized as different persons is equal to or less than a predetermined reference value for both the one person and the other person. The person recognition system according to claim 1, wherein the plurality of persons are determined to be the same person and the recognition result by the person recognition unit is corrected. The determination unit further determines whether or not each of the face images in a plurality of persons recognized as different persons by the person recognition unit is similar with a similarity equal to or higher than a predetermined reference value,
The correction unit is configured so that a difference between the intimacy of the plurality of persons recognized as different persons with respect to the one person is equal to or less than a predetermined reference value, and face images of the plurality of persons Are determined to be similar with a degree of similarity equal to or greater than a predetermined reference value, the plurality of persons are determined to be the same person, and the recognition result by the person recognition unit is corrected. Item 2. The person recognition system according to Item 1. The person recognition unit
For each of a plurality of persons, a feature amount database storing feature amounts of the person's face image;
For each of the plurality of images, a feature amount extracted from an image of a person included in the image is equal to or greater than a predetermined reference value with respect to a feature amount indicating the face image stored in the feature amount database. A recognition processing unit that recognizes the person as a person corresponding to the face image when they are similar in degree of similarity.
When the correction unit determines that the plurality of persons recognized as different persons by the recognition processing unit are the same person, the correction processing unit further recognizes the plurality of persons as the same person. Therefore, the reference value of the similarity in the recognition processing unit is corrected to be lower. A person recognition method for recognizing a person included in each of a plurality of images,
For each of the plurality of images, a person recognition stage for recognizing a plurality of persons included in the image;
A familiarity calculating step of calculating a familiarity between each of the persons included in at least some of the plurality of images based on the recognition result in the person recognition step;
A step of determining whether or not a difference in intimacy with respect to one person of each of a plurality of persons recognized as different persons in the person recognition stage is equal to or less than a predetermined reference value;
When it is determined that a difference in intimacy of each of a plurality of persons recognized as different persons with respect to the one person is equal to or less than a predetermined reference value, the plurality of persons are the same person. A person recognition method comprising: a correction step of determining that there is a correction and correcting a recognition result in the person recognition step. A person recognition program for causing a computer to function as a person recognition system for recognizing a person included in each of a plurality of images,
The computer,
For each of the plurality of images, a person recognition unit that recognizes a plurality of persons included in the image;
Based on the recognition result by the person recognition unit, a closeness calculation unit that calculates a closeness between each of the persons included in at least some of the plurality of images,
A determination unit configured to determine whether or not a difference in the familiarity with respect to one person of each of a plurality of persons recognized as different persons by the person recognition unit is equal to or less than a predetermined reference value;
When it is determined that a difference in intimacy of each of a plurality of persons recognized as different persons with respect to the one person is equal to or less than a predetermined reference value, the plurality of persons are the same person. A person recognition program for determining that the person recognition system is present and causing the person recognition system to function as a person recognition system.

Images