JP5550222B2 - Image processing apparatus and control method thereof - Google Patents

Description

The present invention relates to a control how the image processing apparatus及patron having a face authentication function.

  Conventionally, in the field of personal authentication (face authentication), there is a personal authentication system using biometric information (for example, Patent Document 1). In the example of Patent Document 1, biometric information for authentication is centrally managed by a server or the like, and data is acquired from the server and used at the time of authentication.

  For example, various techniques have been proposed for data acquisition techniques for personal authentication used at the time of authentication (for example, Patent Document 2).

  On the other hand, in a digital camera as an imaging device, a technique for adding information of a person recognized by a face recognition function to an image file as a tag has been proposed (for example, Patent Document 3). In the example of Patent Document 3, the name registered in the camera corresponding to the recognized person is added to the metadata of the image.

In the example of Patent Document 1, authentication data is centrally managed. However, in the case of a portable digital camera, authentication data is often stored in each imaging device (for example, patents). Reference 4). As a result, a problem regarding data duplication occurs.
JP 2003-178031 A JP 2007-249588 A JP 2004-062868 A JP 2006-165822 A

  In the example of the above-mentioned Patent Document 3, there are the following problems when adding the name of a person whose face has been recognized to an image file.

  That is, even if the same name is added to a plurality of images taken by a plurality of imaging devices, the name is defined by each imaging device, and the person shown in the plurality of images is the same person. The problem arises that some guarantees cannot be made.

  For example, as shown in FIG. 14, in the images taken by the two imaging devices A and B of (a) and (b), the person whose name corresponds to ABC is a completely different person. For this reason, if image files stored in a plurality of imaging devices are classified by added names, different people are classified into the same group.

  In view of the above problems, there is a need for an image processing apparatus that can easily manage face recognition information (face feature information) in which an image with face recognition information added is registered by a plurality of apparatuses.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and a control method thereof capable of generating a feature information ID that does not overlap regardless of the image processing apparatus even if facial feature information is registered in each of the plurality of image processing apparatuses. It is to provide.

In order to achieve the above object, an image processing apparatus according to the present invention includes a face image recognition unit that recognizes a face image from an image, a feature extraction unit that extracts feature information of the recognized face image, and a feature information An image processing apparatus comprising: an ID generation unit that generates a corresponding feature information ID; and a feature information storage unit that stores the feature information, the feature information ID, and personal name information corresponding to the feature information in association with each other. Then , it is determined whether the feature information extracted by the feature extraction unit from the face image in the acquired image and the feature information already stored in the feature information storage unit are the feature information for the same face. Obtaining the feature information, the feature information ID corresponding to the feature information, and the personal name information corresponding to the feature information from the external device, and the obtained feature information Characteristic information ID, and the information of the personal name and a obtaining means to be stored in the characteristic information storage means, said ID generating means generating a characteristic information ID including information indicating a unique number of the image processing apparatus Features.

The control method of an image processing apparatus according to the present invention includes a feature extraction step of extracting recognizing facial image recognition process a face image from images, the characteristic information of the recognized face image, corresponding to the feature information An image processing apparatus control method comprising: an ID generation step for generating a feature information ID; and a feature information storage step for storing the feature information, the feature information ID, and personal name information corresponding to the feature information in association with each other. there are, whether the characteristic information from the face image extracted in the feature extraction step in the acquired image, and feature information that has already been remembers the extent the feature information storage Engineering is the feature information for the same face A determination step for determining whether or not the feature information, a feature information ID corresponding to the feature information, and personal name information corresponding to the feature information are obtained from an external device Features information comprises feature information ID, and a step acquisition for personal name information memorize the ID as product engineering is characterized by generating the characteristic information ID including information indicating a unique number of the image processing apparatus And

According to the onset bright, even though registered feature information of the respective faces by a plurality of image processing apparatus, it is possible to generate feature information ID does not overlap regardless of the image processor.

  Hereinafter, the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block configuration diagram of an imaging apparatus according to an embodiment of the present invention. In the present embodiment, an imaging apparatus will be described as an example of an image processing apparatus.

  In FIG. 1, the imaging apparatus includes a photographing lens 101, an imaging element 102, an A / D converter 103, a memory 104, a face image recognition unit 105, a feature extraction unit 106, a feature comparison unit 107, an ID generation unit 108, feature information. A storage unit 109 and a captured image storage unit 110 are provided. The imaging apparatus also includes an information display unit 111, a communication unit 112, and a microcomputer 113.

  Hereinafter, the function and operation of each unit will be described.

  An image (image) incident from the photographing lens 101 is received by the image sensor 102, converted into a digital signal by the A / D converter 103, and stored in the memory 104.

  The face image recognition unit 105 is a part that recognizes a region of the face image from the image stored in the memory 104, and determines whether or not a face image is included in the captured image (captured image). If an image is included, the position is specified.

  A known technique can be used for recognizing the region of the face image. Known facial image recognition techniques include a method based on learning using a neural network, etc., and using template matching to search for parts with features in the shape of eyes, nose, mouth, etc. from the image. There are methods that are considered.

  In addition, many other methods such as a method using wavelet transform and a method using statistical analysis by detecting feature information such as skin color and eye shape have been proposed. In general, a plurality of these methods are combined to improve the accuracy of face detection.

  When a face image is included in the captured image, the feature extraction unit 106 extracts facial features (feature information) from the face image.

  The feature (feature information) extracted by the feature extraction unit 106 is compared with the face feature information already stored in the feature information storage unit 109 by the feature comparison unit 107, and whether or not the feature information is the same face feature information. Determined.

  The ID generation unit 108 is a device that generates a feature information ID that is a management number when the feature information for face recognition is stored in the feature information storage unit 109. The feature information ID and the feature information generated by the ID generation unit 108 Is stored in the feature information storage unit 109.

  The captured image storage unit 110 is a device that stores captured images. The information display unit 111 is a device that displays a captured image at the time of shooting and feature information stored in the feature information storage unit 109.

  The communication unit 112 is a device that communicates information stored in the feature information storage unit 109 and the captured image storage unit 110 to other imaging devices. The microcomputer 113 is a device that controls the above-described operation.

  FIG. 2 is a flowchart showing a procedure of photographing processing executed by the imaging apparatus of FIG.

  This process is executed under the control of the microcomputer 113 in FIG.

  In FIG. 2, first, in step S <b> 201, the microcomputer 113 causes the image sensor 102 to acquire an image.

  Step S201 functions as an acquisition unit that acquires an image of a subject.

  In step S <b> 202, the microcomputer 113 stores the image acquired by the image sensor 102 in the memory 104.

  In step S <b> 203, the microcomputer 113 causes the face image recognition unit 105 to recognize a face image from the image stored in the memory 104.

  In step S <b> 204, the microcomputer 113 determines whether a face image is included from the recognition result of the face image recognition unit 105.

  If a face image is included, in step S205, the microcomputer 113 causes the feature extraction unit 106 to extract facial features (feature information).

  In the next step S206, the microcomputer 113 determines whether or not the features extracted by the feature comparison unit 107 and the feature information stored in advance in the feature information storage unit 109 are feature information for the same face. I do.

  If they do not match, the process proceeds to step 210, and a feature information ID described later is newly generated. In step S208, the microcomputer 113 stores the captured image in the captured image storage unit 110.

  If they match, the microcomputer 113 acquires the feature information ID corresponding to the feature stored in the feature storage unit 109 in step S207.

  In step S208, the microcomputer 113 assigns the feature information ID as meta information to the photographed image and stores it in the photographed image storage unit 110.

  At this time, when a plurality of subjects are photographed, the positional information is also stored at the same time so that the correspondence between the face image and the feature information ID can be specified.

  FIG. 3 is a diagram for explaining a case where the face-recognized image is stored in the imaging apparatus of FIG.

  For example, in FIG. 3, when a captured image is recognized as a face recognition image 301, position information in the recognized face image is added as a feature information ID as in an image storage image 302. In this way, when the feature information ID is read later, it can be immediately determined which feature information ID corresponds to which face.

  If the feature of the photographed face image does not match the feature already stored in the feature information storage unit 109, the feature information ID is not stored in the photographed image, but only the position information is added to the photographed image storage unit 110. You may make it memorize.

  Next, in step S209, the microcomputer 113 displays the captured image on the information display unit 111. At this time, the recognition information of the face image may be displayed. Then, this process ends.

  As described above, the feature information ID that does not overlap is generated regardless of the imaging device, and the meta information is stored in the image, so that the problem of the image file management stored in the plurality of imaging devices can be solved.

  Next, a feature information ID generation method will be described with reference to the drawings.

  FIG. 4 is a diagram showing the feature information ID generated by the ID generation unit in FIG.

  The feature information ID 402 is an imaging device name defined by the model, a unique number (serial number) that can identify each imaging device when there are a plurality of imaging devices even in the same model, and within the imaging device Due to the non-overlapping number, it is the only one that does not depend on the imaging device.

  FIG. 4 is an example in which the feature information ID 402 is generated from the name A of the imaging device 401, the serial number 123, and a number managed independently in each imaging device.

  FIG. 4 is an example in which an ID is generated by connecting an imaging device name, a serial number, and a number that is not duplicated in the imaging device with “_”. However, the present invention is not limited to this example, and there is no “_”. Alternatively, a symbol such as “+” may be used instead of “_”.

  Further, the location to which the name and the serial number are assigned is not limited to the example of FIG. 4, and may be freely determined as long as it is a name that does not overlap regardless of the imaging device, such as (imaging device internal number) _123_A. . Further, the identifier added when generating the ID is not limited to the model name or serial number, and a symbol or the like representing feature information may be freely added as long as the name does not overlap.

  As described above, feature information IDs that do not overlap can be generated regardless of the imaging device.

  Here, information indicating the unique number of the imaging apparatus is given to the feature information ID. Further, the feature information ID is given information indicating an individual model name of the imaging apparatus.

  The feature information ID is shared by a plurality of imaging devices. Further, the feature information ID is associated with personal name information.

  For example, a technique described in Patent Document 2 is used as a technique for acquiring feature information for face recognition. The facial feature information is registered so that the user can easily determine the name and the face image used when registering as an alias so that the user can easily understand the information as shown in the information table 501 of FIG. Also good.

  Next, an example in which feature information is communicated between two imaging apparatuses will be described with reference to the drawings.

  FIG. 6 is a diagram illustrating an example in which feature information is registered in two imaging devices. Note that the external device to be a communication partner may not be an imaging device. As long as it has the characteristic information, it may be a personal computer or an external storage.

  The imaging device 601 whose name is A shown in (a) has a serial number of 123, and facial feature information (information table) 602 for four people registered in the imaging device 601 is registered. According to the example of FIG. 4, the feature information ID is “A_123_ (number in the imaging apparatus)”.

  The imaging device 603 whose name is B shown in (b) has a serial number of 456, and facial feature information (information table) 604 for five people registered in the imaging device B is registered. Similar to the imaging device 601, the feature information ID is “B — 456 — (imaging device internal number)”.

  Since the feature information IDs in the image capturing apparatus 601 and the image capturing apparatus 603 do not overlap, if you want to share and use the feature information registered in the two image capturing apparatuses, care about the overlap of the feature information IDs. It is sufficient to copy the feature information as it is.

  FIG. 7 is a diagram illustrating a state in which two imaging devices are connected by a cable and information is communicated by the communication unit in FIG.

  FIG. 7 shows an example in which two imaging devices 701 and 702 are connected by a cable 703. However, the present invention is not limited to this example. For example, communication may be performed by infrared communication or the like.

  FIG. 8 is a diagram illustrating an example of an information table of the imaging device 601 after the imaging device 601 in FIG. 6 receives the feature information from the imaging device 603.

  An information table 801 shows a feature information table after the imaging device 601 receives the feature information.

  The above example shows an example in which all the feature information is received from the connected imaging device. However, the present invention is not limited to this example, and the user selects and captures only the desired face feature information. Also good.

  As described above, the facial feature information obtained from the two imaging devices can be shared between the two imaging devices 601 and 603.

  FIG. 9 is a diagram illustrating an example in which a part of registered facial feature information is deleted.

  Usually, there is a limit to the memory capacity for storing facial feature information, and it is often better to be able to delete registered facial feature information.

  When deleting information that has already been stored and then registering other feature information, the feature information ID given to the meta information of the captured image is deleted in order to prevent inconsistency. The feature information ID corresponding to the feature information must be prevented from being used again.

  The information table 901 in FIG. 9 is an example in which the feature information corresponding to the feature information ID A_123_2 is deleted. However, after the information is deleted, the feature information ID remains and the deletion flag is turned on as a used mark. Yes.

  Further, there may be a case where it is desired to reuse the feature information deleted as described above. For example, when the feature information is deleted, an image to which the deleted feature information ID is added is displayed on the information display unit 111.

  In order to cope with such a case, when the user performs a deletion operation, if there is a sufficient memory capacity, only the deletion flag may be turned on and the feature information may be left. In this way, when the user wants to use again, the deletion flag can be turned off and reused.

  At this time, if the memory capacity is insufficient when new feature information is to be registered, the feature information whose deletion flag is on may be deleted, and the new feature information may be stored in an empty memory area.

  When the deletion flag is on and the feature information is also deleted, support information indicating that the feature information is input again under the same conditions as when registered previously may be displayed on the information display unit 111.

  As described above, a part of the feature information can be deleted without reusing the feature information ID.

  Next, an example of sharing facial feature information among a plurality of imaging devices will be described with reference to FIGS. 10 and 11.

  FIG. 10 is an explanatory diagram when merging facial feature information registered in three imaging devices.

  The imaging apparatus 1001 shown in FIG. Is 789.

  In the case where the image capturing apparatus 601 and the image capturing apparatus 1001 in FIG. 6 have acquired facial feature information from the image capturing apparatus 603 in which the facial feature information for five people is stored, the image capturing apparatus 601, the image capturing apparatus 603, and the image capturing apparatus. 1001 has the following information table. That is, it has information tables 1002, 1003, and 1004 of (b), (c), and (d), respectively.

  The imaging device 601 deletes the facial feature information copied from the imaging device 603 with the ID B_456_2, and the deletion flag of ID = B_456_2 is turned on as in the information table 1002.

  On the other hand, the imaging apparatus 1001 has feature information whose feature information ID is B_456_2.

  In such a case, the information table of the imaging device 601 when the imaging device 601 acquires data from the imaging device 1001 is like the information table 1101 of FIG.

  Regarding the information whose ID is B_456_2, the state is different between the imaging device 601 and the imaging device 1001, but in such a case, the user can select whether to copy or delete the feature information whose ID is B_456_2. Good.

  Since the feature information is personal information, the feature information acquired from another imaging device may not be transmitted to another imaging device.

  As described above, facial feature information registered in a plurality of imaging devices can be shared.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. Here, the basic configuration of the second exemplary embodiment of the present invention is the same as that of the first exemplary embodiment of the present invention, and thus the description is omitted. However, the method of sharing facial feature information is different.

  FIG. 12 is an explanatory diagram for transferring an image captured by the image capturing apparatus 1201 to the image capturing apparatus 1202.

  In the image photographed by the imaging device 1201 shown in (a), a person whose facial feature is registered as ID = A_123_2 in the information table 1203 of (b) is shown. On the other hand, ID = A_123_2 is not registered in the imaging device 1202.

  In the above case, when the image is transferred from the imaging device 1201 to the imaging device 1202 and the received image is displayed on the imaging device 1202, it is often preferable to display information corresponding to ID = A_123_2.

  In order to solve this problem, when the imaging apparatus 1201 transfers an image, the feature information corresponding to the feature information ID recognized in the transferred image is transmitted.

  In addition, the imaging device 1202 stores the feature information ID and the feature information received from the imaging device 1201 in addition to the information table 1204 in (c) of the imaging device 1202.

  At this time, the ID and the feature information added to the imaging device 1202 are retained even when the image received from the imaging device 1201 is deleted, so that the feature information can be reused.

  In the second embodiment described above, the feature information is stored together with the transferred image. However, if the ID corresponding to the subject imaged by the imaging device 1201 is registered in the imaging device 1202, it is not stored. May be.

  In other words, only when the feature information transferred together with the image is the feature information of a person who does not exist in the information table 1204 of (c) of the imaging device 1202, it may be additionally stored in the information table 1204. .

  In addition, since the facial feature information is a kind of personal information and it may not be desired to copy the feature information to another imaging device, the user can set the feature information so that it cannot be transferred to the other imaging device. If it is, it is even better.

  FIG. 13 is an explanatory diagram showing an example in which it is possible to set whether or not each feature information can be transferred.

  Like the information table 1301, a transfer prohibition setting flag can be set, and the user can freely set on or off.

  When an image is to be transferred from one imaging device to the other imaging device, the transfer prohibition flag is checked before the feature information corresponding to the person recognized in the image is transmitted, and the transfer prohibition flag is turned on. Do not transfer existing feature information.

  When the transfer prohibition flag is turned on and the feature information is not transferred, it is better to display on the information display unit 111 that the transfer has not been performed and notify the user. By doing so, it is possible to prevent unintended feature information from being transferred when an image is transferred.

  As described above, according to the second embodiment of the present invention, the feature information can be easily shared without the user being aware of the facial feature information stored in the feature information storage unit 109. It becomes possible.

(Other embodiments)
Each of the above-described embodiments can be applied to applications other than the imaging device. Any image processing apparatus having a function capable of extracting facial feature information may be a printer, a personal computer, or the like.

  When the feature information of the face is extracted and the feature information ID is generated, information indicating the unique number of the image processing apparatus that has executed these processes may be included in the feature information ID.

  Further, the above-described embodiment can be realized by software by a computer of a system or apparatus (or CPU, MPU, etc.).

  Therefore, the computer program itself supplied to the computer in order to implement the above-described embodiment by the computer also realizes the present invention. That is, the computer program itself for realizing the functions of the above-described embodiments is also one aspect of the present invention.

  The computer program for realizing the above-described embodiment may be in any form as long as it can be read by a computer. For example, it can be composed of object code, a program executed by an interpreter, script data supplied to the OS, but is not limited thereto.

  The computer program for realizing the above-described embodiment is supplied to the computer by a storage medium or wired / wireless communication. Examples of the storage medium for supplying the program include a magnetic storage medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a computer program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program file) that can be a computer program forming the present invention is stored in the server. The program file may be an executable format or a source code.

  Then, the program file is supplied by downloading to a client computer that has accessed the server. In this case, the program file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.

  That is, a server apparatus that provides a client computer with a program file for realizing the above-described embodiment is also one aspect of the present invention.

  In addition, a storage medium in which a computer program for realizing the above-described embodiment is encrypted and distributed is distributed, and key information for decrypting is supplied to a user who satisfies a predetermined condition, and the computer has the user. May be allowed to install. The key information can be supplied by being downloaded from a homepage via the Internet, for example.

  Further, the computer program for realizing the above-described embodiment may use an OS function already running on the computer.

  Further, a part of the computer program for realizing the above-described embodiment may be configured by firmware such as an expansion board attached to the computer, or may be executed by a CPU provided in the expansion board. Good.

It is a block block diagram of the imaging device which concerns on embodiment of this invention. 3 is a flowchart illustrating a procedure of imaging processing executed by the imaging apparatus of FIG. 1. FIG. 3 is an explanatory diagram in the case of storing a face-recognized image in the imaging apparatus of FIG. 1. It is a figure which shows the characteristic information ID produced | generated by the ID production | generation part in FIG. It is explanatory drawing of the registration item of the feature information in the imaging device of FIG. It is a figure which shows the example in which the feature information is registered into two imaging devices. It is a figure which shows the state which connects two imaging devices with a cable and performs communication of information by the communication part in FIG. 7 is a diagram illustrating an example of an information table of the imaging apparatus 601 after the imaging apparatus 601 in FIG. 6 receives feature information from the imaging apparatus 603. FIG. FIG. 3 is a diagram illustrating an example of deleting a part of registered facial feature information in the imaging apparatus of FIG. 1. It is explanatory drawing when merging the facial feature information registered in three imaging devices (1). It is explanatory drawing when merging the facial feature information registered in three imaging devices (2). FIG. 14 is an explanatory diagram when an image captured by the imaging device 1201 is transferred to the imaging device 1202. It is explanatory drawing which shows the example which enabled it to set whether it may transfer for every feature information. It is a figure which shows the malfunction of the conventional imaging device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Shooting lens 102 Image pick-up element 103 A / D converter 104 Memory 105 Face image recognition part 106 Feature extraction part 107 Feature comparison part 108 ID production | generation part 109 Feature information storage part 110 Captured image storage part 111 Information display part 112 Communication part 113 Micro Computer

Claims (8)

Recognizing the face image recognition means a face image from the image, a feature extraction means for extracting characteristic information of the recognized face image, the ID generating means for generating feature information ID corresponding to the previous SL feature information prior Symbol An image processing apparatus comprising: feature information, the feature information ID, and feature information storage means for storing information of an individual name corresponding to the feature information in association with each other ,
Determining whether or not the feature information extracted by the feature extraction unit from the face image in the acquired image and the feature information already stored in the feature information storage unit are feature information for the same face Means,
The feature information, feature information ID corresponding to the feature information, and personal name information corresponding to the feature information are acquired from an external device, and the acquired feature information, feature information ID, and personal name information are used as the feature information. An acquisition means for storing in the information storage means,
The ID generating means, an image processing apparatus and generates the feature information ID including information indicating a unique number of the image processing apparatus. If it is determined that the feature information for the same face by the determination means, the characteristic information ID corresponding to the feature information already stored in the characteristic information storage means as meta information, applied to the acquired image The image processing apparatus according to claim 1, further comprising storage means for storing the information.   The image processing apparatus according to claim 2, wherein the storage unit assigns and stores position information of a face image in the acquired image to the acquired image.   4. The information according to claim 1, wherein the feature information ID includes information indicating an individual model name of the image processing device in addition to a unique number of the image processing device. 5. Image processing device. Having storage means for storing images;
The acquisition means acquires the image from which the feature information is extracted together with the feature information, the feature information ID, and the personal name information from the external device,
The feature information storage unit stores the feature information acquired by the acquisition unit, the feature information ID, and the personal name information, and the storage unit stores the image acquired by the acquisition unit. The image processing apparatus according to claim 1, wherein the image processing apparatus is stored. Even if the storage unit deletes the image acquired by the acquisition unit, the feature information storage unit does not delete the feature information corresponding to the image, the feature information ID , and the personal name information. The image processing apparatus according to claim 5, wherein: The determination unit determines whether the feature information acquired by the acquisition unit and the feature information already stored in the feature information storage unit are feature information for the same face,
The feature information storage means, according to claim 1, characterized in that said only if it is not determined that the feature information for the same face by determining means, for storing the feature information acquired by the acquisition unit The image processing apparatus described. Recognizing the face image recognition process a face image from images, a feature extraction step of extracting characteristic information of the recognized face image, the ID generating step of generating characteristic information ID corresponding to the previous SL characteristic information, the A feature information storage step of storing feature information, the feature information ID, and personal name information corresponding to the feature information in association with each other ,
And feature information extracted in the feature Extraction process from the face image in the acquired image, and the feature information that has already been remembers in the feature information storage 憶step, whether or not the feature information for the same face A determination step for determining;
The feature information from an external device, wherein the information ID corresponding to the characteristic information, and acquires the information of the personal name corresponding to the characteristic information, the acquired characteristic information, characteristic information ID, and personal name information memorize a acquisition step of,
The ID as product engineering, the control method of an image processing apparatus and generates the feature information ID including information indicating a unique number of the image processing apparatus.

Images