JP2012128597A - Information processing apparatus, information processing method and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an apparatus which supports individually matched and well balanced makeup while reducing man-hour for correction.SOLUTION: A video image including a face of a person is captured, a plurality of partial regions are extracted from the captured video image, respective evaluation values of the plurality of extracted partial regions are calculated, and the plurality of evaluation values are compared with each other to calculate a balance correction value. On the basis of the balance correction value, balance correction information is produced which indicates a technique for correcting the balance in appearance of the person, and the balance correction information is outputted.

Description

  The present invention relates to an information processing apparatus for performing a well-balanced makeup suitable for a subject by the subject himself / herself.

  Traditionally, makeup has been learned by telling salespersons and makeup instructors, or searching magazines and the Internet. In recent years, as a method for assisting makeup, a method of assisting makeup by displaying an ideal makeup line as a sample on a face image and tracing it is considered.

JP 2010-73008 A JP 2001-216386 A

C. P. Pageorgiou, M .; Oren, and T.M. Poggio, A general framework for object detection, IEEE Computer Vision and Pattern Recognition, pp. 511-562, 1998. N. Dalal and B.M. Triggs, Histograms of Oriented Gradients for Human Detection, IEEE Computer Vision and Pattern Recognition, pp. 886-893, 2005. C. Harris and M.M. Stephens, A combined corner and edge detector, Proc. 4th Alvey Vision Conference, pp. 147-151, Manchester, U.S. K. Aug. 1988.

  In Patent Document 1, there is a means for capturing a sample image so that an ideal makeup line as a sample can be changed.

  In Patent Document 2, a sample image is superimposed on an input face image and combined, and advice on how to apply makeup is also displayed.

  As a method of correcting the makeup, a method of partially or completely removing the makeup, a method of hiding by overlaying a foundation or the like, and a method of continuing the makeup by changing the sample itself are conceivable. First of all, when removing all makeup, it is quite troublesome to remove all makeup once and apply again. Next, when the makeup is partially removed and when the foundation is overlaid and hidden, except for slight shifts in pale colors, uneven color is likely to occur, and it is difficult and difficult for an amateur to correct properly. Finally, when changing the sample itself, there must be a sample that can continue makeup.

  However, in the above method, the ideal line is basically fixed, and there is a problem that it takes time and effort to make a correction when it deviates from the sample during makeup.

  Moreover, when the ideal line exists, the subject that the difference of the ideal line by an individual difference was not considered occurred. Originally, the arrangement and shape of facial parts varies from person to person, and the face is not completely symmetrical. Thus, even if a sample is prepared for a face having a different arrangement and shape depending on each individual and applied to an individual's face, it is not necessarily a good makeup suitable for the individual.

  In order to solve the above-mentioned problems, the present invention aims to provide a device that supports a well-balanced makeup suitable for an individual by reducing the labor of correction by focusing on making a balanced makeup. To do.

The above object is achieved by the following apparatus.
Imaging means for capturing an image including a human face, extraction means for extracting a plurality of partial areas from the captured video, evaluation means for calculating an evaluation value for each of the extracted partial areas, and the plurality Calculating means for calculating a balance correction value by comparing the evaluation values of each other, and generating means for generating balance correction information indicating a technique for correcting the balance of the appearance of the person based on the balance correction value; An information processing apparatus comprising: output means for outputting the balance correction information.

  With the makeup balance support device according to the present invention, it is possible to reduce the trouble of correction and to make a makeup with a good balance suitable for an individual.

It is a figure which shows the structure of the information processing apparatus which concerns on 1st embodiment. It is a figure which shows the process of the information processing apparatus which concerns on 1st embodiment. It is a figure which shows the structure of the information processing apparatus which concerns on 2nd embodiment. It is a figure which shows the process of the information processing apparatus which concerns on 2nd embodiment. It is a figure which shows the calculation example 1 of an evaluation value in the evaluation part 103 in 1st embodiment. It is a figure which shows the calculation example 2 of an evaluation value in the evaluation part 103 in 1st embodiment. It is a figure which shows the example of calculation of an evaluation value in the evaluation part 103 in 3rd embodiment. It is a figure which shows the output example of the balance correction information at the time of performing balance correction including a hairstyle in the calculation part 104 demonstrated in 1st embodiment. It is a figure which shows the output example of the balance correction information at the time of performing balance correction including fashion in the calculation part 104 demonstrated in 1st embodiment. It is a figure which shows the example of an output of the image which superimposed the information which should be corrected as balance correction information in the output part 106 demonstrated in 1st embodiment. 6 is a diagram illustrating an example of a user interface to be displayed when a target person sets a favorite makeup using an operation unit 204. FIG. It is a figure which shows the structural example of a computer.

  Hereinafter, the present invention will be described in detail according to embodiments with reference to the accompanying drawings.

[First Embodiment]
(Overview)
When performing makeup, the information processing apparatus according to the present embodiment outputs balance correction information for improving the balance of appearance by makeup. The information processing apparatus according to the present embodiment first captures an image including a human face with a camera. Then, a partial area is extracted from the video imaged by the camera. Then, evaluation values of the plurality of partial areas are calculated based on the plurality of partial areas. Then, a balance correction value for improving the balance is calculated from the evaluation value, and the balance correction value is output to the output unit.

(Constitution)
FIG. 1 is a diagram showing an outline of an information processing apparatus according to this embodiment. The configuration of the information processing apparatus according to the present embodiment will be described with reference to FIG. The information processing apparatus includes an imaging unit 101, an extraction unit 102, an evaluation unit 103, a calculation unit 104, a generation unit 105, and an output unit 106.

  The imaging unit 101 captures an image including the target person's face. In FIG. 1, there is one imaging unit, but the number is not particularly limited to one, and a plurality of imaging units may be provided. By having a plurality, it is possible to obtain depth information and expand it in three dimensions.

  The extraction unit 102 extracts a partial region and its partial region information from the video sent from the imaging unit 101. In this embodiment, the partial areas are a face area and a parts area. Parts refer to parts of the face such as eyes, eyebrows, nose, mouth and cheeks. The partial area information is information on the position, shape, and color of the face area and part area.

  For extraction of the face area and the part area, for example, a method such as Haar-like feature quantity (see Non-Patent Document 1) or HOG feature quantity (see Non-Patent Document 2) is used. In addition, a simple technique such as skin color detection can be used to extract the face area. Further, a face area may be input to the subject. The extraction of the makeup area can be obtained by the difference from the state where makeup is not performed.

  When obtaining a difference from a state where makeup is not performed, it is necessary to associate an image without makeup with a current image. As a method of associating feature points, for example, feature points are extracted from two images to be associated by a feature point extraction filter such as a Harris operator (see Non-Patent Document 3). For example, two points having high correlation can be associated with each other based on a normalized correlation value between local regions centered on each point.

  The evaluation unit 103 calculates an evaluation value of the part area extracted by the extraction unit 102. The evaluation value is a value for evaluating each feature of each part area. For example, it may be a value indicating how bad the balance of appearance is. For example, the balance is expressed by rotation, size, shape, position, color distribution, etc. for each part or a combination thereof.

  As a method for obtaining the evaluation value, for example, a reference template serving as a reference for calculating the evaluation value as shown in FIG. 5 is prepared. Part feature points extracted by the extraction unit 102 are extracted. Then, the prepared reference template is associated with the feature point of the corresponding part. Furthermore, the degree of rotation θ, the magnification α, and the degree of deformation β with respect to the reference template are calculated, and the positions Px and Py are acquired and used as evaluation values. The degree of deformation β is, for example, an affine transformation parameter for viewing how to deform.

For example, consider an example in which the left eye evaluation value is calculated as shown in FIG. First, the head and corners of the eyes are extracted and connected with lines. The difference in inclination between this line and the similar line of the template can be calculated as the degree of rotation θ. Next, the width and height of the eye can be calculated by obtaining a perpendicular bisector of a line connecting the top of the eye and the corner of the eye, and obtaining a contact point with the eye contour. From these and the width and height of the eye of the template, magnifications α _w and α _{h of} the eye width and height can be calculated. Find the two points of contact between the eyes and the corners of the eyes, and the perpendicular bisector of the lines connecting them and the contours of the eyes. By using the positions of these points as values before affine transformation and the positions of similar points in the template as values after affine transformation, the affine transformation parameter β, which is the degree of deformation, can be calculated.

  As a method not using the reference template, for example, as shown in FIG. 6, among the parts extracted by the extraction unit 102, for the parts with corresponding parts on the left and right, the one part is reversed and compared with the opposite part. To do. For example, the degree of rotation θ, the magnification α, and the degree of deformation β for the opposite part are calculated by the same method as described above, and the positions Px and Py are obtained. For those that do not have corresponding parts on the left and right, the center line of the part is obtained, the degree of rotation θ relative to the center line of the current face, the magnification α relative to the opposite side of one side of the center line reversed horizontally, deformation The degree β is calculated, and the positions Px and Py are acquired and used as evaluation values.

  At this time, the center line of the face is obtained from the face area extracted by the extraction unit 102 and the position of the part. For example, the face center line may be calculated by calculating a straight line that divides the face area on the left and right sides of the center line into the same area as much as possible. You may calculate the straight line divided so that it may become the same as possible. Similarly, the part center line can be obtained by calculating a straight line that divides the part area so as to have the same area as possible on the left and right sides of the part center line.

  The calculation unit 104 obtains a balance correction value for correcting the appearance balance so that the calculated evaluation value is symmetrical, for example.

  The balance correction value is a value that indicates the amount of correction in the area where makeup should be added or updated in order to make the appearance balanced by applying makeup. There are values such as color. For each part, for example, those values are obtained so as to be balanced and symmetrical.

  If the positions of the left and right parts are asymmetric, they can be corrected by including them in correction values for size and deformation. For example, if the distance from the center line of the face to the position of the eyebrow is different on the left and right, the size to correct for the width of the eyebrow with the larger distance from the center line of the face to the position of the eyebrow, here the width of the eyebrow to be corrected Is set to a correction amount so as to be the same distance as the eyebrows on the opposite side from the face center line. Thereby, it can correct | amend. An example of correction by adding makeup is when a cheek is applied on one side.

  As the balance correction value, the position, size, shape, and color values of the part to be applied are calculated so as to be symmetrical on the opposite cheek, for example. An example of a case where the makeup is updated and corrected is a case where an existing one such as the eyebrows mentioned above is further overwritten from the current state. As the balance correction value, the eyebrow on the opposite side is updated to the position, size, shape, and color values so as to be symmetrical, for example. The balance correction is not limited to being symmetrical. Using the position calculated by the evaluation value and the difference between the positions, the weight for the evaluation value is changed by the distance from the center line of the face, and the weighted left-right symmetry can be calculated.

  In this way, it is possible to consider a balance that includes not only makeup but also hairstyles. For example, in the case of FIG. 8 where the original face contour is not symmetrical and the distortion is wide on one side like the face in FIG. , The overall balance will be better. In this case, the calculation unit 104 calculates a balance correction value from the evaluation value including hair. As for the hair correction value, values such as the size to be corrected and the degree of deformation may be obtained as in the case of the part.

  However, correction of hair is not as accurate as makeup, and even if it is calculated accurately, it is very difficult to achieve it exactly. May only display values roughly, such as what to do with the ratio. When calculating the balance correction value, the hairstyle is handled with a lower weight than the makeup weight, so that the hairstyle can be handled with a rough value compared to the makeup weight.

  Furthermore, you may consider the balance regarding fashion, clothes, and ornaments. As shown in FIG. 9A, when the makeup is balanced and the hairstyle is biased, the balance is improved if there is an accent on the opposite side. In this case, as advice for improving the balance, an instruction regarding fashion, for example, wearing clothes with accents such as a ribbon on the opposite shoulder, or adding lame to the opposite eye, is given.

  Like the hairstyle, the instruction about the fashion seems to be an instruction that is a guideline rather than a strict value, so it seems that a simple instruction such as what to add to what area is good. When calculating the balance correction value, it is possible to handle the fashion with a rough value compared to the makeup by handling the fashion with a lower weight as with the hairstyle with respect to the weight of the makeup.

  The generation unit 105 generates balance correction information for improving balance based on the balance correction value. When the balance correction information is an image, it is as follows. Based on the balance correction value calculated by the calculation unit 104, a balance correction image as shown in FIG. 10B is generated. The generated balance correction image and the video imaged by the imaging unit 101 are superimposed with the position of the face part aligned. The alignment is performed, for example, so that the eye position of the balance correction image matches the eye position of the captured image.

By such a method, the two images shown in FIGS. 10A and 10B are added to create an image superimposed as shown in FIG. When creating a balance-corrected superimposed image, it may be created by emphasizing a correction position and a correction amount that can achieve a balance. In addition, when there is a difference between the shades of the left and right colors, the shade of the current color may be expressed numerically and displayed on the image.
The output unit 106 outputs the generated correction information to, for example, a display unit.

  The generation unit 105 is not limited to generating an image. For example, the correction method may be generated as voice guidance information and output by the output unit 106. Further, the correction information may be transmitted to the robot, and the robot may perform correction according to the correction instruction.

  The subject confirms the correction information output by the output unit 106 and performs makeup. When the target person newly performs makeup, the above-described processing is repeated until the imaging unit 101 captures an image again and the output unit 106 outputs correction information. The correction information is updated at regular intervals.

(processing)
Next, processing performed by the information processing apparatus of this embodiment will be described using the flowchart shown in FIG.

  The process is started in step S100 in a state where the subject's face is in the range where the imaging unit 101 captures an image. In this description, the process start from a state where makeup is not applied will be described.

  In step S <b> 101, a face image is captured by the imaging unit 101. When the captured video is sent to the extraction unit 102, the process proceeds to step S102.

  In step S <b> 102, the extraction unit 102 extracts a partial region such as a face region or a part region from the video sent by the imaging unit 101. When the extracted partial region is sent to the evaluation unit 103, the process proceeds to step S110.

  In step S <b> 110, the balance correction amount setting process is started by sending a partial region such as a face region or a part region sent by the extraction unit 102 to the evaluation unit 103. When the balance correction amount setting process is started, the process proceeds to step S111.

  In step S111, the evaluation unit 103 calculates an evaluation value of the partial area from the partial area sent by the extraction unit 102. When the evaluation value is sent to calculation unit 104, the process proceeds to step S112.

  In step S <b> 112, the calculation unit 104 calculates a balance correction value for improving the balance from the evaluation value sent from the evaluation unit 103. When the calculated balance correction value is sent to the output unit 106, the process proceeds to step S113.

  In step S113, when the generation unit 105 generates a balance correction image from the balance correction value sent from the calculation unit 104, the process proceeds to step S114.

  In step S114, when the generation unit 105 superimposes the video sent by the imaging unit 101 and the balance correction image to generate a balance correction superimposed image, the process proceeds to step S115.

  In step S115, the output unit 106 outputs the balance correction superimposed image to the display unit. When the balance correction superimposed image is displayed on the display unit, the process proceeds to step S116.

  In step S116, the balance correction amount setting process step ends, and the process proceeds to step S103.

  In step S103, the balance correction superimposed image is displayed by the output unit 106 following the actual person's face orientation in real time, and the person who is the target starts makeup. Proceed to S104.

  Actually, at this point in time before makeup, if the subject does not have uneven balance such as distortion, information to be corrected is not displayed, and the output unit 106 displays an image of the subject's face. Is only displayed. The purpose of this apparatus is to present information to be corrected in order to lead to a better balance state after partial makeup has been applied.

  In step S104, the face image is picked up by the image pickup unit 101 again. When the captured video is sent to the extraction unit 102, the process proceeds to step S105.

  In step S <b> 105, the extraction unit 102 extracts a partial region such as a makeup region in addition to the face region and the part region from the video sent by the imaging unit 101. When the extracted partial region is sent to the evaluation unit 103, the process proceeds to step S110.

  The processing in step S110 is the same as that already described, except that the makeup region is added to the partial region. Due to the difference, the balance displayed in the processing step S116 at the end of step S110. The corrected superimposed image is different.

  When the balance correction superimposed image is updated, the update includes the following two contents. One is that the correction amount of the portion where the makeup was applied has disappeared because the subject has applied makeup according to the balance correction superimposed image displayed previously, and the makeup has not been applied yet. The correction amount is updated so that only the correction amount is displayed as a portion to be corrected.

  At this time, it is necessary to associate the balance-corrected superimposed image between the current time and the previous time. This may be performed by tracking feature points of the subject's face image, or by extracting feature points using a feature point extraction filter such as a Harris operator, and based on normalized correlation values between local regions centered on each point. You may match by making two highly correlated points correspond.

  The other is that the balance correction value changes due to the subject applying makeup on a part different from the balance correction superimposed image displayed previously, so that the part to be corrected is newly added or the correction content is changed. The balance correction superimposed image displayed this time is updated.

  The balance correction amount setting processing steps from step S110 to S116 are completed, and the process proceeds to step S106.

  In step S <b> 106, the balance correction superimposed image is updated and displayed from the output unit 106, and it is determined whether or not makeup is completed. The determination at this time may be left to the subject, or a process may be performed to end the correction when the balance correction value falls within a certain range. In this determination, when the makeup is completed, the process proceeds to step S107, and the process ends. In this determination, if makeup is not completed, the process proceeds to step S104, makeup is continued, and steps S104 to S106 are repeated.

[Second Embodiment]
The information processing apparatus according to the present embodiment includes a personal information storage unit and / or a suitable balance information storage unit in the information processing apparatus shown as Example 1. This makes it possible to perform correction using previously used information. Furthermore, a variety of functions can be added by adding a communication unit and an operation unit.
Hereinafter, the configuration and processing of the information processing apparatus according to the present embodiment will be described with reference to the drawings.

(Constitution)
FIG. 3 is a diagram illustrating an outline of the information processing apparatus according to the present embodiment. As illustrated in FIG. 3, the information processing apparatus includes a personal information storage unit 201, a suitable balance information storage unit 202, a communication unit 203, and an operation unit 204 in addition to the information processing device according to the first embodiment illustrated in FIG. 1. It has the added structure. The components common to the information processing apparatus in FIG. 1 are substantially the same as those in the first embodiment, and thus detailed description thereof is omitted.

The imaging unit 101 captures an image including the target person's face.
The extraction unit 102 extracts a face area and a part area from the video sent from the imaging unit 101.

  The personal information storage unit 201 stores information related to individual unique features. For example, it is a unique evaluation value calculated by the evaluation unit 103 of the first embodiment. This unique evaluation value is unique to the individual and includes, for example, information such as part asymmetry and facial distortion.

  When storing in advance, the following procedure is used. A partial area such as a face area or a parts area is extracted from an image including the face of the subject. An evaluation value specific to the subject is calculated from the partial area. This value is stored as personal balance information in the personal information storage unit 201. When using the stored unique balance information, it is necessary to specify which person is stored in the personal information storage unit 201 as the target person.

  In order to realize this, for example, there are the following methods. For example, in the personal information storage unit 201, in addition to the evaluation value of the target person, a face image and a feature amount are also stored together and specified by using a face authentication technique. For example, the personal information storage unit 201 stores the name and ID of the target person in addition to the evaluation value of the target person, and the operation unit 204 allows the target person to input his name and ID. There is also a way to specify which of the stored information to use.

  The suitable balance information storage unit 202 stores, for example, a history of evaluation values at the time of completion of makeup of the subject. Note that what is stored in the preferred balance information storage unit 202 is not limited to the history, but, for example, suitable balance information can be acquired from the outside via the communication unit 203. The communication unit may receive information over a network, or read information from a medium such as a CD-R or USB memory by a reading device.

  The suitable balance information is, for example, an evaluation value calculated from makeup that has been given by a makeup instructor in the past, makeup that has been placed in a magazine, etc., or makeup that the subject likes. In using the preferred balance information stored in the preferred balance information storage unit 202, it is necessary to specify which person the subject is stored in the preferred balance information storage unit 202. About this method, the same method as the method of specifying a person in the personal information storage unit 201 can be used.

  Further, when a plurality of suitable balance information exists as candidates for one individual, for example, a method in which the target person selects desired suitable balance information via the operation unit 204 can be used. For example, in order to make it easy for the target person to understand, it is preferable to display a face image of a model with makeup corresponding to the stored preferable balance information.

  There is a method for receiving an input from a target person using a user interface for selecting a desired makeup state from the displayed face images. As another method, there is a method of setting suitable balance information having the largest number of selections as a default value by storing the number of selections of the suitable balance information together with the suitable balance information.

  In order to have the target person input the desired balance state, calculate a suitable evaluation value, store it in the suitable balance information storage unit 202 as suitable balance information, and calculate the current evaluation value May be used. As a method of having the target person input the desired balance state, for example, there is a method of having the target person adjust the indicator using a user interface as shown in FIG. The desired balance state is not limited to the items displayed in FIG.

  Further, by storing information on the hairstyle and fashion trends as what is stored in the preferred balance information storage unit 202, instructions regarding the hairstyle and fashion balance can be diversified. For example, when the output unit 106 issues an instruction as shown in FIG. 8 or 9, it can be updated to an instruction that matches the fashion.

  The evaluation unit 103 calculates an evaluation value of the part area extracted by the extraction unit 102. The evaluation value calculation method will be described with reference to FIG. 5, for example. Based on suitable balance information, each part of the reference template is updated to a template that is equivalent to the previous one, and each evaluation value is obtained by the procedure described above.

The calculation unit 104
A balance correction value is calculated from the balance information unique to the subject stored in the personal information storage unit 201 and the suitable balance information stored in the suitable balance information storage unit 202. Having the personal information storage unit 201 eliminates the need for the evaluation unit 103 to calculate an evaluation value for a portion that has not been makeupd.

  In addition, when applying makeup similar to the previous makeup, suitable balance information, which is the last final evaluation value stored in the suitable balance information storage unit 202, is calculated in the evaluation unit 103 when calculating the evaluation value. The balance correction value is calculated in consideration.

The generation unit 105 generates a balance correction image based on the calculated balance correction value, and superimposes the generated balance correction image on the video imaged by the imaging unit 101.
The output unit 106 outputs the superimposed image to the display unit.

  Note that the personal information storage unit 201 and the preferred balance information storage unit 202 may have either one or both of the configurations. In addition, the communication unit 203 and the operation unit 204 may have either a configuration having only one, a configuration having both, or a configuration not having both.

  Further, the preferred balance information storage unit 202 may be included in the calculation unit 104 instead of the configuration shown in FIG. An example in that case will be described.

  What is stored in the preferred balance information storage unit 202 is the same as described above. Here, for the purpose of explanation, the history of the evaluation value at the time of completion of the previous makeup of the subject is assumed.

  When the evaluation unit 103 calculates the evaluation value, the evaluation value is calculated from the partial area extracted by the extraction unit 102. Alternatively, the evaluation value is calculated from the partial area extracted by the extraction unit 102 and the unique balance information acquired from the personal information storage unit 201.

  The calculation unit 104 compares this evaluation value with suitable balance information that is a history of stored evaluation values, and calculates a balance correction value. The calculation method of the balance correction value here is, for example, calculating a difference between the evaluation value calculated by the evaluation unit 103 and the stored preferable evaluation value, and calculating a correction value that becomes a preferable evaluation value. To do.

  However, if the area where makeup is currently applied is larger than the area where makeup was applied in the previous history, it is not possible to calculate with the difference alone. Instead, the correction value may be calculated so that the location is, for example, symmetrical. Or, even for places that cannot be realized, the evaluation value in the history, that is, the symmetry, the position relative to other parts, etc. The correction value closest to the history may be calculated.

  Even when a suitable balance information storage unit is included in the calculation unit 104, it does not matter whether the personal information storage unit 201, the communication unit 203, and the operation unit 204 are present. When the communication unit 203 and the operation unit 204 are provided, data transmission / reception between them is performed with the preferred balance information storage unit in the calculation unit 104 instead of the preferred balance information storage unit 202 of FIG. 3 described above.

(processing)
Next, processing performed by the information processing apparatus of this embodiment will be described using the flowchart shown in FIG. Compared with the flow described in the first embodiment, only some steps are different and the others are the same. Therefore, detailed description of common processing is omitted.

  The process is started in step S200 in a state where the subject's face is in a range where the imaging unit 101 captures an image.

  In step S201, information input by the subject is performed. The information input is, for example, selection of suitable balance information stored in the suitable balance information storage unit 202 by the operation unit 204. In addition, an ID or a name for specifying which of the unique balance information and the preferred balance information stored in the personal information storage unit 201 and the preferred balance information storage unit 202 is to be used. When the input information is sent to either or both of the personal information storage unit 201 and the preferred balance information storage unit 202, the process proceeds to step S101. From step S101 to S102, the same processing as in the first embodiment is performed. Subsequently, the process proceeds to step S210.

  In step S <b> 210, the balance correction amount setting process is started by sending a partial area such as a face area or a part area sent by the extraction unit 102 to the evaluation unit 103. When the balance correction amount setting process is started, the process proceeds to step S211, and the evaluation unit 103 stores the unique balance information stored in the personal information storage unit 201 and the suitable balance information storage unit 202. Identify suitable balance information. When these are specified, the process proceeds to step S111.

  In step S111, the evaluation value of the partial area is calculated. At this time, in the evaluation unit 103, the partial balance sent by the extraction unit 102, the unique balance information stored in the personal information storage unit 201, and the suitable balance stored in the suitable balance information storage unit 202 Use information. When the evaluation value is sent to calculation unit 104, the process proceeds to step S112.

  From step S112 to S212, the same processing as in the first embodiment is performed. Subsequently, the process proceeds to step S103. Further, processing similar to that in the first embodiment is performed from step S103 to S105. Subsequently, the process proceeds to step S210. The processing in S210 is the same as that already described. Further, the process proceeds to step S106, and the same process as in the first embodiment is performed. However, unlike Example 1, when makeup is completed in Step 106, the process proceeds to Step S202.

  In step S202, the current evaluation value is stored in the preferred balance information storage unit 202, the process proceeds to step S203, and the process ends.

[Third Embodiment]
The information processing apparatus according to the present embodiment uses the same configuration as that described in the first and second embodiments. In the present embodiment, only the color information and its position in a partial area are used as the partial area of the face.

(Constitution)
The configuration of the present embodiment will be described using the first embodiment described above. The differences from the first embodiment are an extraction unit 102, an evaluation unit 103, and a calculation unit 104. These parts are described in detail below.
The imaging unit 101 captures an image including the target person's face.

  The extraction unit 102 extracts the face area and its color information. The method for extracting the face area has already been described. The face area is finely divided, for example, as shown in FIG. It does not use the concept of parts, but also includes information such as spots and wrinkles other than shapes.

  The evaluation unit 103 acquires color information, for example, a color distribution and a position of the partial region, for each partial region in the face region extracted by the extraction unit 102, and uses it as an evaluation value.

  The calculation unit 104 calculates a balance correction value so that the evaluation value calculated by the evaluation unit 103 as a whole has a good balance. For example, when the color distribution C1 of the partial region shown in FIG. 7 is obtained, the same value as C1 is calculated as a correction value at a position that is symmetrical with good balance. In addition, in order to improve the balance, not only the left-right symmetry but also the weights may be changed according to the colors so that the weights are left-right symmetrical. For example, the weight of the conspicuous color is increased, whereby the correction of the conspicuous color is tightened, and the correction of the inconspicuous color is loosened.

  The generation unit 105 generates a balance correction image based on the calculated balance correction value, and superimposes it on the video imaged by the imaging unit 101 as in the first embodiment.

  The output unit 106 outputs the superimposed video to the display unit.

(processing)
The processing performed by the information processing apparatus of this embodiment will be described with reference to the flowchart shown in FIG. 2 which is the same as that used in the first embodiment. Compared with the flow described in the first embodiment, only some steps are different and the others are the same. Therefore, detailed description of common processing is omitted.

  The process is started in step S100 in a state where the subject's face is in the range where the imaging unit 101 captures an image. When the process is started, first, the process proceeds to step S <b> 101, and a face image is captured by the imaging unit 101. When the captured video is sent to the extraction unit 102, the process proceeds to step S102.

  In step S102, the extraction unit 102 extracts each partial region in the face region as shown in FIG. 7 from the video sent by the imaging unit 101. When the extracted partial region is sent to the evaluation unit 103, the process proceeds to step S110.

  In step S <b> 110, the balance correction amount setting process is started by sending each partial area in the face area sent by the extraction unit 102 to the evaluation unit 103. When the balance correction amount setting process is started, the process proceeds to step S111, and the evaluation unit 103 calculates an evaluation value of the partial region from each partial region sent by the extraction unit 102. When the evaluation value is sent to calculation unit 104, the process proceeds to step S112. In steps S112 to S116 and steps S103 to S104, processing similar to that in the first embodiment is performed. Subsequently, the process proceeds to step S105.

  In step S <b> 105, the extraction unit 102 extracts each partial area in the face area including the makeup area from the video sent by the imaging unit 101. When the extracted partial region is sent to the evaluation unit 103, the process proceeds to step S110. Steps S110 to S106 perform the same processing as in the first embodiment. If it is determined in S106 that makeup is complete, the process proceeds to step S107 and is completed.

[Other Embodiments]
FIG. 12 is a diagram illustrating a configuration example of a computer.
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to the system or apparatus via the networks 1204 and 1207 or the various storage media 1202 and 1203. Then, the computer 1201 (CPU, MPU, etc.) of the system or apparatus reads and executes the program. Further, the program may be provided by being stored in a computer-readable storage medium. Note that an instruction to execute a process may be input from the input unit 1205 to the computer of the apparatus, and the result of the process instructed by the display unit 1206 may be displayed.

Claims (12)

Imaging means for imaging video including a person's face;
Extracting means for extracting a plurality of partial regions from the captured video;
Evaluation means for calculating an evaluation value of each of the extracted partial regions;
Calculating means for calculating a balance correction value by comparing each of the calculated evaluation values with each other;
Generating means for generating balance correction information indicating a method for correcting the balance of the appearance of the person based on the balance correction value;
An information processing apparatus comprising: output means for outputting the balance correction information.   The information processing method according to claim 1, wherein the balance correction information includes a makeup technique.   The information processing method according to claim 1, wherein the calculation unit calculates a balance correction value for correcting a left / right balance of the face of the person who has taken the image.   The information processing apparatus according to claim 1, wherein the balance correction information includes an image.   The generation unit generates a balance correction image indicating a method of correcting the balance from the balance correction value, and generates a balance correction superimposed image in which the video and the balance correction image are superimposed. Item 5. The information processing apparatus according to Item 4. The information processing apparatus according to claim 1, wherein the balance correction information includes voice guidance. Personal information storage means for storing a unique evaluation value of the partial area for each person;
The information processing apparatus according to claim 1, wherein the evaluation unit adds or updates the evaluated evaluation value as the unique evaluation value. A suitable balance information storage means for storing a history of evaluation values for each person as suitable balance information;
The information processing apparatus according to claim 7, wherein the calculation unit calculates the balance correction value based on the preferred balance information. The information processing apparatus according to claim 8, further comprising a communication unit that takes in suitable balance information from the outside into the suitable balance information storage unit.   The information processing apparatus according to claim 1, wherein the calculation unit calculates a balance correction value for correcting a balance of color distribution of the face of the person. An information processing method performed by an information processing apparatus,
An imaging step in which the imaging means of the information processing apparatus captures an image including the face of the person;
An extraction step in which the extraction means of the information processing apparatus extracts a plurality of partial areas from the captured video;
An evaluation step in which the evaluation unit of the information processing apparatus calculates an evaluation value of each of the extracted partial regions;
A calculation step of calculating a balance correction value by the calculation means of the information processing apparatus comparing each of the calculated evaluation values with each other;
A generation step of generating balance correction information indicating a method of correcting the balance of the appearance of the person based on the balance correction value;
An information processing method comprising: an output unit that outputs the balance correction information.   The program for making a computer perform each process of the information processing method of Claim 11.

Images