JP6969622B2 - Shooting game equipment and programs - Google Patents

Description

The present invention relates to a technique for processing a game image.

The game image shooting device called Purikura (registered trademark) is very popular among high school girls. The user photographs himself and makes various decorations on the captured image on the display for editing. The decorated image is printed on the sticker mount (see Patent Documents 1 and 2).

Japanese Unexamined Patent Publication No. 2004-322213 Japanese Unexamined Patent Publication No. 2016-184287 Japanese Unexamined Patent Publication No. 2019-013013

Patent Document 3 discloses a technique for changing the color of the lips of a photographed image to be edited to the color of the lips selected by the user. With this technology, the color of the lips can be changed for each user to create individuality. In Patent Document 3, although it is suggested that the face is highlighted in the stamp image, the highlight of the lips is not mentioned.

Just as gloss is emphasized in actual cosmetics such as lipsticks and lip glosses, the shine of the lips has a great effect on the impression of the face. It is difficult to highlight the lips well with a stamp image. Also, in the stamp image, everyone has the same way of shining, and it is not possible to demonstrate individuality.

In addition, it is conceivable to change the color of the cheeks and hair in the same way.

The present invention is an invention completed based on the above-mentioned problem recognition, and a main object thereof is to make use of the texture of a user's part to add color to a photographed image.

In the photographing game device according to an aspect of the present invention, a photographing unit for photographing a user, an extraction unit for extracting a highlight area in a predetermined part of the user appearing in the photographed image, and a makeup for a predetermined part of the photographed image are used. It has a part processing unit that performs processing to superimpose a highlight image on the highlight area after adding color and / or smoothing processing, and an output unit that outputs the processed image. ..

According to the present invention, it is possible to add color to a captured image by taking advantage of the texture of the user's portion.

It is an overall block diagram of a game image shooting system. It is external perspective view of the game image taking apparatus. It is an external perspective view of a shooting booth. It is a flowchart which shows the flow of play in a game image shooting apparatus. It is a top view of the game image shooting apparatus. It is a screen view of an edit screen. It is a screen view of a lip palette. It is a figure which shows the layer structure. It is a figure which shows the outline of processing of a lip image. It is a functional block diagram of a game image taking apparatus. It is a flowchart which shows the process of a pre-processing. It is a flowchart which shows the process of an editing process. It is a flowchart which shows the process of an image update process. It is a figure which shows the layer composition in the layer composition of the 1st stage in the modification 1. FIG. It is a figure which shows the layer composition in the layer composition of the 2nd stage in the modification 1. FIG. It is a figure which shows the layer composition in the layer composition of the 3rd stage in the modification 1. FIG. It is a flowchart which shows the process of the pre-processing in the modification 1. FIG. It is a flowchart which shows the process of the image update process in the modification 1. It is a figure which shows the layer structure in the modification 2. It is a figure which shows the outline of the processing of the lip image in the modification 2. It is a flowchart which shows the process of the pre-processing in the modification 2. It is a flowchart which shows the process of the image update process in the modification 2. It is a figure which shows the layer composition in the modification 3. It is a flowchart which shows the process of the pre-processing in the modification 3.

The game image shooting device in the present embodiment includes a shooting booth and an editing booth. The user shoots himself at the shooting booth. Hereinafter, the image of the subject (user) taken at the shooting booth is referred to as a "shooting image". The game image shooting device applies image corrections such as skin whitening and hair gloss to the shot images at the shooting booth (hereinafter, such image corrections are referred to as "preliminary corrections"). The game image capturing device in the present embodiment captures not only still images but also moving images at the photographing booth. Of the captured images, those acquired as moving images are called "photographed moving images", and those acquired as still images are called "photographed still images".

The pre-corrected captured image will be sent to the editing booth. At the editing booth, the user can make arbitrary decorations such as pasting stamp images and scribbling with a stylus pen, and image corrections such as eye size and whiteness of the skin (hereinafter, such image decorations). Is called "decorative processing" or "editing"). The captured image that is the subject of editing at the editing booth is called the "target image". Further, among the target images, the one in which the shooting moving image is the editing target is called the "target moving image", and the one in which the shooting still image is the editing target is called the "target still image".

The game image capturing device prints the target still image on a sticker mount (external medium). Further, the game image shooting device uploads the target moving image and the target still image to the server (external medium). The user can also download the target image (target moving image and target still image) after the decoration processing by accessing the server from a user terminal such as a smartphone. As described above, the game image photographing apparatus in the present embodiment has a photographing function, an editing function (decorative processing function), a printing function, and a communication function.
In the following, the setting changes at the time of shooting / editing based on various information of the subject will be mainly described.

FIG. 1 is an overall configuration diagram of the game imaging system 200.
In the game image shooting system 200, the game image shooting device 100 is connected to the server 202 and the user terminal 206 via the Internet 204. In the game image capturing device 100, the user photographs himself / herself. The game image capturing device 100 acquires a captured image including a subject (user), applies pre-correction to the captured image as necessary, and accepts decoration processing by the user. The user is identified by a user ID given in advance. The user makes a play reservation for the game image capturing device 100 by using the user card in which the user ID is registered.

The target image after the decoration processing is uploaded from the game image capturing device 100 to the server 202 together with the user ID. The server 202 stores the user ID, the shooting date and time, and the target image (target moving image and target still image) in association with each other. The user downloads the target image from the server 202 by accessing the server 202 via the authentication information such as the user ID and the password.

The server 202 manages various information related to the user (hereinafter referred to as "user information"). The game image shooting device 100 in the present embodiment automatically gives instructions to the subject and changes the setting of shooting conditions based on the live view image.

FIG. 2 is an external perspective view of the game image capturing device 100.
The game image shooting device 100 includes a reception device 102, a shooting booth 104, an editing booth 106, and a printing device 110. The user reserves the use of the game image capturing device 100 at the reception device 102. The user inputs the user ID in the reception device 102 and pays the play fee in cash or electronic money. The reception device 102 issues a reservation number to the user. The reception device 102 displays the number of people on standby and the reservation number to be called on the reception monitor 108. When the user's reservation number is displayed on the reception monitor 108, the user enters the shooting booth 104. Alternatively, when the shooting booth 104 is in use, the next user may be kept on standby, and the next user may be accepted after the shooting booth 104 is vacant.

The user takes a picture at the shooting booth 104. After the shooting is completed, the user moves to the editing booth 106. An editing device 126 is installed in the editing booth 106. The editing device 126 includes an editing monitor 120. The target image (photographed image to be decorated) is displayed on the editing monitor 120. The editing monitor 120 in this embodiment is formed by a capacitive touch panel. Two stylus pens 122a and 122b are prepared for one editing monitor 120. The user uses the stylus pen 122 and a finger to decorate the target image. After the editing is completed, the printing device 110 prints the target still image on the sticker mount. Further, the editing device 126 uploads the target image to the server 202.

FIG. 3 is an external perspective view of the shooting booth 104.
The shooting booth 104 is shielded from the outside by the curtain 124. A shooting device 128 is installed in the shooting booth 104. The photographing device 128 includes three lights 116 that illuminate the user as a subject from three directions, a camera 118 for acquiring a photographed still image and a photographed moving image, a first photographing monitor 112, a second photographing monitor 114, a left strobe 119a, and the left side strobe 119a. It is equipped with a right strobe 119b.

The first shooting monitor 112 displays a live view image and a shot image (shooting moving image and shooting still image) of the user. The second shooting monitor 114 displays a shot image (hereinafter, referred to as “sample image”) of a person other than the user who is the subject, for example, a model. The user takes an arbitrary shooting pose in front of the camera 118 while referring to the sample image displayed on the second shooting monitor 114. Various explanations regarding the shooting method are also displayed on the second shooting monitor 114. The left strobe 119a and the right strobe 119b emit light at the same time as the captured still image is captured. As described above, in the shooting booth 104, a shooting moving image and a shooting still image can be acquired.
In the present embodiment, the second shooting monitor 114 also indicates a shooting pose by using a sample image (details are omitted).

FIG. 4 is a flowchart showing the flow of play in the game image capturing device 100. FIG. 5 is a plan view of the game image capturing device 100 as viewed from above.
When the user makes a reservation for use in the reception device 102, the reception device 102 executes the reception process (S10). After acceptance, the user enters the shooting booth 104. The photographing device 128 photographs the user and acquires the photographed image of the user (S12). The photographing device 128 captures a still image a predetermined number of times. In this example, a moving image is taken as well as a still image is taken. The shot video and the shot still image are treated as a set. This set is called a captured image set. In this example, up to 6 captured image sets are generated. At the time of shooting, the user can select a pre-correction method from a plurality of pre-correction menus provided by the photographing device 128. The user can also select the background color of the captured image on the photographing device 128. The photographing device 128 executes pre-correction every time a photographed image is acquired. The pre-correction menu may be selected on the reception device 102 before play. Further, the editing booth 106 may accept selection of a background color or the like of a photographed image.

After shooting, the first shooting monitor 112 displays instruction information indicating one of the two editing booths 106. The user moves to the editing booth 106 designated by the first shooting monitor 112 among the two editing booths 106a and 106b. The captured image is transferred from the photographing device 128 to the editing device 126. The user performs decorative processing on the captured image (target image) transferred by the editing device 126 (S14).

After editing, the editing device 126 sends the target image to the printing device 110, and the printing device 110 prints the target image on the sticker mount (S16). Further, the editing device 126 uploads the target image to the server 202 via the Internet 204 (S18). The user can download the target image from the server 202 to the user terminal 206 by accessing the server 202 using the user ID as an access key.

Generally, it takes more time to edit than to shoot. In the game image photographing apparatus 100 of the present embodiment, the customer turnover rate is increased by providing two editing booths 106a and 106b (editing devices 126a and 126b) for one photographing booth 104.

FIG. 6 is a screen view of the edit screen.
This figure shows an edit screen for the left user to operate. On the right side, an edit screen for the right user to operate is displayed. The edit screen for the user to operate on the right side is omitted, but the layout is symmetrical to this figure, and the user operation is received in the same manner, and the captured still image is displayed. Hereinafter, the user A is shown on the left side of the captured still image, the user B is shown on the right side, the user A is directed to the left editing screen shown in FIG. 6, and the user B is facing the right editing screen (not shown). It shall be.

In this example, it is assumed that five captured still images are acquired. The image tabs 300a to 300e (hereinafter, collectively referred to as "image tabs 300" when collectively referred to or not particularly distinguished) correspond to pre-corrected captured still images. When the user A touches the image tab 300, the captured still image corresponding to the image tab 300 is displayed as the target still image in the image display area 302.

The background of the captured still image has been replaced by pre-correction. In the figure, the replaced background is shown in a grid pattern, but it is actually painted in a predetermined color. In the pre-correction, both user A and user B are subjected to skin whitening, hair polishing, lip processing, and face line correction.

The lip palette 304 is used to indicate the degree of processing of the user's lips included in the target still image. When the user operates the lip palette 304, the updated target still image is displayed in the image display area 302.

FIG. 7 is a screen view of the lip palette.
The large tabs 306a to 306d (hereinafter collectively referred to as "large tabs 306" when collectively referred to or not particularly distinguished) are used for selecting a palette. In this example, it is assumed that the large tab 306d of "Makeup" is selected. The palettes of "stamp", "pen" and "moji" other than "makeup" are omitted.

Small tabs 308a, 308b are used to select a "lip" or "cheek" palette. In this example, it is assumed that the small tab 308a of "lip" is selected. The "cheek" palette will be omitted.

The face selection window 310 is used to select a user who is the target of lip processing. In the face selection window 310, a reduced image of the captured still image is displayed. When the user A slides the selection frame 312 to fit the user A's own face, the lip processing on the user A's face is adjusted.

The texture buttons 314a to 314c (hereinafter, collectively referred to as "texture buttons 314" when collectively referred to or not particularly distinguished) are used to specify the texture of the lips, that is, the degree of shine of the lipstick. When the "glossy" texture button 314a is selected, the finish emphasizes the luster of the lipstick. Therefore, it gives a gorgeous impression as if you applied lip gloss. Specifically, a large standard highlight image prepared in advance is placed on the lips. This highlight image has a shape that is manifested by strong reflections on the standard lips.

On the other hand, when the "matte" texture button 314b is selected, the finish is such that the gloss of the lipstick is suppressed. Therefore, it gives a calm impression as if you applied matte lipstick. Specifically, a small standard highlight image prepared in advance is placed on the lips. This highlight image has a shape that is manifested by weak reflections on the standard lip.

On the other hand, when the "original texture" texture button 314c is selected, the user's own original gloss is reproduced. Therefore, the individuality of the user is utilized to give a straightforward impression. The highlight image is extracted from the captured still image. The highlight image extracted from the captured still image has a shape that appears due to the reflection on the user's lips.

In this way, the shape of the highlight image is determined by the selection of the texture button 314, and the glossiness of the lipstick can be adjusted. In the preliminary correction, the "original texture" is used for processing. Therefore, the original luster of the user A is reproduced on the lips of the user A, and the original luster of the user B is reproduced on the lips of the user B.

The lipstick color buttons 316a to 316d (hereinafter, collectively referred to as "lipstick color button 316" when collectively referred to or not particularly distinguished) are used to specify the lipstick color. The lipstick color buttons 316a to 316d correspond to different lipstick colors. There are various lipstick colors, but for example, standard colors belonging to pink, orange, beige, red, etc. are used. In this example, four lipstick-colored buttons 316 are shown, but the number of lipstick-colored buttons 316 may be four or more.

The original color button 318 is assigned a natural lipstick color with less personality. This lipstick color is also used in pretreatment. I don't emphasize the lipstick in particular, but I use this lipstick color to make it look more beautiful than the base material. For example, for young users, emphasizing lipstick can be unnatural, so using the original color will give a better impression. Even when the original color button 318 is selected, the above-mentioned texture can be set.

The lipstick eraser button 320 is used to modify the lip area to which the lipstick is applied. The lip area is automatically extracted, but if the extracted lip area is too large, the user corrects it with a lipstick eraser. Specifically, by sliding the lipstick eraser cursor displayed by selecting the lipstick eraser button 320, the portion where the lipstick eraser cursor has passed is excluded from the lip region. This makes it possible to remove the lipstick in the protruding lip area.

The mechanism of lip processing is the same for both pre-correction and adjustment using the lip palette. In either case, layer composition is performed.

FIG. 8 is a diagram showing a layer configuration. Three layers are set in layer composition. A background layer 500 is provided at the lower level, a person layer 502 is provided at the middle, and a lip layer 503 is provided at the upper level.

A background image 404 is drawn on the background layer 500. The background image 404 is an image having a predetermined background color on one side. A person image 402 is drawn on the person layer 502. The person portion of the person image 402 remains a captured still image. The background portion of the portrait image 402 is transparent. A processed lip image 424 is drawn on the lip layer 503. The processed lip image 424 is transparent except for the lip region, and the lip image is subjected to the processing described later. The generation of the processed lip image 424 will be described in detail in connection with FIG.

Combining each layer gives a composite image. In the background portion of the composite image, the background image 404 of the background layer 500 appears through the lip layer 503 and the person layer 502. In the person portion (excluding the lip portion) of the composite image, the person image 402 of the person layer 502 appears through the lip layer 503. The processed lip image 424 appears in the lip portion of the composite image. In this way, the person is overlaid on the background, and the processed lips are overlaid on it. Hereinafter, the composite image generated in this way is referred to as a total composite image 406.

FIG. 9 is a diagram showing an outline of processing of a lip image.
An image of a face region, that is, a face image 410 is extracted from the person image 402 by face recognition. Further, an image of the lip region of the face image 410, that is, the original lip image 412 is extracted. For example, edge treatment identifies the lip region. The area other than the lip area of the original lip image 412 is transparent.

The highlight image 414 is extracted from the original lip image 412. A region having a brightness higher than a predetermined highlight reference is a highlight region, and an image in the highlight region is a highlight image 414. Areas other than the highlight area of the highlight image 414 are transparent.

The shadow image 416 is extracted from the original lip image 412. A region having a brightness lower than a predetermined shadow reference is a shadow region, and an image of the shadow region is a shadow image 416. The area other than the shadow area of the shadow image 416 is transparent.

The base color is combined with the original lip image 412 to obtain the base lip image 418. The composition of the base color imitates a cosmetic lip concealer. Specifically, the background colors are superimposed with a predetermined background transparency (for example, 50 to 90%). The background color is a standard lip color. For example, even if the original color of the lips is dark, it approaches the standard color of the lips, making the lipstick color more brilliant. In synthesizing the background color, the image may be smoothed so as to hide the unevenness of the lip surface. The background color is also superimposed on the highlight and shadow areas, resulting in a uniform image as a whole. The underlying lip image 418 shown in FIG. 9 does not show highlights and shadows, but in reality it can be slightly discriminated. The area other than the lip area of the base lip image 418 is transparent. The transparency of the base color may be changed according to the lipstick color used.

The lipstick color is combined with the base lip image 418 to obtain the lipstick image 420. Specifically, the lipstick colors are superimposed with a predetermined lipstick color transparency (for example, 10 to 50%). The lipstick color is also superimposed on the highlight and shadow areas, resulting in a more homogeneous image as a whole. Therefore, in the lipstick lip image 420, the original highlights and shadows are obscured. The area other than the lip area of the lipstick lip image 420 is transparent.

The highlight image 414 is superimposed on the lipstick lip image 420 with the highlight transparency (for example, 0 to 40%) to generate the highlight lip image 422. In the case of "original texture", the highlight image 414 is used, but in the case of "glossy" texture, a standard highlight image for "glossy" is used. In the case of a "matte" texture, a standard highlight image for the "matte" is used. By overlaying the highlight images, the highlights become clearer. The area other than the lip area of the highlight lip image 422 is transparent.

A shadow image 416 is superimposed on the highlight lip image 422 with a shadow transparency (for example, 0 to 40%) to generate a processed lip image 424. In the processed lip image 424, the shadow portion becomes clearer. The area other than the lip area of the processed lip image 424 is transparent.

By processing the lips in this way, the highlights and shadows of the original captured still image are reproduced, and the impression of the original lips is maintained. Depending on the shape of the lips, the part where the highlight appears and the part where the shadow appears are different, so by using that part, the individuality of the lips for each user can be utilized.

FIG. 10 is a functional block diagram of the game image capturing device.
Each component of the game image capturing device 100 includes a CPU (Central Processing Unit), a computing unit such as various coprocessors, a storage device such as a memory and a storage, hardware including a wired or wireless communication line connecting them, and storage. It is realized by software that is stored in the device and supplies processing instructions to the processor. The computer program may be composed of a device driver, an operating system, various application programs located on the upper layer thereof, and a library that provides common functions to these programs.
As described above, the game image photographing device 100 is configured as an aggregate of the photographing device 128, the receiving device 102, the editing device 126, and the printing device 110. Each block described below shows a block for each function, not a configuration for each hardware.

The game image capturing device 100 includes a user interface processing unit 130, a data processing unit 132, a communication unit 134, and a data storage unit 136.
The user interface processing unit 130 is in charge of user interface processing via the reception monitor 108, the editing monitor 120, and the like. The communication unit 134 is in charge of communication processing with the server 202 via the Internet 204. The communication unit 134 may directly communicate with the user terminal 206 by short-range wireless communication. For example, the target image may be directly transmitted from the communication unit 134 to the user terminal 206. The data storage unit 136 stores various data. The data processing unit 132 executes various processes based on the data acquired by the communication unit 134 and the data stored in the data storage unit 136. The data processing unit 132 also functions as an interface for the user interface processing unit 130, the communication unit 134, and the data storage unit 136.

The user interface processing unit 130 includes an input unit 140 that accepts operation input by the user, an output unit 142 that presents information to the user, and a photographing unit 138 (camera 118) that acquires captured images. The input unit 140 detects the user's operation input to the touch panel installed in the reception device 102, the photographing device 128, the editing device 126, and the like. The input unit 140 also detects the operation input by the stylus pen 122. The output unit 142 executes various output processes such as audio output and print output in addition to image display in the reception device 102, the photographing device 128, the editing device 126, and the like.

The input unit 140 includes a reception unit 158. The reception unit 158 accepts a user operation in the editing process.

The output unit 142 includes an image output unit 170. The image output unit 170 displays the captured still image as the target still image on the editing monitor 120.

The data processing unit 132 includes a shooting control unit 144, an editing processing unit 146, a reception management unit 148, a printing processing unit 150, an image extraction unit 160, and an image processing unit 162.
The shooting control unit 144 controls the shooting process in the shooting booth 104. The shooting control unit 144 notifies the user of the shooting timing at the shooting booth 104. Assuming that the shooting timing is time t1, the shooting control unit 144 records a moving image (hereinafter referred to as "first half moving image") from 1.5 seconds to 0.1 seconds before time t1 and shoots at time t1. A still image is acquired, and a moving image (hereinafter referred to as "second half moving image") is recorded again from 0.1 second to 1.5 seconds after time t1. The shooting control unit 144 generates a shooting moving image by connecting the first half moving image and the second half moving image. The shooting control unit 144 controls the shooting unit 138 by transmitting a shooting start instruction and a shooting stop instruction to the shooting unit 138. Further, the shooting control unit 144 causes the strobes 119a and 119b to emit light when the shooting still image is acquired. Since the shooting moving image does not include the acquisition period of the shot still image, in other words, the moving image at the time of strobe light emission, the strobe light emission is not reflected.

The editing processing unit 146 decorates the target still image according to the operation input from the user.

The reception management unit 148 manages the payment of the play fee by the user and the reservation order in the reception device 102. The reception management unit 148 guides the reserved user to the shooting booth 104 every time the shooting work is completed in the shooting booth 104. The print processing unit 150 controls printing of the target still image in the printing device 110.

The image extraction unit 160 extracts a lip image, a highlight image, a shadow image, and the like from a captured still image. The image processing unit 162 processes the captured still image.

FIG. 11 is a flowchart showing the process of preprocessing.
As described above, in the preprocessing, the highlight image corresponding to the "original texture" and the lipstick color corresponding to the "original color" are used. The image extraction unit 160 extracts a person image 402 from the captured still image 400 as shown in FIG. 9 (S30). In the case of a plurality, the face image 410 is extracted for each person. In this example, the face images 410 of the user A and the user B are extracted.

The image extraction unit 160 extracts the original lip image 412 from each face image 410 (S32). In this example, the original lip images 412 of the user A and the user B are extracted. The image extraction unit 160 extracts the highlight image 414 from each original lip image 412 (S34). In this example, the highlight images 414 of user A and user B are extracted. Further, the image extraction unit 160 extracts a shadow image 416 from each original lip image 412 (S36). In this example, the shadow images 416 of the user A and the user B are extracted.

The image processing unit 162 synthesizes the base color for each original lip image 412 to generate the base lip image 418 (S38). In this example, the background lip images 418 of the user A and the user B are generated. The image processing unit 162 synthesizes the lipstick color for each base lip image 418 to generate the lipstick lip image 420 (S40). In this example, user A and user B lipstick lip images 420 are generated.

The image processing unit 162 superimposes the highlight image 414 on each lipstick lip image 420 to generate the highlight lip image 422 (S42). In this example, the highlight lip images 422 of user A and user B are generated. The image processing unit 162 superimposes the shadow image 416 on each highlight lip image 422 to generate a processed lip image 424 (S44). In this example, the processed lip images 424 of user A and user B are generated. The processed lip image 424 is combined into one image as shown in FIG. 8, and has both the lips of the user A and the lips of the user B. The image processing unit 162 performs layer composition as shown in FIG. 8 to generate an overall composite image 406. (S46). The whole composite image 406 is used as the target still image.

FIG. 12 is a flowchart showing the process of the editing process.
The image output unit 170 displays the target still image obtained by the preprocessing in the image display area 302 of the edit screen shown in FIG.

In this example, it is assumed that the user A touches the small tab 308a of the "lip" and the reception unit 158 accepts the selection of the lip palette 304 (S12). Therefore, the lip palette 304 (FIG. 7) is displayed. In this example, it is assumed that the user A selects his / her own face in the face selection window 310.

When the reception unit 158 receives the touch of the texture button 314, the applied texture is changed, and when the reception unit 158 receives the touch of the lipstick color button 316 or the original color button 318, the applied lipstick color is changed. (Y in S14), the image update process is executed (S16). In the image update process, the overall composite image 406 is recombined according to the changed texture or lipstick color, and the image output unit 170 displays the recombined overall composite image 406 in the image display area 302. The image update process will be described in relation to FIG. When the image update process is completed, the process returns to S14.

When the reception unit 158 accepts the selection of the lipstick eraser button 320 (Y in S18), the editing processing unit 146 corrects the lip area according to the operation of the lipstick eraser cursor (S20), and the corrected lip area. The image update process is executed in (S22). When the lip region is modified, the original lip image shown in FIG. 9 becomes smaller by that amount. The image output unit 170 displays the overall composite image 406 recombined in the image display area 302. Then, when the image update process is completed, the process returns to S14.

When the reception unit 158 accepts the selection of the large tabs 306a to 306c or the small tabs 308b (Y in S24), the process returns to S12 and switches to the pallet corresponding to the selected tab. The operation on the pallet other than the lip pallet 304 will be omitted. Further, in the case of N in S24, the process returns to S14 and the above-mentioned processing is repeated until the editing is completed.

FIG. 13 is a flowchart showing the process of image update processing.
Since the face image 410, the original lip image 412, the highlight image 414 and the shadow image 416 are obtained in the pre-processing, the processing of person extraction, lip extraction, highlight extraction and shadow extraction can be omitted. However, the process of person extraction, lip extraction, highlight extraction and shadow extraction may be performed again.

The image processing unit 162 synthesizes the base color with the original lip image 412 of the face image 410 selected by the user in the face selection window 310 to generate the base lip image 418 (S50). When the lip region is corrected by the operation of the lipstick eraser, the base color is synthesized with the corrected original lip image 412. In this example, user A's underlying lip image 418 is regenerated.

The image processing unit 162 synthesizes the lipstick color with the base lip image 418 to generate the lipstick lip image 420 (S52). If the lipstick color button 316 or the original color button 318 is selected, the lipstick color corresponding to the selected lipstick color button 316 or the original color button 318 is used. In this example, user A's lipstick lip image 420 is regenerated.

The image processing unit 162 superimposes the highlight image 414 on the lipstick lip image 420 to generate the highlight lip image 422 (S54). When the texture button 314 is selected, the highlight image corresponding to the selected texture button 314 is used. In this example, User A's highlight lip image 422 is regenerated.

The image processing unit 162 superimposes the shadow image 416 on the highlight lip image 422 to generate the processed lip image 424 (S56). In this example, the processed lip image 424 of user A is regenerated. For user B not selected by user A, the lip processing is unchanged. Therefore, the regenerated processed lip image 424 of the user A and the processed lip image 424 of the current user B are combined into one image. The image processing unit 162 performs layer composition and regenerates the entire composite image 406 (S58). The image output unit 170 displays the regenerated whole composite image 406 in the image display area 302 (S60).

[Modification 1]
In the embodiment, an example is shown in which a processed lip image 424 that makes the lips transparent except the lips is generated and superimposed on the person image 402, but the lips may be processed using a mask that makes the lips opaque except the lips.

In the first modification, layer composition is performed three times. That is, the whole composite image 406 is generated by the three-step layer composition.

FIG. 14 is a diagram showing a layer configuration in the layer composition of the first stage in the modification 1.
In the first stage layer composition, four layers are set. A background layer 500 is provided at the lowermost level, a person layer 502 is provided above the background layer 500, a masked base layer 506 is provided above the background layer 502, and a masked lipstick layer 510 is provided at the uppermost level.

In the figure, the base layer mask 504 is shown below the base layer 506 for convenience, but the base layer mask 504 is a set with the base layer 506. Similarly, the lipstick layer mask 508 is shown below the lipstick layer 510, but the lipstick layer mask 508 is a set with the lipstick layer 510.

Similar to the embodiment, a predetermined background image 404 is drawn on the background layer 500, and a person image 402 is drawn on the person layer 502.

The lip region of the base layer mask 504 is transparent, and the region other than the lip region is opaque. A background color image 436 is drawn on the background layer 506. The background color image 436 has a background color on one side, and a predetermined background transparency is set. By setting the base layer 506 and the base layer mask 504, it is possible to superimpose the base color on the lip region with the base transparency without affecting other than the lip region.

Similar to the base layer mask 504, the lip area of the lipstick layer mask 508 is transparent. Areas other than the lip area are opaque. A lipstick color image 438 is drawn on the lipstick layer 510. One side of the lipstick color image 438 is lipstick color, and a predetermined lipstick color transparency is set. By setting the lipstick layer 510 and the lipstick layer mask 508, it is possible to superimpose the lipstick color on the lip region without affecting other than the lip region.

Combining the layers gives a primary composite image 432. In the background portion of the primary composite image 432, the background image 404 of the background layer 500 appears through the person layer 502. The person image 402 of the person layer 502 appears in the person portion (excluding the lip portion) of the primary composite image 432. In the lip portion of the primary composite image 432, an image in which the background color is superimposed on the lip portion of the portrait image 402 of the portrait layer 502 with the background transparency and the lipstick color is superimposed with the lipstick color transparency appears.

In the primary composite image 432, the lipstick color is also superimposed on the highlight and shadow areas, so that the lip image is uniform as a whole. That is, the original highlights and shadows are obscured. In the next layer composition, the highlight image is overlaid.

FIG. 15 is a diagram showing a layer configuration in the layer composition of the second stage in the modification 1.
In the second stage layer composition, two layers are set. A primary composite layer 511 is set at the lower level, and a masked highlight layer 514 is set at the upper level. In the figure, the highlight layer mask 512 is shown below the highlight layer 514 for convenience, but the highlight layer mask 512 is a set with the highlight layer 514.

The highlight area of the highlight layer mask 512 is transparent, and the area other than the highlight area is opaque. A highlight color image 440 is drawn on the highlight layer 514. One side of the highlight color image 440 is a highlight color (white or a color having a lightness higher than the highlight color reference), and the highlight transparency (for example, 0 to 40%) is set. By setting the highlight layer 514 and the highlight layer mask 512, it is possible to superimpose the highlight color on the highlight area without affecting other than the highlight area.

Combining each layer gives a secondary composite image 434. Except for the highlight area, the primary composite image 432 remains. Since the highlight area is colored with highlights, the difference in brightness becomes strong. The image of the lip portion in the secondary composite image 434 is close to the highlight lip image 422 shown in FIG. In the next layer composition, the shadow image is further superimposed.

FIG. 16 is a diagram showing a layer configuration in the layer composition of the third stage in the modification 1.
In the third stage of layer composition, two layers are set. A secondary composite layer 515 is set at the lower level, and a masked shadow layer 518 is set at the upper level. In the figure, the shadow layer mask 516 is shown below the shadow layer 518 for convenience, but the shadow layer mask 516 is a set with the shadow layer 518.

The shadow area of the shadow layer mask 516 is transparent, and the area other than the shadow area is opaque. A shadow color image 442 is drawn on the shadow layer 518. One side of the shadow color image 442 is a shadow color (black or a color having a lightness lower than the shadow color reference), and the shadow transparency (for example, 0 to 40%) is set. By setting the shadow layer 518 and the shadow layer mask 516, the shadow color can be superimposed on the shadow area without affecting other than the shadow area.

Combining each layer gives a whole composite image 406. Except for the shadow area, the secondary composite image 434 remains. Since the shadow area is shaded, the difference in lightness becomes strong. The image of the lip portion in the overall composite image 406 is close to the processed lip image 424 shown in FIG.

FIG. 17 is a flowchart showing the process of preprocessing in the first modification.
The person extraction (S70) and the lip extraction (S72) by the image extraction unit 160 are the same as in the case of FIG.

The image processing unit 162 generates the lipstick layer mask 508 and the base layer mask 504 (S74, S76). Further, the image processing unit 162 generates a highlight layer mask 512 (S78), and further generates a shadow layer mask 516 (S80).

The image processing unit 162 performs three-step layer composition as described above (S82). The whole composite image 406 generated by the three-step layer composition becomes the target still image.

FIG. 18 is a flowchart showing the process of image update processing in the first modification.
The image processing unit 162 generates the lipstick layer mask 508 and the base layer mask 504 (S90, S92). When the lip area is reduced by the lipstick eraser, the transparent portion of the lipstick layer mask 508 and the base layer mask 504 becomes smaller, and the opaque portion becomes larger.

The image processing unit 162 performs three-step layer composition (S94). When the lipstick color is changed, the lipstick color image 438 of FIG. 14 changes. When the texture is changed, the highlight area of the highlight layer mask 512 in FIG. 15 changes. The image output unit 170 displays the entire composite image 406 in the image display area 302 (S96).

[Modification 2]
In the embodiment, an example is shown in which the highlight and the shadow are overlapped after the base and the lipstick are processed on the entire lip, but the base and the lipstick are processed on the lip area excluding the highlight and the shadow. , You may want to leave the original highlights and shadows.

FIG. 19 is a diagram showing a layer configuration in the second modification.
In layer composition, three layers are set as in FIG. A background layer 500 is set at the lower level, a person layer 502 is set at the middle layer, and a lip layer 503 is set at the upper level.

Similar to FIG. 8, a predetermined background image 404 is drawn on the background layer 500, and a person image 402 is drawn on the person layer 502. The processed lip image 430 drawn on the lip layer 503 is different from the processed lip image 424 of FIG. In the processed lip image 430, in addition to the area other than the lip area, the highlight area and the shadow area are also transparent. Similar to the processed lip image 424 of FIG. 8, the base color and the lipstick color are superimposed on the lip region excluding the highlight region and the shadow region.

Combining each layer gives a whole composite image 406. In the background portion of the entire composite image 406, the background image 404 of the background layer 500 appears through the lip layer 503 and the person layer 502. In the person portion (excluding the lip portion) of the entire composite image 406, the person image 402 of the person layer 502 appears through the lip layer 503. The processed lip image 430 appears in the lip portion of the overall composite image 406 except for the highlight of the lips. In the highlights and shadows of the entire composite image 406, the highlights and shadows of the person image 402 of the person layer 502 appear through the lip layer 503. That is, the highlights and shadows of the person image 402 are used as they are.

FIG. 20 is a diagram showing an outline of processing of a lip image in Modification 2.
The face image 410, the original lip image 412, the highlight image 414, and the shadow image 416 are the same as in the case of FIG.

The cut-out lip image 426 is an image in which the highlight region and the shadow region of the original lip image 412 are made transparent. In the second modification, the base color is combined with the cutout lip image 426 to generate the base lip image 428. The highlight area and shadow area of the background lip image 428 are transparent. Further, the lipstick color is synthesized with the base lip image 428 to generate the processed lip image 430. The highlight and shadow areas of the processed lip image 430 are transparent.

FIG. 21 is a flowchart showing the process of preprocessing in the second modification.
The person extraction (S30), lip extraction (S32), highlight extraction (S34), and shadow extraction (S36) by the image extraction unit 160 are the same as in the case of FIG.

The image processing unit 162 cuts out highlights and shadows from the original lip image 412 (S100). That is, the highlight area and the shadow area in the original lip image 412 are set to be transparent, and the cutout lip image 426 is generated. The image processing unit 162 synthesizes the base color with the cutout lip image 426 to obtain the base lip image 428 (S38). The image processing unit 162 further synthesizes a lipstick color with the underlying lip image 428 to obtain a processed lip image 430 (S40). The image processing unit 162 performs layer composition with the layer configuration shown in FIG. 19 to obtain an overall composite image 406 (S46). Then, the entire composite image 406 becomes the target still image.

FIG. 22 is a flowchart showing the process of image update processing in the second modification.
The image extraction unit 160 cuts out highlights and shadows from the original lip image 412 (S100). When the texture is changed, the shape of the highlight area that becomes transparent in the cutout lip image 426 changes. The image processing unit 162 synthesizes the base color with the cutout lip image 426 to obtain the base lip image 428 (S50). The image processing unit 162 further synthesizes a lipstick color with the underlying lip image 428 to obtain a processed lip image 430 (S52). When the lipstick color is changed, the color of the processed lip image 430 changes. The image processing unit 162 performs layer composition with the layer configuration shown in FIG. 19 to obtain an overall composite image 406 (S58). The image output unit 170 displays the entire composite image 406 in the image display area 302 (S60).

[Modification 3]
In the first modification, a layer mask is used to superimpose the base color and the lipstick color on the entire lip, followed by the second step layer compositing of highlights and the third layer compositing of shadows. Although an example of performing stepwise layer composition is shown, a layer mask may be used to perform layer composition in which the base color and the lipstick color are superimposed on the lip region excluding highlights and shadows.

FIG. 23 is a diagram showing a layer configuration in the modified example 3.
As in the case of FIG. 14, four layers are set. The background image 404 drawn on the background layer 500, the person image 402 drawn on the person layer 502, the background color image 436 drawn on the background layer 506, and the lipstick color image 438 drawn on the lipstick layer 510 are also shown in the figure. This is the same as in the case of 14.

The shapes of the base layer mask 520 and the lipstick layer mask 522 are different from those in FIG. Along with the base layer mask 520 and the lipstick layer mask 522, the area of the lips excluding highlights and shadows is transparent. Areas other than the lip area, highlight areas and shadow areas are opaque.

Combining each layer gives a whole composite image 406. The background image 404 of the background layer 500 is transparent to the background portion of the overall composite image 406, and the background image 404 of the background layer 500 appears. Appears. In the lip portion excluding the highlights and shadows of the entire composite image 406, an image in which the background color and the lipstick color are combined appears in the lip portion of the portrait image 402 of the portrait layer 502. The highlights and shadows of the lips of the person image 402 of the person layer 502 appear in the highlights and shadows of the lips of the whole composite image 406. That is, the highlights and shadows of the person image 402 are used as they are.

FIG. 24 is a flowchart showing the process of preprocessing in the modification 3.
The person extraction (S70) and the lip extraction (S72) by the image extraction unit 160 are the same as in the case of FIG.

The image processing unit 162 generates the lipstick layer mask 508 and the base layer mask 504 (S74, S76). In the lipstick layer mask 508 and the base layer mask 504, as described above, the lip region excluding the highlights and shadows is transparent. Other than that, it is opaque.

The image processing unit 162 performs layer composition shown in FIG. 23 to obtain an overall composite image 406 (S82). Then, the entire composite image 406 becomes the target still image.

The process of the image update process in the modification 3 will be described with reference to FIG. The image processing unit 162 generates the above-mentioned lipstick layer mask 508 and the base layer mask 504 (S90, S92). The image processing unit 162 performs layer composition shown in FIG. 23 to obtain an overall composite image 406 (S94). When the texture is changed, the shape of the highlight region that becomes transparent in the base layer mask 504 and the lipstick layer mask 508 changes. When the lipstick color is changed, the lipstick color of the lipstick color image 438 changes. The image output unit 170 displays the entire composite image 406 in the image display area 302 (S96).

[Other variants]
In relation to FIG. 7, an example of using a fixed-form highlight image when the “glossy” texture button 314a is selected is shown, but the highlight image 414 is enlarged to display the enlarged highlight image. It may be placed on the lips.

In relation to FIG. 7, an example of using a fixed-form highlight image when the “matte” texture button 314b is selected is shown, but the highlight image 414 is reduced to obtain a reduced highlight image. It may be placed on the lips.

The treatment of the base color may be omitted and only the treatment of the lipstick may be performed. Further, the processing for the shadow of the lips may be omitted, and only the processing for the highlight of the lips may be performed. Further, the processing for the highlight of the lips may be omitted, and the processing for the shadow of the lips may be performed.

Lipstick color is an example of the tint added to the lips. The color added to the lip portion may be regarded as the color of the lips as a living body, not the lipstick color.

As an instruction for lipstick color, adjustment operations such as "more red" and "more dark" may be performed.

The reason why the highlights of the lips are not clear is that they add color, but it is also largely due to smoothing (blurring). Therefore, even when the process of adding color is omitted and only the smoothing process is performed, the effect of utilizing the texture of the lips is obtained by reproducing the original highlight image in the highlight area.

In the image processing of the cheek, the above-described embodiments and modifications 1 to 3 may be applied. In that case, "lips" in the above description are replaced with "cheeks", and "lipstick" is replaced with "cheek blush". In the case of cheek processing, the processing of the shadow portion may be omitted.

In the image processing of the hair, the above-described embodiments and modifications 1 to 3 may be applied. In that case, "lips" in the above description is replaced with "hair", and "lipstick" is replaced with "hair dyeing".

In the image processing of the nail, the above-described embodiments and modifications 1 to 3 may be applied. In that case, replace "lips" with "nail" and "lipstick" with "manicure" in the above description.

In the image processing of the pupil, the above-described embodiments and modifications 1 to 3 may be applied. In that case, the "lips" in the above description are replaced with "pupils", and the "lipstick" is replaced with "color contacts".

The above-described embodiments and modifications 1 to 3 may be applied to image processing of a chin, a nose, a chin, and the like. In that case, the "lips" in the above explanation are replaced with "hitai, nose and chin", and the "lipstick" is replaced with "foundation".

The present invention is not limited to the above-described embodiment or modification, and the components can be modified and embodied within a range that does not deviate from the gist. Various inventions may be formed by appropriately combining a plurality of components disclosed in the above-described embodiments and modifications. In addition, some components may be deleted from all the components shown in the above embodiments and modifications.

100 play image shooting device, 102 reception device, 104 shooting booth, 106 editing booth, 108 reception monitor, 110 printing device, 112 first shooting monitor, 114 second shooting monitor, 116 lights, 118 camera, 119 strobe, 120 editing monitor , 122 stylus pen, 124 curtain, 126 editing device, 128 imaging device, 130 user interface processing unit, 132 data processing unit, 134 communication unit, 136 data storage unit, 140 input unit, 142 output unit, 144 image processing unit, 146 Editing processing unit, 148 reception processing unit, 150 printing processing unit, 158 reception unit, 160 image extraction unit, 162 image processing unit, 170 image output unit, 200 game image shooting system, 202 server, 204 Internet, 206 user terminal, 300 Image tab, 302 image display area, 304 lip palette, 306 large tab, 308 small tab, 310 face selection window, 312 face selection frame, 314 texture button, 316 lipstick color button, 318 original color button, 320 lipstick eraser button, 400 still image, 402 portrait image, 404 background image, 406 overall composite image, 410 face image, 412 original lip image, 414 highlight image, 416 shadow image, 418 base lip image, 420 lipstick image, 422 highlight lip Image, 424 processed lip image, 426 cutout lip image, 428 background lip image, 430 processed lip image, 432 primary composite image, 434 secondary composite image, 436 background color image, 438 lipstick image, 440 highlight color image , 442 shadow color image, 500 background layer, 502 person layer, 503 lip layer, 504 background layer mask, 506 background layer, 508 lipstick layer mask, 510 lipstick layer, 511 primary composite layer, 512 highlight layer mask, 514 highlights. Layer, 515 secondary composite layer, 516 shadow layer mask, 518 shadow layer, 520 background layer mask, 522 lipstick layer mask.

Claims (4)

The shooting section that shoots the user,
With regard to lip processing, the reception section that accepts the selection of the original texture, the selection of the first texture, or the selection of the second texture,
An extraction unit that extracts the original highlight image from the highlight area of the user's lips in the captured image,
When the selection of the original texture is accepted after the lip portion of the captured image is smoothed, the original highlight image is superimposed on the smoothed lip portion. first processing performed, when receiving the selection of the first texture, the smoothing process has been the lip portion, and facilities the second processing overlaying the highlight image of the first shaped, the first When the selection of the two textures is accepted, the third process of superimposing the second fixed form highlight image having a size different from that of the first fixed form highlight image on the smoothed lip portion. and facilities to the image processing unit,
A shooting play apparatus including an output unit for outputting an image to which the first processing , the second processing, or the third processing has been performed. The image processing portion is subjected to the first processing, the second processing, or the third processing after adding color to the smoothed lip portion.
And the output unit, the first processing, photographing amusement device according to claim 1, wherein the second processing or the third processing and outputs the image subjected. The image processing unit sets a plurality of layers including at least a lip layer, and draws an image subjected to the smoothing process and the first processing, the second processing, or the third processing on the lip layer. Then, the plurality of layers are combined to generate a composite image, and the composite image is generated.
The photographing game device according to claim 1 or 2 , wherein the output unit outputs the generated composite image. The ability to shoot users and
With regard to lip processing, a function that accepts the selection of the original texture, the selection of the first texture, or the selection of the second texture,
A function to extract the original highlight image from the highlight area of the user's lips in the captured image, and
When the selection of the original texture is accepted after the lip portion of the captured image is smoothed, the original highlight image is superimposed on the smoothed lip portion. first processing performed, when receiving the selection of the first texture, the smoothing process has been the lip portion, and facilities the second processing overlaying the highlight image of the first shaped, the first When the selection of the two textures is accepted, the third process of superimposing the second fixed form highlight image having a size different from that of the first fixed form highlight image on the smoothed lip portion. and the facilities to function,
A function to output an image to which the first processing , the second processing, or the third processing has been performed, and
A program characterized by making information processing equipment exhibit.

Images