KR102416500B1 - Electronic device and Method for controlling the electronic device thereof - Google Patents

Abstract

An electronic device and a method for controlling the same are provided. The electronic device acquires a plurality of images by photographing the user's face a plurality of times based on a camera for photographing the user's face, a communication unit for communicating with an external user terminal, and a plurality of photographing parameters, and uses the plurality of photographing parameters. Extracting a user face region corresponding to each of a plurality of photographed images, synthesizing the extracted user face regions, and controlling the communication unit to transmit data on the synthesized image to an external user terminal for skin analysis on the user's face includes a processor.

Description

Electronic device and method for controlling the same

The present invention relates to an electronic device and a method for controlling the same, and more particularly, to an electronic device for acquiring an image for skin analysis of a user's face by photographing the user's face multiple times, and a method for controlling the same.

In recent years, as interest in beauty increases, interest in skin beauty of a user's face is increasing. In particular, an electronic device for photographing a user's facial skin and analyzing various skin troubles (eg, wrinkles, pores, acne, etc.) on the user's face has been developed.

Meanwhile, in a conventional electronic device for skin analysis, a user's face is photographed based on preset shooting parameters (eg, intensity of light source, direction, aperture value, shutter speed, etc.) while the user's face is placed in the electronic device. did That is, the conventional electronic device for skin analysis cuts off external light and evenly spreads the light over the entire face of the user through light reflection in the electronic device to photograph the user's face.

However, if the intensity of the light source is weak when photographing the user's face, it is difficult to distinguish between dark areas and pigmentation during image analysis. existed. In addition, since the conventional electronic device for skin analysis has a large size in a shape that covers the entire user's face, there is a problem in that it is inconvenient to carry and carry.

An object of the present disclosure is to provide an electronic device for acquiring a plurality of images by photographing a user's skin multiple times based on a plurality of photographing parameters, and synthesizing the plurality of images into one face image for skin analysis, and a control method thereof is in providing.

According to an embodiment of the present disclosure for achieving the above object, an electronic device includes: a camera for photographing a user's face; a communication unit for performing communication with an external user terminal; and capturing a plurality of images by photographing the user's face a plurality of times based on the plurality of photographing parameters, extracting a user face region corresponding to each of the plurality of images photographed with the plurality of photographing parameters, and the extracted user face and a processor that controls the communication unit to synthesize regions and transmit data on the synthesized image to an external user terminal for skin analysis on the user's face.

According to an embodiment of the present disclosure for achieving the above object, a control method of an electronic device includes: acquiring a plurality of images by photographing a user's face a plurality of times based on a plurality of photographing parameters; extracting a user face region corresponding to each of the plurality of images captured with the plurality of shooting parameters, and synthesizing the extracted user face regions; and transmitting data on the synthesized image to an external user terminal for skin analysis on the user's face.

In a non-transitory computer-readable recording medium storing a program for executing a control method of an electronic device according to an embodiment of the present disclosure for achieving the above object, the control method includes: a user based on a plurality of shooting parameters; acquiring a plurality of images generated by photographing a face a plurality of times; extracting a user face region corresponding to each of the plurality of images captured with the plurality of shooting parameters, and synthesizing the extracted user face regions; determining a skin condition of a user's face by analyzing the synthesized image; and providing information on the skin condition.

As described above, by acquiring a face image having an even brightness distribution over the entire region of the user's face, it is possible to increase the accuracy of the facial skin analysis result. In addition, since there is no need for the user's face to be inserted into the electronic device, the electronic device can be made smaller and lighter, so that the convenience of carrying and carrying can be improved.

1 is a view for explaining a system for analyzing a user's skin according to an embodiment of the present disclosure;
2 is a block diagram schematically illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
3 is a block diagram illustrating the configuration of an electronic device in detail according to an embodiment of the present disclosure;
4A to 6E are diagrams for explaining an embodiment of analyzing a user's skin using a skin analysis application, according to an embodiment of the present disclosure;
7 is a sequence diagram for explaining a control method of a system for analyzing a user's facial skin, according to an embodiment of the present disclosure;
8 and 9 are flowcharts for explaining a method of controlling an electronic device, according to various embodiments of the present disclosure.

Terms used in this specification will be briefly described, and the present invention will be described in detail.

The terms used in the embodiments of the present invention have been selected as currently widely used general terms as possible while considering the functions in the present invention, which may vary depending on the intention or precedent of a person skilled in the art, the emergence of new technology, etc. . In addition, in a specific case, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term and the overall content of the present invention, rather than the name of a simple term.

Embodiments of the present invention can apply various transformations and can have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the specific embodiments, and it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the invention. In describing the embodiments, if it is determined that a detailed description of a related known technology may obscure the subject matter, the detailed description thereof will be omitted.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as "comprises" or "consisting of" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and are intended to indicate that one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

In an embodiment of the present invention, a 'module' or 'unit' performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of 'modules' or a plurality of 'units' are integrated into at least one module and implemented with at least one processor (not shown) except for 'modules' or 'units' that need to be implemented with specific hardware. can be

In an embodiment of the present invention, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" with another element interposed therebetween. also includes In addition, it includes not only a physical connection but also a case of wireless connection. Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Hereinafter, the present disclosure will be described in detail with reference to the drawings. 1 is a view for explaining a system for analyzing a user's skin, according to an embodiment of the present disclosure. As shown in FIG. 1 , the system 10 may include an electronic device 100 and a user terminal 200 . In this case, the electronic device 100 may be a device dedicated to photographing the user's skin, but this is only an exemplary embodiment and may be implemented as another electronic device (eg, a smart phone, etc.) having a camera and a light source. have. The user terminal 200 may be implemented as a smart phone, but this is only an exemplary embodiment, and may be implemented as various terminals such as a tablet PC, a notebook PC, and a desktop PC.

The electronic device 100 may set a plurality of photographing parameters for analyzing the user's skin. Specifically, the electronic device 100 photographs the user's face a plurality of times based on the plurality of preliminary photographing parameters, and analyzes the brightness of a plurality of images taken based on the plurality of preliminary photographing parameters to obtain a plurality of the plurality of preliminary photographing parameters. Preliminary photographing parameters corresponding to the user's face region of may be set as a plurality of photographing parameters. In addition, the electronic device 100 may store a plurality of set shooting parameters.

In addition, the electronic device 100 may acquire a plurality of images by photographing the user's face a plurality of times based on the plurality of stored photographing parameters.

In addition, the electronic device 100 may extract a user face region corresponding to each of a plurality of images captured with a plurality of shooting parameters, and may synthesize the extracted user face regions.

In addition, the electronic device 100 may transmit data on the image synthesized for skin analysis of the user's face to the user terminal 200 .

The user terminal 200 may determine the skin condition of the user's face based on the data on the composite image. Specifically, the user terminal 200 may analyze whether there are skin troubles (eg, wrinkles, acne, pores) on the user's face through image analysis.

The user terminal 200 may provide information on the skin condition of the user's face.

Meanwhile, in the above-described embodiment, it has been described that the electronic device 100 transmits data on the composite image to the user terminal 200 , but this is only an example, and information about the composite image in the electronic device 100 is provided. The data may be analyzed to determine the skin condition of the user's face, and information on the determined skin condition may be output.

2 is a block diagram schematically illustrating a configuration of an electronic device 100 according to an embodiment of the present disclosure. As shown in FIG. 2 , the electronic device 100 includes a camera 110 , a communication unit 120 , and a processor 130 .

The camera 110 captures the user's face. Specifically, the camera 110 may acquire a plurality of images by photographing a user's face based on a plurality of photographing parameters.

The communication unit 120 communicates with an external device. In particular, the communication unit 120 may extract a plurality of regions from a plurality of images photographed based on a plurality of photographing parameters and transmit the synthesized image to the external user terminal 200 .

The processor 130 controls the overall operation of the electronic device 100 . In particular, the processor 130 acquires a plurality of images by photographing the user's face a plurality of times based on the plurality of photographing parameters, extracts a user face region corresponding to each of the plurality of images photographed with the plurality of photographing parameters, and extracts The communicator 120 may be controlled to synthesize the user's face regions and transmit data on the synthesized image to the external user terminal for skin analysis on the user's face.

First, the processor 130 sets shooting parameters for analyzing the skin of the user's face. Specifically, the processor 130 may photograph the user's face a plurality of times based on the plurality of preliminary photographing parameters. In this case, the plurality of preliminary photographing parameters may be determined according to an external lighting environment. In addition, the processor 130 may analyze the brightness of a plurality of images taken based on the plurality of preliminary shooting parameters to set preliminary shooting parameters corresponding to a plurality of user face regions among the plurality of preliminary shooting parameters as a plurality of shooting parameters. have.

In one embodiment of the present invention, the image captured based on the first preliminary shooting parameter among the plurality of preliminary shooting parameters is a brightness value of a preset range for the first region (eg, face center region) of the user's face. If yes, the processor 130 may set the first preliminary photographing parameter as the first photographing parameter corresponding to the first area of the user's face. When the image captured based on the second preliminary shooting parameter among the plurality of preliminary shooting parameters has a brightness value within a preset range for the second region (eg, the outer face region) of the user's face, the processor 130 is The second preliminary photographing parameter may be set as the second photographing parameter corresponding to the second area of the user's face.

In this case, the photographing parameter may include at least one of the intensity of the light source provided in the electronic device 100 , the direction of the light source, the shutter speed, and the aperture value, and may be set differently for each user.

After the plurality of photographing parameters are set, the processor 130 may acquire a plurality of images by photographing the user's face a plurality of times based on the set plurality of photographing parameters. In an embodiment of the present disclosure, the processor 130 acquires a first image by photographing the user's face based on the first photographing parameter, and obtains a second image by photographing the user's face based on the second photographing parameter. can

In addition, the processor 130 may extract a user face region corresponding to each of the plurality of images from the plurality of images, and may obtain a composite image by synthesizing the extracted user face regions. In an embodiment of the present disclosure, the processor 130 extracts a first region of the user's face from the first image captured by the first shooting parameter, and extracts the user's face from the second image captured by the second shooting parameter The second region may be extracted. In addition, the processor 130 may obtain a composite image by synthesizing the extracted first and second regions.

In this case, the processor 130 may correct and synthesize the first region and the second region according to the detected motion information of the electronic device. That is, when the motion of the electronic device 100 is detected while capturing the second image, the processor 130 may correct the second region according to the detected motion of the electronic device. According to another embodiment, the processor 130 may correct the second image before extracting the second region according to the movement of the electronic device 100 .

Also, when synthesizing the first region and the second region, the processor 130 may synthesize the first region and the second region by performing brightness correction. Specifically, the processor 130 may determine a brightness difference value between the first area and the second area near the boundary between the first area and the second area. In addition, the processor 130 may determine a synthesis weight for at least one of the first region and the second region based on the brightness difference value. In this case, the composite weight may be a weight for adjusting brightness. In addition, the processor 130 may synthesize by adjusting the brightness of at least one of the first region and the second region based on the synthesis weight.

In addition, the processor 130 may control the communication unit 120 to transmit the composite image to the external user terminal 200 .

Meanwhile, in the above-described embodiment, it has been described that the electronic device 100 transmits the composite image to the external user terminal 200, but this is only an embodiment, and a plurality of images obtained based on a plurality of photographing parameters are described. may be transmitted to the external user terminal 200 . In this case, the user terminal 200 may analyze the skin state of the user's face by synthesizing the plurality of images in the same manner as described above.

Meanwhile, according to another embodiment of the present invention, the processor 130 directly analyzes the synthesized image to obtain skin condition information on at least one of pigmentation, acne, sebum, pores, and wrinkles on the user's face, and obtain It is possible to output the skin condition information.

3 is a block diagram illustrating in detail the configuration of the electronic device 100 according to an embodiment of the present disclosure. 3 , the electronic device 100 includes a camera 110 , a communication unit 120 , a light source 140 , an output unit 150 , a memory 160 , an input unit 170 , and a sensing unit 180 . and a processor 120 . Meanwhile, since the configuration of the electronic device 100 illustrated in FIG. 3 is only an example, it is not necessarily limited to the above-described block diagram. Accordingly, it goes without saying that a part of the configuration of the electronic device 100 illustrated in FIG. 3 may be omitted, modified, or added according to the type of the electronic device 100 or the purpose of the electronic device 100 .

As shown in FIG. 1 , the camera 110 may be positioned at the upper front of the electronic device 100 to photograph the user's face. In this case, the camera 110 may acquire a plurality of images by photographing the user's face multiple times based on different shooting parameters (eg, shutter speed value, aperture value, etc.).

The light source 140 provides light toward the user's face. In particular, as shown in FIG. 1 , a plurality of light sources 140 may be provided around the camera 110 . Also, while photographing the user's face, the light source 140 may provide different intensities according to the control of the processor 130 .

The output unit 150 outputs various information. In particular, the output unit 151 may include a display 151 and an audio output unit 153 as shown in FIG. 3 .

The display 151 displays various image data and UI. In particular, the display 140 may provide information on the skin condition of the user's face.

Meanwhile, according to an embodiment of the present invention, when the electronic device 100 is implemented as a portable terminal such as a smart phone, the display 151 may be implemented as a touch screen by combining with a touch panel, and a flexible display (flexible display). It may be coupled to at least one of a front area, a side area, and a rear area of the electronic device 100 in the form of a display. The flexible display may be characterized as being able to bend, bend, or roll without damage through a thin and flexible substrate like paper.

The audio output unit 153 is configured to output various types of audio data on which various processing operations such as decoding, amplification, and noise filtering have been performed by an audio processing unit (not shown) as well as various notification sounds or voice messages. According to another embodiment of the present invention, the electronic device 100 may further include an output terminal as a configuration for outputting audio.

In particular, the audio output unit 153 may provide information on the skin condition of the user's face in the form of audio.

The communication unit 120 may communicate with the external user terminal 200 . In particular, the communication unit 120 may receive a request to photograph a user's face from the external user terminal 200 , and may transmit a composite image to the external user terminal 200 . In this case, the communication unit 120 may be implemented as a Bluetooth module, but this is only an embodiment, and may be implemented with various communication modules such as a Zigbi communication module, a Wi-Fi communication module, and an NFC communication module.

The memory 160 stores various modules for driving the electronic device 100 . For example, software including a base module, a sensing module, a communication module, a presentation module, a web browser module, and a service module may be stored in the memory 160 . In this case, the base module is a basic module that processes signals transmitted from each hardware included in the electronic device 100 and transmits them to a higher layer module. The sensing module is a module for collecting information from various sensors and analyzing and managing the collected information, and may include a face recognition module, a voice recognition module, a motion recognition module, and the like. The presentation module is a module for configuring a display screen, and may include a multimedia module for playing and outputting multimedia content, and a UI rendering module for performing UI and graphic processing. The communication module is a module for performing communication with the outside. The web browser module refers to a module that accesses a web server by performing web browsing. The service module is a module including various applications for providing various services.

As described above, the memory 160 may include various program modules, but of course, some of the various program modules may be omitted, modified, or added according to the type and characteristics of the electronic device 100 .

Also, the memory 160 may store a plurality of shooting parameters for each user.

Meanwhile, the memory 160 may be implemented as a non-volatile memory, a volatile memory, a flash-memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 160 is accessed by the processor 130 , and reading/writing/modification/deletion/update of data by the processor 130 may be performed. In the present disclosure, the term "memory" refers to a memory 160, a ROM (not shown) in the processor 130, a RAM (not shown), or a memory card (not shown) mounted in the electronic device 100 (eg, micro SD). card, memory stick).

The input unit 170 may input a user command to operate the electronic device 100 and transmit information about the user command to the processor 130 . In particular, the input unit 180 may be a button (eg, a button for taking a picture, etc.) provided in the electronic device 100 , but this is only an exemplary embodiment and may be implemented as another input device. .

In an embodiment of the present disclosure, the input unit 170 may be implemented as a touch panel for sensing a user's touch, a (digital) pen sensor, or the like. The touch panel may use, for example, at least one of a capacitive type, a pressure-sensitive type, an infrared type, and an ultrasonic type. In addition, the touch panel may further include a control circuit. However, implementation of the input unit 170 as a touch panel, a pen sensor, etc. is only an exemplary embodiment, and may be implemented as various electronic devices such as a microphone or a pointing device for receiving a user's utterance.

The sensing unit 180 includes various sensors for detecting the state of the electronic device 100 . In particular, the sensing unit 180 includes a motion sensor (eg, an acceleration sensor, a gyro sensor, an electromagnetic sensor, etc.) for detecting the movement of the electronic device 100 , and for detecting a lighting environment around the electronic device 100 . It may include an illuminance sensor and the like.

The processor 130 is electrically connected to various components (eg, the output unit 150 , the memory 140 , and the camera 110 ) of the electronic device 100 , and thus the overall operation and function of the electronic device 100 . can be controlled. In particular, the processor 130 may control the overall operation of the electronic device 100 using various programs stored in the memory 160 .

Specifically, the processor 130 acquires a plurality of images by photographing the user's face a plurality of times based on the plurality of photographing parameters, and extracts a user face region corresponding to each of the plurality of images photographed with the plurality of photographing parameters, The communicator 120 may be controlled to synthesize the extracted user face regions and transmit data on the synthesized image to the external user terminal 200 for skin analysis on the user's face.

An embodiment in which the processor 130 provides information on the skin condition of the user's face in cooperation with the external user terminal 200 will be described in detail with reference to FIGS. 4A to 6E .

In addition, the user terminal 200 may execute a skin analysis application in response to a user command, and may display a start screen of the skin analysis application as shown in FIG. 4A . In this case, the start screen of the skin analysis application may include a UI for user login.

And, when user login is performed, the user terminal 200 may search for an external electronic device 100 capable of communicating. In this case, the user terminal 200 may display a screen guiding the search of the external electronic device 100 as shown in FIG. 4B .

When the external electronic device 100 is found, the user terminal 200 may communicate with the external electronic device 100 . In this case, the user terminal 200 may communicate with the external device 100 using Bluetooth communication. When communication is connected between the user terminal 200 and the external device 100 , the user terminal 200 may transmit information about the logged-in user to the external device 100 .

The processor 130 of the electronic device 100 determines whether a plurality of shooting parameters for the logged-in user are preset.

When the plurality of photographing parameters corresponding to the logged-in user are not previously set, the processor 130 may set a plurality of photographing parameters corresponding to the user's face region among the plurality of preliminary photographing parameters.

Specifically, the processor 130 may acquire a plurality of images by photographing a user's face based on a plurality of preliminary photographing parameters through the camera 110 . For example, the processor 130 may acquire first to nth images by photographing the user's face based on the first through n preliminary photographing parameters. In this case, the preliminary shooting parameters may include the intensity and direction of the light source 140 , the shutter speed of the camera 110 , the aperture value, the exposure value, and the like, and may be determined according to the external light detected by the sensing unit 180 . .

Meanwhile, in the above embodiment, it has been described that a plurality of images are acquired by photographing a user's face based on a plurality of preliminary photographing parameters, but this is only an embodiment, and the user's face is photographed by changing the preliminary photographing parameters in real time And, it is possible to obtain a video of the user's face.

In addition, the processor 130 may analyze the brightness of a plurality of images taken based on the plurality of preliminary shooting parameters to set preliminary shooting parameters corresponding to a plurality of user face regions among the plurality of preliminary shooting parameters as a plurality of shooting parameters. have.

In an embodiment of the present invention, when an image photographed based on the first preliminary photographing parameter among the plurality of preliminary photographing parameters has a brightness value within a preset range for the central region of the user's face, the processor 130 may perform the first The preliminary photographing parameter may be set as the first photographing parameter corresponding to the central region of the user's face. And, when an image captured based on the second preliminary shooting parameter among the plurality of preliminary shooting parameters has a brightness value within a preset range for the outer region of the user's face, the processor 130 sets the second preliminary shooting parameter to the user's face. may be set as the second shooting parameter corresponding to the outer region of . That is, the processor 130 may set the preliminary shooting parameter having the optimal brightness value in the central region and the outer region of the user's face as the shooting parameter corresponding to each region.

In this case, the intensity of the light source included in the shooting parameter corresponding to the central region of the user's face may be weaker than the intensity of the light source included in the shooting parameter corresponding to the outer region of the user's face. Also, the shutter speed value included in the shooting parameter corresponding to the central region of the user's face may be greater than the shutter speed value included in the shooting parameter corresponding to the outer region of the user's face. Also, the aperture (open) value included in the photographing parameter corresponding to the central region of the user's face may be smaller than the aperture (open) value included in the photographing parameter corresponding to the outer region of the user's face. In addition, the exposure value included in the shooting parameter for the central region of the user's face may be smaller than the exposure value included in the shooting parameter corresponding to the outer region of the user's face.

Meanwhile, in the above-described embodiment, the user's face is divided into two regions (eg, a central region and an outer region), but this is only an embodiment. The user's face is divided into three or more regions, each region You can set the corresponding shooting parameters.

In addition, the processor 130 may acquire a plurality of images by photographing the user's face based on a plurality of preset shooting parameters through the camera 110 . For example, the processor 130 may acquire a first image photographed using the first photographing parameter and a second image photographed using the second photographing parameter.

In this case, as shown in FIG. 5A , the first image may include a central region 510-1 having a brightness value in the first range and an outer region 520-1 having a brightness value in the second range, As shown in FIG. 5B , the second image may include a central area 510 - 2 having a brightness value in the third range and an outer area 520 - 2 having a brightness value in the fourth range. In this case, an overlapping section may exist between the first range and the fourth range. That is, the first range and the fourth range may have the same or similar brightness values.

In addition, the processor 130 may extract corresponding user face regions from a plurality of images, and synthesize the extracted regions to generate a composite image.

Specifically, as shown in FIG. 5C , the processor 130 extracts the central region 510-1 corresponding to the first image from the first image, and the outer region ( ) corresponding to the second image from the second image. 520-2) and synthesizing the central region 510-1 corresponding to the first image and the outer region 520-2 corresponding to the second image to generate a composite image 530 of the user's face. can In this case, the entire face region of the composite image 530 may have a uniform brightness value as a whole, as shown in FIG. 5C .

In this case, the processor 130 may correct and synthesize the first region and the second region according to the motion information of the electronic device 100 sensed through the sensing unit 180 .

Specifically, when the user takes a picture of the user's face using the electronic device 100 , the electronic device 100 may shake. Accordingly, when synthesizing the first region extracted from the first image and the second region extracted from the second image, the processor 130 according to the motion information of the electronic device 100 sensed through the sensing unit 180 . At least one of the first region and the second region may be motion-corrected to be synthesized. In this case, the processor 130 may correct the first area and the second area, but this is only an example, and at least one of the first image and the second image without extracting the first area and the second area can be corrected

In addition, the processor 130 determines a brightness difference value between the first area and the second area in the vicinity of the boundary between the first area and the second area, and applies the brightness difference value to at least one of the first area and the second area based on the brightness difference value. It is possible to determine a synthesis weight for the syntheses, and to synthesize by adjusting the brightness of at least one of the first region and the second region based on the synthesis weight. For example, when the brightness value of the first area is greater than the brightness value of the second area near the boundary between the first area and the second area by a, the processor 130 sets the brightness value a as a composite weight for the second area. The first region and the second region can be synthesized by increasing by

As described above, a more accurate composite image of a user's face can be obtained through motion correction and brightness correction.

In addition, the processor 130 may control the communication unit 120 to transmit the composite image of the user's face to the external user terminal 200 .

When the composite image of the user's face is received, the user terminal 200 may display the composite image of the user's face as shown in FIG. 6A .

In addition, the user terminal 200 may analyze the skin condition of the user's face based on the composite image.

In an embodiment of the present invention, the user terminal 200 may separate the composite image into a melanin image and a hemoglobin image by using a pigment separation technique. And, the user terminal 200 may analyze the melanin image to obtain information on the degree of blemishes and skin tone. Also, the user terminal 200 may analyze the hemoglobin image to obtain information on erythema present on the user's face.

In addition, according to another embodiment of the present invention, the user terminal 200 analyzes the RGB of the composite image to obtain information on acne present on the user's face (eg, whether acne occurs, probability of occurrence of acne, etc.) can

Also, according to another embodiment of the present invention, the user terminal 200 may obtain information on sebum present on the user's face from the UV image of the composite image.

In addition, as another embodiment of the present invention, the user terminal 200 may obtain information about pores existing on the user's face (eg, whether pores exist, depth of pores, etc.) by analyzing RGB of the composite image. have.

In addition, as another embodiment of the present invention, the user terminal 200 may obtain information about wrinkles present on the user's face (eg, whether wrinkles exist, the depth of wrinkles, etc.) by analyzing the RGB of the composite image. have.

In addition, the user terminal 200 may output the obtained information on the skin state of the user's face. For example, the user terminal 200 as shown in FIG. 6B. A UI that visually provides information about the skin condition may be displayed.

In addition, the user terminal 200 may display a UI including history information on the analyzed skin condition, as shown in FIG. 6C . In addition, the user terminal 200 may display a UI that provides skin trouble prevention information and information on recommended cosmetics, as shown in FIG. 6D . Also, the user terminal 200 may provide a chatting service with a skin counselor as shown in FIG. 6E .

According to various embodiments of the present invention as described above, the accuracy of the facial skin analysis result may be increased by acquiring a face image having an even brightness distribution over the entire region of the user's face. In addition, since the user's face does not need to be inserted into the electronic device, the electronic device can be made smaller and lighter, so that the convenience of carrying and carrying can be improved.

Meanwhile, in the above-described embodiment, the electronic device 100 generates a composite image and transmits it to the user terminal 200, and the user terminal 200 analyzes the composite image to obtain information on the skin condition of the user's face. Although described, this is only an exemplary embodiment, and the electronic device 100 may directly analyze the composite image to obtain information on the skin condition of the user's face.

7 is a sequence diagram illustrating a control method of a system for analyzing a user's facial skin, according to an embodiment of the present disclosure.

First, the electronic device 100 sets a plurality of photographing parameters (S710). In this case, the electronic device 100 may set preliminary photographing parameters for obtaining an image having an optimal brightness value for each region of the user's face among the plurality of preliminary photographing parameters as the plurality of photographing parameters.

Then, the electronic device 100 and the Huda terminal 200 perform communication (S720). Meanwhile, in another embodiment of the present invention, step S710 may be performed after step S720.

Then, the electronic device 100 acquires a plurality of images by using the plurality of photographing parameters (S730). Specifically, the electronic device 100 may acquire a plurality of images by photographing a user's face based on a plurality of photographing parameters.

Then, the electronic device 100 generates a composite image (S740). Specifically, the electronic device 100 may extract a user face region corresponding to each of a plurality of images obtained by a plurality of shooting parameters, and synthesize the extracted user face regions to generate a composite image.

Then, the electronic device 100 transmits the composite image to the portable terminal 200 (S750).

Then, the user terminal 200 analyzes the composite image (S760). In particular, the user terminal 200 may obtain information (eg, acne, color tone, wrinkles, etc.) on the skin condition of the user's face by analyzing the composite image.

Then, the user terminal 200 provides information on the skin condition of the user's face (S770).

Meanwhile, in the above-described embodiment, it has been described that the electronic device 100 synthesizes a plurality of images captured by a plurality of photographing parameters into a single image, but this is only an embodiment, and the electronic device 100 provides a plurality of An image of may be acquired and transmitted to the user terminal 200 , and the user terminal 200 may synthesize a plurality of images into one image.

FIG. 8 is a flowchart illustrating a method in which the electronic device 100 transmits a composite image to the external user terminal 200 to obtain information on the skin condition of the user's face, according to an embodiment of the present disclosure.

First, the electronic device 100 acquires a plurality of images by photographing a user's face a plurality of times based on a plurality of photographing parameters (S810). In this case, a plurality of shooting parameters may be preset. For example, the electronic device 100 may obtain a first image by using the first photographing parameter and obtain a second image by using the second photographing parameter.

Then, the electronic device 100 extracts a user face region corresponding to each of a plurality of images captured with a plurality of photographing parameters ( S820 ). For example, the electronic device 100 may extract a central region of the user's face from the first image and extract an outer region of the user's face from the second image.

Then, the electronic device 100 synthesizes the extracted user face regions (S830). For example, the electronic device 100 may obtain a composite image of the user's face by synthesizing the central region extracted from the first image and the outer region extracted from the second image.

Then, the electronic device 100 transmits data on the synthesized image to the external user terminal 200 (S840).

9 is a flowchart illustrating a method for the electronic device 100 to directly acquire a skin state of a user's face, according to an embodiment of the present disclosure.

Meanwhile, since S910 to S930 illustrated in FIG. 9 are the same as S810 to S830 described with reference to FIG. 8 , overlapping descriptions will be omitted.

Then, the electronic device 100 analyzes the synthesized image to determine the skin state of the user's face (S940). Specifically, the electronic device 100 may obtain information on various skin conditions such as skin tone as well as skin troubles on the user's face (eg, acne, pores, wrinkles, etc.) by analyzing the synthesized image.

Then, the electronic device 100 provides information on the determined skin condition (S950). In this case, the electronic device 100 may provide information and services on the skin condition visually through various UIs as shown in FIGS. 6A to 6E , but this is only an example, and the skin It can provide information about status.

The methods described above may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The above hardware devices may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, although the present disclosure has been described with reference to the limited embodiments and drawings, the present disclosure is not limited to the above embodiments, and those skilled in the art to which the present disclosure pertains various modifications and variations from these descriptions. This is possible. Therefore, the scope of the present disclosure should not be limited to the described embodiments, and should be defined by the claims described below as well as the claims and equivalents.

110: camera 120: communication unit
130: processor 140: light source
150: output unit 160: memory
170: input unit 180: sensing unit

Claims (19)

In an electronic device,
a camera for photographing a user's face;
a communication unit for performing communication with an external user terminal; and
A plurality of images are obtained by photographing a user's face a plurality of times based on a plurality of photographing parameters, a first area of the user's face is extracted from a first image photographed by a first photographing parameter, and a second photographing parameter is used. Extracting a second area of the user's face from the captured second image, synthesizing the first area extracted from the first image and the second area extracted from the second image, and performing skin analysis on the user's face An electronic device comprising a; processor for controlling the communication unit to transmit the data for the synthesized image to an external user terminal for the purpose. According to claim 1,
The processor is
The user's face is photographed a plurality of times based on a plurality of preliminary photographing parameters, and brightness of a plurality of images taken based on the plurality of preliminary photographing parameters is analyzed to correspond to a plurality of user face regions among the plurality of preliminary photographing parameters and setting preliminary photographing parameters to the plurality of photographing parameters. 3. The method of claim 2,
The processor is
When an image photographed based on a first preliminary photographing parameter among the plurality of preliminary photographing parameters has a brightness value within a preset range for the first region of the user's face, the first preliminary photographing parameter is set to the image of the user's face. set as the first shooting parameter corresponding to the first area,
When an image photographed based on a second preliminary photographing parameter among the plurality of preliminary photographing parameters has a brightness value in the preset range for the second region of the user's face, the second preliminary photographing parameter is set to the value of the user's face. and setting the second photographing parameter corresponding to the second area. delete According to claim 1,
The processor is
The electronic device of claim 1, wherein the first region and the second region are corrected and synthesized according to the detected motion information of the electronic device. According to claim 1,
The processor is
A brightness difference value between the first area and the second area is determined near the boundary between the first area and the second area, and based on the brightness difference value, at least one of the first area and the second area is applied. The electronic device of claim 1, wherein the synthesis weight is determined, and the brightness of at least one of the first region and the second region is adjusted based on the synthesis weight for synthesis. 3. The method of claim 2,
The plurality of preliminary shooting parameters,
An electronic device, characterized in that it is determined according to an external lighting environment. According to claim 1,
The shooting parameters are
The electronic device according to claim 1, wherein the electronic device is at least one of an intensity of a light source provided in the electronic device, a shutter speed, and an aperture value. According to claim 1,
Display; further comprising,
The processor is
Analyzing the synthesized image to obtain skin condition information on at least one of pigmentation, acne, sebum, pores, and wrinkles on the user's face, and controlling the display to output the obtained skin condition information electronic device. A method for controlling an electronic device, comprising:
acquiring a plurality of images by photographing a user's face a plurality of times based on a plurality of photographing parameters;
extracting a first region of the user's face from a first image captured by a first shooting parameter and extracting a second region of the user's face from a second image captured by a second shooting parameter;
synthesizing the first region extracted from the first image and the second region extracted from the second image; and
and transmitting data on the synthesized image to an external user terminal for skin analysis on the user's face. ◈Claim 11 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The user's face is photographed a plurality of times based on a plurality of preliminary photographing parameters, and brightness of a plurality of images taken based on the plurality of preliminary photographing parameters is analyzed to correspond to a plurality of user face regions among the plurality of preliminary photographing parameters and setting preliminary shooting parameters to the plurality of shooting parameters. ◈Claim 12 was abandoned when paying the registration fee.◈ 12. The method of claim 11,
The setting step is
When an image photographed based on a first preliminary photographing parameter among the plurality of preliminary photographing parameters has a brightness value within a preset range for the first region of the user's face, the first preliminary photographing parameter is set to the image of the user's face. set as the first shooting parameter corresponding to the first area,
When an image photographed based on a second preliminary photographing parameter among the plurality of preliminary photographing parameters has a brightness value in the preset range for the second region of the user's face, the second preliminary photographing parameter is set to the value of the user's face. and setting the second shooting parameter corresponding to the second area. delete ◈Claim 14 was abandoned at the time of payment of the registration fee.◈ 11. The method of claim 10,
The synthesizing step is
Compensating and synthesizing the first region and the second region according to the detected motion information of the electronic device. ◈Claim 15 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The synthesizing step is
determining a difference in brightness between the first area and the second area near a boundary between the first area and the second area;
determining a composite weight for at least one of the first area and the second area based on the brightness difference value;
and synthesizing by adjusting the brightness of at least one of the first region and the second region based on the synthesis weight. ◈Claim 16 was abandoned when paying the registration fee.◈ 12. The method of claim 11,
The plurality of preliminary shooting parameters,
A control method, characterized in that it is determined according to an external lighting environment. ◈Claim 17 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
The shooting parameters are
The control method according to claim 1, wherein the control method is at least one of an intensity of a light source provided in the electronic device, a shutter speed, and an aperture value. ◈Claim 18 was abandoned when paying the registration fee.◈ 11. The method of claim 10,
obtaining skin condition information on at least one of pigmentation, acne, sebum, pores, and wrinkles on the user's face by analyzing the synthesized image; and
and outputting the acquired skin condition information. In a non-transitory computer-readable recording medium storing a program for executing a control method of an electronic device,
The control method is
acquiring a plurality of images generated by photographing a user's face a plurality of times based on a plurality of photographing parameters;
extracting a first region of the user's face from a first image captured by a first shooting parameter and extracting a second region of the user's face from a second image captured by a second shooting parameter;
synthesizing the first region extracted from the first image and the second region extracted from the second image;
determining a skin condition of a user's face by analyzing the synthesized image; and
A recording medium comprising a; providing information on the skin condition.

Images