JP7635025B2 - Skin condition measurement system - Google Patents

Description

The present invention relates to a skin condition measurement system that can provide a user with information related to the user's skin condition.

There has long been a service in which stores equipped with skin condition measuring devices and analytical equipment measure the customer's skin condition, analyze the measurement data to obtain analysis results, and then recommend and sell skin care cosmetics and beauty tools to customers that correspond to the skin condition indicated by the analysis results.

In response to this, in recent years, systems have been proposed that use small, inexpensive devices that can be used personally by the user to detect the user's skin condition with a sensor, diagnose the skin condition based on the obtained data, output the diagnosis results on a display, etc., and set up and execute electronic treatments based on the diagnosis results.
An example of such a conventional system for diagnosing skin conditions is disclosed in Japanese Patent Application Laid-Open No. 2004-141259.

JP 2004-141259 A

A conventional system for diagnosing skin conditions has the configuration shown in the above-mentioned patent document, in which a sensor is placed in contact with the user's skin, the skin condition detected by the sensor is diagnosed for the area of skin that the sensor contacts, and the diagnosis result is output, allowing the user to select an appropriate type of treatment and its execution time according to their own skin condition.

However, in the conventional systems, the detection range of the sensor is small relative to the area of skin where the skin condition is to be ascertained, and the skin that is detected by the sensor is only a part of the entire area. As a result, it is usually necessary to perform detection only on a part of the skin area where the skin condition is to be ascertained, and a diagnosis based on the detection results does not reflect the skin condition of the entire skin area, making it difficult to ensure the accuracy of the diagnosis.

The present invention has been made to solve the above-mentioned problems, and aims to provide a skin condition measurement system that obtains data related to a user's skin condition by capturing images using multiple imaging means, and improves the quality of information related to the user's skin condition that is presented to the user based on multiple sets of skin image data.

The skin condition measuring system disclosed in the present invention comprises a first imaging means capable of imaging a predetermined area of the user's body skin, a second imaging means capable of imaging a predetermined part of the user's skin narrower than the predetermined area of the body, and a control unit that acquires information related to the skin condition based on skin image data of the predetermined area of the body obtained by imaging with the first imaging means and skin image data of the predetermined part obtained by imaging with the second imaging means.

Thus, according to the disclosure of the present invention, a predetermined area of the user's body is imaged with a first imaging means, and a predetermined part of the user's skin is imaged with a second imaging means, and the control unit obtains information about the user's skin condition based on the obtained skin image data of the predetermined area of the body and the skin image data of the predetermined part, respectively. In analyzing the image data about the skin to derive information about the skin condition, the quality of the information about the user's skin condition based on the image data obtained from multiple images can be improved, and processing such as displaying and analyzing the skin condition can be performed with high accuracy, and the skin condition can be diagnosed appropriately and efficiently, allowing the user to understand the skin condition more accurately.

In addition, in the skin condition measuring system disclosed in the present invention, the second imaging means captures an image of a specific part of the skin that overlaps with the specific body area, as necessary.

As disclosed in this invention, a specific part of the user's skin imaged by the second imaging means is overlapped with a specific area of the user's body imaged by the first imaging means, and two sets of skin image data are obtained for the same specific part in the specific area of the user's body. This makes it possible to understand the relationship between the image data from the common specific parts in the image data obtained by the different imaging means, and to infer the detailed skin condition of each part of the specific area of the body other than the overlapping parts. Detailed information regarding the skin condition can be obtained by analysis using each image data, and this information regarding the skin condition allows the user to correctly understand their skin condition, further encouraging the user to take appropriate measures such as skin care.

In addition, in the skin condition measuring system disclosed in the present invention, the second imaging means may, if necessary, image a specific part of the skin that does not overlap with the specific body area.

As disclosed in this invention, the specified part of the user's skin imaged by the second imaging means does not overlap with the specified area of the user's body imaged by the first imaging means, and by obtaining two sets of skin image data, one for the specified area of the user's body and one for the specified area of the skin outside this area, it is possible to derive information regarding the skin condition by comparing these two sets of skin image data. For example, by using the skin condition based on the skin image data of the narrower specified area of the skin as a standard and comparing it with the skin condition of each part of the specified area of the body based on the skin image data of the specified area of the body, it is possible to determine the relative skin condition. Information regarding the skin condition can be easily obtained by analysis using the two sets of skin image data, allowing the user to reliably understand their skin condition and further encouraging the user to take appropriate care of the skin in the specified area of the body.

In addition, in the skin condition measuring system disclosed in the present invention, the imaging of the specified area of the body by the first imaging means and the imaging of the specified part of the skin by the second imaging means are performed at different times, as necessary.

As disclosed in this invention, the imaging of a predetermined area of the user's body by the first imaging means and the imaging of a predetermined part of the user's skin by the second imaging means are performed at different times, eliminating the need for devices constituting each imaging means to be present around the user at the same time and perform each imaging separately. This eliminates the need to place the imaging means separately as devices for simultaneous imaging, and makes it possible to reuse devices, for example, by imaging each imaging target sequentially with an imaging device in which imaging means are arranged together, thereby minimizing the number of imaging devices and reducing device costs.

In addition, in the skin condition measuring system disclosed in the present invention, the first imaging means and the second imaging means are arranged in a specific terminal device, as necessary.

As described above, according to the disclosure of the present invention, by providing the first imaging means together with the second imaging means in one terminal device and being able to perform imaging using both the first imaging means and the second imaging means on one terminal device, imaging of the skin of the specified area of the user's body and imaging of the specified part of the skin can be handled by one terminal device, and the cost associated with imaging can be reduced by using only one imaging device.

In addition, in the skin condition measuring system disclosed in the present invention, the first imaging means can be switched between a state of imaging the skin of the specified area of the body and a state of imaging the specified part, as necessary, and also serves as the second imaging means.

As described above, according to the disclosure of the present invention, a first imaging means that can be switched between a state of imaging the specified body area and a state of imaging the specified part is used, and this first imaging means also serves as a second imaging means. By imaging the specified body area and the specified part with one imaging means in one terminal device and acquiring image data for each, imaging can be performed with a minimum necessary configuration, and information related to skin condition can be acquired at low cost.

In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the first imaging means is a specified imaging device that is part of or connected to the user's terminal device and captures at least the specified range of the user's body, image data obtained by imaging can be transmitted to another person's terminal device, communication is performed between the user's terminal device and the other person's terminal device, and an image of the user including the specified range of the body is displayed on the other person's terminal device based on the transmitted image data of the user, while the control unit obtains information related to the skin condition based on the skin image data of the specified range of the body obtained by imaging with the first imaging means and the skin image data of the specified part obtained by imaging with the second imaging means.

Thus, according to the disclosure of the present invention, image data of the user is transmitted to the other person's terminal device from a first imaging means constituting an imaging device that is part of the user's terminal device or that is connected to this terminal device, and while the user's image is displayed on the other person's terminal device, the control unit acquires information regarding the skin condition based on the skin image data of the specified body area and the specified part, and in some cases, it is possible to handle not only the user's image but also information regarding the skin condition in communication between the user's terminal device and the other person's terminal device, thereby making it possible to refer to information regarding the user's skin condition while performing communication involving image display between the user's terminal device and the other person's terminal device, such as a video call, and to display the user's image on the other person's terminal device, and to utilize the information regarding the skin condition while communicating with the user via communication.

In addition, in the skin condition measuring system disclosed in the present invention, if necessary, the first imaging means is a specified imaging device that is part of or connected to another person's terminal device and captures images of at least the specified range of the user's body, and while the image data of the user obtained by the first imaging means is used on the other person's terminal device for a main purpose other than obtaining information regarding the user's skin condition, the control unit obtains information regarding the skin condition based on the skin image data of the specified range of the body obtained by imaging with the first imaging means and the skin image data of the specified part obtained by imaging with the second imaging means.

Thus, according to the disclosure of the present invention, a user is imaged with a first imaging means constituting an imaging device that is part of another person's terminal device or that is connected to this terminal device, and the user's image data is used for a specified purpose in the other person's terminal device, while the control unit acquires information regarding the skin condition based on skin image data of a specified area of the body from the first imaging means and skin image data of the specified part from the second imaging means, and this information regarding the skin condition is made available. By making it possible to use the image data captured and acquired for purposes such as personal authentication in an imaging device used together with the other person's terminal device, for skin diagnosis, there is no need to prepare a new first imaging means, and information regarding the skin condition based on each image data can be acquired efficiently at low cost.

In addition, in the skin condition measurement system disclosed in the present invention, the first imaging means is a predetermined imaging device that is part of or connected to the user's terminal device and captures at least the predetermined area of the user's body, as necessary, the second imaging means is operable to capture images by the user's operation, the control unit first acquires information related to the skin condition based on skin image data of the predetermined area of the body obtained by imaging with the first imaging means, and uses the information to set a predetermined area of the skin to be imaged with the second imaging means and notifies the user of the information of the set predetermined area via the user's terminal device, and the control unit acquires skin image data of the predetermined area in conjunction with the imaging of the predetermined area of the skin with the second imaging means, and then acquires information related to the skin condition again based on the skin image data of the predetermined area of the body and the skin image data of the predetermined area.

In this way, according to the disclosure of the present invention, the user is imaged by a first imaging means which is part of the user's terminal device or an imaging device connected to this terminal device, and the control unit first acquires information about the skin condition based on the obtained skin image data of the predetermined area of the user's body, and then sets the predetermined area of the skin to be imaged by the second imaging means based on this information, and the information about this predetermined area is notified to the user from the user's terminal device, causing the user to image the predetermined area of the skin with the second imaging means. When skin image data of the predetermined area is obtained by imaging with this second imaging means, the control unit acquires information about the skin condition again and makes this information about the skin condition available, so that the predetermined area of the skin to be imaged by the second imaging means is set according to the skin condition of the predetermined area of the user's body. First, an area where more detailed information about the skin condition is required, such as a problem with the skin condition such as a blemish or a sign of a problem, is extracted, and the corresponding area is imaged by the second imaging means as the predetermined area of the skin. Using the obtained skin image data, more detailed information about the skin condition focusing on the problem area can be acquired, and the user can properly grasp the skin condition.

In addition, in the skin condition measuring system disclosed in the present invention, the control unit, as necessary, acquires information regarding the skin condition of the specified part of the skin based on skin image data of the specified part obtained by imaging with the second imaging means, and then acquires, as information regarding the skin condition, information regarding the relative skin condition of each part of the specified range of the body based on the information regarding the skin condition of the specified part.

In this way, according to the disclosure of the present invention, the second imaging means captures an image of a predetermined part of the skin that does not overlap with the predetermined body area, and the control unit acquires information about the skin condition of the predetermined part of the skin based on the obtained skin image data of the predetermined part. Furthermore, the control unit acquires information about the skin condition based on the skin image data of the predetermined body area as relative skin condition information for each part of the predetermined body area based on the information about the skin condition of the predetermined part, and makes the information about the skin condition available. This makes it possible to show the skin condition of the predetermined body area of the user relatively based on the skin condition of the predetermined part. If the predetermined part of the skin is a part that is suitable as a standard for evaluation, for example, a part that is hidden from sunlight such as the surface of the arm close to the torso or the underside of the chin, which is less affected by ultraviolet rays and has a small degree of deterioration, the skin condition of each part of the predetermined body area can be shown as the degree of deterioration from the reference part, and the skin condition can be expressed in a form that is easy for the user to understand, allowing the user to more accurately grasp the skin condition.

In addition, in the skin condition measuring system disclosed in the present invention, as necessary, the first imaging means captures a mirror image of the specified area of the body reflected on a mirror surface located in front of the user, and the second imaging means captures an image of a specified part of the user's skin simultaneously with the image captured by the first imaging means, and the control unit acquires information related to the skin condition based on skin image data of the specified area of the body obtained by capturing the mirror image by the first imaging means and skin image data of the specified part obtained by capturing the image by the second imaging means.

Thus, according to the disclosure of the present invention, a first imaging means captures an image of the mirror reflected on a mirror surface in front of the user to obtain skin image data of the specified area of the body, and at the same time, a second imaging means captures an image to obtain skin image data of the specified area, and a control unit obtains information related to the skin condition from each piece of skin image data, making the information related to the skin condition available. By capturing images simultaneously with each imaging means, skin image data of the specified area of the body and skin image data of the specified area can be obtained at once, shortening the time from capturing images to obtaining information related to the skin condition, reducing the burden on the user related to capturing images, encouraging the user to continue activities such as capturing images to obtain information related to the skin condition, and obtaining information related to the skin condition without wasting time, making it possible to quickly grasp the user's skin condition.

In addition, the skin condition measuring system disclosed in the present invention is capable of capturing an image of the specified part of the user with the first imaging means disposed on one surface of a specified terminal device, the second imaging means disposed on the terminal device, and the first imaging means on the terminal device facing the mirror surface, as required.

Thus, according to the disclosure of the present invention, a first imaging means is disposed on one surface of a specified terminal device, and this first imaging means is directed toward a mirror surface and capable of imaging the specified part of the user, and a second imaging means is disposed in the same terminal device, and one device can handle imaging of the specified area of the user's body and the specified part, so that skin image data of the specified area of the body and skin image data of the specified part can be obtained together by imaging with each imaging means of the terminal device, and when imaging, the user only needs to handle the terminal device equipped with each imaging means, which significantly reduces the burden on the user related to imaging and allows the user to continue activities to obtain information about their skin condition without difficulty.

In addition, in the skin condition measuring system disclosed in the present invention, the control unit records information on the specified area of the skin to be imaged by the set second imaging means as necessary, and when setting a new specified area of the skin to be imaged by the second imaging means, the control unit uses information on the skin condition that was once acquired based on the skin image data of the specified area of the body, and refers to the information on the recorded specified area to set the specified area from among the areas of the specified area of the body that have not been recorded as a specified area.

In this way, according to the disclosure of the present invention, the control unit first acquires information regarding the skin condition based on skin image data of the specified range of the user's body obtained by imaging with the first imaging means, and when setting the specified area of the skin to be imaged by the second imaging means based on the acquired information, the control unit records the information of the set specified area for each such setting and makes the information of the set specified area available for later reference. If recorded information of a specified area exists, at a new imaging opportunity with the first imaging means, in addition to the information regarding the skin condition acquired based on the skin image data of the specified range of the body, the control unit sets the specified area of the skin to be imaged by the second imaging means from among the areas of the specified range of the body that have not been recorded as specified areas based on the recorded information of the specified area. In this way, the control unit always images the areas of the specified range of the user's body that have not yet been imaged by the second imaging means as the specified area of the skin to be imaged by the second imaging means. When the set specified area of the skin is imaged with the second imaging means, detailed information regarding the skin condition centered on the new specified area can be acquired using the obtained skin image data, and the user can properly grasp the skin condition. Furthermore, when imaging is performed multiple times with the second imaging means for each imaging with the first imaging means, skin image data of different areas without overlapping is obtained for each imaging with the second imaging means, making it possible to obtain more detailed and accurate information regarding the skin condition based on the skin image data.

In addition, in the skin condition measuring system disclosed in the present invention, the control unit, as necessary, records information about the specific part of the skin to be imaged by the second imaging means, as well as skin image data obtained by imaging the specific part by the second imaging means, in such a manner that the date and time when the skin image data was obtained can be determined, and in parallel with setting the specific part of the skin to be imaged by the second imaging means at a new imaging opportunity with the first imaging means, the control unit refers to the recorded information about the specific part and skin image data, and acquires information about the skin condition based on the recorded skin image data of the specific part whose data acquisition date and time falls within a past specified period corresponding to the turnover cycle of the user's skin, the skin image data of the specific part of the skin obtained by imaging the specific part of the skin set as the part to be imaged by the second imaging means, and the skin image data of the specific area of the body.

In this way, according to the disclosure of the present invention, the control unit records information about the specific part of the skin to be imaged by the second imaging means, and records the skin image data obtained by imaging the set specific part by the second imaging means. When the newly set specific part of the skin is imaged by the second imaging means to obtain skin image data of the specific part, the information about the recorded specific part and the skin image data are referenced, and the skin image data of the specific part, whose data acquisition date and time are included in a past specific period corresponding to the turnover cycle of the user's skin, is considered to have been imaged at the same timing as the new image. Information about the skin condition is obtained based on the recorded skin image data acquired during the specific period, the skin image data of the specific part obtained by the latest image, and the skin image data of the specific range of the body. Each time the specific part of the skin is imaged by the second imaging means within the specific period corresponding to the turnover cycle of the skin, the skin image data of the specific part is recorded sequentially, and a state in which each part of the specific range of the body is widely covered is obtained. The information about the skin condition obtained based on these and the skin image data of the specific range of the body reflects the user's skin condition in detail and with good accuracy, allowing the user to properly grasp the skin condition.

In addition, in the skin condition measuring system disclosed in the present invention, while the control unit records information on the specific area of the skin to be imaged by the second imaging means and skin image data of the specific area obtained by imaging the specific area by the second imaging means, the control unit deletes, as necessary, recorded skin image data obtained before the previous specific period corresponding to the turnover cycle of the user's skin and recorded information on the specific area corresponding to the skin image data for each new imaging opportunity.

As described above, according to the disclosure of the present invention, by deleting skin image data recorded by the control unit from a predetermined period before the time when new imaging is to be performed, which corresponds to the skin turnover cycle, accurate information regarding the skin condition can be derived using only skin image data from a period considered to be the same skin condition, together with skin image data obtained by imaging with the second imaging means, and the amount of recording can be limited to reduce the burden associated with recording.

1 is a schematic configuration diagram of a terminal device in a skin condition measuring system according to a first embodiment of the present invention. FIG. 1 is a block diagram showing the configuration of a skin condition measuring system according to a first embodiment of the present invention. 4A to 4C are explanatory diagrams illustrating a state in which a conversion lens is attached to a second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. 11 is an explanatory diagram of a display screen of the terminal device immediately before capturing an image of a first imaging unit in the skin condition measuring system according to the first embodiment of the present invention. FIG. FIG. 11 is an explanatory diagram of a display screen of the terminal device immediately before capturing an image by the second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. 5A to 5C are explanatory diagrams illustrating an imaging state by a second imaging unit in the skin condition measuring system according to the first embodiment of the present invention. 3 is an explanatory diagram showing a display state of diagnostic information of a skin condition based on images captured by each imaging unit in the skin condition measuring system according to the first embodiment of the present invention; FIG. 13 is an explanatory diagram of the display screen of the terminal device immediately before capturing an image by the second imaging unit at a new imaging opportunity in the skin condition measuring system according to the first embodiment of the present invention. FIG. 10A to 10C are explanatory diagrams illustrating an image capturing state by another terminal device in the skin condition measuring system according to the first embodiment of the present invention. FIG. 11 is a schematic configuration diagram of a skin condition measuring system according to a second embodiment of the present invention. 13 is an explanatory diagram of information transmission and reception between a terminal device and a server in a skin condition measuring system according to a second embodiment of the present invention. FIG. FIG. 11 is an explanatory diagram of the state of information transmission and reception between each terminal device of a user and another person and a server in a first alternative example of a skin condition measuring system according to a second embodiment of the present invention. 13 is an explanatory diagram of an imaging state of another person's terminal device having a first imaging unit in a second alternative example of the skin condition measuring system according to the second embodiment of the present invention. FIG. FIG. 11 is an explanatory diagram of the state of information transmission and reception between each terminal device and the server in a second example of the skin condition measuring system according to the second embodiment of the present invention. 13A to 13C are explanatory diagrams illustrating an imaging state by a second imaging unit in a skin condition measuring system according to a third embodiment of the present invention. 13 is an explanatory diagram of a display screen of a terminal device immediately before imaging by each imaging unit in another example of the skin condition measuring system according to the third embodiment of the present invention. FIG. 13 is an explanatory diagram of a usage state of a terminal device in a skin condition measuring system according to a fourth embodiment of the present invention. FIG. FIG. 13 is a block diagram showing the configuration of a skin condition measuring system according to a fourth embodiment of the present invention. 13A to 13C are explanatory diagrams illustrating image capturing states by each image capturing unit in a skin condition measuring system according to a fourth embodiment of the present invention. 13A and 13B are explanatory diagrams of an image capturing state by another terminal device in the skin condition measuring system according to the fourth embodiment of the present invention.

(First embodiment of the present invention)
A skin condition measuring system according to a first embodiment of the present invention will be described below with reference to FIGS.
In each of the above figures, the skin condition measuring system 1 of this embodiment comprises a first imaging unit 11 as the first imaging means capable of imaging the skin of a specified area of the user's body, a second imaging unit 12 as the second imaging means capable of imaging a specified part of the user's skin narrower than the specified area of the body, and a control unit 13 that acquires information regarding the skin condition based on skin image data of the specified area of the body obtained by imaging with the first imaging unit 11 and skin image data of the specified part obtained by imaging with the second imaging unit 12, and the first imaging unit 11, second imaging unit 12, and control unit 13 are configured to be arranged in a single terminal device 10.

In this embodiment, the user who is the subject of skin condition measurement enters into a contract to receive skin condition diagnosis services with a business that provides skin condition diagnosis services using the skin condition measurement system 1 (hereinafter, referred to as the provider).

As a result, the user can receive a skin condition diagnosis using the skin condition measuring system 1, obtain the skin condition diagnosis results by the skin condition measuring system 1, and, if necessary, enjoy the benefits of obtaining advice information according to the diagnosis results. In addition, the provider can obtain data regarding the user's skin condition, and can enjoy the benefit of realizing a sales promotion effect for skin care products and skin care services provided by a specified provider (e.g., a sponsor of the provider) by recommending these products to the user through advertising, etc. Skin care products are products used for skin care, such as lotions, beauty serums, supplements, facial massagers, and facial cleansers. Skin care services are services that provide skin care, such as aesthetic services, etc.

The first imaging unit 11 is disposed in the terminal device 10 as the first imaging means, and is capable of capturing an image of a predetermined area of the user's body, such as the face, to measure the user's skin condition.

The second imaging unit 12, as the second imaging means, is arranged in the same terminal device 10 as the first imaging unit 11, and is capable of imaging a specified part of the user's skin, such as a part of the cheek or forehead, which is narrower than the specified range of the user's body and overlaps with this specified range of the body, in order to measure the user's skin condition.
This second imaging unit 12 is equipped with a conversion lens 19 so that it is possible to obtain image data having a higher resolution per unit area of the imaging target than the first imaging unit 11 .

The first imaging unit 11 and the second imaging unit 12 are capable of performing imaging by user operation via the terminal device 10. The imaging of the specified area of the body by the first imaging unit 11 and the imaging of the specified part of the skin by the second imaging unit 12 are performed at different times and with different relative positional relationships between the imaging units and the imaging subject.

Note that the first imaging unit 11 and the second imaging unit 12 are each part of the user's terminal device 10, but this is not limited thereto, and either or both of the first imaging means and the second imaging means may be independent imaging devices that are connected to the user's terminal device, perform imaging, and obtain skin image data.

The second imaging unit 12 is one of multiple camera units in a terminal device 10 such as a smartphone that can capture an image of the user's skin, and is not limited to a configuration in which a conversion lens 19 suitable for capturing images of the skin is attached to the device, but can also be a camera unit in a terminal device such as a smartphone that can directly capture an image of a specific part of the skin, without requiring a conversion lens.

The terminal device 10 is configured to have a first imaging unit 11 that constitutes the first imaging means and a second imaging unit 12 that constitutes the second imaging means. In addition, the terminal device can be configured to have one imaging unit that can switch between a state of imaging the skin of the specified area of the user's body and a state of imaging the specified part. In other words, the imaging unit that is the first imaging means that can image the skin of the specified area of the user's body can also be configured to image the specified part as necessary, and serve as the second imaging means.

In addition, the second imaging unit 12 constituting the second imaging means is configured to be provided in the same terminal device 10 as the first imaging unit 11 constituting the first imaging means, but it is also possible to provide the second imaging means separately from the first imaging means, for example, to use another device having a camera unit constituting the second imaging means in combination with a terminal device having a camera unit constituting the first imaging means.

In addition, the second imaging unit 12 constituting the second imaging means is configured to be provided on a different surface of the terminal device 10 from the first imaging unit 11 constituting the first imaging means, but the second imaging unit 12 and the first imaging unit 11 may be provided on the same surface of the terminal device 10, in other words, one of multiple imaging units provided on the same surface of the terminal device may be the first imaging means and the others may be the second imaging means.

The second imaging unit 12, which constitutes the second imaging means, can capture images at a position closer to the user than the first imaging unit 11, which constitutes the first imaging means, and can be configured so that when the second imaging unit 12 is brought closer to the user together with the terminal device 10, or when the user approaches the second imaging unit 12 and the distance between the second imaging unit 12 and the specified part of the skin reaches a specified value, the second imaging unit 12 automatically captures an image of the specified part of the skin.

In addition, one or more lighting means that irradiate light in conjunction with each of the first imaging unit 11 and the second imaging unit 12 can be provided, and lighting corresponding to when a predetermined area of the body is imaged by the first imaging unit 11 and lighting corresponding to when a predetermined part is imaged by the second imaging unit 12 can be performed, respectively. Furthermore, the lighting means corresponding to the second imaging unit 12 can be configured to irradiate unpolarized light as lighting for detecting the texture of the user's skin when imaging the predetermined part of the user, and to irradiate linearly polarized light as lighting for detecting blemishes on the user's skin.

In this way, by capturing images by selectively irradiating unpolarized light and linearly polarized light as illumination for the second imaging unit 12, skin texture detection and blemish detection can be performed appropriately. Alternatively, the illumination may be configured to irradiate circularly polarized light, and specifically, right-handed circularly polarized light may be used as illumination for detecting skin texture, and left-handed circularly polarized light may be used as illumination for detecting blemishes on the skin. Alternatively, the color (wavelength) of the light irradiated as illumination may be made to correspond to skin texture detection and blemish detection, respectively. For example, a light source capable of irradiating skin-colored light for texture detection and white light for blemish detection may be provided in conjunction with the second imaging unit 12.

The terminal device 10 has the first imaging unit 11 and the second imaging unit 12, and is a device capable of capturing images of a predetermined area of the user's body and the predetermined part of the body, and acquiring skin image data, and also has the control unit 13.

The terminal device 10 is a known portable terminal device such as a mobile phone, a so-called smartphone, or a tablet device that has wireless communication means 14 for connecting to and communicating with an IP communication network, and among these, it has a display unit 16 that can execute application programs on the device and display various information. The terminal device 10 is equipped with two camera units that can function as the first imaging unit 11 and the second imaging unit 12.

The display unit 16 of the terminal device 10 is capable of displaying an image representing at least the predetermined area of the user's body. The display unit 16 is a touch panel type display device and also serves as an input device.

The terminal device 10 is an information processing device that realizes the functions and processing of the control unit 13 by executing a pre-stored program, and is not limited to a portable device as long as it is equipped with the first imaging unit 11 and the second imaging unit 12 and can measure the skin condition, and may be a computer such as a notebook type or desktop type with a camera.

The control unit 13 of the terminal device 10 is capable of analyzing the skin image data obtained by imaging using an application program for diagnosing a skin condition, and acquiring diagnostic information, which can then be referred to by the user. In this case, the control unit 13 of the terminal device 10 executes the application program to display the acquired diagnostic information as content on the display unit 16, making it possible for the user to refer to the information.

The control unit 13 acquires information regarding the skin condition based on the skin image data obtained by imaging, i.e., the skin image data of the specified area of the body obtained by imaging with the first imaging unit 11 and the skin image data of the specified part obtained by imaging with the second imaging unit 12.

The control unit 13 analyzes the skin image data of the specified body area and the skin image data of the specified part to obtain diagnostic information on the skin condition of the specified body area of the user as information on the skin condition. The function of the control unit 13 to analyze each piece of skin image data and obtain diagnostic information on the user's skin condition is realized by the control unit 13 executing processing according to an application program.

As part of the above process, the control unit 13 evaluates the user's skin based on the skin image data, and determines an index value indicating the user's skin condition. For example, the control unit 13 evaluates at least one of the following items as the skin condition: skin age, fineness of skin texture, degree of skin whiteness, degree of openness of skin pores, amount of skin blemishes, amount of moisture contained in the skin (degree of moisturization), amount of oil contained in the skin, etc., and determines an index value indicating the skin condition from the evaluation result.

In this embodiment, the control unit 13 performs predetermined image processing on the images captured by the first imaging unit 11 and the second imaging unit 12 to obtain an index value indicating the skin condition of the user.
For example, the control unit 13 performs edge detection on the captured image, detects skin grooves in the image, identifies the area of the skin ridge surrounded by the detected skin grooves, and calculates an index value indicating the fineness of the skin texture (index value indicating the skin condition) such that the smaller the size of the identified skin ridge area is and the more uniform it is, the larger the index value will be.

In this embodiment, the control unit 13 determines, as the diagnostic information, information indicating that the diagnostic result has been obtained as an index value indicating the user's skin condition. The control unit 13 then performs control so that the acquired diagnostic information can be appropriately displayed on the display unit 16 as the display content of the display unit 16. For example, the control unit 13 can create a graph for some of the index values indicating the user's skin condition from the diagnostic result, and display this on the display unit 16 as the display content.

The control unit 13 also receives data including information on the skin condition, skin care action information, skin images, and the time when the skin was imaged on the terminal device 10, associates the received index value indicating the skin condition, the skin care action information, the skin image, and the time when the skin was imaged, records the data temporarily in the recording unit 17 for recording, and can diagnose the user's skin condition based on the recorded index value of the skin condition and the time when the skin was imaged.

Specifically, the control unit 13 identifies the change over time in the skin condition index value based on the skin condition index value recorded in the recording unit 17 and information on the time when the skin condition measurement was performed, and diagnoses whether the skin condition has improved based on the identified change in the nature of the change. For example, the control unit 13 identifies the change over time in the skin condition index value during the period of the user's skin metabolism (turnover) immediately before processing (for example, about 28 days ± 1 to 7 days), and diagnoses the skin condition.

The control unit 13 may also identify the time-dependent change in the index value of the skin condition during a period different from the period required for the user's skin turnover immediately prior to the treatment (e.g., a period of a predetermined length immediately prior to the treatment (e.g., one week, ten days, one month, etc.), a specified period, etc.) and diagnose the skin condition.

When the skin condition index value changes over time in a direction that indicates an improvement in the skin condition, the control unit 13 gives a diagnosis result that the user's skin condition has improved. On the other hand, when the skin condition index value does not change over time in a direction that indicates an improvement in the skin condition (changing in a direction that indicates deterioration or showing no change), the control unit 13 gives a diagnosis result that the user's skin condition has not improved.

The control unit 13 may also perform a skin condition diagnostic process that takes into account the user's attributes, such as age. In addition, the control unit 13 may change the diagnostic criteria for some or all of the diagnostic information based on the analysis of the skin image data of the specified area obtained by the second imaging unit 12, depending on the location imaged by the second imaging unit 12.

The control unit 13 controls the display unit 16 of the terminal device 10 to appropriately display images including graphs and text showing diagnostic information. In addition, the control unit 13 may control the display unit 16 to appropriately display images including a progression graph showing the progression of skin condition, a group of images that are skin images, a group of action information showing information on skin care actions corresponding to the skin images, and an advertisement showing an advertisement (with a link) for a specific skin care product or skin care service.

The control unit 13 may also create content indicating the diagnosis results as part of the diagnosis information, and display the content on the display unit 16 so that the content can be referenced from the terminal device 10. For example, the control unit 13 may create content including bar objects indicating index values of the skin condition for each of the degree of moisturization, the degree of pore opening, the degree of whitening, and the skin age (aging), and display the content on the display unit 16.

In addition, the control unit 13 acquires, as other information related to the skin condition, a display image of a specified area of the user's body based on skin image data of the specified area of the body obtained by imaging with the first imaging unit 11, and a display image of the specified area of the user based on skin image data of the specified area of the skin obtained by imaging with the second imaging unit 12.

Each time a new image is captured by the first imaging unit 11 and the second imaging unit 12, the control unit 13 obtains a display image of the specified area or part of the user's body based on skin image data of the specified area or part of the body obtained by the image capture, and causes the display unit 16 to display this display image.

The control unit 13 can also associate a portion of information in the display image of the specified body area that corresponds to the specified part of the skin with the display image of the specified part, and display the display image of the specified part simultaneously with the display of a part or all of the display image of the specified body area on the display unit 16.

The control unit 13 can also perform image correction processing (e.g., color calibration) on a part or all of the display image of the specified body range derived from the image data of the specified body range, based on the display image of the specified part derived from the image data of the specified part.

Furthermore, the control unit 13 can execute a process for synthesizing a predetermined guide design portion, which indicates that the position is to be imaged by the second imaging unit 12, on the display image of the predetermined body range derived from the image data of the predetermined body range, and display the processed image on the display unit 16. As a result, the guide design portion is displayed as part of the display image of the predetermined body range displayed on the display unit 16, and the user can visually recognize and understand the position in the predetermined body range that should be imaged by the second imaging unit 12.

The control unit 13 also first acquires diagnostic information on the skin condition of the specified area of the user's body as information on the skin condition based on skin image data of the specified area of the body obtained by imaging with the first imaging unit 11, and uses this diagnostic information to set a specified area of the skin that is included in the specified area of the body and should be imaged by the second imaging unit 12. The control unit 13 can notify the user of the information on the set specified area via the user's terminal device 10.

Then, when the second imaging unit 12 captures an image of the specified area of the skin based on the user's operation referring to the information on the specified area, skin image data of the specified area is obtained. The control unit 13 then acquires diagnostic information on the skin condition of the specified area of the body again based on the skin image data of the specified area of the body and the skin image data of the specified area.

In setting the predetermined part of the skin to be imaged by the second imaging unit 12 by the control unit 13, the AI may be used to suggest the predetermined part of the skin to be imaged from the current overall face image, which is known from the diagnostic information of the skin condition once acquired based on the facial skin image data as the predetermined body range. In that case, in addition to the diagnostic information, the necessary information can be appropriately obtained by the user's information input, information input based on cooperation with other IOT devices, or access to so-called big data on the network. For example, the past and present wrinkles and age of wrinkles and age spots of the user, as well as future predictions of wrinkles and age spots, can be obtained and used. In addition, statistical data based on the location where the user lives (e.g., Okinawa, Hokkaido, etc.) and weather-related information based on the GPS of the area where the user lives (e.g., ultraviolet light amount, temperature, humidity, weather, etc.) can be obtained. Furthermore, various types of information may be obtained and used as information about the user, such as information based on GPS on whether the user spends more time outdoors or indoors (proportion), information about the user's lifestyle (e.g., sleep, exercise, bathing, stress, etc.), the user's biometric information (blood pressure, body temperature, blood flow, electrocardiogram, etc.), the user's eating habits (likes fried food, dislikes vegetables, etc.), information about cosmetics and skin care devices used by the user, genetic information related to the user (e.g., age spots and wrinkles on the grandparents' or parents' skin), the user's skin color, skin temperature, etc. (including temporary changes due to exposure to ultraviolet rays, etc.), and information related to improving the user's facial impression (e.g., physiognomy, facial muscles, smile, etc.).

When using AI to suggest specific areas of the skin, it is particularly capable of suggesting areas (including hidden blemishes) that are likely to develop spots or wrinkles in the future based on various information, suggesting areas that will improve the impression of the face if improved through care, and suggesting areas where favorable results can be expected if improvements are started from now, so that the user can remain beautiful several years or decades into the future.

In addition, the control unit 13 may use AI to suggest effective skin care and makeup plans at the stage where it has roughly grasped blemishes, wrinkles, dark circles under the eyes, freckles, etc. on the entire face as diagnostic information of the user's facial skin condition based on the facial skin image data as the specified body area obtained by imaging with the first imaging unit 11, for example, suggesting that blemishes in certain areas should be treated with cosmetics, and blemishes in other areas should be covered with foundation for the time being since they cannot be improved immediately.

The control unit 13 has a recording unit 17 that records information about a specific area of the skin to be imaged by the second imaging unit 12 when the control unit 13 sets the specific area of the skin to be imaged by the second imaging unit 12 using diagnostic information about the skin condition of the specific area of the body that has been acquired based on skin image data of the specific area of the body.

The recording unit 17 records information about the specific area of the skin to be imaged by the second imaging unit 12, as well as skin image data obtained by imaging this specific area by the second imaging unit 12, in a manner that makes it possible to determine the date and time that this skin image data was obtained.

When setting a new specific area of the skin to be imaged by the second imaging unit 12, the control unit 13 uses the diagnostic information on the skin condition that was previously acquired based on the skin image data of the specific area of the body that was previously acquired, and if information on a previously set specific area has been recorded in the recording unit 17, it also refers to the information on the specific area already recorded in the recording unit 17 and sets the specific area from among the areas of the specific area of the body that have not been recorded as a specific area.

When imaging is performed multiple times with the second imaging unit 12 for each imaging with the first imaging unit 11 to obtain skin image data for multiple specific areas, the settings of the specific areas of the skin to be imaged are repeated for each imaging with the second imaging unit 12 so that they do not overlap with each other. In addition, for each imaging with the second imaging unit 12, information on the set specific areas is notified to the user via the terminal device 10 and is recorded in the recording unit 17 to be reflected in the next settings.

The control unit 13 also has a mechanism for referencing information on the specific area of the skin that has already been recorded in the recording unit 17 and skin image data of the specific area, in parallel with setting a specific area of the skin to be imaged by the second imaging unit 12 during a new imaging opportunity by the first imaging unit 11.

The control unit 13 then regards the skin image data of the recorded specific area, whose data acquisition date and time falls within a specific period of the past that corresponds to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture.

In this case, the control unit 13 obtains diagnostic information (information related to the skin condition) of the specified area of the body based on the recorded skin image data acquired during the specified period, the skin image data of the specified area of the skin set at this new imaging opportunity obtained by imaging the specified area by the second imaging unit 12, and the skin image data of the specified area of the body.

On the other hand, the control unit 13 may delete from the recording unit 17 the information on the specified area and the skin image data of the specified area, if the data acquisition date and time is not included in the specified period and is older than the turnover cycle of the user's skin, since the data is from a time when the skin turnover does not overlap with the user's skin in the new image capture opportunity by the first imaging unit 11 and cannot be considered to have been captured at the same time as the new image capture, and so the corresponding information on the specified area and the skin image data of the specified area are deleted from the recording unit 17.

Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, the entire face of the user is set as the predetermined body range to be imaged by the first imaging unit 11, and the predetermined part imaged by the second imaging unit 12 is the main part of the skin of the face (cheeks, eyes, mouth, etc.). The terminal device 10 is a smartphone, and the camera unit (so-called in-camera) provided on the side where the display unit 16 is located is used as the first imaging unit 11, while the camera unit (so-called out-camera) provided on the opposite side of the display unit 16 of the terminal device 10 is used as the second imaging unit 12, so that each can perform imaging. When imaging the predetermined part, a conversion lens 19 is attached in front of the second imaging unit 12 in the terminal device 10, so that the imaging range of the second imaging unit 12 is narrower than that of the first imaging unit 11, while imaging can be performed at a higher resolution, and an extremely narrow area of the skin of the face is imaged as the predetermined part.

In order to capture an image using each imaging unit of the terminal device 10, first, the user attaches the conversion lens 19 to the terminal device 10.
When a user starts an application program on the terminal device 10 that already exists in the terminal device and provides a user interface for using each imaging unit, the user can use each imaging unit for imaging through the terminal device 10.

When the user performs an operation to start capturing images on the user interface displayed on the terminal device 10, the first imaging unit 11 of the terminal device 10 captures an image of the face as a specified area of the user's body.

In detail, when the terminal device 10 executes an application program for measuring skin condition, the user operates the user interface displayed on the display unit 16, and first, the first imaging unit 11 as the first imaging means starts imaging the user's face, and the control unit 13 of the terminal device 10 continuously acquires image data. In this case, the control unit 13 of the terminal device 10 causes the user to maintain a state in which the terminal device 10 is held so that the face is displayed in a predetermined orientation and size by a predetermined guide display on the display unit 16. For example, the control unit 13 can display on the display unit 16 an image of a frame that fits the contours of the face on the screen of the display unit 16 in a state where the face is correctly imaged, superimposed on the image captured by the first imaging unit 11, and prompt the user to adjust the relative positional relationship between the first imaging unit 11 and the face, using the image of the frame as a guide while looking at the display unit 16, so that the face fits within it.

When the user's face is captured by the first imaging unit 11, the control unit 13 causes the display unit 16 to display an image of the face based on the image data obtained by the imaging. The control unit 13 causes the display unit 16 to display a predetermined guide display or the like, to have the user maintain a state in which the terminal device 10 is held so that the face is displayed in a predetermined orientation and size.
Furthermore, as the user's face is imaged by the first imaging unit 11 of the terminal device 10, an image of the face is displayed on the display unit 16 based on image data obtained by imaging.

The user refers to the facial image displayed on the display unit 16, and performs a specified operation on the user interface of the terminal device 10 at a timing when the face as a specified body area is deemed to be appropriately captured by the first imaging unit 11, to determine a still image of the face. The data of the still image of the face at this determination moment is handled by the control unit 13 as skin image data of the specified body area obtained by imaging with the first imaging unit 11. Note that the control unit 13 may automatically determine the facial image at a timing when the face as a specified body area is appropriately captured by the first imaging unit 11, regardless of the user's operation.

When the skin image data based on the image of the face as the specified body area by the first imaging unit 11 is thus finalized, the second imaging unit 12 may transition to a state suitable for imaging the specified area of the skin.

The control unit 13 acquires a display image of the face based on the skin image data obtained by the first imaging unit 11, and displays this on the display unit 16 while analyzing the skin image data and temporarily acquiring diagnostic information on the user's facial skin condition.

The control unit 13 also refers to the information already recorded in the recording unit 17 to determine whether or not there is information on the specified area that was set during a previous image capture, that is, information on the specified area that was recorded when the control unit 13 set the specified area of the skin to be imaged by the second imaging unit 12 using diagnostic information on the facial skin condition that was previously obtained in the same manner as above.

If information on a previously set specific area has been recorded, the control unit 13 uses the diagnostic information based on the recorded information to identify areas within the specified body range other than the areas previously set as specific areas, and determines areas for which detailed images need to be obtained by imaging with the second imaging unit 12 in order to more appropriately diagnose the skin condition, for example, when signs of the presence of blemishes or the like have been found on a part of the skin and confirmation of the presence or absence of blemishes is required.

On the other hand, when information about the specified area to be imaged by the second imaging unit 12 that was previously set has not been recorded in the recording unit 17, such as when imaging the user's skin for the first time, the control unit 13 uses the diagnostic information about the facial skin condition previously acquired for the entire specified area of the body to determine areas that require detailed images to be acquired by imaging with the second imaging unit 12 in order to more appropriately diagnose the skin condition, as described above.

The control unit 13 sets the portion thus determined as the predetermined portion to be imaged by the second imaging unit 12, and records information about the set predetermined portion in the recording unit 17. The control unit 13 then generates, for this predetermined portion, for example, a predetermined guide design portion (mark) indicating the predetermined portion on the face to be imaged by the second imaging unit 12, performs processing to synthesize this guide design portion onto a display image of the face based on the facial skin image data, and causes the synthesized image to be displayed on the display unit 16 (see FIG. 5).
By looking at this display unit 16, the user recognizes a specific area to be imaged by the second imaging unit 12, and adjusts the position of the terminal device 10 including the second imaging unit 12 so that this area can be imaged.

When the guide design section indicates the position to be imaged by the second imaging unit 12, the user can easily understand the specific part of the face that should be imaged by the second imaging unit 12 as the specified body area, and the user can appropriately perform imaging using the second imaging unit 12 of the terminal device 10 whose position has been adjusted.

In addition, instead of setting a specific area to be imaged by the second imaging unit 12 and synthesizing an image showing this specific area with an image of the face and displaying it, the control unit 13 can also display guidance or provide audio guidance to the user from the terminal device 10 so that the user can image an area of the face that is of concern to the user as the specific body area of the user using the second imaging unit 12. Alternatively, a location where skin condition is generally measured can be selected in advance and registered as an initial measurement position, and a process can be performed to synthesize a specific guide design area showing one or more areas of the skin that are the initial position with the display image of the face, and the synthesized image can be displayed on the display unit 16.

In this case, the user will also recognize any areas of the face that are of concern, or will look at and confirm the initial measurement positions on the face indicated by the guide design displayed on the display unit 16, and then adjust the position of the terminal device 10 with the second imaging unit 12 so that these can be imaged (see Figure 6).

When the user adjusts the position of the terminal device 10 held in the hand and determines that the second imaging unit 12 is facing the specified part of the face, etc., the user performs a specified operation on the user interface of the terminal device 10, such as the shutter button, to cause the second imaging unit 12 to capture an image.

In addition, the user may bring the second imaging unit 12 together with the terminal device 10 closer to the specified area, or bring the specified area closer to the second imaging unit 12, so that when the distance between the second imaging unit 12 and the specified area of the skin reaches a specified value, the second imaging unit 12 automatically captures an image of the specified area of the skin, or the terminal device 10 may notify the user that it is the optimal time to capture an image, prompting the user to operate the terminal device 10 for image capture.

During this imaging stage, the second imaging unit 12 images the predetermined area of the skin under a predetermined lighting condition using an external light source (e.g., a built-in light source of a conversion lens) or a built-in light source of the terminal device 10, and outputs skin image data. This skin image data is acquired by the control unit 13.

For example, the built-in light source of the conversion lens 19 attached to the terminal device 10, which can adjust the light emitted as illumination, optimizes the light illumination state on the part to be imaged through verification of the skin image obtained by trial imaging by the second imaging unit 12, and then imaging is performed by the second imaging unit 12. The control unit 13 can acquire image data for a texture diagnosis image in which the skin is imaged under illumination with unpolarized light irradiation, and a blemish diagnosis image in which the skin is imaged under illumination with linearly polarized light irradiation.

The control unit 13 records the acquired skin image data of the specific area in the recording unit 17 together with the previously recorded information of the specific area in a state in which the date and time when the skin image data was acquired can be determined.
Here, when capturing images multiple times at once using the second imaging unit 12, for each image captured by the second imaging unit 12, the control unit 13 sets the specific area of the skin to be captured based on the information recorded in the recording unit 17 so that it does not overlap with the previous image, and the capture of images by the second imaging unit 12 is repeated through operation by the user in accordance with the information on the specific area notified to the user via the terminal device 10.

In addition, when information on a previously set specific area is recorded in the recording unit 17, the control unit 13 sets the specific area of the skin to be imaged by the second imaging unit 12, and while this specific area is being imaged by the second imaging unit 12, it refers to the recording unit 17 and regards the skin image data of the specific area, whose data acquisition date and time is included in a past specific period corresponding to the turnover cycle of the user's skin, as having been imaged at the same time as the new image captured by the second imaging unit 12.

Then, after acquiring skin image data of the specified area of the skin from the second imaging unit 12, the control unit 13 analyzes this skin image data, the previously recorded skin image data of the specified area acquired during the specified period that is assumed to have been acquired at the same time, and the facial image data acquired by the previously acquired image acquisition by the first imaging unit 11, to acquire diagnostic information on the skin condition of the user's face as the specified area of the body.

If information about a previously set specific area has not been recorded in the recording unit 17, the control unit 13 acquires skin image data of the specific area of the skin from the second imaging unit 12, and then analyzes this skin image data and the facial skin image data previously acquired by the first imaging unit 11 to acquire formal diagnostic information about the user's facial skin condition.

Then, the control unit 13 causes the display unit 16 to display the diagnostic information on the facial skin condition simultaneously with the facial image or in place of the facial image (see FIG. 7).
The display of diagnostic information on the facial skin condition on the display unit 16 is not limited to text-based information display, and the information can be provided by specifically indicating areas and symptoms of the skin that require attention with marking images on the facial image, etc. For example, the position and size of hidden blemishes derived as diagnostic information analyzed from the fine image data obtained by the second imaging unit 12 can be indicated by marks of a predetermined color superimposed on the corresponding areas on the facial image, and can be used as reference information when carrying out skin care such as blemish prevention.
When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 11 and the second imaging unit 12 is completed.

After this, in order to grasp the progress of the skin condition, the user repeats the above-mentioned process of capturing images of the face as the specified area of the body with the first imaging unit 11 and the specified part with the second imaging unit 12 at specified intervals, and the control unit 13 obtains diagnostic information on the skin condition based on the obtained skin image data for each image capture and displays it on the display unit 16.

While repeating such imaging, the control unit 13 records information on the specified part set in the recording unit 17, and when the next imaging is performed, the control unit 13 refers to the recorded information to set a new specified part that does not overlap with the previous part (see FIG. 8). This allows the face area as the specified body range to be gradually filled with the part for which detailed skin image data was obtained by the second imaging unit 12 for each imaging, and multiple high-precision skin image data can be continuously arranged in a map shape, allowing for more accurate analysis of the skin condition based on the skin image data, leading to accurate and reliable diagnostic information. Here, if the second imaging unit 12 performs imaging multiple times for each imaging performed by the first imaging unit 11, detailed skin image data covering each part of the face can be obtained in a short period of time (for example, about one week). However, if such imaging behavior is burdensome for the user, the imaging may be distributed to reduce the burden on the user. For example, imaging can be performed once a week with the first imaging unit 11 and once with the second imaging unit 12, and in one week the cheek can be imaged as the specified area, and in the next week the forehead can be imaged as the specified area, and imaging can be repeated every week, so that imaging with the second imaging unit 12 covers the entire face in about a month.

Incidentally, each time the second imaging unit 12 captures an image of the specified area, the control unit 13 records information about the specified area and skin image data obtained by imaging the specified area with the second imaging unit 12. At the time of a new imaging opportunity, the control unit 13 can determine that, among the information and skin image data recorded in the recording unit 17, data acquisition dates and times that exceed a specified period of time in the past that corresponds to the turnover cycle of the user's skin do not overlap with the turnover of the user's skin at this new imaging opportunity, and delete such old information and skin image data, allowing new recording to be performed repeatedly without difficulty.

In addition, when a new image is captured by the second imaging unit 12, the control unit 13 refers to the information recorded in the recording unit 17 so that the second imaging unit 12 can capture a new predetermined area that does not overlap with the previous area. However, the present invention is not limited to this. While the first imaging unit 11 captures the predetermined area of the body and the second imaging unit 12 captures the predetermined area, the second imaging unit 12 can repeatedly capture the same predetermined area many times. From the skin condition diagnostic information and display image obtained by the control unit 13 based on the skin image data focusing on the same area of the skin, the detailed progress of the skin condition in a specific area can be grasped, and the degree of improvement due to skin care can be easily confirmed.

In this case, the control unit 13 records in the recording unit 17 images of the face or the specified part for display on the display unit 16 based on the skin image data obtained by repeated imaging for each imaging, and if necessary, the images can be arranged in chronological order and displayed as a list, or displayed in chronological order in a sequential switching manner, which is preferable as it allows the user to easily grasp changes in the skin.

In addition, the facial skin image data obtained by the first imaging unit 11, which is the basis for the display image and diagnostic information on the skin condition for the face as the specified body area, is the skin image data obtained by the first imaging unit 11 before imaging by the second imaging unit 12. Alternatively, the control unit 13 may obtain the skin image data of the specified area from the second imaging unit 12, and then perform image correction processing on the facial skin image data based on the skin image data of the specified area, generate a display facial image based on the corrected facial skin image data, and display the facial image on the display unit 16, or analyze the corrected facial skin image data together with the skin image data of the specified area to obtain diagnostic information on the facial skin condition.

In this case, since the imaging range is limited and imaging conditions are easily adjusted, image correction processing is performed on the image data of the specified body range based on the image data of the specified part based on the imaging of the second imaging unit 12, and the skin color of the specified body range shown in this image data can be matched to that of the skin in the image data of the specified part. Therefore, regardless of the imaging conditions of the first imaging unit 11, the image data of the specified body range obtained by imaging can be made in a state that makes it easy to display the image and perform analysis related to diagnosing the skin condition, and this helps the user to correctly understand the skin condition.

In this way, the skin condition measurement system according to this embodiment images the face of the user as a specified area of the user's body with the first imaging unit 11, and images a specified part of the user's face with the second imaging unit 12, and the control unit 13 obtains information about the user's skin condition based on the obtained skin image data of the face and the skin image data of the specified part. Therefore, when analyzing the skin image data to derive information about the skin condition, the quality of information about the user's skin condition, such as diagnostic information about the skin condition, based on the skin image data obtained from multiple images, can be improved, and processing such as displaying and analyzing the skin condition can be performed with high accuracy, the skin condition diagnosis can be carried out appropriately and efficiently, and the reliability of the information obtained can be increased, allowing the user to grasp the skin condition more accurately.

In addition, the first imaging unit 11 is provided in the terminal device 10 together with the second imaging unit 12, and imaging can be performed using both the first imaging unit 11 and the second imaging unit 12 in one terminal device 10. This allows one terminal device 10 to capture images of the face as the specified body range of the user and the specified part of the face, and the cost associated with imaging can be reduced by using only one device for imaging.

In the skin condition measuring system according to the embodiment, the control unit 13 recognizes the image captured by the second imaging unit 12 as an image of the set predetermined part, on the premise that the user correctly captures the part indicated by the terminal device 10 as the predetermined part of the skin to be captured by the second imaging unit 12. In addition, the control unit can extract parts of the image of the predetermined body area obtained by the first imaging unit that have characteristics common to those appearing in the image captured by the second imaging unit (for example, the shape of characteristic parts such as skin texture and blemishes), and can set such parts as the predetermined part and recognize the image captured by the second imaging unit as the image of the predetermined part. The position of the image captured by the second imaging unit in the predetermined body area can be correctly identified, and the information on the skin condition based on the image data can be accurate. In this way, if the control unit does not particularly set the predetermined part of the skin to be captured by the second imaging unit, the image captured by the second imaging unit can be captured before or after the image captured by the first imaging unit.

In addition, in the skin condition measurement system according to the embodiment, the first imaging unit 11, the second imaging unit 12, and the control unit 13 are configured to be provided in a portable terminal device 10 such as a smartphone, but the present invention is not limited to this, and the terminal device may be a computer such as a notebook type or desktop type with a camera, as long as it is an information processing device that realizes the functions and processing of the control unit 13 by executing a program stored in advance and is equipped with the first imaging unit 11 and the second imaging unit 12 and can measure the skin condition. The terminal device may also be another information processing device specialized for a specific purpose such as beauty, for example, an electronic device that incorporates a camera and a display into the mirror part (half mirror) of a dressing table or a tabletop mirror for makeup or beauty, and that can display, in addition to being used as a normal mirror, a user's image captured by a camera, an image that has been processed, information obtained by analyzing the user's image, etc., on the mirror surface as needed.

In the case of a terminal device 50 having such a mirror unit 55, as shown in FIG. 9, multiple camera units constituting a first imaging unit 51 and a second imaging unit 52, or a camera unit serving as both a first imaging unit and a second imaging unit, can be provided on the upper part of the mirror unit 55, all facing directly towards the user. In addition, a light source 53 for illumination during imaging may be provided together with the first imaging unit 51 and the second imaging unit 52. In particular, a light source unit that can switch between irradiating non-polarized light as illumination for detecting the texture of the user's skin and irradiating linearly polarized light as illumination for detecting blemishes on the user's skin near the second imaging unit 52 when imaging the specified area, or a light source unit that can switch between irradiating right-handed circularly polarized light for detecting skin texture and irradiating left-handed circularly polarized light for detecting blemishes on the skin, is provided near the second imaging unit 52. This is preferable because it allows the second imaging unit 52 to properly detect skin texture and blemishes by selectively irradiating non-polarized light and linearly polarized light, or right-handed circularly polarized light and left-handed circularly polarized light, as illumination.

A hood 54 may be provided above the mirror section where the first and second imaging sections 51 and 52 are located, covering the first and second imaging sections to block the entry of unnecessary light from the surroundings. When a light source 53 for illumination during imaging is provided near the first and second imaging sections 51 and 52, the hood 54 can also be used as a reflector that directs illumination light toward the user.

In addition, when a hood 54 is provided, the size of the hood can be set so that when a part of the user's face (e.g., the forehead) touches the tip of the hood, the face as the specified area of the user's body is positioned relative to the first imaging unit 51 in a position suitable for imaging by this, or when the specified area of the user or the skin in the vicinity thereof touches the tip of the hood, the specified area of the user is positioned relative to the second imaging unit 52 in a position suitable for imaging by these.

In such a terminal device 50, when imaging is performed with the second imaging unit 52, the distance between the imaging unit in focus and the imaging target is usually shorter than in the case of the first imaging unit 51 because the imaging unit captures a narrower range in detail than in the case of imaging with the first imaging unit 51, so the user needs to bring the specified part to be imaged with the second imaging unit 52 closer to the terminal device 50. In order to facilitate imaging with the second imaging unit 52, the distance between the terminal device 50 and the user is measured by a specified measuring means of the terminal device 50, and when the user brings the specified part closer to the terminal device 50 and the distance becomes suitable for imaging with the second imaging unit 52, imaging with the second imaging unit 52 is performed, or when the distance between the terminal device 50 and the user is too far or too close during imaging, a notification is issued to the user through a notification means of the terminal device 50 to encourage the user to move closer or farther away from the terminal device 50 to maintain an appropriate distance. Also, to avoid the user having to move their body to change the distance between the user and the terminal device 50 for each image captured by the first imaging unit 51 and the second imaging unit 52, the distance between the terminal device 50 and the image capture target portion of the user may be set to be the same in each focus state of the first imaging unit 51 and the second imaging unit 52. In this case, imaging can be performed simultaneously by the first imaging unit 51 and the second imaging unit 52, reducing the burden on the user related to imaging.

Second Embodiment of the Invention
A skin condition measuring system according to a second embodiment of the present invention will be described with reference to FIGS.
In each of the figures, the skin condition measuring system 2 of this embodiment comprises a first imaging unit 21, a second imaging unit 22, and a control unit 93, just like the first embodiment, but differs in that the first imaging unit 21 and the second imaging unit 22 are arranged in a terminal device 20, while the control unit 93 is arranged in a server 90 that is set up to be able to communicate data with the terminal device 20 via a network 100.

In this embodiment, as in the first embodiment, the user who is the subject of skin condition measurement has entered into a contract with the provider of the skin condition measurement system 2 to receive skin condition diagnosis services, and is able to use the skin condition measurement system 2, so that the user can obtain skin condition diagnosis information from the skin condition measurement system 2.

The first imaging unit 21 is disposed in the terminal device 20 as the first imaging means, similarly to the first embodiment, and is capable of capturing an image of a predetermined range of the user's body.
As in the first embodiment, the second imaging unit 22 is arranged in the same terminal device 20 as the first imaging unit 21 as the second imaging means, and is capable of imaging a specific part of the user's skin.

The first imaging unit 21 and the second imaging unit 22 have the same configuration as those shown in the first embodiment, including the fact that they each form part of the user's terminal device 20 as a camera unit of the terminal device 20 capable of capturing an image of the user's skin, and detailed description thereof will be omitted.

The terminal device 20 has the first imaging unit 21 and the second imaging unit 22, and is a device capable of capturing images of a predetermined range and the predetermined part of the user's body and acquiring skin image data.
As in the first embodiment, the terminal device 20 is a known portable terminal device such as a mobile phone, a so-called smartphone, or a tablet device having wireless communication means 24 for connecting to and communicating with an IP communication network, and among these, an application program can be executed on the device and a display unit 26 can display various information. The terminal device 20 is equipped with two camera units, which can function as the first imaging unit 21 and the second imaging unit 22.

The display unit 26 of the terminal device 20 is capable of displaying an image representing at least the specified area of the user's body. This display unit 26 is a touch panel type display device, and is configured to also function as an input device. More specifically, the display unit 26 displays the display image of the specified area of the body and the display image of the specified part acquired by the control unit 93 of the server 90, as well as displaying diagnostic information on the skin condition of the specified area of the body acquired by the control unit 93.

The terminal device 20 is an information processing device that realizes various functions and processes by executing a pre-stored program, and is capable of communicating via the network 100. As long as it is equipped with the first imaging unit 21 and the second imaging unit 22 and can measure the skin condition, it is not limited to a portable device as in the first embodiment, but may be a notebook or desktop computer, or another information processing device specialized for a specific purpose.

The terminal device 20 starts an application program for measuring skin condition, and captures an image of the face of the user as the specified area of the body with the first imaging unit 21 to obtain skin image data of the face, and captures an image of the specified area of the skin with the second imaging unit 22 to obtain skin image data of the specified area, and transmits these skin image data to the control unit 93 of the server 90.

Furthermore, by executing an application program, the terminal device 20 communicates to request that the diagnostic information acquired by the server 90 be made accessible to the terminal device 20, and acquires the diagnostic information as content from the server 90 in response and displays it on the display unit 26. In addition, in order to make the diagnostic information accessible to the terminal device 20, the terminal device 20 can receive diagnostic information data transmitted from the server 90 and display it, or can download and display the diagnostic information data. Furthermore, the terminal device 20 receives from the server 90 a display image of the specified body range or a display image of the specified body part acquired by the control unit 93 of the server 90, and causes the display unit 26 to display them.

The server 90 is arranged to be capable of data communication with the terminal device 20 via the network 100, receives the skin image data of the specified body area and the skin image data of the specified part acquired by the terminal device 20, and as a control unit, analyzes the skin image data to acquire diagnostic information on the user's skin condition, acquires display images based on each piece of skin image data, transmits the diagnostic information and display images to the terminal device 20, and causes the display unit 26 of the terminal device 20 to display the diagnostic information and the skin image, respectively.

This server 90 is a computer equipped with a CPU and the like that functions as a server. The computer executes processes according to the recorded programs, thereby realizing the functions of the server.

Note that the server 90 in this embodiment is not necessarily limited to a single physical computer, but may be a group of multiple computer devices connected to a network and working together to provide one or more requested services as a whole, or may be a virtual device that, although it is only a portion of the functions of a computer device, has the function of providing one or more requested services as if they were a single device and can be treated as a single device by the service recipient.

The server 90 is configured to include a control unit 93 that acquires diagnostic information on the skin condition and information on images to be displayed on the terminal device, as information on the skin condition, based on skin image data obtained by imaging with the first imaging unit 21 and the second imaging unit 22 of the terminal device 20, and a recording unit 97 that records the information. In other words, the server 90 realizes the functions of the control unit 93 and the recording unit 97 by the computer that constitutes the server 90 executing processing according to a program.

The control unit 93 in the server 90 acquires information regarding the skin condition based on image data of the skin obtained by imaging.
This control unit 93 is capable of acquiring information regarding the skin condition of a specified area of the body based on skin image data of the specified area of the body obtained by imaging with the first imaging unit 21 received from the terminal device 20, and is also capable of acquiring information regarding the skin condition of the specified area based on skin image data of the specified area obtained by imaging with the second imaging unit 22 also received from the terminal device 20.

In detail, the control unit 93 acquires diagnostic information on the skin condition of the specified area of the body by analyzing the skin image data of the specified area of the body and the skin image data of the specified part as information on the skin condition. The function of the control unit 93 to analyze each image data and acquire diagnostic information on the user's skin condition is based on the same mechanism as in the first embodiment, except that the server 90 executes the process according to the application program, and a detailed description will be omitted.

The control unit 93 can then transmit diagnostic information on the skin condition of the specified area of the body to the terminal device 20. The terminal device 20 can display the diagnostic information on the skin condition of the specified area of the body on the display unit 26 based on the control of the control unit 93.

In addition, the control unit 93 acquires, as other information related to the skin condition, a display image of the specified body area derived from the skin image data of the specified body area, and also acquires a display image of the specified part derived from the skin image data of the specified part.

The control unit 93 can then transmit information about such display images to the terminal device 20. Based on the control of the control unit 93, the terminal device 20 can display an image of the specified body area, specifically the face, on the display unit 26, while displaying an image of the specified body part with a larger magnification than the image of the specified body area, either in a state where the image is replaced in the same display area on the screen of the display unit 26 where the image is displayed, or in an area separate from the display area, simultaneously and in parallel with the display of the image of the face.

The control unit 93 can also perform image correction processing (e.g., color calibration) on a part or all of the display image of the specified body area derived from the skin image data of the specified body area, based on the display image of the specified area derived from the skin image data of the specified area.

Furthermore, the control unit 93 executes a process for synthesizing a predetermined guide design portion, which indicates that the position is to be imaged by the second imaging unit 22, on the display image of the predetermined body area derived from the skin image data of the predetermined body area, at a position on this image corresponding to the predetermined part to be imaged by the second imaging unit 22, and can also transmit the processed display image to the terminal device 20 and display it on the display unit 26 of the terminal device 20. As a result, the guide design portion is displayed as part of the display image of the predetermined body area displayed on the display unit 26 of the terminal device 20, and the user can visually recognize and understand the predetermined part in the predetermined body area to be imaged by the second imaging unit 22.

In addition, the control unit 93 first acquires diagnostic information on the skin condition of the specified area of the body as information regarding the skin condition based on skin image data of the specified area of the body obtained by imaging with the first imaging unit 21 received from the terminal device 20, and from this diagnostic information sets a specified area of the skin to be imaged by the second imaging unit 22, transmits information related to this specified area of the skin to the terminal device 20, and shows it to the user on the display unit 26 of the terminal device 20, thereby making it possible to perform imaging of the specified area of the skin by the second imaging unit 22.

Then, when the second imaging unit 22 captures an image of the specified area of the skin based on the user's operation referring to the information on the specified area, the skin image data of the specified area is sent to the server 90. The control unit 93 then acquires new diagnostic information on the skin condition of the specified area of the body based on the newly acquired skin image data of the specified area and the skin image data of the specified area of the body.

The recording unit 97 records information about the specific area of the skin to be imaged by the second imaging unit 12 set by the control unit 93, using diagnostic information about the skin condition of the specific area of the body that was acquired by the control unit 93 based on skin image data of the specific area of the body.

In addition to information about the specific area of skin to be imaged by the second imaging unit 22, the recording unit 97 records the skin image data obtained by imaging this specific area by the second imaging unit 22 in a manner that makes it possible to determine the date and time when this skin image data was obtained.

Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, as in the first embodiment, the entire face of the user is the predetermined body area to be imaged by the first imaging unit 21, and the predetermined part included therein to be imaged by the second imaging unit 22 is the main part of the skin of the face. The terminal device 20 is a smartphone, and the camera unit on the display unit 26 side is used as the first imaging unit 21, while the camera unit on the opposite side to the display unit 26 is used as the second imaging unit 22, and each is capable of performing imaging and communicating with the server 90 through the network 100. When imaging the predetermined part, a conversion lens 29 is attached in front of the second imaging unit 22 in the terminal device 20, making the imaging range of the second imaging unit 22 narrower than that of the first imaging unit 21, while enabling imaging at a higher resolution, and an extremely narrow area of the skin of the face is imaged as the predetermined part.

As in the first embodiment, in order to capture images using the imaging units of the terminal device 20, a conversion lens (not shown) is first attached to the terminal device 20 by the user.
Next, the user operates the terminal device 20 to access a specific web page prepared by a skin condition measurement service providing server (e.g., server 90), and this access allows the terminal device 20 to use online services such as web applications provided by the server, and an application program that provides a user interface for using each imaging unit is launched in the terminal device 20, allowing the user to use each imaging unit through the terminal device 20 to capture images for skin condition measurement.

In addition, a user may operate the terminal device 20 to start an application program on the terminal device 20 that is pre-recorded on the terminal device and provides a user interface for using each imaging unit, so that the user can use each imaging unit to capture skin images via the terminal device 20.

When the user operates the user interface displayed on the display unit 26 of the terminal device 20 to start measurement, the terminal device 20 will capture an image of the face as a specified area of the user's body.

In detail, in the terminal device 20, a user interface is displayed on the display unit 26 by executing an application program for measuring skin condition. When the user operates this user interface, the first imaging unit 21, which serves as the first imaging means in the terminal device 20, starts capturing an image of the user's face, and the skin image data is continuously acquired by the terminal device 20 and transmitted to the server 90.

Then, the control unit 93 of the server 90 causes the display unit 26 of the terminal device 20 to display an image of the face based on the image data obtained by imaging. The control unit 93 also causes the user to maintain a state in which the terminal device 20 is held so that the face is displayed in a predetermined orientation and size by displaying a predetermined guide on the display unit 26 of the terminal device 20, etc.

The user refers to the facial image displayed on the display unit 26, and at the timing when the user believes that the face as a predetermined body area has been properly captured by the first imaging unit 21, performs a predetermined operation on the user interface of the terminal device 20 to determine a still image of the face. Data of the still image of the face at the determination moment is transmitted to the server 90, and is handled by the control unit 93 as skin image data of the predetermined body area obtained by imaging with the first imaging unit 21.

The control unit 93 acquires a display image of the face based on the skin image data obtained by the first imaging unit 21, transmits this to the terminal device 20, and displays it on the display unit 26 while analyzing the skin image data and temporarily acquiring diagnostic information on the user's facial skin condition.

The control unit 93 also refers to the information already recorded in the recording unit 97 to determine whether or not there is information on the specified area that was set during a previous image capture, that is, information on the specified area that was recorded when the control unit 13 set the specified area of the skin to be imaged by the second imaging unit 12 using diagnostic information on the facial skin condition that was previously obtained in the same manner as described above.

If information on a previously set specific area has been recorded, the control unit 93 uses the diagnostic information based on the recorded information to identify areas within the specified body range other than the areas previously set as specific areas, and determines areas for which detailed images need to be obtained by imaging with the second imaging unit 22 in order to more appropriately diagnose the skin condition, for example, when signs of the presence of blemishes or the like have been found on a part of the skin and confirmation of the presence or absence of blemishes is required.

On the other hand, when information about the specified area to be imaged by the second imaging unit 22 that was previously set has not been recorded in the recording unit 97, such as when imaging the user's skin for the first time, the control unit 93 uses the diagnostic information to determine areas for the entire specified area of the body that require detailed images to be obtained by imaging with the second imaging unit 22 in a similar manner to that described above, in order to more appropriately diagnose the skin condition.

The control unit 93 sets the portion thus determined as the predetermined portion to be imaged by the second imaging unit 22, and records information about the set predetermined portion in the recording unit 97. The control unit 93 then generates a predetermined guide design portion indicating the predetermined portion, performs processing to synthesize the guide design portion onto a display image of the face based on the facial skin image data, and transmits the synthesized image to the terminal device 20, which then displays the image on the display unit 26 of the terminal device 20.
By looking at this display unit 26, the user recognizes a specific area to be imaged by the second imaging unit 22, and adjusts the position of the terminal device 20 including the second imaging unit 22 so that this area can be imaged.

In addition, as in the first embodiment, the control unit 93 sets a predetermined part to be imaged by the second imaging unit 22, and instead of synthesizing an image of a guide design part or the like showing this predetermined part with the display image of the face and displaying it, it can also send an instruction to the terminal device 20 to guide the user from the terminal device 20 so that the user takes an image of a part of the face as the predetermined body range of the user, which part is concerned about the skin condition after viewing the face display by the display unit 26, and have the terminal device 20 execute a guide display or audio guide for the user. Alternatively, a part where the skin condition is generally measured is selected in advance and registered as an initial measurement position, and a process is performed to synthesize a predetermined guide design part showing one or more parts of the skin, which are the initial measurement position, with the display image of the face, and the synthesized image is sent to the terminal device 20 and displayed on the display unit 26.

When the user adjusts the position of the terminal device 20 held in the hand and determines that the second imaging unit 22 is facing the specified part of the face, etc., the user performs a specified operation on the user interface of the terminal device 20, such as the shutter button, to cause the second imaging unit 22 to capture an image.

During this imaging stage, the second imaging unit 22 images the predetermined area of the skin under a predetermined lighting condition using an external light source (e.g., a built-in light source of a conversion lens) or a built-in light source of the terminal device 20, and outputs skin image data. This skin image data of the predetermined area is transmitted from the terminal device 20 to the server 90 and acquired by the control unit 93.

For example, the built-in light source of the conversion lens attached to the terminal device 20, which can adjust the light emitted as illumination, optimizes the light illumination state on the part to be imaged through verification of the skin image obtained by trial imaging by the second imaging unit 22, and then imaging is performed. Each piece of skin image data, an image for texture diagnosis in which the skin is imaged under illumination with unpolarized light irradiation, and an image for blemish diagnosis in which the skin is imaged under illumination with linearly polarized light irradiation, are transmitted from the terminal device 20 to the server 90 and acquired by the control unit 93.

The control unit 93 records the acquired skin image data of the specific area in the recording unit 97 in a manner that makes it possible to determine the date and time when this skin image data was acquired, in addition to the information of the specific area that was previously recorded.
In addition, when information on a previously set specific area is recorded in the recording unit 97, the control unit 93 refers to the recording unit 97 and regards the skin image data of the specific area, whose data acquisition date and time is included in a past specific period corresponding to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture by the second imaging unit 22.

Then, after acquiring skin image data of the specified area of the skin from the second imaging unit 22, the control unit 93 analyzes this skin image data, the previously recorded skin image data of the specified area acquired during the specified period, which is assumed to have been acquired at the same time, and the facial image data acquired by the first imaging unit 21, to acquire formal diagnostic information on the skin condition of the face as the specified area of the user's body.

If information about a previously set specific area has not been recorded in the recording unit 97, the control unit 93 acquires skin image data of the specific area of the skin from the second imaging unit 22, and then analyzes this skin image data and the facial skin image data previously acquired by the first imaging unit 21 to acquire formal diagnostic information about the user's facial skin condition.

The control unit 93 then transmits diagnostic information on the facial skin condition to the terminal device 20 (see FIG. 11 ) and displays the diagnostic information on the display unit 26 of the terminal device 20 either simultaneously with the facial image or in place of the facial image.

In addition, the control unit 93 can generate a display image of the specified area of the skin based on image data of the specified area, transmit information about the display image of the specified area to the terminal device 20, and display the image of the specified area in any area on the screen of the display unit 26 of the terminal device 20 as necessary.

The display of diagnostic information on the facial skin condition on the display unit 26 of the terminal device 20 is not limited to text-based information display, as in the first embodiment, but can also be provided in a format in which areas of the skin requiring attention or symptoms are specifically indicated by marking images or the like on the facial image, based on display control of the diagnostic information by the control unit 93. For example, image information can be displayed in which the positions and sizes of hidden blemishes derived as diagnostic information analyzed from the fine image data obtained by the second imaging unit 22 are indicated by marks of a predetermined color superimposed on the corresponding areas on the facial image, and this can be used as reference information when carrying out skin care such as blemish prevention.
When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 21 and the second imaging unit 22 is completed.

After this, in order to grasp the skin condition in more detail, for example, the skin condition of each part of the face and the progress of the skin condition with care, the user repeats the same procedure at predetermined intervals, capturing images of the face as the predetermined area of the body with the first imaging unit 21 and capturing images of the predetermined part with the second imaging unit 22, and the control unit 93 of the server 90 that has acquired each piece of skin image data acquires diagnostic information on the skin condition based on each piece of skin image data for each image capture, and displays it on the display unit 26 of the terminal device 20.

As this imaging is repeated, in particular by recording information about the specified area set by the control unit 93 in the recording unit 97, and then referencing the recorded information at the time of the next imaging, a new specified area that does not overlap with the previous one is set. As in the first embodiment, the face area as the specified body range is gradually filled in with areas for which detailed skin image data has been obtained by imaging with the second imaging unit 22 with each imaging, and multiple high-precision skin image data are connected in a map-like manner, allowing analysis of the skin condition based on the skin image data to be performed with higher precision, leading to accurate and reliable diagnostic information.

Incidentally, each time an image of the specified part is captured by the second imaging unit 22, the control unit 93 records information about the specified part and skin image data obtained by capturing the specified part by the second imaging unit 22. At the time of a new imaging opportunity, the control unit 93 can determine that, among the information about the specified part and skin image data recorded in the recording unit 97, data acquisition dates and times that exceed a specified period of time in the past that corresponds to the turnover cycle of the user's skin do not overlap in time with the user's skin at this new imaging opportunity, and erase such old information and skin image data so that new recording can be performed repeatedly without difficulty.

Facial skin image data obtained by the first imaging unit 21, which is the basis for the display image and diagnostic information on the skin condition of the face as the specified body area acquired by the server 90, is skin image data obtained by the first imaging unit 21 before imaging by the second imaging unit 22. Alternatively, the control unit 93 of the server 90 may acquire the skin image data of the specified area acquired by the second imaging unit 22 from the terminal device 20, and then perform image correction processing of the facial skin image data based on the skin image data of the specified area, generate a display image of the face based on the corrected facial skin image data, and transmit the display image of the face to the terminal device 20 to be displayed on the display unit 26 of the terminal device 20, or analyze the corrected facial skin image data together with the skin image data of the specified area to acquire diagnostic information on the facial skin condition.

In this case, since the imaging range is limited and imaging conditions are easily adjusted, image correction processing is performed on the image data of the specified body range based on the image data of the specified part based on the imaging of the second imaging unit 22, and the color of the skin image of the specified body range shown in this image data can be matched to that of the skin image in the image data of the specified part. Therefore, regardless of the imaging conditions of the first imaging unit 21, the image data of the specified body range obtained by imaging can be made in a state that makes it easy to display the image and perform analysis related to diagnosing the skin condition, and this helps the user to correctly understand the skin condition.

In this way, the skin condition measurement system according to this embodiment captures the face of the user as a specified area of the user's body with the first imaging unit 21, and captures a specified part of the user's face with the second imaging unit 22, and based on the obtained skin image data of the face and the skin image data of the specified part, the server 90 obtains diagnostic information on the skin condition and an image for display as information on the user's skin condition. Therefore, when analyzing the skin image data to derive information on the skin condition, the quality of information on the user's skin condition, such as diagnostic information on the skin condition, based on the skin image data obtained from multiple images, can be improved, and processing on the server 90 related to the display and analysis of the skin condition can be performed with high accuracy, the skin condition diagnosis can be carried out appropriately and efficiently, the reliability of the information obtained can be increased, and the user can grasp the skin condition more accurately.

In the skin condition measuring system according to the embodiment, the facial skin image data obtained by the first imaging unit 21 as the first imaging means is transmitted only from the terminal device 20 to the server 90 and used for processing in the control unit 93 to obtain information on the skin condition, such as diagnostic information. However, this is not limited to this. As shown in FIG. 12, the facial skin image data as the specified body range can be transmitted to another person's terminal device 60 together with the server 90, and the control unit 93 of the server 90 can obtain information on the skin condition based on the skin image data, while the other person's terminal device 60 displays an image of the user including the face based on the transmitted skin image data.

This can be applied to usage situations, such as video calls, in which images based on images captured by each terminal device are continuously displayed on the other party's terminal device during communication between the user's terminal device 20 and the other party's terminal device 60. As a specific example, a user can capture an image of his or her face with the first imaging unit 21 of his or her own terminal device 20 to obtain diagnostic information, and while viewing the diagnostic information ultimately acquired by the server 90, make a video call with a specific other party through the terminal device, and the image of the user displayed on the other party's terminal device 60 is based on facial skin image data obtained by imaging with the first imaging unit 21.

As an example of this application, a user may receive skin counseling through a video call with a counselor via a terminal device, and while receiving support from the counselor via the call, the user may operate the terminal device 20 to capture images of the face and the specified body part as the specified area with the first imaging unit 21 and the second imaging unit 22 of the terminal device 20, and the skin image data obtained by the imaging may be transmitted from the terminal device 20 to the server 90, and the control unit 93 of the server 90 may obtain diagnostic information on the skin condition.

In this case, the skin condition diagnostic information acquired by the control unit 93 of the server 90 can be transmitted to the counselor, i.e., to the terminal device 60 of a person other than the user, so that the user and the counselor can share the diagnostic information and reach a common understanding of the user's skin condition, and the counselor can then provide detailed counseling about the user's skin based on the diagnostic information. Also, the skin condition diagnostic information may not be transmitted from the server 90 to the user's terminal device 20, but may be viewable only on the counselor's terminal device 60, which prevents the user from being given preconceived ideas due to the diagnostic information, and makes it easier for the user to smoothly accept the counselor's advice. Furthermore, if the second imaging unit as the second imaging means is disposed on the same side as the first imaging unit in the user's terminal device in the same orientation, or is shared by the first imaging unit of the terminal device, or is an imaging device independent of the user's terminal device, when the user images the specified part of the face with the second imaging unit, the user can adjust the distance between the second imaging unit and the specified part of the face of the user, or appropriately point the second imaging unit at the specified part, while receiving imaging support by telephone from a counselor who can check the user's imaging status through the display screen of the terminal device, and then perform imaging, which is preferable.

In addition, in the skin condition measurement system according to the embodiment, the first imaging unit 21 as the first imaging means is disposed in the user's terminal device 20, and the user operates the first imaging unit 21 to capture an image of the face as the specified area of the user's body. However, the present invention is not limited to this, and the first imaging means may be a specified imaging device that is part of or connected to another person's terminal device and captures at least the specified area of the user's body, and the image data of the user obtained by such first imaging means is used on the other person's terminal device for a main purpose other than obtaining information about the user's skin condition. The server 90 may receive the skin image data of the specified area of the user's body obtained by imaging with the first imaging means, and the control unit 93 of the server 90 may obtain information about the user's skin condition, such as diagnostic information, based on the skin image data of the specified area of the body and the skin image data of the specified part obtained by imaging with the second imaging means.

As a specific example, as shown in Figures 13 and 14, the image capturing unit of another person's terminal device 61 captures an image of the user, and the image data obtained is used primarily for personal authentication processing to identify the user. The server 90 obtains the image data captured by the other person's terminal device 61 as skin image data of a specified area of the user's body, and the control unit 93 analyzes this data together with skin image data of the specified area obtained separately from a second imaging means to obtain diagnostic information on the skin condition.

In this case, the image data of the user acquired by imaging with the imaging unit of the other person's terminal device 61 can be used for skin diagnosis, eliminating the need to prepare a new first imaging means for imaging the specified area of the user's body, and enabling information on skin condition, such as diagnostic information based on each image data, to be acquired efficiently and at low cost.

Third embodiment of the present invention
In the skin condition measuring system according to the first embodiment, the second imaging unit 12 as the second imaging means is configured to be capable of imaging a predetermined part of the face (e.g., part of the cheek, forehead, etc.) that overlaps with the face as the predetermined area of the user's body. In addition, as a third embodiment, the second imaging means can also be configured to image a predetermined part of the skin that does not overlap with the predetermined area of the user's body.

In this case, the control unit acquires information regarding the skin condition of the specified area of the skin based on the skin image data of the specified area obtained by imaging with the second imaging means, and then acquires information regarding the relative skin condition of each part of the specified body area based on the information regarding the skin condition of the specified area as information regarding the skin condition of the specified body area.

As a specific example, the face of the user as the predetermined body area is imaged by the first imaging unit 31 as the first imaging means, and a predetermined part of the skin that does not overlap with the face, for example, the side of the arm close to the torso or the underside of the chin, which is less exposed to sunlight and less affected by ultraviolet rays, is imaged by the second imaging unit 32 as the second imaging means provided in the terminal device 30 (see FIG. 15). Note that the imaging by the second imaging unit 32 may be performed before or after the imaging by the first imaging unit 31.

Based on the skin image data of the specified area obtained by imaging with the second imaging unit 32, a control unit (not shown) obtains information regarding the skin condition of the specified area of the skin. Furthermore, when the control unit obtains information regarding the skin condition of the face based on the skin image data of the face as the specified body range obtained by imaging with the first imaging unit 31, the control unit obtains information regarding the skin condition of the face as information regarding the relative skin condition of each part of the face based on the previously obtained information regarding the skin condition of the specified area.

This allows the user's facial skin condition to be shown relatively to the skin condition of the specified area, which is less affected by ultraviolet rays and has a smaller degree of skin deterioration, making it suitable as a standard for skin evaluation. This allows the degree of deterioration of the skin condition of each part of the face within the specified body area to be shown in a form that is easy for the user to understand, making it possible to more accurately grasp the facial skin condition as a difference from the user's ideal skin condition, and encouraging the user to take appropriate care of their facial skin by using the skin condition of the specified area as a target state.

The terminal device may be configured to suggest to the user skin care that will bring the facial skin condition closer to the target skin condition of the specified area, based on information about the facial skin condition, such as relative diagnostic information about the skin condition based on information about the skin condition of the specified area acquired by the control unit. In this case, an external AI may be used to perform a more detailed analysis based on various information about the skin condition in addition to the diagnostic information, leading to more effective skin care suggestions.

In the above embodiment, a first imaging unit 31 as a first imaging means and a second imaging unit 32 as a second imaging means are arranged in one terminal device 30, and imaging by each imaging unit is performed at different timings, making it possible to easily perform imaging with a single terminal device 30. However, in addition to this, a configuration in which the second imaging means is provided separately from the first imaging means, for example, a configuration in which another imaging device 36 having a camera unit forming the second imaging means is used in combination with a terminal device 35 having a camera unit forming the first imaging means, may be used. In this case, imaging can be performed simultaneously by the first imaging means and the second imaging means (see FIG. 16), shortening the time from imaging to obtaining information on the skin condition, reducing the burden on the user related to imaging, and making it easier for the user to continue activities such as imaging to obtain information on the skin condition.

(Fourth embodiment of the present invention)
A skin condition measuring system according to a fourth embodiment of the present invention will be described with reference to FIGS.
In each of the above figures, the skin condition measuring system 4 of this embodiment is equipped with a first imaging unit 41, a second imaging unit 42, and a control unit 43, like the first embodiment, but differs in that the terminal device 40 in which the first imaging unit 41, the second imaging unit 42, and the control unit 43 are arranged is configured to simultaneously capture images using the first imaging unit 41 and the second imaging unit 42, and the first imaging unit 41 is configured to capture a mirror image of the user reflected on a mirror surface 80 in front of the user.

In this embodiment, as in the first embodiment, the user who is the subject of skin condition measurement has entered into a contract with the provider of the skin condition measurement system 4 to receive skin condition diagnosis services, and is able to use the skin condition measurement system 4, so that the user can obtain skin condition diagnosis information from the skin condition measurement system 4.

The first imaging unit 41 is disposed in the terminal device 40 as the first imaging means, and is capable of imaging a predetermined area of the user's body, such as the face, to measure the user's skin condition. However, this first imaging unit 41 images a mirror image of the predetermined area of the user's body reflected on a mirror surface 80 located in front of the user.

The second imaging unit 42, as the second imaging means, is disposed in the same terminal device 40 as the first imaging unit 41, and is capable of imaging a specific part of the user's skin, such as a part of the cheek or forehead, which is narrower than the specific body area of the user and overlaps with the specific body area, in order to measure the user's skin condition.

This second imaging unit 42 is arranged on the other side of the surface on the terminal device 40 opposite to the surface on which the first imaging unit 41 is arranged, and is capable of imaging the specified part of the user when the first imaging unit 41 on the terminal device 40 is directed toward the mirror surface 80 in front of the user.
In addition, the second imaging section 42 is capable of acquiring image data having a higher resolution per unit area of the imaging target than the first imaging section 41 .

The first imaging unit 41 and the second imaging unit 42 are capable of performing imaging by user operation via the terminal device 40, and the imaging of the specified area of the body by the first imaging unit 41 and the imaging of the specified part of the skin by the second imaging unit 42 are performed at the same time.

The terminal device 40 has the first imaging unit 41 and the second imaging unit 42, and is a device capable of capturing images of a predetermined area of the user's body and the predetermined part of the body and acquiring skin image data, and also has the control unit 43.

The terminal device 40 is a handheld device capable of executing application programs, and has a display unit 46 capable of displaying various types of information.
The display unit 46 of the terminal device 40 is capable of displaying an image of the user. The display unit 46 is a touch panel type display device, and also serves as an input device.

The terminal device 40 is an information processing device that realizes the functions and processing of the control unit 43 by executing a pre-stored program, and is not limited to being a handheld device, but may also be a fixed device, as long as it is positioned between the user and the mirror surface 80 in front of the user and can simultaneously capture images using the first imaging unit 41 and the second imaging unit 42. Also, considering that the mirror surface 80 may be that of a mirror that is often installed in places where there is moisture, such as bathrooms, washbasins, and vanities, it is preferable that the terminal device 40 has a waterproof structure since it will be used in the same space.

The control unit 43 of the terminal device 40 is capable of analyzing the skin image data obtained by imaging and acquiring diagnostic information using an application program for diagnosing a skin condition, and the acquired diagnostic information can be referred to by the user. In this case, the control unit 43 of the terminal device 40 executes the application program to display the acquired diagnostic information as content on the display unit 46, making it possible for the user to refer to the information.

The control unit 43 acquires information regarding the skin condition based on the skin image data obtained by imaging, i.e., the skin image data of the specified area of the body obtained by imaging with the first imaging unit 41 and the skin image data of the specified part obtained by imaging with the second imaging unit 42.

The control unit 43 analyzes the skin image data of the specified body area and the skin image data of the specified part to obtain diagnostic information on the skin condition of the specified body area of the user as information on the skin condition. The function of the control unit 43 to analyze each skin image data and obtain diagnostic information on the user's skin condition in this manner is based on a mechanism similar to that of the first embodiment, which is realized by the control unit 43 executing processing according to an application program, and detailed description will be omitted.

In addition, the control unit 43 obtains, as other information related to the skin condition, an image for display of the specified body area based on the skin image data of the specified body area, and obtains an image for display of the specified area based on the skin image data of the specified area.

The control unit 43 can then display a display image of the specified body area, specifically the face, on the display unit 46 of the terminal device 40, while replacing the image in the same area as the display area on the screen of the display unit 46 where the display image is displayed, or can display a display image of the specified part with a larger magnification than the display image of the specified body area, simultaneously and in parallel with the display of the display image of the face, in an area separate from the display area.

In addition, the control unit 43 can use previously acquired diagnostic information on the skin condition of the specified area of the user's body as information on the skin condition to set a specified area of the skin that is to be imaged by the second imaging unit 42 and is included in the specified area of the body, and can notify the user of the information on this specified area via the terminal device 40.

In this case, when skin image data of the specified area is obtained by the second imaging unit 42 imaging the specified area of the skin based on the user's operation referring to the information of the specified area, the control unit 43 can obtain diagnostic information on the skin condition of the specified area of the body based on the skin image data of the specified area and the skin image data of the specified area of the body obtained by simultaneous imaging.

The control unit 43 also has a recording unit 47 that records information about a specific area of the skin to be imaged by the second imaging unit 42 when the control unit 43 uses the acquired diagnostic information about the skin condition of the specific area of the body to set the specific area of the skin to be imaged by the second imaging unit 42.

The recording unit 47 records information about the specific area of the skin to be imaged by the second imaging unit 42, as well as skin image data obtained by imaging this specific area by the second imaging unit 42, in a manner that makes it possible to determine the date and time when this skin image data was obtained.

When information on a previously set specific part is recorded in the recording unit 47, the control unit 43 uses the previously acquired diagnostic information on the skin condition of the specific body area during a new image capture opportunity with the first imaging unit 41, and references the information on the specific body area already recorded in the recording unit 47 to set a new specific skin area to be imaged by the second imaging unit 42 from among the areas of the specific body area that have not been recorded as a specific part.

The control unit 43 also has a mechanism for referencing information on the specific area of the skin that has already been recorded in the recording unit 47, the skin image data of the specific area, and the date and time of data acquisition in parallel with setting a specific area of the skin to be imaged by the second imaging unit 42 during a new imaging opportunity by the first imaging unit 41.

The control unit 43 then regards the skin image data of the recorded specific area, whose data acquisition date and time falls within a specific period of the past that corresponds to the turnover cycle of the user's skin, as having been captured at the same time as the new image capture.

In this case, the control unit 43 obtains diagnostic information (information related to the skin condition) of the skin condition of the specified area of the body based on the recorded skin image data acquired during the specified period, the skin image data of the specified area of the skin set at this new imaging opportunity obtained by imaging the specified area by the second imaging unit 42, and the skin image data of the specified area of the body.

On the other hand, the control unit 43 may delete from the recording unit 47 the information on the specified area and the skin image data of the specified area, if the data acquisition date and time is not included in the specified period and is older than the turnover period of the user's skin, since the skin turnover does not overlap with the skin of the user in the new image capture opportunity by the first imaging unit 41 and the data cannot be regarded as having been captured at the same time as the new image capture.

Next, the usage state of the skin condition measuring system according to this embodiment will be described. As a premise, as in the first embodiment, the entire face of the user is set as the predetermined body range imaged by the first imaging unit 41, and the predetermined part included therein and imaged by the second imaging unit 42 is the main part of the facial skin. It is also assumed that the first imaging unit 41 and the second imaging unit 42 of the terminal device 40 are each capable of performing imaging.

First, a user operates the terminal device 40 to start an application program that already exists in the terminal device and that provides a user interface for using each imaging unit, and the user is then able to use each imaging unit for imaging via the terminal device 40.

The user then holds the terminal device 40 so that the first imaging unit 41, as the first imaging means, faces the mirror image of the face, as a specified area of the user's body, reflected on the mirror surface 80 located in front of the user, and the second imaging unit 42, as the second imaging means, faces a part of the user's face.

When the user holds the terminal device 40 in this manner, the first imaging unit 41 captures a mirror image of the face as the specified body range reflected on the mirror surface 80 located in front of the user, while the second imaging unit 42 captures an image of a portion of the user's face, and the control unit 43 continuously acquires the image data obtained by each imaging unit.

As the first imaging unit 41 and the second imaging unit 42 capture images, the images captured by each imaging unit are displayed on the display unit 46 of the terminal device 40 based on the image data obtained by the capture.
In this case, the control unit 43, by displaying a predetermined guide on the display unit 46, etc., causes the user to maintain a state in which the terminal device 40 is supported and a distance between the user and the mirror surface 80 so that at least a mirror image of the face is captured by the first imaging unit 41 in a predetermined orientation and size, and the control unit 43 obtains a display image of the face based on the facial skin image data obtained by imaging by the first imaging unit 41, and causes this display image of the face to be appropriately displayed on the display unit 46.

The control unit 43 then displays guidance and provides audio guidance to the user from the terminal device 40, so that the user can have the second imaging unit 42 capture an image of the area of the face that is of concern to the user, which is the specified area of the user's body. Alternatively, a location where the skin condition is generally measured is selected in advance and registered as an initial measurement position, and a process is performed to synthesize a specified guide design portion showing one or more areas of the skin that are the initial measurement positions onto the display image of the face, and the synthesized image is displayed on the display unit 46.

The user then recognizes any areas of the face that are of concern, or checks the initial measurement positions on the face indicated by the guide design displayed on the display unit 46, and then fine-tunes the position and orientation of the terminal device 40 with the second imaging unit 42 so that these areas can be imaged.

The user aims the first imaging unit 41 and the second imaging unit 42 at each imaging target, and referring to the mirror image of the face and the image display of a specified part of the face on the display unit 46, performs a specified operation on the user interface of the terminal device 40, such as the shutter button, at the time when each imaging target is deemed to have been appropriately captured by the corresponding imaging unit, and determines the mirror image of the face and a still image of the specified part. The data of the mirror image of the face and the still image of the specified part at the moment of determination are handled by the control unit 43 as skin image data of the specified area of the body and skin image data of the specified part.

At the stage of determining the image in this way, the second imaging unit 42 can also be configured to image the predetermined part of the skin under a predetermined lighting condition using an external light source or a built-in light source of the terminal device 40. For example, in the terminal device 40, a built-in light source capable of adjusting the light emitted as illumination optimizes the light illumination state for the predetermined part of the face of the subject to be imaged through verification of the skin image obtained by trial imaging of the second imaging unit 42, and then imaging is performed under illumination, and the control unit 43 can acquire image data for a texture diagnosis image obtained by imaging the skin under illumination using unpolarized light irradiation, and a blemish diagnosis image obtained by imaging the skin under illumination using linearly polarized light irradiation.

The control unit 43 obtains display images of the face and the specified parts based on the skin image data obtained by the first imaging unit 41 and the second imaging unit 42, and displays these on the display unit 46 while analyzing the skin image data and obtaining diagnostic information on the user's facial skin condition.

The control unit 43 then displays the diagnostic information on the facial skin condition on the display unit 46 either simultaneously with the facial image or in place of the facial image. The control unit 43 also records information on the position of the specific part on the face and the skin image data of the specific part in the recording unit 47 in a state in which the date and time when the skin image data was obtained can be determined.

The display of diagnostic information on the facial skin condition on the display unit 46 is not limited to displaying text-based information, but can also provide information by specifically indicating areas and symptoms of the skin that require attention with marking images on the facial image, etc. For example, the position and size of hidden blemishes derived as diagnostic information analyzed from the detailed image data obtained by the second imaging unit 42 can be indicated by marks of a predetermined color superimposed on the corresponding areas on the facial image, and can be used as reference information when carrying out skin care such as blemish prevention.
When such diagnostic information is displayed, imaging for one skin condition measurement by the first imaging unit 41 and the second imaging unit 42 is completed.

After this, in order to grasp the progress of the skin condition, the user repeats the above-mentioned process of capturing images of the face as the specified area of the body with the first imaging unit 41 and the specified part with the second imaging unit 42 at specified intervals, and the control unit 43 obtains diagnostic information on the skin condition based on the obtained skin image data for each capture and displays it on the display unit 46.

In this case, the control unit 43 refers to the information already recorded in the recording unit 47 and determines whether or not there is information on the specific part of the face that was set during a previous image capture, that is, information on this set specific part that was recorded when the control unit 43 set the specific part of the skin to be imaged by the second imaging unit 42 using skin condition diagnostic information, etc.

If information on previously set specific areas has been recorded, the control unit 43 uses the previous diagnostic information for each part of the specified body range other than the parts previously set as specific areas based on the recorded information to determine areas for which detailed images need to be obtained by imaging with the second imaging unit 42 in order to more appropriately diagnose the skin condition, for example, when signs of the presence of blemishes or the like have been found on part of the skin and confirmation of the presence or absence of blemishes is required.

On the other hand, if information on the specified area to be imaged by the previously set second imaging unit 42 has not been recorded in the recording unit 47, the control unit 43 uses the previous skin condition diagnosis information for the entire specified body area to determine areas that require detailed images to be obtained by imaging with the second imaging unit 42 in order to more appropriately diagnose the skin condition, as described above.

The control unit 43 sets the portion determined as above as the predetermined portion to be imaged by the second imaging unit 42, and records information about the set predetermined portion in the recording unit 47. Then, the control unit 43 generates, for this predetermined portion, for example, a predetermined guide design portion (mark) indicating the predetermined portion on the face to be imaged by the second imaging unit 42, performs processing to synthesize this guide design portion with a display image of the face based on the facial skin image data, and causes the display unit 46 to display the synthesized image.
By looking at this display unit 46 , the user recognizes the specific area to be imaged by the second imaging unit 42 , and fine-tunes the position and orientation of the terminal device 40 so that this area can be imaged by the second imaging unit 42 .

As imaging etc. are repeated in this manner, the control unit 43 records information about the specified areas set in the recording unit 47, and when imaging next time, the recorded information is referenced to set a new specified area that does not overlap with the previous one. This allows the face area as the specified body range to be gradually filled with areas for which detailed skin image data has been obtained by imaging with the second imaging unit 42 with each imaging, resulting in multiple high-precision skin image data being connected in a map-like manner, allowing analysis of the skin condition based on the skin image data to be performed with greater precision, leading to accurate and reliable diagnostic information.

In this way, each time an image of the specified area is taken by the second imaging unit 42, the control unit 43 records information about the specified area and skin image data obtained by imaging the specified area by the second imaging unit 42. At the time of a new imaging opportunity, for each piece of information and skin image data recorded in the recording unit 47 whose data acquisition date and time exceeds a specified period of time in the past corresponding to the turnover cycle of the user's skin, the control unit 43 can determine that the skin turnover does not overlap with the user's skin at the time of this new imaging opportunity, and can erase such old information and skin image data so that new recording can be performed repeatedly without difficulty.

When the control unit 43 captures a new image using the second imaging unit 42, it refers to the information recorded in the recording unit 47 so that the second imaging unit 42 can capture a new, non-overlapping, specified area. However, the present invention is not limited to this. While the first imaging unit 41 captures the specified area of the body and the second imaging unit 42 captures the specified area, the second imaging unit 42 can repeatedly capture the same specified area. From the skin condition diagnostic information and display image obtained based on the skin image data focusing on the same area of the skin by the control unit 43, the detailed progress of the skin condition in a specific area can be grasped, and the degree of improvement due to skin care can be easily confirmed.

In this way, the skin condition measuring system according to this embodiment captures the mirror image reflected on the mirror surface 80 in front of the user with the first imaging unit 41 to obtain skin image data of the specified area of the body, and at the same time, captures the skin image data of the specified area with the second imaging unit 42, and the control unit 43 obtains information regarding the skin condition from each piece of skin image data and makes this information regarding the skin condition available. Therefore, by capturing images simultaneously with each imaging unit, skin image data of the specified area of the body and skin image data of the specified area can be obtained at once, shortening the time from capturing images to obtaining information regarding the skin condition, reducing the burden on the user related to capturing images, and also obtaining information regarding the skin condition without wasting time, making it possible to quickly grasp the user's skin condition.

In addition, a first imaging unit 41 is disposed on one surface of the terminal device 40, and with this first imaging unit 41 facing the mirror surface 80, it is possible to image the specified part of the user, and a second imaging unit 42 is disposed on the same terminal device 40, and one device can handle imaging of the specified range of the user's body and the specified part. Therefore, skin image data of the specified range of the body and skin image data of the specified part can be obtained together by imaging with each imaging unit of the terminal device 40, and when imaging, the user only needs to handle the terminal device 40 equipped with each imaging unit, which significantly reduces the burden on the user related to imaging and allows the user to continue activities to obtain information about their skin condition without difficulty.

In the skin condition measuring system according to the embodiment, the terminal device 40, which is provided with the first imaging unit 41 as the first imaging means and the second imaging unit 42 as the second imaging means, is configured as a device that acquires diagnostic information on the skin condition by capturing images using the first imaging unit 41 and the second imaging unit 42 exclusively. However, this is not limited to this, and a communication terminal device such as a smartphone that can be used for various purposes and has multiple camera units can also be configured to be used as a terminal device in which the multiple camera units function as the first imaging means and the second imaging means, respectively, if the multiple camera units are configured to capture images for each camera unit simultaneously and acquire image data for each camera unit separately.

In addition, in the terminal device 40, the second imaging unit 42 as the second imaging means is configured to be disposed on the opposite side of the surface on which the first imaging unit 41 as the first imaging means in the terminal device 40 is disposed, but this is not limited thereto, and as long as the second imaging means disposed in the terminal device is capable of imaging the predetermined part of the user in a state in which the first imaging means in the terminal device is directed toward the mirror surface 80 that reflects the user in front of the user, the second imaging means may be disposed in any location on the terminal device. In particular, when the direction of the imaging target can be changed by adding a conversion lens or the like, the second imaging means may be disposed on the surface on which the first imaging means in the terminal device is disposed. In addition, the second imaging unit 42 as the second imaging means is not limited to being provided in the same terminal device 40 as the first imaging unit 41 as the first imaging means, but may be provided separately from the first imaging means, for example, a terminal device having a camera unit as the first imaging means may be used in combination with another device having a camera unit as the second imaging means.

In addition, in the skin condition measuring system according to the embodiment, the second imaging unit 42 as the second imaging means is capable of imaging a predetermined part of the face (e.g., a part of the cheek, forehead, etc.) that overlaps with the face as the predetermined body area of the user, while the imaging range is a part hidden by the terminal device 40 in the mirror image of the face reflected on the mirror surface 80 in front of the user, and does not overlap with the imaging range of the first imaging unit 42 as the first imaging means that images the mirror image of the face, and no part is commonly imaged by the first imaging unit 41 and the second imaging unit 42. In addition, as shown in FIG. 20, the terminal device 4 The second imaging unit 49 in No. 5 can be configured to be able to image the specified part of the user from a position a little distance away, and can also be configured to be able to image the specified part of the face even when the terminal device 45 is positioned away from the front of the face, for example, diagonally in front of or to the side of the face, within a range in which the first imaging unit 48 can image the mirror image of the face reflected on the mirror surface 80 in front of the user, so that the imaging range of the first imaging unit 48, which can image a mirror image of the entire face, overlaps with the imaging range of the second imaging unit 49, which can image the specified part of the face, and there are parts that are commonly imaged by the first imaging unit 48 and the second imaging unit 49.

In this case, the relationship between the skin image data obtained by the two different imaging units can be understood from the common specific areas in the skin image data, and the detailed skin condition of each part of the face as a specific area of the body other than the overlapping areas can be inferred. Detailed information regarding the skin condition can be obtained by analysis using the skin image data in the control unit, and this information regarding the skin condition allows the user to correctly understand their skin condition, further encouraging the user to take appropriate measures regarding skin care, etc.

In addition, in the skin condition measurement system according to the embodiment, the first imaging unit 41 as the first imaging means in the terminal device is directed to a mirror surface 80 reflecting the user in front of the user, and captures a mirror image of the user's face reflected on the mirror surface 80 as the specified body area of the user to obtain skin image data of the face. This skin image data is then analyzed by the control unit 43 to obtain information regarding the skin condition. In addition, the image is not processed in any particular way, and the image of the face is displayed on the display unit 46 as it was captured. However, the control unit can also process the image data obtained by the first imaging means to such an extent that the displayed content cannot be discerned even if a third party views the image, for parts of the image other than the mirror image of the specified body area (face) of the user, such as body parts other than the specified body area of the user or a wall behind the user, which are reflected as a background. This can reliably prevent the leakage of personal information of the user that is not related to obtaining information regarding the skin condition, such as diagnostic information.

In addition, in the skin condition measurement system according to the embodiment, the control unit 43 analyzes both the skin image data of the face as the specified body range obtained by imaging with the first imaging unit 41 as the first imaging means and the skin image data of the specified part obtained by imaging with the second imaging unit 42 as the second imaging means to obtain diagnostic information on the skin condition. However, the skin image data of the specified body range of the user obtained by imaging with the first imaging means can be used only to identify the position of the specified part imaged by the second imaging means of the terminal device based on the position of the terminal device contained therein, and only the skin image data of the specified part obtained by imaging with the second imaging means can be used for image analysis focusing on the skin by the control unit. This ensures that the specified part imaged by the second imaging means in the specified body range of the user can be correctly identified, and information on the skin condition such as diagnostic information can be obtained accurately based on the skin characteristics of each part of the specified body range. This simplifies the process of obtaining information on the skin condition of the control unit based on each skin image data, reduces the load on the control unit, and speeds up processing.

REFERENCE SIGNS LIST 1, 2, 4 Skin condition measuring system 10, 20, 30, 40 Terminal device 11, 21, 31, 41 First imaging section 12, 22, 32, 42 Second imaging section 13, 23, 43 Control section 14, 24 Wireless communication means 16, 26, 46 Display section 17, 27, 47 Recording section 19 Conversion lens 35 Terminal device 36 Imaging device 45 Terminal device 48 First imaging section 49 Second imaging section 50 Terminal device 51 First imaging section 52 Second imaging section 53 Illumination light source 54 Hood 55 Mirror section 60, 61 Terminal device 80 Mirror surface 90 Server 93 Control section 97 Recording section 100 Network

Claims (1)

A first imaging means capable of imaging a predetermined area of the skin of a user's body;
A second imaging means capable of imaging a predetermined part of the user's skin that is narrower than the predetermined body area;
a control unit that acquires information about a skin condition based on skin image data of the predetermined body area obtained by imaging with the first imaging means and skin image data of the predetermined part obtained by imaging with the second imaging means,
the second imaging means is capable of imaging at a position closer to the user than the first imaging means and capable of acquiring image data having a higher resolution per unit area of the imaging target than the first imaging means;
The control unit is configured to analyze the skin image data of the predetermined body area and the skin image data of the predetermined part to obtain diagnostic information on the skin condition of the predetermined body area of the user as information on the skin condition,
The control unit acquires information regarding the skin condition of the specified part of the skin based on skin image data of the specified part obtained by imaging with the second imaging means, and then acquires, as the information regarding the skin condition, information regarding the relative skin condition of each part of the specified range of the body based on the information regarding the skin condition of the specified part .

Images