KR102842211B1 - Apparatus for provinding guidance to sell cosmetics unattended - Google Patents

Abstract

The present invention relates to a device for providing guidance for unmanned cosmetics sales, and according to one aspect of the present invention, the device for providing guidance for unmanned cosmetics sales comprises: an image input unit for receiving image data from a camera filming the inside of a store; a diagnosis result input unit for receiving a diagnosis result including a score for an evaluation item including at least one of moisture, pigmentation, elasticity, and oil from a skin diagnosis device installed in the store; an object identification unit for detecting at least one object corresponding to a user from the image data inputted to the image input unit, selecting one object located at a first position preset in the image data at the time when the diagnosis result is inputted to the diagnosis result input unit, and assigning a user code for distinguishing the object from other users; a matching unit for matching and storing the diagnosis result inputted from the diagnosis result input unit and the user code assigned by the object identification unit in response to the input of the diagnosis result; an object tracking unit for tracking a change in the position of each object assigned a user code by the object identification unit among objects detected from the image data inputted to the image input unit; a cosmetic barcode for recognizing a cosmetic barcode in response to a user placing a cosmetic at a preset position, and storing a pre-stored corresponding to each cosmetic according to the cosmetic barcode. It includes a cosmetic recognition unit that selects one cosmetic information from among a plurality of cosmetic information including a cosmetic name, a diagnosis result extraction unit that extracts a diagnosis result stored in a matching unit according to the location of an object being tracked by the object tracking unit when cosmetic information is selected in the cosmetic recognition unit, a guidance generation unit that generates a guidance using the diagnosis result extracted from the diagnosis result extraction unit and the cosmetic information selected by the cosmetic recognition unit, and a display unit that outputs the guidance generated by the guidance generation unit.

Description

{APPARATUS FOR PROVINDING GUIDANCE TO SELL COSMETICS UNATTENDED}

The present invention relates to a technology that enables unmanned cosmetics sales, and more specifically, to a technology that provides guidance on cosmetics unmanned.

When visiting a small cosmetics store, customers face the challenge of trying out products in confined spaces and interacting with staff, making it difficult to freely browse the selection. Consequently, customers are flocking to larger stores, where they can experience products with less interference from staff. However, these larger stores require more space and incur higher management and labor costs.

In this regard, unmanned vending technology is being developed that eliminates the burden on employees and reduces operating costs such as labor costs. The COVID-19 pandemic has led to active development and application of technologies for contactless sales, with unmanned vending services now being offered in various sectors, including food service, convenience stores, and movie theaters.

However, cosmetics require consulting services, such as recommending products that suit the customer's skin type, depending on the customer's needs, making it difficult to apply conventional unmanned sales technology that only provides simple sales services.

In relation to this, an unmanned cosmetics sales technology has been introduced that recommends cosmetics based on the results of diagnosing the user's skin. However, this conventional technology has a limitation in that it can only accept one user at a time because it cannot initiate a configuration to distinguish and recommend users when multiple users enter.

Republic of Korea Patent Application No. 10-2021-0063303 Republic of Korea Patent Application No. 10-2021-0098459

The purpose of the present invention is to solve the above problem, and to provide a guidance provision device for unmanned cosmetics sales that distinguishes users based on their locations detected in an image and provides customized guidance based on the diagnosis results and recognized cosmetics of each user.

The purpose of the present invention is not limited to the purposes mentioned above, and other purposes not mentioned can be clearly understood from the description below.

In order to achieve the above-described object, a device for providing guidance for unmanned cosmetics sales according to one aspect of the present invention comprises: an image input unit for receiving image data from a camera filming the inside of a store; a diagnosis result input unit for receiving a diagnosis result including a score for an evaluation item including at least one of moisture, pigmentation, elasticity, and oil from a skin diagnosis device installed in the store; an object identification unit for detecting at least one object corresponding to a user from the image data inputted to the image input unit and selecting one of the objects located at a first position preset in the image data at the time when the diagnosis result is inputted to the diagnosis result input unit and assigning a user code for distinguishing the object from other users; a matching unit for matching and storing the diagnosis result inputted from the diagnosis result input unit and the user code assigned by the object identification unit in response to the input of the diagnosis result; an object tracking unit for tracking a change in the position of each object assigned a user code by the object identification unit among the objects detected from the image data inputted to the image input unit; a cosmetic barcode for recognizing a cosmetic barcode in response to a user placing a cosmetic at a predetermined position and storing one of a plurality of cosmetic information including a cosmetic name pre-stored in response to each cosmetic according to the cosmetic barcode. It includes a cosmetics recognition unit that selects cosmetics information, a diagnosis result extraction unit that extracts a diagnosis result stored in a matching unit according to the location of an object being tracked by the object tracking unit when cosmetics information is selected in the cosmetics recognition unit, a guidance generation unit that generates a guidance using the diagnosis result extracted in the diagnosis result extraction unit and the cosmetics information selected in the cosmetics recognition unit, and a display unit that outputs the guidance generated in the guidance generation unit.

According to the present invention, there is provided an effect of providing a guidance provision device for unmanned cosmetics sales that distinguishes users based on their locations detected in an image and provides customized guidance based on the diagnosis results and recognized cosmetics of each user.

According to the present invention, the usability of an unmanned cosmetics sales store is improved by providing customized guidance to each user even when multiple users use the store.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

FIG. 1 is a block diagram of a device for providing guidance for unmanned cosmetics sales according to one embodiment of the present invention.
Figure 2 is a flowchart of a method for providing guidance for unmanned cosmetics sales according to another embodiment of the present invention.
FIG. 3 is a first example showing guidance provided to a user in embodiments of the present invention.
FIG. 4 is a second example showing guidance provided to a user in embodiments of the present invention.
FIG. 5 is an exemplary diagram showing the internal layout of an unmanned cosmetics sales store to which a guide provision device for unmanned cosmetics sales according to an embodiment of the present invention is applied.

The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. These embodiments are provided only to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the description of the claims. Meanwhile, the terminology used in this specification is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular includes the plural unless specifically stated otherwise.

FIG. 1 is a block diagram of a device for providing information for unmanned cosmetics sales according to one embodiment of the present invention, and FIG. 2 is a flowchart of a method for providing information for unmanned cosmetics sales according to another embodiment of the present invention.

A method for providing guidance for unmanned cosmetics sales according to another embodiment of the present invention can be performed by a guidance providing device for unmanned cosmetics sales according to one embodiment of the present invention.

Meanwhile, the blocks in the attached block diagram and the steps in the flowchart may be implemented as computer instructions that are loaded into the processor or memory of an electronic device capable of data processing (e.g., a general-purpose computer, a special-purpose computer, a portable laptop computer, a network computer) and perform designated functions. Since these computer program instructions can be stored in a computer-readable memory, the functions described in the blocks in the block diagram or the steps in the flowchart may also be produced as a product that includes instruction means for performing them.

For the convenience of explanation below, the drawing symbols for identical configurations are used interchangeably and duplicate explanations are avoided.

A device (30) for providing guidance for unmanned cosmetics sales can be installed in an unmanned cosmetics sales store equipped with a camera (10) that captures the interior of the store and generates image data, and a skin diagnosis device (20) that generates a diagnosis result of diagnosing the user's skin according to the user's input.

Referring to FIG. 5, an unmanned cosmetics sales store to which an information provision device (30) for unmanned cosmetics sales according to the present invention is applied may include a display space for displaying cosmetics, an experience space for experiencing cosmetics directly and confirming the appearance of applying cosmetics, a cash register for paying for cosmetics, a camera (10) for photographing the inside of the store, and a skin diagnosis device (20) for generating a diagnosis result of diagnosing the user's skin according to the user's input.

A camera (10) is installed inside an unmanned cosmetics sales store and can generate image data by filming the inside of the unmanned cosmetics sales store and output the generated image data to the outside.

A skin diagnosis device (20) is installed in an unmanned cosmetics sales store and is equipped with a photographing unit for taking pictures of the user's skin and an analysis unit for analyzing the image of the user's skin to generate a diagnosis result including a score for each evaluation item including at least one of moisture, pigmentation, elasticity, and oil, so that the diagnosis result can be generated according to the user's input and the generated diagnosis result can be output to the outside.

The diagnostic results may include scores for multiple evaluation items corresponding to moisture, pigmentation, elasticity, and oiliness, respectively.

The skin diagnosis device (20) may be any of the existing skin diagnosis devices that generate diagnosis results including scores for each evaluation item including at least one of moisture, pigmentation, elasticity, and oil.

A device (30) for providing guidance for unmanned cosmetics sales is electrically connected to a camera (10) and a skin diagnosis device (20) to receive image data from the camera (10) and diagnosis results from the skin diagnosis device (20).

In addition, the device (30) for providing guidance for unmanned cosmetics sales can recognize cosmetics in response to a user placing the cosmetics in a predetermined location, and generate and display guidance using the diagnosis results and information on the recognized cosmetics.

According to the present invention, by providing guidance based on the skin diagnosis results of a user visiting an unmanned cosmetics sales store and the cosmetics recognized by the user, customized cosmetics counseling is made possible unmanned.

In particular, the device (30) for providing guidance for unmanned cosmetics sales according to the present invention is characterized in that it identifies a user based on the location of an object detected in image data at each time point when a diagnosis result is input and when a cosmetic is recognized, and provides customized guidance to a user who has recognized a cosmetic among users in the store.

Specifically, a device (30) for providing guidance for unmanned cosmetics sales may include an image input unit (310), a diagnosis result input unit (320), an object identification unit (330), a matching unit (340), an object tracking unit (350), a cosmetics recognition unit (360), a diagnosis result extraction unit (370), a guidance generation unit (380), and a display unit (390).

The video input unit (310) can receive video data from a camera (10) that films the inside of the store (S101).

The diagnosis result input unit (320) can receive a diagnosis result including a score for an evaluation item including at least one of moisture, pigmentation, elasticity, and oil from a skin diagnosis device (20) installed in a store (S103).

The diagnostic results may include scores for multiple evaluation items corresponding to moisture, pigmentation, elasticity, and oiliness, respectively.

For example, if the minimum score for multiple evaluation items corresponding to moisture, pigmentation, elasticity, and oil is 0 and the maximum score is 10, the diagnosis result may include a score of 3 for the evaluation item moisture, a score of 6 for the evaluation item pigmentation, a score of 5 for the evaluation item elasticity, and a score of 4 for the evaluation item oil.

The object identification unit (330) detects at least one object corresponding to a user from image data inputted to the image input unit (310), and at the time when the diagnosis result is inputted to the diagnosis result input unit (320), selects one object located at a preset first position in the image data and assigns a user code to distinguish it from other users (S105).

The object identification unit (330) detects an object corresponding to a user from image data input into the image input unit (310) in response to the diagnosis result input into the diagnosis result input unit (320), and assigns a user code to one of the detected objects located at a preset first position to distinguish the user from other users (S105).

Here, the first position can be preset to correspond to the point where the skin diagnosis device is displayed in the image data input to the image input unit (310).

The first position can be preset as the coordinates of any pixel included in the area where the skin diagnosis device is displayed in the image data.

The object identification unit (330) receives learning data labeling objects corresponding to people in an image in response to the diagnosis results input from the diagnosis result input unit (320), and uses a learning model learned in advance to detect objects corresponding to people in image data input from the outside, thereby detecting at least one object corresponding to a user from image data input to the image input unit (310).

In addition, the object identification unit (330) can assign a user code to the object that is closest to the coordinates of the first location where the center coordinates of the object are preset among the detected objects to distinguish the user from other users (S105).

At this time, the closest object means the object with the smallest distance calculated based on the difference between the center coordinate and the coordinate of the first position.

According to the above configuration, when a user uses a skin diagnosis device (20) and a diagnosis result is input, the object identification unit (330) assigns a user code to an object detected at a location where the skin diagnosis device is displayed in the image data, so that the user who diagnosed the skin can be distinguished by the user code.

That is, the object identification unit (330) enables an object located near the skin diagnosis device to be identified as the user who performed the skin diagnosis when the skin diagnosis result is input among multiple users.

The matching unit (340) can match and store the diagnosis results input from the diagnosis result input unit (320) and the user code assigned to the object detected by the object identification unit (330) corresponding to the input diagnosis results (S107).

The object tracking unit (350) can track the change in position of each object detected from the image data input by the image input unit (310) and assigned a user code by the object identification unit (330) (S109).

The object tracking unit (350) matches and stores features according to the pixel value distribution of an area including an object to which a user code has been assigned in response to the object identification unit (330) assigning a user code to an object in the image data, and detects an object corresponding to each user code in the image data according to the similarity with the features stored in advance for each user code, and can track the change in the location of each object to which a user code has been assigned (S109).

The object tracking unit (350) detects an object among the objects detected in the image data inputted to the image input unit (310) after the object identification unit (330) assigns a user code to the object, and the object whose features according to the distribution of pixel values have the highest similarity to the features stored in advance for the user code is detected as an object corresponding to each user code, thereby tracking the change in the location of the object assigned the user code (S109).

Here, the similarity between features can be calculated by applying the mean square difference similarity and summing the squares of the differences in pixel values in each pixel included in the area containing each object.

The cosmetics recognition unit (360) recognizes a cosmetics barcode in response to a user placing a cosmetics at a predetermined location, and selects one of a plurality of cosmetics information items including the name of the cosmetics stored in advance in response to each cosmetic according to the cosmetics barcode.

The cosmetics recognition unit (360) can store a plurality of cosmetic information consisting of a cosmetic code, a cosmetic name, and a plurality of ingredient names indicating ingredients contained in the cosmetics for each of a plurality of cosmetics.

The cosmetics recognition unit (360) is equipped with a barcode recognition means for receiving a cosmetics barcode in response to a user placing a cosmetics at a predetermined location, and can determine whether a cosmetics barcode is input (S111).

The cosmetics recognition unit (360) can select cosmetics information that includes a cosmetics code corresponding to the cosmetics code recognized in the input cosmetics barcode among the pre-stored cosmetics information when a cosmetics barcode is input in response to a user's action of placing a cosmetics on a barcode recognition means (S113).

When cosmetic information is selected in the cosmetic recognition unit (360), the diagnosis result extraction unit (370) can extract the diagnosis result stored in the matching unit (340) according to the location of the object to which the user code is assigned and being tracked in the object tracking unit (350) (S115).

The diagnosis result extraction unit (370) selects an object located at a preset second position in the image data at the time when the cosmetic information is selected in the cosmetic recognition unit (360) among at least one object whose position change was tracked in the object identification unit (330) in response to the selection of cosmetic information in the cosmetic recognition unit (360), and extracts one diagnosis result in which the user code of the object located at the second position is matched among the diagnosis results matched with each user code in the matching unit (340) (S115).

Here, the second position can be preset to correspond to the point where the device (30) providing guidance for unmanned cosmetics sales is displayed in the video data.

The second position can be preset as the coordinates of any one pixel included in the area where the device (30) providing guidance for unmanned cosmetics sales is displayed in the image data.

The diagnosis result extraction unit (370) selects one object whose center coordinates are closest to the coordinates of the second position preset in the image data at the time when cosmetic information is selected in the cosmetic recognition unit (360) among at least one object whose position change was tracked in the object identification unit (330), and extracts one diagnosis result that matches the user code of the selected object among the diagnosis results stored in the matching unit (340) (S115).

That is, the diagnosis result extraction unit (370) can extract the diagnosis result of the user who recognized the cosmetics based on an object adjacent to the device providing information for unmanned cosmetics sales and assigned a user code at the time when the cosmetics information is selected from the image data.

The guidance generation unit (380) can generate a guidance message using the diagnosis results extracted from the diagnosis results extraction unit (370) and the cosmetic information selected from the cosmetic recognition unit (360) (S117).

The guidance generation unit (380) selects an evaluation item corresponding to the lowest score among the evaluation items including at least one of moisture, pigmentation, elasticity, and oil included in the diagnosis results extracted from the diagnosis result extraction unit (370), and generates a guidance by inputting the cosmetic name of the cosmetic information selected from the cosmetic recognition unit (360) and the evaluation item text preset for the selected evaluation item into a preset template (S117).

The guidance generation unit (380) can store a plurality of valid information consisting of a plurality of evaluation item texts preset to display each evaluation item for each of a plurality of evaluation items, a preset template to generate a guidance, and an item code for one of moisture, pigmentation, elasticity, and oil for each of a plurality of cosmetic names.

Valid information may consist of the cosmetic product name and one of multiple item codes corresponding to moisture, pigmentation, elasticity, and oil.

The item code for each cosmetic name can be preset based on the user's input to indicate the item for which the cosmetic provides an improvement effect among moisture, pigmentation, elasticity, and oil corresponding to each cosmetic name.

The guidance generating unit (380) selects an evaluation item corresponding to the lowest score among the scores for evaluation items including at least one of moisture, pigmentation, elasticity, and oil included in the diagnosis results extracted by the diagnosis result extracting unit (370), extracts valid information on the cosmetic name of the cosmetic information selected by the cosmetic recognition unit (360), and, depending on whether the item according to the item code of the extracted valid information matches the selected evaluation item, if the item according to the item code of the valid information matches the selected evaluation item, recommends the use of the cosmetic recognized by the user for the item with the lowest score, and brings in a first template including a text area in which a preset first basic text is displayed, a first input area for displaying the evaluation item name in the text area, separated from the first basic text, and a second input area for displaying the cosmetic name in the text area, separated from the first basic text and the first input area, and inputs the preset evaluation item text for the selected evaluation item in the first input area of the first template and inputs the selected cosmetic information in the second input area. A guide can be generated by entering the name of the cosmetic product (S117).

The guidance generation unit (380) selects a second cosmetic name, which is one of the cosmetic names that matches the valid information including the item code of the item corresponding to the selected evaluation item, if the item according to the item code of the valid information extracted for the first cosmetic name, which is the cosmetic name of the cosmetic information selected by the cosmetic recognition unit (360), does not match the selected evaluation item, and for the item with the lowest score, it can bring in a second template that is stored in advance to recommend a cosmetic other than the cosmetic recognized by the user, and generate a guidance by inputting the preset evaluation item text, the first cosmetic name, and the second cosmetic name for the selected evaluation item into the second template.

Here, the second template may include a text area in which a preset second basic text is displayed, a first input area for displaying an evaluation item name in the text area and separated from the second basic text, a second input area for displaying a first cosmetic product name in the text area and separated from the second basic text and the first input area, and a second input area for displaying a second cosmetic product name in the text area and separated from the second basic text, the first input area, and the second input area.

The guidance generation unit (380) can generate a guidance text recommending cosmetics other than the cosmetics recognized by the user by inputting a preset evaluation item text for the selected evaluation item in the first input area of the second template, inputting the name of the first cosmetic product in the first input area, and inputting the name of the second cosmetic product in the second input area.

The guidance generation unit (380) can generate a guidance recommending a cosmetic product recognized by the user or a guidance recommending a cosmetic product other than the cosmetic product recognized by the user, depending on whether the cosmetic product recognized by the user is the cosmetic product that is set to be effective for the evaluation item with the lowest score in the user's skin diagnosis results.

Therefore, through guidance, the use of cosmetics recognized by the user is recommended based on the user's diagnosis results, or cosmetics other than those recognized by the user are recommended, enabling unmanned customized counseling for the user.

The display unit (390) can output the guidance text generated by the guidance text generation unit (380) (S119).

The display unit (390) can display the guidance text generated by the guidance text generation unit (380) so that the user can check it (S119).

Referring to FIG. 3, when the matching item for the cosmetic recognized by the user matches the evaluation item with the lowest score in the user's diagnosis result, a guide (1100) including a first basic text for recommending the use of the cosmetic recognized by the user, a first input area (1110) in which the evaluation item with the lowest score in the user's diagnosis result is displayed, and a second input area (1110) in which the name of the cosmetic recognized by the user is displayed may be displayed on the display unit (390).

Referring to FIG. 4, if the matching item for the cosmetic recognized by the user does not match the evaluation item with the lowest score in the user's diagnosis result, a display unit (390) may display a guide (1200) including a second basic text for recommending the use of a cosmetic other than the cosmetic recognized by the user, a first input area (1210) in which the evaluation item with the lowest score in the user's diagnosis result is displayed, a second input area (1220) in which the name of the cosmetic recognized by the user is displayed, and a third input area (1230) in which the name of another cosmetic that provides an improvement function for the evaluation item with the lowest score in the user's diagnosis result is displayed, in addition to the cosmetic recognized by the user.

According to the present invention, it is possible to distinguish users who have recognized cosmetics based on the location of the user detected in the image, and to provide guidance on whether it is okay for the user to use the recognized cosmetics or whether it is okay to use other cosmetics based on the user's skin diagnosis results.

According to the present invention, by distinguishing users who have recognized customized cosmetics based on skin diagnosis based on image analysis, customized guidance can be provided to each user even when multiple users use an unmanned store.

Accordingly, it is expected that the usability and convenience of unmanned cosmetics stores will be improved.

Those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering its technical spirit or essential characteristics. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims below rather than the detailed description above, and all changes or modifications derived from the claims and their equivalents should be construed as being included within the scope of the present invention.

10: Camera
20: Skin diagnostic device
30: Device for providing guidance for unmanned cosmetics sales
310: Video input section
320: Diagnostic result input section
330: Object identification section
340: Matching Department
350: Object Tracking Unit
360: Cosmetics Recognition Department
370: Diagnostic result extraction section
380: Guidance generation section
390: Display

Claims (3)

A video input unit that receives video data from a camera filming the inside of a store;
A diagnosis result input unit that receives a diagnosis result including a score for at least one of the evaluation items including moisture, pigmentation, elasticity, and oil from a skin diagnosis device installed in a store;
An object identification unit that detects at least one object corresponding to a user from image data input to the image input unit, selects one object located at a preset first position from the image data at the time when the diagnosis result is input to the diagnosis result input unit, and assigns a user code to distinguish it from other users;
A matching unit that matches and stores the diagnosis results input in the above diagnosis result input unit and the user code assigned in the above object identification unit in response to the input diagnosis results;
An object tracking unit that tracks changes in the location of each object detected from the image data input to the image input unit and assigned a user code by the object identification unit;
A cosmetic recognition unit that recognizes a cosmetic barcode in response to a user placing a cosmetic at a predetermined location and selects one cosmetic information from among a plurality of cosmetic information items including a cosmetic name stored in advance corresponding to each cosmetic according to the cosmetic barcode;
A diagnosis result extraction unit that extracts the diagnosis result stored in the matching unit according to the location of the object being tracked by the object tracking unit when cosmetic information is selected in the cosmetic recognition unit;
A guidance generation unit that generates a guidance using the diagnosis results extracted from the above diagnosis result extraction unit and the cosmetic information selected from the above cosmetic recognition unit; and
A device for providing guidance for unmanned cosmetics sales, including a display unit that outputs guidance generated by the guidance generation unit. In the first paragraph,
The above diagnostic result extraction section
In response to the selection of cosmetic information in the cosmetic recognition unit, the object located at the second position preset at the time when the cosmetic information was selected in the cosmetic recognition unit is selected from among at least one object whose position change was being tracked in the object identification unit, and in the matching unit, one diagnosis result is extracted in which the user code of the object located at the second position matches among the diagnosis results matched with each user code.
A device for providing guidance for unmanned cosmetics sales. In the second paragraph,
The above guidance generation section
Selecting an evaluation item corresponding to the lowest score among the scores for evaluation items including at least one of moisture, pigmentation, elasticity, and oil included in the diagnosis results extracted from the above diagnosis result extraction unit, and generating a guide by entering the cosmetic name of the cosmetic information selected from the above cosmetics recognition unit and the preset evaluation item text for the selected evaluation item into a preset template.
A device for providing guidance for unmanned cosmetics sales.